Apache Mesos

cxin

http://www.mesosproject.org/
Apache Mesos
Dynamic Resource Sharing for Clusters



Download ·   Documentation ·   Mailing Lists ·   Research ·   FAQ
　　Apache Mesos is a cluster manager that provides efficient resource    isolation and sharing across distributed applications,   or frameworks.   It can run Hadoop,   MPI,    Hypertable,   Spark (a new   framework for low-latency interactive and   iterative jobs), and other applications.   Mesos is open source in the Apache Incubator.
http://www.mesosproject.org/images/mesos.png
You can use Mesos to:

• Run Hadoop, MPI, Spark, and other frameworks on a dynamically shared pool of nodes.
  
• Run multiple instances of Hadoop on the same cluster to       isolate production and experimental jobs,       or even multiple versions of Hadoop.
  
• Run long-lived services (e.g. Hypertable and HBase) on the same nodes       as batch applications and share resources between them.
  
• Build new cluster computing frameworks without reinventing low-level        facilities for farming out tasks, and have them coexist with        existing ones.
  

Features

• Fault-tolerant replicated master using     ZooKeeper.
  
• Scalability to 10,000s of nodes using fast, event-driven C++ implementation.
  
• Isolation between tasks with    Linux Containers.
  
• Multi-resource scheduling (memory and CPU aware).
  
• Efficient application-controlled scheduling mechanism (resource offers) that lets frameworks achieve their own placement goals (e.g. data locality).
  
• Java, Python and C++ APIs for developing new parallel applications.
  
• Web UI for viewing cluster state.
  

Users
　　Mesos is being used to manage clusters at Twitter, Conviva, UC Berkeley, and UC San Francisco.
Research Project
　　Mesos started out as a research project in the UC Berkeley AMP Lab before being open sourced and joining the Apache Incubator. You can find more about the research behind Mesos, including our papers, on our research page.
Apache Incubator Notice
　　Apache Mesos is an effort undergoing incubation at The Apache Software Foundation (ASF), sponsored by the name of Apache Incubator. Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF.