1 1. Solr
1.1 Features
1.2 Pros & Cons
1.3 References
2 2. Senseidb
2.1 Features
2.2 Pros & Cons
2.3 为何没有直接用Solr?
2.4 References
3 3. elasticsearch
3.1 Features
3.2 Pros & Cons
3.3 References
4 4. Conclusion
5 其它参考文献
比较的时候,主要关注以下几个方面:
Clustering
Scalability on Storage and Service
High Availability Considerations
Features
Flexibility
1. Solr
很显然, Solr跟Lucene是一家,所以,对Lucene做了很多扩展,与lucene的集成也比较好,而且,业界貌似求稳的都会选择Solr来构建他们的搜索体系。
但SolrCloud才是最终的理想解决方案,而SolrCloud还没有production-ready。
下面是Solr相关的架构图:
1.1 Features
Solr的首页上对自己的特性罗列阐述的很详细了,这里不再赘述。
1.2 Pros & Cons
Pros
成熟且验证过的方案
文档资料丰富
社区活跃
plugin extension points
Cons
貌似体系比较庞杂, replication的架构扩展有稍许问题?!
1.3 References
New SolrCloud Design
Scaling Lucene and Solr
Turbocharging Solr Index Replication with BitTorrent
funny and sparkling idea by introducing BitTorrent replication mechanism *****
Distributed Searching
Carrot2-OSS framework for building search clustering engine
Solr search results clustering is based on the Carrot2 real-time document clustering engine.
Clustering Component
New SolrCloud Design
SolrCloud
UniqueKey
Solr Near Realtime Search
will be added in Solr4, currently available in trunk
Scaling Solr Indexing with SolrCloud, Hadoop and Behemoth
2. Senseidb
architecture of sensei
2.1 Features
主要解决高速索引更新的问题;
底层是zoie的“2-swapping-in-memory-index + 1-on-disk-index”索引结构支持
需要定义schema;
通过Gateway可以接入多种数据源;
通过BQL或者REST API,甚至各种语言bindings进行数据查询;
支持通过hadoop MR job批量更新数据索引;
2.2 Pros & Cons
Pros
高速索引更新
多数据源接入
灵活的访问接口
与hadoop生态的集成
优秀的分布式扩展能力
Cons
static schema
application side versioning maitaining
2.3 为何没有直接用Solr?
摘录在John Wang的访谈片段:
Sensei leverages Lucene.
We weren’t able to leverage Solr because of the following requirements:
* High update requirement, 10’s of thousands updates per second in to the system
* Real distributed solution, current Solr’s distributed story has a SPOF at the master, and Solr Cloud is not yet completed.
* Complex faceting support. Not just your standard terms based faceting. We needed to facet on social graph, dynamic time ranges and many other interesting faceting scenarios. Faceting behavior also needs to be highly customizable, which is not available via Solr.
2.4 References
Introducing SenseiDB 1.0: an open-source, distributed, realtime, semi-structured database
Sensei: distributed, realtime, semi-structured database
3. elasticsearch
很新,当前0.19RC3版本, 文档缺乏。不过, ES确实有很多值得喝彩的地方。
3.1 Features
Schema-Free | Schemaless
feed index engine with JSON formatted documents
Query by Lucene based query string or JSON based query DSL over HTTP or Native Java;
shards and replicas, LB and routings
cloud integration
multiple search types
multiple data sources integration with River
many more...
3.2 Pros & Cons
Pros
许多灵活, 优秀的特性(见features列表)
作者拥有多年在搜索领域的涉猎经验
senseidb的pros它也基本都有
Cons
3.3 References
quick intro to elastic search
Flume, Hive and realtime indexing via ElasticSearch
The Future of Compass & ElasticSearch
Elastic Search: Distributed, Lucene-based Search Engine
ElasticSearch at berlinbuzzwords 2010
Elastic Search Vs. Apache Solr
Your Data, Your Search
Search Engine Time Machine
transient状态与持久化状态的结合, write behind策略
NoSQL, Yes Search
Geo Location and Search
Zero Conf Persistency
Local Gateway (Local Storage | Local FileSystem)
The River
ES里River的概念跟Senseidb里Gateway的概念相近, 是数据源通道的意思,可以根据不同的数据源给出不同的River实现,比如基于MysqlBinlog的River, 基于Hbase的River,或者RabbitMQ River,CouchDB River etc.
Percolator
这个Percolator是ES里的概念,不要跟Google的Percolator混淆
Versioning
Optimistic Concurrency Control
New Search Types
Introduce count and scan search types, the latter can be used to scroll large result set
Data Visualization with ElasticSearch and Protovis
Distributed Diagram (Video)
Road to a Distributed Search Engine (Video)*****
4. Conclusion
All are based on Lucene.
All are distributed.
senseidb shards with multi-write?!
solr shards with master-slaves and slave pull strategy;
elasticsearch shards with primary-secondary push strategy;
All do partitioning in document granularity, All require some unique key for each document(optional for some situations);
Sensei is good at real-time index update; Solr is good at stable and wide adoption; Elasticsearch is good at flexible and good ideas;
5 其它参考文献
Lily架构简介
在自己的lily node里实现了multiwrite + wal+ message queue的数据分发, 没有充分利用现有系统中各个组件/系统的能力(虽然是基于hbase的table实现的), 部分上来讲把事情搞复杂了。
引自:http://afoo.me/notes-on-senseidb-solr-and-elasticsearch.html
运维网声明
1、欢迎大家加入本站运维交流群:群②:261659950 群⑤:202807635 群⑦870801961 群⑧679858003
2、本站所有主题由该帖子作者发表,该帖子作者与运维网 享有帖子相关版权
3、所有作品的著作权均归原作者享有,请您和我们一样尊重他人的著作权等合法权益。如果您对作品感到满意,请购买正版
4、禁止制作、复制、发布和传播具有反动、淫秽、色情、暴力、凶杀等内容的信息,一经发现立即删除。若您因此触犯法律,一切后果自负,我们对此不承担任何责任
5、所有资源均系网友上传或者通过网络收集,我们仅提供一个展示、介绍、观摩学习的平台,我们不对其内容的准确性、可靠性、正当性、安全性、合法性等负责,亦不承担任何法律责任
6、所有作品仅供您个人学习、研究或欣赏,不得用于商业或者其他用途,否则,一切后果均由您自己承担,我们对此不承担任何法律责任
7、如涉及侵犯版权等问题,请您及时通知我们,我们将立即采取措施予以解决
8、联系人Email:admin@iyunv.com 网址:www.yunweiku.com