Solr:Text analysis

rhces

　　

 
 
　　

 

 

 

 

 
　　------------------------------------------------------------------------------------------------------------------------------------

Defining a custom field type for microblog text

　　

 
　　

 
　　

 
　　------------------------------------------------------------------------------------------------------------------------------------

Advanced text analysis

　　

 
 
 
　　How do you select the right text analyzer during indexing? Assuming you want to index all your documents regardless of language in the same index, a simple solution would be to use a unique field for each language. Suppose we want to index French tweets in our microblog search application. We could define the following field:

<field name="text_fr" type="text_microblog_fr"
indexed="true" stored="true" />
　　

-------------------------------------------------------------------------------------------------------------------------------------
　　Integrate jcseg with solr to deal with chinese tokenizer
　　1.cp  jcseg-core-1.9.5.jar and   jcseg-solr-1.9.5.jar  to solr-4.7.0/example/solr-webapp/webapp/WEB-INF/lib/
　　2. cp lexicon dir to solr-4.7.0/example/solr-webapp/webapp/WEB-INF/lib/
　　3. alter schema.xml add fildtype

<fieldtype name="textComplex" class="solr.TextField">
<analyzer>
<tokenizer class="org.lionsoul.jcseg.solr.JcsegTokenizerFactory" mode="complex"/>
    <filter class="solr.StopFilterFactory" ignoreCase="true" words="lang/stopwords_ch.txt"/>
</analyzer>
</fieldtype>