php-ext-trie-filter 过滤关键词

昊漫玉

关键词过滤扩展，用于检查一段文本中是否出现敏感词，基于Double-Array Trie 树实现

 

安装 libdatrie , 需要 libdatrie-0.2.4 或更新的版本

 

它依赖 libiconv .

安装:

================================


wget http://ftp.gnu.org/pub/gnu/libiconv/libiconv-1.14.tar.gz

tar -zxf libiconv-1.14.tar.gz

cd libiconv-1.14

./configure

make && make install

 

libdatrie 下载地址:

http://linux.thai.net/~thep/datrie/datrie.html#Download

 

安装:

================================


wget ftp://linux.thai.net/pub/ThaiLinux/software/libthai/libdatrie-0.2.4.tar.gz

tar -zxf libdatrie-0.2.4.tar.gz

cd libdatrie-0.2.4

 

./configure --prefix=/usr/local/libdatrie/

make ICONV_LIBS='/usr/local/lib/libiconv.so'

make install

 

安装 PHP 扩展

================================


wget https://github.com/wulijun/php-ext-trie-filter/archive/master.zip

unzip master.zip

cd php-ext-trie-filter-master/

phpize

./configure --with-php-config=/usr/local/php/bin/php-config --with-trie_filter=/usr/local/libdatrie/

make && make install

 

将生成的 trie_filter.so 文件复制到 php 扩展目录,并在 php.ini 中添加该文件

参考: http://blog.sina.com.cn/s/blog_5f54f0be0101dxqp.html

 

安装完后，在 phpinfo 里可以看到 trie_filter 扩展

 

使用示例

$arrWord = array('word1', 'word2', 'word3');
$resTrie = trie_filter_new(); //create an empty trie tree
foreach ($arrWord as $k => $v) {
trie_filter_store($resTrie, $v);
}
// 生成关键词词典.
trie_filter_save($resTrie, __DIR__ . '/blackword.tree');

// 加载关键词词典
$resTrie = trie_filter_load(__DIR__ . '/blackword.tree');

$strContent = 'hello word2 word1';
// 查找关键词
$arrRet = trie_filter_search($resTrie, $strContent);
print_r($arrRet); //Array(0 => 6, 1 => 5)

echo substr($strContent, $arrRet[0], $arrRet[1]); //word2

$arrRet = trie_filter_search_all($resTrie, $strContent);
print_r($arrRet); //Array(0 => Array(0 => 6, 1 => 5), 1 => Array(0 => 12, 1 => 5))

$arrRet = trie_filter_search($resTrie, 'hello word');
print_r($arrRet); //Array()

// 获得查找结果树
trie_filter_free($resTrie);