python html parse

吴贺华

• bs4:转换成unicode编码，http://www.crummy.com/software/BeautifulSoup/
  
  
  • 
    from bs4 import BeautifulSoup
    soup = BeautifulSoup(open("index.html"))
    soup = BeautifulSoup("data")
    
  • Beautiful Soup将复杂HTML文档转换成一个复杂的树形结构,每个节点都是Python对象,所有对象可以归纳为4种: Tag , NavigableString ,BeautifulSoup , Comment .
    
  • 
    from bs4 import SoupStrainer
    only_a_tags = SoupStrainer("a")
    only_tags_with_id_link2 = SoupStrainer(id="link2")
    def is_short_string(string):
    return len(string) < 10
    only_short_strings = SoupStrainer(text=is_short_string)
    
  • 
    BeautifulSoup(html_doc, "html.parser", parse_only=only_a_tags)
    
  
  
• lxml: python 对 libxml 的包装
  
• html5lib：纯python实现