Abstract: The distribution and meaning of natural annotations on large datasets are discussed. The proposed research on word extraction shows the positive potential of both implicit and explicit natural annotation in word segmentation. Experiments on word extraction indicates that the implicit natural annotation derived from language laws and patterns are more powerful in splitting character strings in raw corpora.

Key words: natural annotation, Chinese word segmentation, word extraction, large-scale corpora

摘要： 以中文分词为应用目标, 将大规模语料库上存在的自然标注信息分为显性标注信息与隐性标注信息, 分别考察了它们的分布和对大数据集上语言计算的影响。结果表明, 两者都直接或间接地表达了作者对语言的分割意志, 因而对分词具有积极的影响。通过词语抽取测试, 发现在缺乏丰富显性标注信息的文本中, 来自语言固有规律的自然标注信息对字符串有着强大的分割性能。

关键词: 自然标注信息, 中文分词, 词语抽取, 大规模语料库