brill标注器案例

http://tieba.baidu.com/p/6070002023

from nltk.corpus import brown
brown_tagged_sents=brown.tagged_sents(categories=‘news’)
brown_sents = brown.sents(categories=‘news’)

import nltk
nltk.download(‘brown’)
nltk.download(‘universal_tagset’)
import nltk.tag.brill
from nltk.corpus import brown

brown_tagged_sents = brown.tagged_sents(categories=‘news’, tagset=‘universal’)
brown_sents = brown.sents(categories=‘news’)
size = int(len(brown_tagged_sents) * 0.9)
train_sents = brown_tagged_sents[:size]
#set up first stage of tagging
print(size)

[nltk_data] Downloading package brown to
[nltk_data] C:\Users\Lenovo\AppData\Roaming\nltk_data…
[nltk_data] Package brown is already up-to-date!
[nltk_data] Downloading package universal_tagset to
[nltk_data] C:\Users\Lenovo\AppData\Roaming\nltk_data…
[nltk_data] Package universal_tagset is already up-to-date!
4160

import re
patterns=[
(r’.*ingKaTeX parse error: Expected 'EOF', got '#' at position 13: ', 'VBG'), #̲ gerunds (r…’, ‘VBD’), # simple past
(r’.esKaTeX parse error: Expected 'EOF', got '#' at position 13: ', 'VBZ'), #̲ 3rd singular p…’, ‘MD’), # modals
(r’.‘s′,′NN', 'NN′,′NN’), # possessive nouns
(r’.sKaTeX parse error: Expected 'EOF', got '#' at position 13: ', 'NNS'), #̲ plural nouns …’, ‘CD’), # cardinal numbers
(r’.’, ‘NN’) # nouns (default)
]

backoff=nltk.RegexpTagger(patterns)

baseline_tagger = nltk.UnigramTagger(train_sents, backoff=backoff)
baseline_tagger = nltk.UnigramTagger(train_sents, backoff=backoff)
#set up second stage of tagging
tt = nltk.tag.brill_trainer.BrillTaggerTrainer(baseline_tagger,
nltk.tag.brill.brill24())
brill_tagger = tt.train(train_sents, max_rules=15)
#evaluation
print(brill_tagger.rules())
(Rule(‘090’, ‘PRT’, ‘ADP’, [(Pos([1]),‘DET’)]), Rule(‘099’, ‘PRT’, ‘ADP’, [(Pos([1]),‘NOUN’), (Pos([2]),‘NOUN’)]), Rule(‘097’, ‘NOUN’, ‘VERB’, [(Pos([-1]),‘PRT’), (Pos([1]),‘DET’)]), Rule(‘099’, ‘PRT’, ‘ADP’, [(Pos([1]),‘NOUN’), (Pos([2]),’.’)]), Rule(‘108’, ‘ADP’, ‘PRT’, [(Word([0]),‘all’)]), Rule(‘090’, ‘PRT’, ‘ADP’, [(Pos([1]),‘NUM’)]), Rule(‘108’, ‘ADP’, ‘PRT’, [(Word([0]),‘up’)]), Rule(‘099’, ‘PRT’, ‘ADP’, [(Pos([1]),‘ADJ’), (Pos([2]),‘NOUN’)]), Rule(‘103’, ‘ADP’, ‘ADV’, [(Word([2]),‘as’)]), Rule(‘097’, ‘VERB’, ‘NOUN’, [(Pos([-1]),‘DET’), (Pos([1]),‘ADP’)]), Rule(‘109’, ‘NOUN’, ‘VERB’, [(Word([-1]),‘would’), (Pos([-1]),‘VERB’)]), Rule(‘108’, ‘ADP’, ‘PRT’, [(Word([0]),‘out’)]), Rule(‘100’, ‘NOUN’, ‘VERB’, [(Word([-1]),‘will’)]), Rule(‘099’, ‘ADP’, ‘PRON’, [(Pos([1]),‘VERB’), (Pos([2]),‘VERB’)]), Rule(‘097’, ‘VERB’, ‘NOUN’, [(Pos([-1]),‘ADJ’), (Pos([1]),’.’)]))
[(‘Various’, ‘ADJ’), (‘of’, ‘ADP’), (‘the’, ‘DET’), (‘apartments’, ‘NOUN’), (‘are’, ‘VERB’), (‘of’, ‘ADP’), (‘the’, ‘DET’), (‘terrace’, ‘NOUN’), (‘type’, ‘NOUN’), (’,’, ‘.’), (‘being’, ‘VERB’), (‘on’, ‘ADP’), (‘the’, ‘DET’), (‘ground’, ‘NOUN’), (‘floor’, ‘NOUN’), (‘so’, ‘ADV’), (‘that’, ‘ADP’), (‘entrance’, ‘NOUN’), (‘is’, ‘VERB’), (‘direct’, ‘ADJ’), (’.’, ‘.’)]
0.8535831755207814

print(brill_tagger.tag(brown_sents[4200]))
print(brill_tagger.evaluate(brown_tagged_sents[size:]))
[(‘and’, ‘CONJ’), (‘it’, ‘PRON’), (‘was’, ‘VERB’), (‘filled’, ‘VERB’), (‘then’, ‘ADV’), (‘as’, ‘ADP’), (‘now’, ‘ADV’), (‘by’, ‘ADP’), (‘quarreling’, ‘VBG’), (‘tribes’, ‘VBZ’), (‘with’, ‘ADP’), (‘no’, ‘DET’), (‘political’, ‘ADJ’), (‘or’, ‘CONJ’), (‘historical’, ‘NN’), (‘unity’, ‘NOUN’), (’.’, ‘.’)]
0.8535831755207814

brill标注器案例相关推荐

ArcGIS模型构建器案例学习笔记-字段处理模型集
ArcGIS模型构建器案例学习笔记-字段处理模型集联系方式:谢老师,135-4855-4328,xiexiaokui@qq.com 由四个子模型组成子模型1:判断字段是否存在方法:python工 ...
从头开始训练一个 NER 标注器
文章目录从头开始训练一个 NER 标注器一.自定义模型 1.导入所需要的包与模块 2.导入训练样本二.训练模型 1.对现有的模型进行优化 2.创建内置管道组件 3.添加train data的标签 ...
第7课微信小程序实现图片搜索器案例：
第7课微信小程序实现图片搜索器案例: 效果图如下: 手机运行效果: 来我们来看看代码的全局样式: 首先在app.json内的pages把需要的index1.index2页面注册之后我们只需写ind ...
Android（安卓）简单拨号器案例
第5天今天整理下1.0版本的拨号器案例,一个 TextView,一个EditText,还有一个Button,控件就是那么简单. 1.activity_main.xml代码 <TextView ...
编程学习笔记之python深入之装饰器案例及说明文档[图]
编程学习笔记之python深入之装饰器案例及说明文档[图] 装饰器即在不对一个函数体进行任何修改,以及不改变整体的原本意思的情况下,增加函数功能的新函数,因为这个新函数对旧函数进行了装饰,所以称为装饰 ...
交互式医学影像标注器--MD.ai概述
今天给大家介绍一款交互式医学影像标注工具MD.ai,他的新颖之处在于一方面他是基于web模式的一款专业的医学影像标注工具,另一方面就是他是第一个创建的交互标注的工具,是一个多机构,多国专家可以联合标注 ...
杨老师课堂_Java核心技术下之控制台模拟文件管理器案例
背景需求介绍: 编写一个模拟文件管理器的程序,实现控制台对文件和文件夹的管理操作. 要求在此程序中: 当用户输入指令 1 时,代表"指定关键字检索文件",此时需要用户输入检索的目录 ...
Python-装饰器-案例-获取文件列表
1 import os 2 3 4 def get_all_path(fun): 5 '''装饰器.功能:获取全路径文件名.如:D:/tmp/12.txt 6 :param fun: 7 :retur ...
百度地图--展示行政区划（省市区县）和添加多个标注点案例
百度地图JavaScript API支持HTTP和HTTPS,免费对外开放,可直接使用.接口使用无次数限制.在使用前,您需先申请密钥(ak)才可使用. 申请密钥链接: http://lbsyun.ba ...

brill标注器案例

brill标注器案例相关推荐

最新文章

热门文章