BERT-based Lexical Substitution论文阅读

摘要：

之前的词汇替换是通过查询目标词汇的同义词来实现的（e.g. WordNet), 然后基于文本对候选词打分。这种方法有两个限制：

忽略了不是同义词的但是效果很好的候选词汇
没有考虑同意替换对整个文本的影响

主体：

对目标词进行embedding dropout，使得可以平衡目标词的语义信息和上下文信息。

如果完全遮盖，返回的候选词可能与原词意思不同，却能满足上下文信息；

如果不遮盖，返回的候选词大约99.99%会预测到原词。

计算相似度时，将BERT前四层的表示做拼接来计算相似度。

推荐时使用了词的似然和句子的余弦相似度求和：

Reference:

Zhou, W., Ge, T., Xu, K., Wei, F., & Zhou, M. (2019, July). BERT-based lexical substitution. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (pp. 3368-3373).

BERT-based Lexical Substitution论文阅读相关推荐

论文阅读——《Exposure Control using Bayesian Optimization based on Entropy Weighted Image Gradient》
论文阅读--<Exposure Control using Bayesian Optimization based on Entropy Weighted Image Gradient> ...
[论文阅读] (07) RAID2020 Cyber Threat Intelligence Modeling Based on Heterogeneous GCN
<娜璋带你读论文>系列主要是督促自己阅读优秀论文及听取学术讲座,并分享给大家,希望您喜欢.由于作者的英文水平和学术能力不高,需要不断提升,所以还请大家批评指正,非常欢迎大家给我留言评论,学 ...
目标检测论文阅读：GHM(anchor based)
目标检测论文阅读:GHM(anchor based) 论文链接:https://arxiv.org/abs/1811.05181 代码链接:https://github.com/libuyu/GHM_ ...
论文阅读06——《CaEGCN: Cross-Attention Fusion based Enhanced Graph Convolutional Network for Clustering》
欢迎到我的个人博客看原文论文阅读06--<CaEGCN: Cross-Attention Fusion based Enhanced Graph Convolutional Network f ...
【论文阅读】Attention Based Spatial-Temporal GCN...Traffic Flow Forecasting[基于注意力的时空图卷积网络交通流预测]（1）
[论文阅读]Attention Based Spatial-Temporal Graph Convolutional Networks for Traffic Flow Forecasting[基于注 ...
论文阅读 [TPAMI-2022] Grid Anchor Based Image Cropping: A New Benchmark and An Efficient Model
论文阅读 [TPAMI-2022] Grid Anchor Based Image Cropping: A New Benchmark and An Efficient Model 论文搜索(stud ...
论文阅读：Saliency-Guided Region Proposal Network for CNN Based Object Detection
论文阅读:Saliency-Guided Region Proposal Network for CNN Based Object Detection (1)Author (2)Abstract (3 ...
《论文阅读》RoBERTa: A Robustly Optimized BERT Pretraining Approach
<论文阅读>RoBERTa: A Robustly Optimized BERT Pretraining Approach 简介想法来源数据集的选择创新点总结期刊:arxiv 2 ...
论文阅读：Visual Semantic Localization based on HD Map for AutonomousVehicles in Urban Scenarios
题目:Visual Semantic Localization based on HD Map for Autonomous Vehicles in Urban Scenarios 中文:基于高清地图 ...

BERT-based Lexical Substitution论文阅读

BERT-based Lexical Substitution论文阅读相关推荐

最新文章

热门文章