神经机器翻译WMT14英法基准系统 WMT14 English-French Baseline

最近（2017年以来）的WMT14 English-French Baseline记录

1. GNMT

https://arxiv.org/pdf/1609.08144.pdf

语料处理：a shared source and target vocabulary of 32K wordpieces

For the wordpiece models, we train 3 different models with vocabulary sizes of 8K, 16K, and 32K. Table 4 summarizes our results on the WMT En→Fr dataset. In this table, we also compare against other strong baselines without model ensembling. As can be seen from the table, “WPM-32K”, a wordpiece model with a shared source and target vocabulary of 32K wordpieces, performs well on this dataset and achieves the best quality as well as the fastest inference speed.

On WMT En→Fr, the training set contains 36M sentence pairs. In both cases, we use newstest2014 as the test sets to compare against previous work. The combination of newstest2012 and newstest2013 is used as the development set.

实验结果：Table 4 in Page 16: En→Fr WPM-32K 38.95

or Table 6 in Page 17: En→Fr Trained with log-likelihood 38.95

2. Transformer

https://papers.nips.cc/paper/7181-attention-is-all-you-need.pdf

语料处理： 32000 joint word-piece vocabulary

For English-French, we used the significantly larger WMT 2014 English-French dataset consisting of 36M sentences and split tokens into a 32000 word-piece vocabulary.

实验结果：Table 2 in Page 8: Transformer (base model) 38.1 Transformer (big) 41.0

3. RNMT+

http://aclweb.org/anthology/P18-1008

语料处理：32K joint sub-word units (其实是32K wordpieces)

We train our models on the standard WMT’14 En→Fr and En→De datasets that comprise 36.3M and 4.5M sentence pairs, respectively. Each sentence was encoded into a sequence of sub-word units obtained by first tokenizing the sentence with the Moses tokenizer, then splitting tokens into subword units (also known as “wordpieces”) using the approach described in (Schuster and Nakajima, 2012). We use a shared vocabulary of 32K sub-word units for each source-target language pair.

实验结果： Table 1 in Page 81: RNMT+ 41.00 ± 0.05

4. ConvS2S

https://arxiv.org/pdf/1705.03122.pdf

github：https://github.com/facebookresearch/fairseq/

https://github.com/facebookresearch/fairseq/issues/59 （语料处理）

语料处理：40K joint BPE

We use the full training set of 36M sentence pairs, and remove sentences longer than 175 words as well as pairs with a source/target length ratio exceeding 1.5. This results in 35.5M sentence-pairs for training. Results are reported on newstest2014. We use a source and target vocabulary with 40K BPE types.

注意validation set的设置: In all setups a small subset of the training data serves as validation set (about 0.5-1% for each dataset) for early stopping and learning rate annealing.

实验结果： Table 1： ConvS2S (BPE 40K) 40.51

5. Fairseq

https://arxiv.org/pdf/1806.00187.pdf

github：https://github.com/pytorch/fairseq

语料处理： 40K joint BPE

For En–Fr, we train on WMT’14 and borrow the setup of Gehring et al. (2017) with 36M training sentence pairs. We use newstest12+13 for validation and newstest14 for test. The 40K vocabulary is based on a joint source and target BPE factorization.

validation set: newstest12+13 for validation

实验结果： Table2: Our result 43.2

神经机器翻译WMT14英法基准系统 WMT14 English-French Baseline相关推荐

神经机器翻译系统资料
作者:zhbzz2007 出处:http://www.cnblogs.com/zhbzz2007 欢迎转载,也请保留这段声明.谢谢! 1 简介自2013年提出了神经机器翻译系统之后,神经机器翻译系统 ...
使用fairseq从头开始训练一个中英神经机器翻译模型
前言本文在news-commentary-v15语料上训练了中英NMT模型,并将整个流程,包括工具和数据的准备.数据的预处理.训练及解码,以及中途遇到的问题和解决方案记录在此,希望能够给予别人一些帮 ...
统计机器翻译与神经机器翻译区别_如果每个人都献出一点爱，就会拥有一套超级牛的机器翻译系统...
所谓机器翻译技术就是利用计算机软件技术实现不同语言之间的自动翻译,目的为了帮助解决或缓解人工翻译代价过高和效率过低的问题.特别是针对大规模数据的实时和低成本翻译的应用场景,非人工翻译所为,有效利用机器 ...
哈佛大学 NLP 组开源神经机器翻译系统 OpenNMT
今天,Harvard NLP (哈佛大学自然语言处理研究组) 宣布开源其研发的神经机器翻译系统 OpenNMT,该系统使用了 Torch 数学工具包,已达 industrial-strength 可生 ...
最简版Seq2Seq的英法机器翻译实践和详细代码解释
Seq2Seq的英法机器翻译实践本文的内容主要是基于英法平行语料库来实现一个简单的英法翻译模型.没有使用注意力机制和双向LSTM等技术,主要是为了掌握基本的Seq2Seq结构和TensorFlow函 ...
【论文翻译】联合学习对齐和翻译的神经机器翻译
这篇论文在采用RNN编码器-解码器结构做端到端的神经机器翻译的基础上,使得模型可以在预测下一个词的时候,自动地搜索源句子相关的部分,这就是attention机制. 原论文链接: Neural Mach ...
《Effective Approaches to Attention-based Neural Machine Translation》—— 基于注意力机制的有效神经机器翻译方法
目录 <Effective Approaches to Attention-based Neural Machine Translation> 一.论文结构总览二.论文背景知识 2.1 ...
深度学习与自然语言处理教程(6) - 神经机器翻译、seq2seq与注意力机制（NLP通关指南·完结）
作者:韩信子@ShowMeAI 教程地址:https://www.showmeai.tech/tutorials/36 本文地址:https://www.showmeai.tech/article-d ...
[转]神经机器翻译（NMT）相关资料整理
1 简介自2013年提出了神经机器翻译系统之后,神经机器翻译系统取得了很大的进展.最近几年相关的论文,开源系统也是层出不穷.本文主要梳理了神经机器翻译入门.进阶所需要阅读的资料和论文,并提供了相关链 ...

神经机器翻译WMT14英法基准系统 WMT14 English-French Baseline

神经机器翻译WMT14英法基准系统 WMT14 English-French Baseline相关推荐

最新文章

热门文章