Commit 2ec567fe by 曹润柘

update chapter 1 and 3

parent 8b250298
......@@ -1162,7 +1162,7 @@ p_0+p_1 & = & 1 \label{eqC3.62-new}
\end{figure}
%-------------------------------------------
\parinterval 一般来说,HMM包含下面三个问题{\color{red} 参考文献 !!!引用- 《Foundations of Statistical Natural Language Processing》}
\parinterval 一般来说,HMM包含下面三个问题\cite{manning1999foundations}
\begin{itemize}
\item 估计。即给定模型(硬币种类和转移概率),根据可见状态链(抛硬币的结果),计算在该模型下得到这个结果的概率,这个问题的解决需要用到前后向算法。
......@@ -1225,12 +1225,10 @@ p_0+p_1 & = & 1 \label{eqC3.62-new}
\parinterval 本质上说,IBM模型的词对齐的不``完整''问题是IBM模型本身的缺陷。解决这个问题有很多思路,第一种方法就是,反向训练后,合并源语言单词,然后再正向训练。这里用汉英翻译为例来解释这个方法。首先反向训练,就是把英语当作待翻译语言,而把汉语当作目标语言进行训练(参数估计)。这样可以得到一个词对齐结果(参数估计的中间结果)。在这个词对齐结果里面,一个汉语单词可对应多个英语单词。之后,扫描每个英语句子,如果有多个英语单词对应同一个汉语单词,就把这些英语单词合并成一个英语单词。处理完之后,再把汉语当作源语言言把英语当作目标语言进行训练。这样就可以把一个汉语词对应到合并的英语单词上。虽然从模型上看,还是一个汉语单词对应一个英语``单词'',但实质上已经把这个汉语单词对应到多个英语单词上了。训练完之后,再利用这些参数进行翻译(解码)时,就能把一个中文单词翻译成多个英文单词了。但是反向训练后再训练也存在一些问题。首先,合并英语单词会使数据变得更稀疏,使训练不充分。其次,由于IBM模型的词对齐结果并不是高精度的,利用它的词对齐结果来合并一些英文单词可能造成严重的错误,比如:把本来很独立的几个单词合在了一起。因此,此方法也并不完美。具体使用时还要考虑实际需要和问题的严重程度来决定是否使用这个方法。
\parinterval 另一种方法是双向对齐之后进行词对齐\textbf{对称化}(Symmetrization)。这个方法可以帮助我们在IBM词对齐的基础上获得对称的词对齐结果。思路很简单,用正向(汉语为源语言,英语为目标语言)和反向(汉语为目标语言,英语为源语言)同时训练。这样可以得到两个词对齐结果。然后利用一些启发性方法用这两个词对齐生成对称的结果(比如,取``并集''、``交集''等),这样就可以得到包含1对多和多对多的词对齐结果。比如,在基于短语的统计机器翻译中已经很成功地使用了这种词对齐信息进行短语的获取。直到今天,对称化仍然是很多自然语言处理系统中的一个关键步骤。\\ \\ \\
\parinterval 另一种方法是双向对齐之后进行词对齐\textbf{对称化}(Symmetrization)。这个方法可以帮助我们在IBM词对齐的基础上获得对称的词对齐结果。思路很简单,用正向(汉语为源语言,英语为目标语言)和反向(汉语为目标语言,英语为源语言)同时训练。这样可以得到两个词对齐结果。然后利用一些启发性方法用这两个词对齐生成对称的结果(比如,取``并集''、``交集''等),这样就可以得到包含1对多和多对多的词对齐结果。比如,在基于短语的统计机器翻译中已经很成功地使用了这种词对齐信息进行短语的获取。直到今天,对称化仍然是很多自然语言处理系统中的一个关键步骤。
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{Deficiency}\index{Chapter3.6.2}
\parinterval Deficiency问题是指翻译模型会把一部分概率分配给一些根本不存在的源语言字符串。如果用$\textrm{P}(\textrm{well}|\mathbf{t})$表示$\textrm{P}(\mathbf{s}| \mathbf{t})$在所有的正确的(可以理解为语法上正确的)$\mathbf{s}$上的和,即
\begin{eqnarray}
\textrm{P}(\textrm{well}|\mathbf{t})=\sum_{s\textrm{\;is\;well\;formed}}{\textrm{P}(\mathbf{s}| \mathbf{t})}
......
......@@ -361,3 +361,153 @@ year={2015}
year={1996},
organization={Association for Computational Linguistics}
}
@book{manning1999foundations,
title={Foundations of statistical natural language processing},
author={Manning, Christopher D and Manning, Christopher D and Sch{\"u}tze, Hinrich},
year={1999},
publisher={MIT press}
}
@article{SPhilipp,
title={Philipp Koehn, Statistical machine translation},
author={Sánchez-Martínez, Felipe and Juan Antonio Pérez-Ortiz},
volume={24},
number={3-4},
pages={273-278},
}
@article{SIDDHARTHANChristopher,
title={Christopher D. Manning and Hinrich Schutze. Foundations of Statistical Natural Language Processing. MIT Press, 2000. ISBN 0-262-13360-1. 620 pp.},
author={SIDDHARTHAN and ADVAITH},
journal={Natural Language Engineering},
volume={8},
number={01},
}
@book{宗成庆2013统计自然语言处理,
title={统计自然语言处理},
author={宗成庆},
year={2013},
}
@article{HeatonIan,
title={Ian Goodfellow, Yoshua Bengio, and Aaron Courville: Deep learning},
author={Heaton and Jeff},
journal={Genetic Programming \& Evolvable Machines},
pages={s10710-017-9314-z},
}
@article{周志华2018《机器学习》,
title={《机器学习》},
author={周志华},
journal={航空港},
number={2},
pages={94-94},
year={2018},
}
@inproceedings{Tong2012NiuTrans,
title={NiuTrans: an open source toolkit for phrase-based and syntax-based machine translation},
author={Tong, Xiao and Zhu, Jingbo and Hao, Zhang and Qiang, Li},
booktitle={Proceedings of the ACL 2012 System Demonstrations},
year={2012},
}
@article{Koehn2007Moses,
title={Moses: Open Source Toolkit for Statistical Machine Translation},
author={Koehn, Philipp and Hoang, Hieu and Birch, Alexandra and Callisonburch, Chris and Federico, Marcello and Bertoldi, Nicola and Cowan, Brooke and Shen, Wade and Moran, Christine and Zens, Richard},
volume={9},
number={1},
pages={177--180},
year={2007},
}
@inproceedings{Dyer2010cdec,
title={cdec: A Decoder, Alignment, and Learning Framework for Finite-State and Context-Free Translation Models},
author={Dyer, Chris and Lopez, Adam and Ganitkevitch, Juri and Weese, Jonathan and Resnik, Philip},
booktitle={ACL 2010, Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, July 11-16, 2010, Uppsala, Sweden, System Demonstrations},
year={2010},
}
@article{SennrichNematus,
title={Nematus: a Toolkit for Neural Machine Translation},
author={Sennrich, Rico and Firat, Orhan and Cho, Kyunghyun and Birch, Alexandra and Haddow, Barry and Hitschler, Julian and Junczys-Dowmunt, Marcin and Läubli, Samuel and Barone, Antonio Valerio Miceli and Mokry, Jozef},
}
@article{Ottfairseq,
title={fairseq: A Fast, Extensible Toolkit for Sequence Modeling},
author={Ott, Myle and Edunov, Sergey and Baevski, Alexei and Fan, Angela and Gross, Sam and Ng, Nathan and Grangier, David and Auli, Michael},
}
@article{VaswaniTensor2Tensor,
title={Tensor2Tensor for Neural Machine Translation},
author={Vaswani, Ashish and Bengio, Samy and Brevdo, Eugene and Chollet, Francois and Gomez, Aidan N. and Gouws, Stephan and Jones, Llion and Kaiser, Łukasz and Kalchbrenner, Nal and Parmar, Niki},
}
@article{KleinOpenNMT,
title={OpenNMT: Open-Source Toolkit for Neural Machine Translation},
author={Klein, Guillaume and Kim, Yoon and Deng, Yuntian and Senellart, Jean and Rush, Alexander M.},
}
@article{ZhangTHUMT,
title={THUMT: An Open Source Toolkit for Neural Machine Translation},
author={Zhang, Jiacheng and Ding, Yanzhuo and Shen, Shiqi and Cheng, Yong and Sun, Maosong and Luan, Huanbo and Liu, Yang},
}
@article{WangCytonMT,
title={CytonMT: an Efficient Neural Machine Translation Open-source Toolkit Implemented in C++},
author={Wang, Xiaolin and Utiyama, Masao and Sumita, Eiichiro},
}
@article{Germann2016Modern,
title={Modern MT: A New Open-Source Machine Translation Platform for the Translation Industry},
author={Germann, Ulrich and Barbu, E and Bentivoglio, M and Bogoychev, Nikolay and Buck, C and Caroselli, D and Carvalho, L and Cattelan, A and Cattoni, R and Cettolo, M},
year={2016},
abstract={Modern MT (www.modernmt.eu) is a three-year Horizon 2020 innovation action(2015–2017) to develop new open-source machine translation technology for use in translation production environments, both fully automatic and as a back-end in interactive post-editing scenarios. Led by Translated srl, the project consortium also includes the Fondazione Bruno Kessler (FBK), the University of Edinburgh, and TAUS B.V. Modern MT has received funding from the European Union’s Horizon 2020 research and innovation programme under Grant Agreement No. 645487 (call ICT-17-2014).},
}
@article{JunczysMarian,
title={Marian: Fast Neural Machine Translation in C++},
author={Junczys-Dowmunt, Marcin and Grundkiewicz, Roman and Dwojak, Tomasz and Hoang, Hieu and Heafield, Kenneth and Neckermann, Tom and Seide, Frank and Germann, Ulrich and Aji, Alham Fikri and Bogoychev, Nikolay},
}
@article{hieber2017sockeye,
title={Sockeye: A Toolkit for Neural Machine Translation.},
author={Hieber, Felix and Domhan, Tobias and Denkowski, Michael and Vilar, David and Sokolov, Artem and Clifton, Ann and Post, Matt},
journal={arXiv: Computation and Language},
year={2017}}
@article{KuchaievMixed,
title={Mixed-Precision Training for NLP and Speech Recognition with OpenSeq2Seq},
author={Kuchaiev, Oleksii and Ginsburg, Boris and Gitman, Igor and Lavrukhin, Vitaly and Li, Jason and Nguyen, Huyen and Case, Carl and Micikevicius, Paulius},
}
@inproceedings{肖桐2011CWMT2011,
title={CWMT2011东北大学参评系统NiuTrans介绍(英文)},
author={肖桐 and 张浩 and 李强 and 路琦 and 朱靖波 and 任飞亮 and 王会珍},
booktitle={机器翻译研究进展——第七届全国机器翻译研讨会论文集},
year={2011},
}
@article{luong2016achieving,
title={Achieving Open Vocabulary Neural Machine Translation with Hybrid Word-Character Models},
author={Luong, Minhthang and Manning, Christopher D},
journal={arXiv: Computation and Language},
year={2016}}
@article{luong2015effective,
title={Effective Approaches to Attention-based Neural Machine Translation},
author={Luong, Minhthang and Pham, Hieu and Manning, Christopher D},
journal={arXiv: Computation and Language},
year={2015}}
@article{see2016compression,
title={Compression of Neural Machine Translation Models via Pruning},
author={See, Abigail and Luong, Minhthang and Manning, Christopher D},
journal={arXiv: Artificial Intelligence},
year={2016}}
@article{bahdanau2015neural,
title={Neural Machine Translation by Jointly Learning to Align and Translate},
author={Bahdanau, Dzmitry and Cho, Kyunghyun and Bengio, Yoshua},
year={2015}}
\ No newline at end of file
......@@ -55,7 +55,7 @@
\IfFileExists{C:/WINDOWS/win.ini}
{\newcommand{\mycfont}{song}}
{\newcommand{\mycfont}{gbsn}}
%{\newcommand{\mycfont}{gbsn}}
\begin{CJK}{UTF8}{\mycfont}
\end{CJK}
......@@ -103,9 +103,9 @@
% CHAPTERS
%----------------------------------------------------------------------------------------
\include{Chapter1/chapter1}
%\include{Chapter2/chapter2}
%\include{Chapter3/chapter3}
\include{Chapter2/chapter2}
\include{Chapter3/chapter3}
%\include{Chapter6/chapter6}
%----------------------------------------------------------------------------------------
......
......@@ -553,8 +553,10 @@ addtohook={%
\usepackage{appendix}
\usepackage{pgfplots}
\usepackage{tikz}
%----------------------------------------------------------------------------------------
% Chapter 6
%----------------------------------------------------------------------------------------
\usepackage{multirow}
\usepackage{tcolorbox}
Markdown 格式
0%
您添加了 0 到此讨论。请谨慎行事。
请先完成此评论的编辑!
注册 或者 后发表评论