Skip to content

T
Toy-MT-Introduction
Commit f504468e by xiaotong
Options

updates

parent dfc7b3c5
正在显示 包含 229 行增加 和 228 行删除
Book/Chapter1/chapter1.tex
...@@ -13,19 +13,19 @@ ...@@ -13,19 +13,19 @@
\chapter{机器翻译简介} \chapter{机器翻译简介}
\section{机器翻译的概念}\index{Chapter1.1} \section{机器翻译的概念}\index{Chapter1.1}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\parinterval 广义上来讲，``翻译''是指把一个事物转化为另一个事物的过程。这个概念多使用在对可序列化符号串的转化上，比如计算机程序的编译、文字翻译和蛋白质生物合成等。在程序编译中，高级语言编写的程序经过一系列的处理后转化为可执行目标程序，这是一种从高级程序语言到低级程序语言的``翻译''。在人类语言的翻译中，一种语言的文字通过某种方式转化为另一种语言表达，这是一种自然语言的``翻译''。在蛋白质生成合成的第一步，RNA分子序列转化到特定氨基酸序列，这是一种生物学遗传信息的``翻译''。甚至说，给上联对出下联、给一幅图片写出图片的主题等都可以被看作是``翻译''的过程。 \parinterval 广义上来讲，``翻译''是指把一个事物转化为另一个事物的过程。这个概念多使用在对符号串的转化上，比如计算机程序的编译、文字翻译、蛋白质生物合成等。在程序编译中，高级语言编写的程序经过一系列的处理后转化为可执行的目标程序，这是一种从高级程序语言到低级程序语言的``翻译''。在人类语言的翻译中，一种语言文字通过某种方式转化为另一种语言表达，这是一种自然语言的``翻译''。在蛋白质生成合成的第一步，RNA分子序列转化为特定的氨基酸序列，这是一种生物学遗传信息的``翻译''。甚至说，给上联对出下联、给一幅图片写出图片的主题等都可以被看作是``翻译''的过程。
\vspace{0.5em} \vspace{0.5em}
%---------------------------------------------- %----------------------------------------------
% 图1.1 % 图1.1
\begin{figure}[htp] \begin{figure}[htp]
\centering \centering
\includegraphics[scale=0.4]{./Chapter1/Figures/figure-zh_en-example.png} \includegraphics[scale=0.4]{./Chapter1/Figures/figure-zh_en-example.png}
\caption{在机器翻译中，中文被翻译为英语的例子} \caption{通过计算机将中文翻译为英文}
\label{fig:zh_en-example} \label{fig:zh_en-example}
\end{figure} \end{figure}
%------------------------------------------- %-------------------------------------------
\parinterval 当然，这里我们更加关注人类语言之间的翻译问题，即自然语言的翻译。如图\ref{fig:zh_en-example}所示，通过计算机可以将一段中文文字自动转化为英文文字，其中中文被称为源语言，英文被称为目标语言，系统的输入是一个源语言（即中文）的文字序列，系统的输出是一个目标语言（即英文）的文字序列。 \parinterval 这里我们更加关注人类语言之间的翻译问题，即自然语言的翻译。如图\ref{fig:zh_en-example}所示，通过计算机可以将一段中文文字自动转化为英文文字，其中中文被称为{\small\sffamily\bfseries{源语言}}（Source Language），英文被称为{\small\sffamily\bfseries{目标语言}}（Target Language）。
\parinterval 一直以来，自然语言文字的翻译往往是由人工完成。让计算机像人一样进行翻译似乎还是电影中的桥段，因为很难想象人类语言的多样性和复杂性可以用计算机语言进行描述。但是时至今日，人工智能技术的发展已经大大超越了人类传统的认知，用计算机进行自动翻译也不再是一种想象，它已经深入到人们生活的很多方面，并发挥着重要作用。而这种由计算机进行自动翻译的过程也被称作机器翻译，类似的，自动翻译、智能翻译、多语言自动转换等概念也是指同样的事情。如果将今天的机器翻译和人工翻译进行对比，可以发现机器翻译系统所生成的译文还并不完美，有时翻译质量甚至非常差，但是其优点在于速度快并且成本低，更为重要的是机器翻译系统可以从大量数据中不断学习和进化，随着学习其翻译质量不断得到提升。人工翻译尽管翻译的精度很高，但是费时费力。当需要翻译大量的文本且精度要求不那么高时，比如海量数据的浏览型任务，机器翻译的优势就体现了出来。对于人工作业无法完成的事情，使用机器翻译可能只需花几个小时甚至几分钟就能完成。这就类似于拿着锄头耕地种庄稼和使用现代化机器作业。 \parinterval 一直以来，自然语言文字的翻译往往是由人工完成。让计算机像人一样进行翻译似乎还是电影中的桥段，因为很难想象人类语言的多样性和复杂性可以用计算机语言进行描述。但是时至今日，人工智能技术的发展已经大大超越了人类传统的认知，用计算机进行自动翻译也不再是一种想象，它已经深入到人们生活的很多方面，并发挥着重要作用。而这种由计算机进行自动翻译的过程也被称作机器翻译，类似的，自动翻译、智能翻译、多语言自动转换等概念也是指同样的事情。如果将今天的机器翻译和人工翻译进行对比，可以发现机器翻译系统所生成的译文还并不完美，有时翻译质量甚至非常差，但是其优点在于速度快并且成本低，更为重要的是机器翻译系统可以从大量数据中不断学习和进化，随着学习其翻译质量不断得到提升。人工翻译尽管翻译的精度很高，但是费时费力。当需要翻译大量的文本且精度要求不那么高时，比如海量数据的浏览型任务，机器翻译的优势就体现了出来。对于人工作业无法完成的事情，使用机器翻译可能只需花几个小时甚至几分钟就能完成。这就类似于拿着锄头耕地种庄稼和使用现代化机器作业。
......
Book/mt-book-xelatex.ptc
\boolfalse {citerequest}\boolfalse {citetracker}\boolfalse {pagetracker}\boolfalse {backtracker}\relax \boolfalse {citerequest}\boolfalse {citetracker}\boolfalse {pagetracker}\boolfalse {backtracker}\relax
\babel@toc {english}{}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {part}{\@mypartnumtocformat {I}{机器翻译基础}}{9}{part.1}% \select@language {english}
\defcounter {refsection}{0}\relax
\contentsline {part}{\@mypartnumtocformat {I}{机器翻译基础}}{9}{part.1}
\ttl@starttoc {default@1} \ttl@starttoc {default@1}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {chapter}{\numberline {1}机器翻译简介}{11}{chapter.1}% \contentsline {chapter}{\numberline {1}机器翻译简介}{11}{chapter.1}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {section}{\numberline {1.1}机器翻译的概念}{11}{section.1.1}% \contentsline {section}{\numberline {1.1}机器翻译的概念}{11}{section.1.1}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {section}{\numberline {1.2}机器翻译简史}{14}{section.1.2}% \contentsline {section}{\numberline {1.2}机器翻译简史}{14}{section.1.2}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {section}{\numberline {1.3}机器翻译现状}{19}{section.1.3}% \contentsline {section}{\numberline {1.3}机器翻译现状}{19}{section.1.3}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {section}{\numberline {1.4}机器翻译方法}{20}{section.1.4}% \contentsline {section}{\numberline {1.4}机器翻译方法}{20}{section.1.4}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {1.4.1}基于规则的机器翻译}{20}{subsection.1.4.1}% \contentsline {subsection}{\numberline {1.4.1}基于规则的机器翻译}{20}{subsection.1.4.1}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {1.4.2}基于实例的机器翻译}{22}{subsection.1.4.2}% \contentsline {subsection}{\numberline {1.4.2}基于实例的机器翻译}{22}{subsection.1.4.2}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {1.4.3}统计机器翻译}{23}{subsection.1.4.3}% \contentsline {subsection}{\numberline {1.4.3}统计机器翻译}{23}{subsection.1.4.3}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {1.4.4}神经机器翻译}{24}{subsection.1.4.4}% \contentsline {subsection}{\numberline {1.4.4}神经机器翻译}{24}{subsection.1.4.4}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {1.4.5}对比分析}{25}{subsection.1.4.5}% \contentsline {subsection}{\numberline {1.4.5}对比分析}{25}{subsection.1.4.5}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {section}{\numberline {1.5}翻译质量评价}{25}{section.1.5}% \contentsline {section}{\numberline {1.5}翻译质量评价}{25}{section.1.5}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {1.5.1}人工评价}{26}{subsection.1.5.1}% \contentsline {subsection}{\numberline {1.5.1}人工评价}{26}{subsection.1.5.1}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {1.5.2}自动评价}{27}{subsection.1.5.2}% \contentsline {subsection}{\numberline {1.5.2}自动评价}{27}{subsection.1.5.2}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsubsection}{BLEU}{27}{section*.15}% \contentsline {subsubsection}{BLEU}{27}{section*.15}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsubsection}{TER}{28}{section*.16}% \contentsline {subsubsection}{TER}{28}{section*.16}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsubsection}{基于检测点的评价}{29}{section*.17}% \contentsline {subsubsection}{基于检测点的评价}{29}{section*.17}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {section}{\numberline {1.6}机器翻译应用}{30}{section.1.6}% \contentsline {section}{\numberline {1.6}机器翻译应用}{30}{section.1.6}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {section}{\numberline {1.7}开源项目与评测}{32}{section.1.7}% \contentsline {section}{\numberline {1.7}开源项目与评测}{32}{section.1.7}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {1.7.1}开源机器翻译系统}{33}{subsection.1.7.1}% \contentsline {subsection}{\numberline {1.7.1}开源机器翻译系统}{33}{subsection.1.7.1}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsubsection}{统计机器翻译开源系统}{33}{section*.19}% \contentsline {subsubsection}{统计机器翻译开源系统}{33}{section*.19}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsubsection}{神经机器翻译开源系统}{34}{section*.20}% \contentsline {subsubsection}{神经机器翻译开源系统}{34}{section*.20}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {1.7.2}常用数据集及公开评测任务}{36}{subsection.1.7.2}% \contentsline {subsection}{\numberline {1.7.2}常用数据集及公开评测任务}{36}{subsection.1.7.2}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {section}{\numberline {1.8}推荐学习资源}{39}{section.1.8}% \contentsline {section}{\numberline {1.8}推荐学习资源}{39}{section.1.8}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {chapter}{\numberline {2}词法、语法及统计建模基础}{43}{chapter.2}% \contentsline {chapter}{\numberline {2}词法、语法及统计建模基础}{43}{chapter.2}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {section}{\numberline {2.1}问题概述 }{44}{section.2.1}% \contentsline {section}{\numberline {2.1}问题概述 }{44}{section.2.1}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {section}{\numberline {2.2}概率论基础}{45}{section.2.2}% \contentsline {section}{\numberline {2.2}概率论基础}{45}{section.2.2}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {2.2.1}随机变量和概率}{46}{subsection.2.2.1}% \contentsline {subsection}{\numberline {2.2.1}随机变量和概率}{46}{subsection.2.2.1}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {2.2.2}联合概率、条件概率和边缘概率}{47}{subsection.2.2.2}% \contentsline {subsection}{\numberline {2.2.2}联合概率、条件概率和边缘概率}{47}{subsection.2.2.2}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {2.2.3}链式法则}{48}{subsection.2.2.3}% \contentsline {subsection}{\numberline {2.2.3}链式法则}{48}{subsection.2.2.3}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {2.2.4}贝叶斯法则}{49}{subsection.2.2.4}% \contentsline {subsection}{\numberline {2.2.4}贝叶斯法则}{49}{subsection.2.2.4}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {2.2.5}KL距离和熵}{51}{subsection.2.2.5}% \contentsline {subsection}{\numberline {2.2.5}KL距离和熵}{51}{subsection.2.2.5}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsubsection}{（一）信息熵}{51}{section*.27}% \contentsline {subsubsection}{（一）信息熵}{51}{section*.27}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsubsection}{（二）KL距离}{52}{section*.29}% \contentsline {subsubsection}{（二）KL距离}{52}{section*.29}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsubsection}{（三）交叉熵}{53}{section*.30}% \contentsline {subsubsection}{（三）交叉熵}{53}{section*.30}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {section}{\numberline {2.3}中文分词}{53}{section.2.3}% \contentsline {section}{\numberline {2.3}中文分词}{53}{section.2.3}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {2.3.1}基于词典的分词方法}{54}{subsection.2.3.1}% \contentsline {subsection}{\numberline {2.3.1}基于词典的分词方法}{54}{subsection.2.3.1}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {2.3.2}基于统计的分词方法}{55}{subsection.2.3.2}% \contentsline {subsection}{\numberline {2.3.2}基于统计的分词方法}{55}{subsection.2.3.2}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsubsection}{统计模型的学习与推断}{56}{section*.34}% \contentsline {subsubsection}{统计模型的学习与推断}{56}{section*.34}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsubsection}{掷骰子游戏}{56}{section*.36}% \contentsline {subsubsection}{掷骰子游戏}{56}{section*.36}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsubsection}{全概率分词方法}{58}{section*.40}% \contentsline {subsubsection}{全概率分词方法}{58}{section*.40}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {section}{\numberline {2.4}$n$-gram语言模型 }{61}{section.2.4}% \contentsline {section}{\numberline {2.4}$n$-gram语言模型 }{61}{section.2.4}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {2.4.1}建模}{61}{subsection.2.4.1}% \contentsline {subsection}{\numberline {2.4.1}建模}{61}{subsection.2.4.1}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {2.4.2}未登录词和平滑算法}{63}{subsection.2.4.2}% \contentsline {subsection}{\numberline {2.4.2}未登录词和平滑算法}{63}{subsection.2.4.2}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsubsection}{加法平滑方法}{64}{section*.47}% \contentsline {subsubsection}{加法平滑方法}{64}{section*.47}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsubsection}{古德-图灵估计法}{65}{section*.49}% \contentsline {subsubsection}{古德-图灵估计法}{65}{section*.49}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsubsection}{Kneser-Ney平滑方法}{66}{section*.51}% \contentsline {subsubsection}{Kneser-Ney平滑方法}{66}{section*.51}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {section}{\numberline {2.5}句法分析（短语结构分析）}{68}{section.2.5}% \contentsline {section}{\numberline {2.5}句法分析（短语结构分析）}{68}{section.2.5}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {2.5.1}句子的句法树表示}{68}{subsection.2.5.1}% \contentsline {subsection}{\numberline {2.5.1}句子的句法树表示}{68}{subsection.2.5.1}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {2.5.2}上下文无关文法}{70}{subsection.2.5.2}% \contentsline {subsection}{\numberline {2.5.2}上下文无关文法}{70}{subsection.2.5.2}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {2.5.3}规则和推导的概率}{75}{subsection.2.5.3}% \contentsline {subsection}{\numberline {2.5.3}规则和推导的概率}{75}{subsection.2.5.3}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {section}{\numberline {2.6}小结及深入阅读}{77}{section.2.6}% \contentsline {section}{\numberline {2.6}小结及深入阅读}{77}{section.2.6}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {part}{\@mypartnumtocformat {II}{统计机器翻译}}{79}{part.2}% \contentsline {part}{\@mypartnumtocformat {II}{统计机器翻译}}{79}{part.2}
\ttl@stoptoc {default@1} \ttl@stoptoc {default@1}
\ttl@starttoc {default@2} \ttl@starttoc {default@2}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {chapter}{\numberline {3}基于词的机器翻译模型}{81}{chapter.3}% \contentsline {chapter}{\numberline {3}基于词的机器翻译模型}{81}{chapter.3}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {section}{\numberline {3.1}什么是基于词的翻译模型}{81}{section.3.1}% \contentsline {section}{\numberline {3.1}什么是基于词的翻译模型}{81}{section.3.1}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {section}{\numberline {3.2}构建一个简单的机器翻译系统}{83}{section.3.2}% \contentsline {section}{\numberline {3.2}构建一个简单的机器翻译系统}{83}{section.3.2}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {3.2.1}如何进行翻译？}{83}{subsection.3.2.1}% \contentsline {subsection}{\numberline {3.2.1}如何进行翻译？}{83}{subsection.3.2.1}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsubsection}{（二）机器翻译流程}{84}{section*.66}% \contentsline {subsubsection}{（二）机器翻译流程}{84}{section*.66}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsubsection}{（三）人工 vs. 机器}{85}{section*.68}% \contentsline {subsubsection}{（三）人工 vs. 机器}{85}{section*.68}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {3.2.2}基本框架}{85}{subsection.3.2.2}% \contentsline {subsection}{\numberline {3.2.2}基本框架}{85}{subsection.3.2.2}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {3.2.3}单词翻译概率}{86}{subsection.3.2.3}% \contentsline {subsection}{\numberline {3.2.3}单词翻译概率}{86}{subsection.3.2.3}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsubsection}{（一）什么是单词翻译概率？}{86}{section*.70}% \contentsline {subsubsection}{（一）什么是单词翻译概率？}{86}{section*.70}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsubsection}{（二）如何从一个双语平行数据中学习？}{87}{section*.72}% \contentsline {subsubsection}{（二）如何从一个双语平行数据中学习？}{87}{section*.72}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsubsection}{（三）如何从大量的双语平行数据中学习？}{88}{section*.73}% \contentsline {subsubsection}{（三）如何从大量的双语平行数据中学习？}{88}{section*.73}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {3.2.4}句子级翻译模型}{89}{subsection.3.2.4}% \contentsline {subsection}{\numberline {3.2.4}句子级翻译模型}{89}{subsection.3.2.4}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsubsection}{（一）句子级翻译的基础模型}{89}{section*.75}% \contentsline {subsubsection}{（一）句子级翻译的基础模型}{89}{section*.75}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsubsection}{（二）生成流畅的译文}{91}{section*.77}% \contentsline {subsubsection}{（二）生成流畅的译文}{91}{section*.77}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {3.2.5}解码}{92}{subsection.3.2.5}% \contentsline {subsection}{\numberline {3.2.5}解码}{92}{subsection.3.2.5}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {section}{\numberline {3.3}基于词的翻译建模}{95}{section.3.3}% \contentsline {section}{\numberline {3.3}基于词的翻译建模}{95}{section.3.3}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {3.3.1}噪声信道模型}{95}{subsection.3.3.1}% \contentsline {subsection}{\numberline {3.3.1}噪声信道模型}{95}{subsection.3.3.1}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {3.3.2}统计机器翻译的三个基本问题}{98}{subsection.3.3.2}% \contentsline {subsection}{\numberline {3.3.2}统计机器翻译的三个基本问题}{98}{subsection.3.3.2}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsubsection}{词对齐}{99}{section*.86}% \contentsline {subsubsection}{词对齐}{99}{section*.86}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsubsection}{基于词对齐的翻译模型}{100}{section*.89}% \contentsline {subsubsection}{基于词对齐的翻译模型}{100}{section*.89}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsubsection}{基于词对齐的翻译实例}{101}{section*.91}% \contentsline {subsubsection}{基于词对齐的翻译实例}{101}{section*.91}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {section}{\numberline {3.4}IBM模型1-2}{102}{section.3.4}% \contentsline {section}{\numberline {3.4}IBM模型1-2}{102}{section.3.4}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {3.4.1}IBM模型1}{102}{subsection.3.4.1}% \contentsline {subsection}{\numberline {3.4.1}IBM模型1}{102}{subsection.3.4.1}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {3.4.2}IBM模型2}{104}{subsection.3.4.2}% \contentsline {subsection}{\numberline {3.4.2}IBM模型2}{104}{subsection.3.4.2}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {3.4.3}解码及计算优化}{105}{subsection.3.4.3}% \contentsline {subsection}{\numberline {3.4.3}解码及计算优化}{105}{subsection.3.4.3}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {3.4.4}训练}{106}{subsection.3.4.4}% \contentsline {subsection}{\numberline {3.4.4}训练}{106}{subsection.3.4.4}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsubsection}{（一）目标函数}{106}{section*.96}% \contentsline {subsubsection}{（一）目标函数}{106}{section*.96}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsubsection}{（二）优化}{107}{section*.98}% \contentsline {subsubsection}{（二）优化}{107}{section*.98}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {section}{\numberline {3.5}IBM模型3-5及隐马尔可夫模型}{112}{section.3.5}% \contentsline {section}{\numberline {3.5}IBM模型3-5及隐马尔可夫模型}{112}{section.3.5}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {3.5.1}基于产出率的翻译模型}{113}{subsection.3.5.1}% \contentsline {subsection}{\numberline {3.5.1}基于产出率的翻译模型}{113}{subsection.3.5.1}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {3.5.2}IBM 模型3}{115}{subsection.3.5.2}% \contentsline {subsection}{\numberline {3.5.2}IBM 模型3}{115}{subsection.3.5.2}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {3.5.3}IBM 模型4}{117}{subsection.3.5.3}% \contentsline {subsection}{\numberline {3.5.3}IBM 模型4}{117}{subsection.3.5.3}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {3.5.4} IBM 模型5}{118}{subsection.3.5.4}% \contentsline {subsection}{\numberline {3.5.4} IBM 模型5}{118}{subsection.3.5.4}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {3.5.5}隐马尔可夫模型}{120}{subsection.3.5.5}% \contentsline {subsection}{\numberline {3.5.5}隐马尔可夫模型}{120}{subsection.3.5.5}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsubsection}{隐马尔可夫模型}{120}{section*.110}% \contentsline {subsubsection}{隐马尔可夫模型}{120}{section*.110}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsubsection}{词对齐模型}{121}{section*.112}% \contentsline {subsubsection}{词对齐模型}{121}{section*.112}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {3.5.6}解码和训练}{122}{subsection.3.5.6}% \contentsline {subsection}{\numberline {3.5.6}解码和训练}{122}{subsection.3.5.6}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {section}{\numberline {3.6}问题分析}{123}{section.3.6}% \contentsline {section}{\numberline {3.6}问题分析}{123}{section.3.6}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {3.6.1}词对齐及对称化}{123}{subsection.3.6.1}% \contentsline {subsection}{\numberline {3.6.1}词对齐及对称化}{123}{subsection.3.6.1}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {3.6.2}Deficiency}{124}{subsection.3.6.2}% \contentsline {subsection}{\numberline {3.6.2}Deficiency}{124}{subsection.3.6.2}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {3.6.3}句子长度}{125}{subsection.3.6.3}% \contentsline {subsection}{\numberline {3.6.3}句子长度}{125}{subsection.3.6.3}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {3.6.4}其它问题}{125}{subsection.3.6.4}% \contentsline {subsection}{\numberline {3.6.4}其它问题}{125}{subsection.3.6.4}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {section}{\numberline {3.7}小结及深入阅读}{125}{section.3.7}% \contentsline {section}{\numberline {3.7}小结及深入阅读}{125}{section.3.7}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {part}{\@mypartnumtocformat {III}{神经机器翻译}}{127}{part.3}% \contentsline {part}{\@mypartnumtocformat {III}{神经机器翻译}}{127}{part.3}
\ttl@stoptoc {default@2} \ttl@stoptoc {default@2}
\ttl@starttoc {default@3} \ttl@starttoc {default@3}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {chapter}{\numberline {4}人工神经网络和神经语言建模}{129}{chapter.4}% \contentsline {chapter}{\numberline {4}人工神经网络和神经语言建模}{129}{chapter.4}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {section}{\numberline {4.1}深度学习与人工神经网络}{130}{section.4.1}% \contentsline {section}{\numberline {4.1}深度学习与人工神经网络}{130}{section.4.1}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {4.1.1}发展简史}{130}{subsection.4.1.1}% \contentsline {subsection}{\numberline {4.1.1}发展简史}{130}{subsection.4.1.1}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsubsection}{（一）早期的人工神经网络和第一次寒冬}{130}{section*.114}% \contentsline {subsubsection}{（一）早期的人工神经网络和第一次寒冬}{130}{section*.114}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsubsection}{（二）神经网络的第二次高潮和第二次寒冬}{131}{section*.115}% \contentsline {subsubsection}{（二）神经网络的第二次高潮和第二次寒冬}{131}{section*.115}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsubsection}{（三）深度学习和神经网络的崛起}{132}{section*.116}% \contentsline {subsubsection}{（三）深度学习和神经网络的崛起}{132}{section*.116}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {4.1.2}为什么需要深度学习}{133}{subsection.4.1.2}% \contentsline {subsection}{\numberline {4.1.2}为什么需要深度学习}{133}{subsection.4.1.2}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsubsection}{（一）端到端学习和表示学习}{133}{section*.118}% \contentsline {subsubsection}{（一）端到端学习和表示学习}{133}{section*.118}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsubsection}{（二）深度学习的效果}{134}{section*.120}% \contentsline {subsubsection}{（二）深度学习的效果}{134}{section*.120}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {section}{\numberline {4.2}神经网络基础}{134}{section.4.2}% \contentsline {section}{\numberline {4.2}神经网络基础}{134}{section.4.2}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {4.2.1}线性代数基础}{134}{subsection.4.2.1}% \contentsline {subsection}{\numberline {4.2.1}线性代数基础}{134}{subsection.4.2.1}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsubsection}{（一）标量、向量和矩阵}{135}{section*.122}% \contentsline {subsubsection}{（一）标量、向量和矩阵}{135}{section*.122}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsubsection}{（二）矩阵的转置}{136}{section*.123}% \contentsline {subsubsection}{（二）矩阵的转置}{136}{section*.123}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsubsection}{（三）矩阵加法和数乘}{136}{section*.124}% \contentsline {subsubsection}{（三）矩阵加法和数乘}{136}{section*.124}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsubsection}{（四）矩阵乘法和矩阵点乘}{137}{section*.125}% \contentsline {subsubsection}{（四）矩阵乘法和矩阵点乘}{137}{section*.125}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsubsection}{（五）线性映射}{138}{section*.126}% \contentsline {subsubsection}{（五）线性映射}{138}{section*.126}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsubsection}{（六）范数}{139}{section*.127}% \contentsline {subsubsection}{（六）范数}{139}{section*.127}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {4.2.2}人工神经元和感知机}{140}{subsection.4.2.2}% \contentsline {subsection}{\numberline {4.2.2}人工神经元和感知机}{140}{subsection.4.2.2}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsubsection}{（一）感知机\ \raisebox {0.5mm}{------}\ 最简单的人工神经元模型}{141}{section*.130}% \contentsline {subsubsection}{（一）感知机\ \raisebox {0.5mm}{------}\ 最简单的人工神经元模型}{141}{section*.130}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsubsection}{（二）神经元内部权重}{141}{section*.133}% \contentsline {subsubsection}{（二）神经元内部权重}{141}{section*.133}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsubsection}{（三）神经元的输入\ \raisebox {0.5mm}{------}\ 离散 vs 连续}{142}{section*.135}% \contentsline {subsubsection}{（三）神经元的输入\ \raisebox {0.5mm}{------}\ 离散 vs 连续}{142}{section*.135}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsubsection}{（四）神经元内部的参数学习}{143}{section*.137}% \contentsline {subsubsection}{（四）神经元内部的参数学习}{143}{section*.137}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {4.2.3}多层神经网络}{144}{subsection.4.2.3}% \contentsline {subsection}{\numberline {4.2.3}多层神经网络}{144}{subsection.4.2.3}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsubsection}{（一）线性变换和激活函数}{144}{section*.139}% \contentsline {subsubsection}{（一）线性变换和激活函数}{144}{section*.139}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsubsection}{（二）单层神经网络$\rightarrow $多层神经网络}{146}{section*.146}% \contentsline {subsubsection}{（二）单层神经网络$\rightarrow $多层神经网络}{146}{section*.146}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {4.2.4}函数拟合能力}{148}{subsection.4.2.4}% \contentsline {subsection}{\numberline {4.2.4}函数拟合能力}{148}{subsection.4.2.4}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {section}{\numberline {4.3}神经网络的张量实现}{151}{section.4.3}% \contentsline {section}{\numberline {4.3}神经网络的张量实现}{151}{section.4.3}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {4.3.1} 张量及其计算}{151}{subsection.4.3.1}% \contentsline {subsection}{\numberline {4.3.1} 张量及其计算}{151}{subsection.4.3.1}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsubsection}{（一）张量}{151}{section*.156}% \contentsline {subsubsection}{（一）张量}{151}{section*.156}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsubsection}{（二）张量的矩阵乘法}{153}{section*.159}% \contentsline {subsubsection}{（二）张量的矩阵乘法}{153}{section*.159}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsubsection}{（三）张量的单元操作}{154}{section*.161}% \contentsline {subsubsection}{（三）张量的单元操作}{154}{section*.161}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {4.3.2}张量的物理存储形式}{155}{subsection.4.3.2}% \contentsline {subsection}{\numberline {4.3.2}张量的物理存储形式}{155}{subsection.4.3.2}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {4.3.3}使用开源框架实现张量计算}{156}{subsection.4.3.3}% \contentsline {subsection}{\numberline {4.3.3}使用开源框架实现张量计算}{156}{subsection.4.3.3}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {4.3.4}神经网络中的前向传播}{160}{subsection.4.3.4}% \contentsline {subsection}{\numberline {4.3.4}神经网络中的前向传播}{160}{subsection.4.3.4}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {4.3.5}神经网络实例}{161}{subsection.4.3.5}% \contentsline {subsection}{\numberline {4.3.5}神经网络实例}{161}{subsection.4.3.5}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {section}{\numberline {4.4}神经网络的参数训练}{162}{section.4.4}% \contentsline {section}{\numberline {4.4}神经网络的参数训练}{162}{section.4.4}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {4.4.1}损失函数}{163}{subsection.4.4.1}% \contentsline {subsection}{\numberline {4.4.1}损失函数}{163}{subsection.4.4.1}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {4.4.2}基于梯度的参数优化}{164}{subsection.4.4.2}% \contentsline {subsection}{\numberline {4.4.2}基于梯度的参数优化}{164}{subsection.4.4.2}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsubsection}{（一）梯度下降}{165}{section*.179}% \contentsline {subsubsection}{（一）梯度下降}{165}{section*.179}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsubsection}{（二）梯度获取}{167}{section*.181}% \contentsline {subsubsection}{（二）梯度获取}{167}{section*.181}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsubsection}{（三）基于梯度的方法的变种和改进}{169}{section*.185}% \contentsline {subsubsection}{（三）基于梯度的方法的变种和改进}{169}{section*.185}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {4.4.3}参数更新的并行化策略}{172}{subsection.4.4.3}% \contentsline {subsection}{\numberline {4.4.3}参数更新的并行化策略}{172}{subsection.4.4.3}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {4.4.4}梯度消失、梯度爆炸和稳定性训练}{174}{subsection.4.4.4}% \contentsline {subsection}{\numberline {4.4.4}梯度消失、梯度爆炸和稳定性训练}{174}{subsection.4.4.4}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsubsection}{（一）梯度消失现象及解决方法}{174}{section*.188}% \contentsline {subsubsection}{（一）梯度消失现象及解决方法}{174}{section*.188}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsubsection}{（二）梯度爆炸现象及解决方法}{175}{section*.192}% \contentsline {subsubsection}{（二）梯度爆炸现象及解决方法}{175}{section*.192}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsubsection}{（三）稳定性训练}{175}{section*.193}% \contentsline {subsubsection}{（三）稳定性训练}{175}{section*.193}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {4.4.5}过拟合}{177}{subsection.4.4.5}% \contentsline {subsection}{\numberline {4.4.5}过拟合}{177}{subsection.4.4.5}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {4.4.6}反向传播}{178}{subsection.4.4.6}% \contentsline {subsection}{\numberline {4.4.6}反向传播}{178}{subsection.4.4.6}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsubsection}{（一）输出层的反向传播}{179}{section*.196}% \contentsline {subsubsection}{（一）输出层的反向传播}{179}{section*.196}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsubsection}{（二）隐藏层的反向传播}{181}{section*.200}% \contentsline {subsubsection}{（二）隐藏层的反向传播}{181}{section*.200}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsubsection}{（三）程序实现}{182}{section*.203}% \contentsline {subsubsection}{（三）程序实现}{182}{section*.203}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {section}{\numberline {4.5}神经语言模型}{184}{section.4.5}% \contentsline {section}{\numberline {4.5}神经语言模型}{184}{section.4.5}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {4.5.1}基于神经网络的语言建模}{184}{subsection.4.5.1}% \contentsline {subsection}{\numberline {4.5.1}基于神经网络的语言建模}{184}{subsection.4.5.1}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsubsection}{（一）基于前馈神经网络的语言模型}{185}{section*.206}% \contentsline {subsubsection}{（一）基于前馈神经网络的语言模型}{185}{section*.206}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsubsection}{（二）基于循环神经网络的语言模型}{187}{section*.209}% \contentsline {subsubsection}{（二）基于循环神经网络的语言模型}{187}{section*.209}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsubsection}{（三）基于自注意力机制的语言模型}{188}{section*.211}% \contentsline {subsubsection}{（三）基于自注意力机制的语言模型}{188}{section*.211}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsubsection}{（四）语言模型的评价}{189}{section*.213}% \contentsline {subsubsection}{（四）语言模型的评价}{189}{section*.213}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {4.5.2}单词表示模型}{190}{subsection.4.5.2}% \contentsline {subsection}{\numberline {4.5.2}单词表示模型}{190}{subsection.4.5.2}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsubsection}{（一）One-hot编码}{190}{section*.214}% \contentsline {subsubsection}{（一）One-hot编码}{190}{section*.214}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsubsection}{（二）分布式表示}{190}{section*.216}% \contentsline {subsubsection}{（二）分布式表示}{190}{section*.216}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {4.5.3}句子表示模型及预训练}{192}{subsection.4.5.3}% \contentsline {subsection}{\numberline {4.5.3}句子表示模型及预训练}{192}{subsection.4.5.3}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsubsection}{（一）简单的上下文表示模型}{192}{section*.220}% \contentsline {subsubsection}{（一）简单的上下文表示模型}{192}{section*.220}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsubsection}{（二）ELMO模型}{194}{section*.223}% \contentsline {subsubsection}{（二）ELMO模型}{194}{section*.223}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsubsection}{（三）GPT模型}{194}{section*.225}% \contentsline {subsubsection}{（三）GPT模型}{194}{section*.225}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsubsection}{（四）BERT模型}{195}{section*.227}% \contentsline {subsubsection}{（四）BERT模型}{195}{section*.227}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsubsection}{（五）为什么要预训练？}{196}{section*.229}% \contentsline {subsubsection}{（五）为什么要预训练？}{196}{section*.229}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {section}{\numberline {4.6}小结及深入阅读}{196}{section.4.6}% \contentsline {section}{\numberline {4.6}小结及深入阅读}{196}{section.4.6}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {chapter}{\numberline {5}神经机器翻译模型}{199}{chapter.5}% \contentsline {chapter}{\numberline {5}神经机器翻译模型}{199}{chapter.5}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {section}{\numberline {5.1}神经机器翻译的发展简史}{199}{section.5.1}% \contentsline {section}{\numberline {5.1}神经机器翻译的发展简史}{199}{section.5.1}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {5.1.1}神经机器翻译的起源}{201}{subsection.5.1.1}% \contentsline {subsection}{\numberline {5.1.1}神经机器翻译的起源}{201}{subsection.5.1.1}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {5.1.2}神经机器翻译的品质 }{203}{subsection.5.1.2}% \contentsline {subsection}{\numberline {5.1.2}神经机器翻译的品质 }{203}{subsection.5.1.2}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {5.1.3}神经机器翻译的优势 }{206}{subsection.5.1.3}% \contentsline {subsection}{\numberline {5.1.3}神经机器翻译的优势 }{206}{subsection.5.1.3}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {section}{\numberline {5.2}编码器-解码器框架}{208}{section.5.2}% \contentsline {section}{\numberline {5.2}编码器-解码器框架}{208}{section.5.2}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {5.2.1}框架结构}{208}{subsection.5.2.1}% \contentsline {subsection}{\numberline {5.2.1}框架结构}{208}{subsection.5.2.1}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {5.2.2}表示学习}{209}{subsection.5.2.2}% \contentsline {subsection}{\numberline {5.2.2}表示学习}{209}{subsection.5.2.2}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {5.2.3}简单的运行实例}{210}{subsection.5.2.3}% \contentsline {subsection}{\numberline {5.2.3}简单的运行实例}{210}{subsection.5.2.3}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {5.2.4}机器翻译范式的对比}{211}{subsection.5.2.4}% \contentsline {subsection}{\numberline {5.2.4}机器翻译范式的对比}{211}{subsection.5.2.4}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {section}{\numberline {5.3}基于循环神经网络的翻译模型及注意力机制}{212}{section.5.3}% \contentsline {section}{\numberline {5.3}基于循环神经网络的翻译模型及注意力机制}{212}{section.5.3}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {5.3.1}建模}{214}{subsection.5.3.1}% \contentsline {subsection}{\numberline {5.3.1}建模}{214}{subsection.5.3.1}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {5.3.2}输入（词嵌入）及输出（Softmax）}{216}{subsection.5.3.2}% \contentsline {subsection}{\numberline {5.3.2}输入（词嵌入）及输出（Softmax）}{216}{subsection.5.3.2}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {5.3.3}循环神经网络结构}{220}{subsection.5.3.3}% \contentsline {subsection}{\numberline {5.3.3}循环神经网络结构}{220}{subsection.5.3.3}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsubsection}{循环神经单元（RNN）}{220}{section*.251}% \contentsline {subsubsection}{循环神经单元（RNN）}{220}{section*.251}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsubsection}{长短时记忆网络（LSTM）}{220}{section*.252}% \contentsline {subsubsection}{长短时记忆网络（LSTM）}{220}{section*.252}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsubsection}{门控循环单元（GRU）}{222}{section*.255}% \contentsline {subsubsection}{门控循环单元（GRU）}{222}{section*.255}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsubsection}{双向模型}{223}{section*.257}% \contentsline {subsubsection}{双向模型}{223}{section*.257}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsubsection}{多层循环神经网络}{225}{section*.259}% \contentsline {subsubsection}{多层循环神经网络}{225}{section*.259}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {5.3.4}注意力机制}{225}{subsection.5.3.4}% \contentsline {subsection}{\numberline {5.3.4}注意力机制}{225}{subsection.5.3.4}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsubsection}{翻译中的注意力机制}{226}{section*.262}% \contentsline {subsubsection}{翻译中的注意力机制}{226}{section*.262}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsubsection}{上下文向量的计算}{227}{section*.265}% \contentsline {subsubsection}{上下文向量的计算}{227}{section*.265}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsubsection}{注意力机制的解读}{230}{section*.270}% \contentsline {subsubsection}{注意力机制的解读}{230}{section*.270}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {5.3.5}训练}{232}{subsection.5.3.5}% \contentsline {subsection}{\numberline {5.3.5}训练}{232}{subsection.5.3.5}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsubsection}{损失函数}{233}{section*.273}% \contentsline {subsubsection}{损失函数}{233}{section*.273}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsubsection}{长参数初始化}{233}{section*.274}% \contentsline {subsubsection}{长参数初始化}{233}{section*.274}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsubsection}{优化策略}{234}{section*.275}% \contentsline {subsubsection}{优化策略}{234}{section*.275}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsubsection}{梯度裁剪}{234}{section*.277}% \contentsline {subsubsection}{梯度裁剪}{234}{section*.277}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsubsection}{学习率策略}{235}{section*.278}% \contentsline {subsubsection}{学习率策略}{235}{section*.278}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsubsection}{并行训练}{236}{section*.281}% \contentsline {subsubsection}{并行训练}{236}{section*.281}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {5.3.6}推断}{237}{subsection.5.3.6}% \contentsline {subsection}{\numberline {5.3.6}推断}{237}{subsection.5.3.6}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsubsection}{贪婪搜索}{239}{section*.285}% \contentsline {subsubsection}{贪婪搜索}{239}{section*.285}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsubsection}{束搜索}{240}{section*.288}% \contentsline {subsubsection}{束搜索}{240}{section*.288}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsubsection}{长度惩罚}{241}{section*.290}% \contentsline {subsubsection}{长度惩罚}{241}{section*.290}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {5.3.7}实例-GNMT}{242}{subsection.5.3.7}% \contentsline {subsection}{\numberline {5.3.7}实例-GNMT}{242}{subsection.5.3.7}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {section}{\numberline {5.4}Transformer}{244}{section.5.4}% \contentsline {section}{\numberline {5.4}Transformer}{244}{section.5.4}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {5.4.1}自注意力模型}{245}{subsection.5.4.1}% \contentsline {subsection}{\numberline {5.4.1}自注意力模型}{245}{subsection.5.4.1}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {5.4.2}Transformer架构}{246}{subsection.5.4.2}% \contentsline {subsection}{\numberline {5.4.2}Transformer架构}{246}{subsection.5.4.2}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {5.4.3}位置编码}{249}{subsection.5.4.3}% \contentsline {subsection}{\numberline {5.4.3}位置编码}{249}{subsection.5.4.3}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {5.4.4}基于点乘的注意力机制}{251}{subsection.5.4.4}% \contentsline {subsection}{\numberline {5.4.4}基于点乘的注意力机制}{251}{subsection.5.4.4}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {5.4.5}掩码操作}{252}{subsection.5.4.5}% \contentsline {subsection}{\numberline {5.4.5}掩码操作}{252}{subsection.5.4.5}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {5.4.6}多头注意力}{253}{subsection.5.4.6}% \contentsline {subsection}{\numberline {5.4.6}多头注意力}{253}{subsection.5.4.6}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {5.4.7}残差网络和层正则化}{255}{subsection.5.4.7}% \contentsline {subsection}{\numberline {5.4.7}残差网络和层正则化}{255}{subsection.5.4.7}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {5.4.8}前馈全连接网络子层}{256}{subsection.5.4.8}% \contentsline {subsection}{\numberline {5.4.8}前馈全连接网络子层}{256}{subsection.5.4.8}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {5.4.9}训练}{257}{subsection.5.4.9}% \contentsline {subsection}{\numberline {5.4.9}训练}{257}{subsection.5.4.9}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {5.4.10}推断}{260}{subsection.5.4.10}% \contentsline {subsection}{\numberline {5.4.10}推断}{260}{subsection.5.4.10}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {section}{\numberline {5.5}序列到序列问题及应用}{260}{section.5.5}% \contentsline {section}{\numberline {5.5}序列到序列问题及应用}{260}{section.5.5}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {5.5.1}自动问答}{261}{subsection.5.5.1}% \contentsline {subsection}{\numberline {5.5.1}自动问答}{261}{subsection.5.5.1}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {5.5.2}自动文摘}{261}{subsection.5.5.2}% \contentsline {subsection}{\numberline {5.5.2}自动文摘}{261}{subsection.5.5.2}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {5.5.3}文言文翻译}{262}{subsection.5.5.3}% \contentsline {subsection}{\numberline {5.5.3}文言文翻译}{262}{subsection.5.5.3}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {5.5.4}对联生成}{262}{subsection.5.5.4}% \contentsline {subsection}{\numberline {5.5.4}对联生成}{262}{subsection.5.5.4}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {5.5.5}古诗生成}{263}{subsection.5.5.5}% \contentsline {subsection}{\numberline {5.5.5}古诗生成}{263}{subsection.5.5.5}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {section}{\numberline {5.6}小结及深入阅读}{264}{section.5.6}% \contentsline {section}{\numberline {5.6}小结及深入阅读}{264}{section.5.6}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {part}{\@mypartnumtocformat {IV}{附录}}{267}{part.4}% \contentsline {part}{\@mypartnumtocformat {IV}{附录}}{267}{part.4}
\ttl@stoptoc {default@3} \ttl@stoptoc {default@3}
\ttl@starttoc {default@4} \ttl@starttoc {default@4}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {chapter}{\numberline {A}附录A}{269}{appendix.1.A}% \contentsline {chapter}{\numberline {A}附录A}{269}{Appendix.1.A}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {chapter}{\numberline {B}附录B}{271}{appendix.2.B}% \contentsline {chapter}{\numberline {B}附录B}{271}{Appendix.2.B}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {section}{\numberline {B.1}IBM模型3训练方法}{271}{section.2.B.1}% \contentsline {section}{\numberline {B.1}IBM模型3训练方法}{271}{section.2.B.1}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {section}{\numberline {B.2}IBM模型4训练方法}{273}{section.2.B.2}% \contentsline {section}{\numberline {B.2}IBM模型4训练方法}{273}{section.2.B.2}
\defcounter {refsection}{0}\relax \defcounter {refsection}{0}\relax
\contentsline {section}{\numberline {B.3}IBM模型5训练方法}{274}{section.2.B.3}% \contentsline {section}{\numberline {B.3}IBM模型5训练方法}{274}{section.2.B.3}
\contentsfinish \contentsfinish
Book/mt-book-xelatex.tex
...@@ -58,7 +58,7 @@ ...@@ -58,7 +58,7 @@
\node[inner sep=0pt] (background) at (current page.center) {\includegraphics[width=\paperwidth]{background.pdf}}; \node[inner sep=0pt] (background) at (current page.center) {\includegraphics[width=\paperwidth]{background.pdf}};
\draw (current page.center) node [fill=ocre!30!white,fill opacity=0.6,text opacity=1,inner sep=1cm]{\Huge\centering\bfseries\sffamily\parbox[c][][t]{\paperwidth}{\centering 机器翻译：统计建模与深度学习方法\\[15pt] % Book title \draw (current page.center) node [fill=ocre!30!white,fill opacity=0.6,text opacity=1,inner sep=1cm]{\Huge\centering\bfseries\sffamily\parbox[c][][t]{\paperwidth}{\centering 机器翻译：统计建模与深度学习方法\\[15pt] % Book title
%{\Large 副标题是否需要}\\[20pt] % Subtitle %{\Large 副标题是否需要}\\[20pt] % Subtitle
{\huge 肖桐\ \ 朱靖波}}}; % Author name {\LARGE 肖桐\ \ 朱靖波}}}; % Author name
\end{tikzpicture} \end{tikzpicture}
\vfill \vfill
\endgroup \endgroup
......
Markdown 格式
0% 或
您添加了 0 人 到此讨论。请谨慎行事。
请先完成此评论的编辑！
请 注册 或者 后发表评论