Skip to content

T
Toy-MT-Introduction
Commit 30bf3446 by xiaotong
Options

update

parent 48700c4f
正在显示 包含 12 行增加 和 77 行删除
Section04-Phrasal-and-Syntactic-Models/section04-test.tex
...@@ -155,71 +155,6 @@ ...@@ -155,71 +155,6 @@
%%%------------------------------------------------------------------------------------------------------------ %%%------------------------------------------------------------------------------------------------------------
\subsection{改进方法} \subsection{改进方法}
%%%------------------------------------------------------------------------------------------------------------
%%% 对于句法系统的改进
\begin{frame}{改进方法}
\begin{itemize}
\item \textbf{基于森林的翻译模型}。句法分析会出现错误，因此只使用一棵句法树进行规则抽取和解码会放大句法分析错误的影响。一种解决方法是使用多棵句法树增加覆盖度，句法森林是一种有效的数据结构表示指数级树结构，因此也被用于基于句法的机器翻译。
\vspace{0.3em}
\item \textbf{句法软约束和规则模糊匹配}。前面提到的模型都要求模型严格遵循句法结构，很多时候由于句法结构可能不完全适合翻译任务甚至有错误，这种模型过``硬''。因此可以使用句法软约束或者放松规则匹配时的约束。
\vspace{0.3em}
\item \textbf{控制句法使用的程度}。句法模型比较适合捕捉句法上层的表示，而短语模型更适合处理局部依赖。因此可以使用二者的混合来达到更好的效果，比如，可以让句法模型处理上层骨架的翻译，之后让短语模型处理简单短语片段的翻译。
\end{itemize}
\end{frame}
%%%------------------------------------------------------------------------------------------------------------
%%% 实验结果
\begin{frame}{翻译效果}
\begin{center}
\begin{tabular}{l | l | l | l}
\multicolumn{2}{c|}{模型} & 开发集 & 测试集 \\
\multicolumn{2}{c|}{} & (BLEU[\%]) & (BLEU[\%]) \\ \hline
\multicolumn{2}{l|}{短语(Moses)} & 36.51 & 34.93 \\
\multicolumn{2}{l|}{短语(NiuTrans)} & 36.99 & 35.29 \\ \hline
\multicolumn{2}{l|}{层次短语(Moses)} & 36.65 & 34.79 \\
\multicolumn{2}{l|}{层次短语(NiuTrans)} & 37.41 & 35.35 \\ \hline
& 基于串的解码 & 36.48 & 34.71 \\
树到串 & 基于树的解码 & 35.54 & 33.99 \\
(NiuTrans) & 基于森林的解码 & 36.14 & 34.25 \\ \hline
& 基于串的解码 & 35.99 & 34.01 \\
树到树 & 基于树的解码 & 35.04 & 33.21 \\
(NiuTrans) & 基于森林的解码 & 35.56 & 33.45 \\ \hline
串到树 & 基于串的解码 & 37.63 & 35.65 \\
(NiuTrans) & & &
\end{tabular}
\end{center}
\vspace{-0.5em}
\scriptsize{* 以上结果来自 NiuTrans: An Open Source Toolkit for Phrase-based and Syntax-based Machine Translation}
\end{frame}
%%%------------------------------------------------------------------------------------------------------------
%%% 总结
\begin{frame}{小结一下}
\begin{itemize}
\item 基于短语和基于句法的模型是机器翻译2000年之后的重要进展之一
\begin{itemize}
\item 研究热度持续十余年，至今仍有使用(无指导机器翻译中使用SMT做初始模型)
\item 一些方法代表了NLP中的原始创新，比如最小错误率训练
\end{itemize}
\item 相关技术和方法对现在研究仍然有很好的借鉴意义
\begin{itemize}
\item 对翻译的统计建模方法，比如，基于翻译文法和推导的机器翻译建模思想
\item 翻译调序等机器翻译特有问题的描述方法
\item 先验知识的使用，句法结构、篇章等等
\end{itemize}
\item<2-> 在深度学习时代下重新审视统计机器翻译
\begin{itemize}
\item 注意，统计机器翻译并不是简单几套系统，更重要的是思想，这种建模方法更接近人类对翻译的认知
\item 深度学习方法从另一个视角看待机器翻译，二者必然存在结合的可能，只是结合的方法需要探索
\end{itemize}
\end{itemize}
\end{frame}
%%%------------------------------------------------------------------------------------------------------------ %%%------------------------------------------------------------------------------------------------------------
%%% last slide %%% last slide
......
Section04-Phrasal-and-Syntactic-Models/section04.tex
...@@ -5055,14 +5055,14 @@ NP-BAR(NN$_1$ NP-BAR$_2$) $\to$ NN$_1$ NP-BAR$_2$ ...@@ -5055,14 +5055,14 @@ NP-BAR(NN$_1$ NP-BAR$_2$) $\to$ NN$_1$ NP-BAR$_2$
\vspace{0.4em} \vspace{0.4em}
\begin{center} \begin{center}
喜欢 VP$_1$ NP$_2$ $\to$ VP(VBZ(likes) VP$_1$ NP$_2$ \\ 喜欢 VP$_1$ NP$_2$ $\to$ VP(VBZ(likes) VP$_1$ NP$_2$) \\
\end{center} \end{center}
\vspace{0.4em} \vspace{0.4em}
二叉化之后变为 二叉化之后变为
\vspace{0.4em} \vspace{0.4em}
\begin{center} \begin{center}
\hspace{1em} 喜欢 V103 $\to$ VP(VBZ(likes) V103 \\ \hspace{1em} 喜欢 V103 $\to$ VP(VBZ(likes) V103) \\
\vspace{0.4em} \vspace{0.4em}
VP$_1$ NP$_2$ $\to$ V103(VP$_1$ NP$_2$) \\ VP$_1$ NP$_2$ $\to$ V103(VP$_1$ NP$_2$) \\
\end{center} \end{center}
......
Section06-Neural-Machine-Translation/section06.tex
...@@ -4060,7 +4060,7 @@ $\textrm{``you''} = \argmax_{y} \textrm{P}(y|\textbf{s}_1, \alert{\textbf{C}})$ ...@@ -4060,7 +4060,7 @@ $\textrm{``you''} = \argmax_{y} \textrm{P}(y|\textbf{s}_1, \alert{\textbf{C}})$
\node [Resnode,anchor=south] (res2) at ([yshift=0.3em]ffn1.north) {\tiny{$\textbf{Add \& LayerNorm}$}}; \node [Resnode,anchor=south] (res2) at ([yshift=0.3em]ffn1.north) {\tiny{$\textbf{Add \& LayerNorm}$}};
\node [inputnode,anchor=north west] (input1) at ([yshift=-1em]sa1.south west) {\tiny{$\textbf{Embedding}$}}; \node [inputnode,anchor=north west] (input1) at ([yshift=-1em]sa1.south west) {\tiny{$\textbf{Embedding}$}};
\node [posnode,anchor=north east] (pos1) at ([yshift=-1em]sa1.south east) {\tiny{$\textbf{Postion}$}}; \node [posnode,anchor=north east] (pos1) at ([yshift=-1em]sa1.south east) {\tiny{$\textbf{Postion}$}};
\node [anchor=north] (inputs) at ([yshift=-3em]sa1.south) {\tiny{$\textbf{Inputs: 我 很 好}$}}; \node [anchor=north] (inputs) at ([yshift=-3em]sa1.south) {\tiny{$\textbf{编码器输入: 我 很 好}$}};
\node [anchor=south] (encoder) at ([xshift=0.2em,yshift=0.6em]res2.north west) {\scriptsize{\textbf{编码器}}}; \node [anchor=south] (encoder) at ([xshift=0.2em,yshift=0.6em]res2.north west) {\scriptsize{\textbf{编码器}}};
\draw [->] (sa1.north) -- (res1.south); \draw [->] (sa1.north) -- (res1.south);
...@@ -4079,7 +4079,7 @@ $\textrm{``you''} = \argmax_{y} \textrm{P}(y|\textbf{s}_1, \alert{\textbf{C}})$ ...@@ -4079,7 +4079,7 @@ $\textrm{``you''} = \argmax_{y} \textrm{P}(y|\textbf{s}_1, \alert{\textbf{C}})$
\node [outputnode,anchor=south] (o1) at ([yshift=1em]res5.north) {\tiny{$\textbf{Output layer}$}}; \node [outputnode,anchor=south] (o1) at ([yshift=1em]res5.north) {\tiny{$\textbf{Output layer}$}};
\node [inputnode,anchor=north west] (input2) at ([yshift=-1em]sa2.south west) {\tiny{$\textbf{Embedding}$}}; \node [inputnode,anchor=north west] (input2) at ([yshift=-1em]sa2.south west) {\tiny{$\textbf{Embedding}$}};
\node [posnode,anchor=north east] (pos2) at ([yshift=-1em]sa2.south east) {\tiny{$\textbf{Postion}$}}; \node [posnode,anchor=north east] (pos2) at ([yshift=-1em]sa2.south east) {\tiny{$\textbf{Postion}$}};
\node [anchor=north] (outputs) at ([yshift=-3em]sa2.south) {\tiny{$\textbf{Outputs: I am fine}$}}; \node [anchor=north] (outputs) at ([yshift=-3em]sa2.south) {\tiny{$\textbf{解码器输入: $<$SOS$>$ I am fine}$}};
\node [anchor=east] (decoder) at ([xshift=-1em,yshift=-1.5em]o1.west) {\scriptsize{\textbf{解码器}}}; \node [anchor=east] (decoder) at ([xshift=-1em,yshift=-1.5em]o1.west) {\scriptsize{\textbf{解码器}}};
...@@ -4141,7 +4141,7 @@ $\textrm{``you''} = \argmax_{y} \textrm{P}(y|\textbf{s}_1, \alert{\textbf{C}})$ ...@@ -4141,7 +4141,7 @@ $\textrm{``you''} = \argmax_{y} \textrm{P}(y|\textbf{s}_1, \alert{\textbf{C}})$
\node [Resnode,anchor=south] (res2) at ([yshift=0.3em]ffn1.north) {\tiny{$\textbf{Add \& LayerNorm}$}}; \node [Resnode,anchor=south] (res2) at ([yshift=0.3em]ffn1.north) {\tiny{$\textbf{Add \& LayerNorm}$}};
\node [inputnode,anchor=north west] (input1) at ([yshift=-1em]sa1.south west) {\tiny{$\textbf{Embedding}$}}; \node [inputnode,anchor=north west] (input1) at ([yshift=-1em]sa1.south west) {\tiny{$\textbf{Embedding}$}};
\node [posnode,anchor=north east] (pos1) at ([yshift=-1em]sa1.south east) {\tiny{$\textbf{Postion}$}}; \node [posnode,anchor=north east] (pos1) at ([yshift=-1em]sa1.south east) {\tiny{$\textbf{Postion}$}};
\node [anchor=north] (inputs) at ([yshift=-3em]sa1.south) {\tiny{$\textbf{Inputs: 我 很 好}$}}; \node [anchor=north] (inputs) at ([yshift=-3em]sa1.south) {\tiny{$\textbf{编码器输入: 我 很 好}$}};
\node [anchor=south] (encoder) at ([xshift=0.2em,yshift=0.6em]res2.north west) {\scriptsize{\textbf{编码器}}}; \node [anchor=south] (encoder) at ([xshift=0.2em,yshift=0.6em]res2.north west) {\scriptsize{\textbf{编码器}}};
\draw [->] (sa1.north) -- (res1.south); \draw [->] (sa1.north) -- (res1.south);
...@@ -4160,7 +4160,7 @@ $\textrm{``you''} = \argmax_{y} \textrm{P}(y|\textbf{s}_1, \alert{\textbf{C}})$ ...@@ -4160,7 +4160,7 @@ $\textrm{``you''} = \argmax_{y} \textrm{P}(y|\textbf{s}_1, \alert{\textbf{C}})$
\node [outputnode,anchor=south] (o1) at ([yshift=1em]res5.north) {\tiny{$\textbf{Output layer}$}}; \node [outputnode,anchor=south] (o1) at ([yshift=1em]res5.north) {\tiny{$\textbf{Output layer}$}};
\node [inputnode,anchor=north west] (input2) at ([yshift=-1em]sa2.south west) {\tiny{$\textbf{Embedding}$}}; \node [inputnode,anchor=north west] (input2) at ([yshift=-1em]sa2.south west) {\tiny{$\textbf{Embedding}$}};
\node [posnode,anchor=north east] (pos2) at ([yshift=-1em]sa2.south east) {\tiny{$\textbf{Postion}$}}; \node [posnode,anchor=north east] (pos2) at ([yshift=-1em]sa2.south east) {\tiny{$\textbf{Postion}$}};
\node [anchor=north] (outputs) at ([yshift=-3em]sa2.south) {\tiny{$\textbf{Outputs: I am fine}$}}; \node [anchor=north] (outputs) at ([yshift=-3em]sa2.south) {\tiny{$\textbf{解码器输入: $<$SOS$>$ I am fine}$}};
\node [anchor=east] (decoder) at ([xshift=-1em,yshift=-1.5em]o1.west) {\scriptsize{\textbf{解码器}}}; \node [anchor=east] (decoder) at ([xshift=-1em,yshift=-1.5em]o1.west) {\scriptsize{\textbf{解码器}}};
...@@ -4372,7 +4372,7 @@ PE_{(pos,2i+1)} = cos(pos/10000^{2i/d_{model}}) ...@@ -4372,7 +4372,7 @@ PE_{(pos,2i+1)} = cos(pos/10000^{2i/d_{model}})
\node [Resnode,anchor=south] (res2) at ([yshift=0.3em]ffn1.north) {\tiny{$\textbf{Add \& LayerNorm}$}}; \node [Resnode,anchor=south] (res2) at ([yshift=0.3em]ffn1.north) {\tiny{$\textbf{Add \& LayerNorm}$}};
\node [inputnode,anchor=north west] (input1) at ([yshift=-1em]sa1.south west) {\tiny{$\textbf{Embedding}$}}; \node [inputnode,anchor=north west] (input1) at ([yshift=-1em]sa1.south west) {\tiny{$\textbf{Embedding}$}};
\node [posnode,anchor=north east] (pos1) at ([yshift=-1em]sa1.south east) {\tiny{$\textbf{Postion}$}}; \node [posnode,anchor=north east] (pos1) at ([yshift=-1em]sa1.south east) {\tiny{$\textbf{Postion}$}};
\node [anchor=north] (inputs) at ([yshift=-3em]sa1.south) {\tiny{$\textbf{Inputs: 我 很 好}$}}; \node [anchor=north] (inputs) at ([yshift=-3em]sa1.south) {\tiny{$\textbf{编码器输入: 我 很 好}$}};
\node [anchor=south] (encoder) at ([xshift=0.2em,yshift=0.6em]res2.north west) {\scriptsize{\textbf{编码器}}}; \node [anchor=south] (encoder) at ([xshift=0.2em,yshift=0.6em]res2.north west) {\scriptsize{\textbf{编码器}}};
\draw [->] (sa1.north) -- (res1.south); \draw [->] (sa1.north) -- (res1.south);
...@@ -4391,7 +4391,7 @@ PE_{(pos,2i+1)} = cos(pos/10000^{2i/d_{model}}) ...@@ -4391,7 +4391,7 @@ PE_{(pos,2i+1)} = cos(pos/10000^{2i/d_{model}})
\node [outputnode,anchor=south] (o1) at ([yshift=1em]res5.north) {\tiny{$\textbf{Output layer}$}}; \node [outputnode,anchor=south] (o1) at ([yshift=1em]res5.north) {\tiny{$\textbf{Output layer}$}};
\node [inputnode,anchor=north west] (input2) at ([yshift=-1em]sa2.south west) {\tiny{$\textbf{Embedding}$}}; \node [inputnode,anchor=north west] (input2) at ([yshift=-1em]sa2.south west) {\tiny{$\textbf{Embedding}$}};
\node [posnode,anchor=north east] (pos2) at ([yshift=-1em]sa2.south east) {\tiny{$\textbf{Postion}$}}; \node [posnode,anchor=north east] (pos2) at ([yshift=-1em]sa2.south east) {\tiny{$\textbf{Postion}$}};
\node [anchor=north] (outputs) at ([yshift=-3em]sa2.south) {\tiny{$\textbf{Outputs: I am fine}$}}; \node [anchor=north] (outputs) at ([yshift=-3em]sa2.south) {\tiny{$\textbf{解码器输入: $<$SOS$>$ I am fine}$}};
\node [anchor=east] (decoder) at ([xshift=-1em,yshift=-1.5em]o1.west) {\scriptsize{\textbf{解码器}}}; \node [anchor=east] (decoder) at ([xshift=-1em,yshift=-1.5em]o1.west) {\scriptsize{\textbf{解码器}}};
...@@ -4759,7 +4759,7 @@ PE_{(pos,2i+1)} = cos(pos/10000^{2i/d_{model}}) ...@@ -4759,7 +4759,7 @@ PE_{(pos,2i+1)} = cos(pos/10000^{2i/d_{model}})
\node [Resnode,anchor=south] (res2) at ([yshift=0.3em]ffn1.north) {\tiny{$\textbf{Add \& LayerNorm}$}}; \node [Resnode,anchor=south] (res2) at ([yshift=0.3em]ffn1.north) {\tiny{$\textbf{Add \& LayerNorm}$}};
\node [inputnode,anchor=north west] (input1) at ([yshift=-1em]sa1.south west) {\tiny{$\textbf{Embedding}$}}; \node [inputnode,anchor=north west] (input1) at ([yshift=-1em]sa1.south west) {\tiny{$\textbf{Embedding}$}};
\node [posnode,anchor=north east] (pos1) at ([yshift=-1em]sa1.south east) {\tiny{$\textbf{Postion}$}}; \node [posnode,anchor=north east] (pos1) at ([yshift=-1em]sa1.south east) {\tiny{$\textbf{Postion}$}};
\node [anchor=north] (inputs) at ([yshift=-3em]sa1.south) {\tiny{$\textbf{Inputs: 我 很 好}$}}; \node [anchor=north] (inputs) at ([yshift=-3em]sa1.south) {\tiny{$\textbf{编码器输入: 我 很 好}$}};
\node [anchor=south] (encoder) at ([xshift=0.2em,yshift=0.6em]res2.north west) {\scriptsize{\textbf{编码器}}}; \node [anchor=south] (encoder) at ([xshift=0.2em,yshift=0.6em]res2.north west) {\scriptsize{\textbf{编码器}}};
\draw [->] (sa1.north) -- (res1.south); \draw [->] (sa1.north) -- (res1.south);
...@@ -4778,7 +4778,7 @@ PE_{(pos,2i+1)} = cos(pos/10000^{2i/d_{model}}) ...@@ -4778,7 +4778,7 @@ PE_{(pos,2i+1)} = cos(pos/10000^{2i/d_{model}})
\node [outputnode,anchor=south] (o1) at ([yshift=1em]res5.north) {\tiny{$\textbf{Output layer}$}}; \node [outputnode,anchor=south] (o1) at ([yshift=1em]res5.north) {\tiny{$\textbf{Output layer}$}};
\node [inputnode,anchor=north west] (input2) at ([yshift=-1em]sa2.south west) {\tiny{$\textbf{Embedding}$}}; \node [inputnode,anchor=north west] (input2) at ([yshift=-1em]sa2.south west) {\tiny{$\textbf{Embedding}$}};
\node [posnode,anchor=north east] (pos2) at ([yshift=-1em]sa2.south east) {\tiny{$\textbf{Postion}$}}; \node [posnode,anchor=north east] (pos2) at ([yshift=-1em]sa2.south east) {\tiny{$\textbf{Postion}$}};
\node [anchor=north] (outputs) at ([yshift=-3em]sa2.south) {\tiny{$\textbf{Outputs: I am fine}$}}; \node [anchor=north] (outputs) at ([yshift=-3em]sa2.south) {\tiny{$\textbf{解码器输入: $<$SOS$>$ I am fine}$}};
\node [anchor=east] (decoder) at ([xshift=-1em,yshift=-1.5em]o1.west) {\scriptsize{\textbf{解码器}}}; \node [anchor=east] (decoder) at ([xshift=-1em,yshift=-1.5em]o1.west) {\scriptsize{\textbf{解码器}}};
...@@ -4984,7 +4984,7 @@ x_{l+1} = x_l+\mathcal{F}(x_l) ...@@ -4984,7 +4984,7 @@ x_{l+1} = x_l+\mathcal{F}(x_l)
\node [Resnode,anchor=south] (res2) at ([yshift=0.3em]ffn1.north) {\tiny{$\textbf{Add \& LayerNorm}$}}; \node [Resnode,anchor=south] (res2) at ([yshift=0.3em]ffn1.north) {\tiny{$\textbf{Add \& LayerNorm}$}};
\node [inputnode,anchor=north west] (input1) at ([yshift=-1em]sa1.south west) {\tiny{$\textbf{Embedding}$}}; \node [inputnode,anchor=north west] (input1) at ([yshift=-1em]sa1.south west) {\tiny{$\textbf{Embedding}$}};
\node [posnode,anchor=north east] (pos1) at ([yshift=-1em]sa1.south east) {\tiny{$\textbf{Postion}$}}; \node [posnode,anchor=north east] (pos1) at ([yshift=-1em]sa1.south east) {\tiny{$\textbf{Postion}$}};
\node [anchor=north] (inputs) at ([yshift=-3em]sa1.south) {\tiny{$\textbf{Inputs: 我 很 好}$}}; \node [anchor=north] (inputs) at ([yshift=-3em]sa1.south) {\tiny{$\textbf{编码器输入: 我 很 好}$}};
\node [anchor=south] (encoder) at ([xshift=0.2em,yshift=0.6em]res2.north west) {\scriptsize{\textbf{编码器}}}; \node [anchor=south] (encoder) at ([xshift=0.2em,yshift=0.6em]res2.north west) {\scriptsize{\textbf{编码器}}};
\draw [->] (sa1.north) -- (res1.south); \draw [->] (sa1.north) -- (res1.south);
...@@ -5003,7 +5003,7 @@ x_{l+1} = x_l+\mathcal{F}(x_l) ...@@ -5003,7 +5003,7 @@ x_{l+1} = x_l+\mathcal{F}(x_l)
\node [outputnode,anchor=south] (o1) at ([yshift=1em]res5.north) {\tiny{$\textbf{Output layer}$}}; \node [outputnode,anchor=south] (o1) at ([yshift=1em]res5.north) {\tiny{$\textbf{Output layer}$}};
\node [inputnode,anchor=north west] (input2) at ([yshift=-1em]sa2.south west) {\tiny{$\textbf{Embedding}$}}; \node [inputnode,anchor=north west] (input2) at ([yshift=-1em]sa2.south west) {\tiny{$\textbf{Embedding}$}};
\node [posnode,anchor=north east] (pos2) at ([yshift=-1em]sa2.south east) {\tiny{$\textbf{Postion}$}}; \node [posnode,anchor=north east] (pos2) at ([yshift=-1em]sa2.south east) {\tiny{$\textbf{Postion}$}};
\node [anchor=north] (outputs) at ([yshift=-3em]sa2.south) {\tiny{$\textbf{Outputs: I am fine}$}}; \node [anchor=north] (outputs) at ([yshift=-3em]sa2.south) {\tiny{$\textbf{解码器输入: $<$SOS$>$ I am fine}$}};
\node [anchor=east] (decoder) at ([xshift=-1em,yshift=-1.5em]o1.west) {\scriptsize{\textbf{解码器}}}; \node [anchor=east] (decoder) at ([xshift=-1em,yshift=-1.5em]o1.west) {\scriptsize{\textbf{解码器}}};
......
Markdown 格式
0% 或
您添加了 0 人 到此讨论。请谨慎行事。
请先完成此评论的编辑！
请 注册 或者 后发表评论