Skip to content

T
Toy-MT-Introduction
Commit e10537de by 曹润柘
Options

合并分支 'caorunzhe' 到 'master' 

Caorunzhe

查看合并请求 !273
parents 35ddc010 b64ef0db
正在显示 包含 39 行增加 和 23 行删除
Book/Chapter6/Figures/figure-example-of-context-vector-calculation-process.tex
...@@ -104,7 +104,7 @@ ...@@ -104,7 +104,7 @@
%\visible<3-> %\visible<3->
{ {
% coverage score formula node % coverage score formula node
\node [anchor=north west] (formula) at ([xshift=-0.3\hnode,yshift=-1.5\hnode]attn11.south) {\small{不同$\textbf{C}_i$所对应的源语言词的权重是不同的}}; \node [anchor=north west] (formula) at ([xshift=-0.3\hnode,yshift=-1.5\hnode]attn11.south) {\small{不同$\textbf{C}_j$所对应的源语言词的权重是不同的}};
\node [anchor=north west] (example) at (formula.south west) {\footnotesize{$\textbf{C}_2=0.4 \times \textbf{h}(\textrm{``你''}) + 0.4 \times \textbf{h}(\textrm{``什么''}) +$}}; \node [anchor=north west] (example) at (formula.south west) {\footnotesize{$\textbf{C}_2=0.4 \times \textbf{h}(\textrm{``你''}) + 0.4 \times \textbf{h}(\textrm{``什么''}) +$}};
\node [anchor=north west] (example2) at ([yshift=0.4em]example.south west) {\footnotesize{$\ \ \ \ \ \ \ \ 0 \times \textbf{h}(\textrm{``都''}) + 0.1 \times \textbf{h}(\textrm{`` 没''}) + ..$}}; \node [anchor=north west] (example2) at ([yshift=0.4em]example.south west) {\footnotesize{$\ \ \ \ \ \ \ \ 0 \times \textbf{h}(\textrm{``都''}) + 0.1 \times \textbf{h}(\textrm{`` 没''}) + ..$}};
} }
......
Section01-Introduction/section01.tex
...@@ -304,8 +304,8 @@ ...@@ -304,8 +304,8 @@
\visible<3->{ \visible<3->{
\begin{center} \begin{center}
\begin{tikzpicture} \begin{tikzpicture}
\node [anchor=south west, fill=red, minimum width=1.5cm, minimum height=2.3cm] (mt) at (1,0) {{\color{white} \textbf{机器}}}; \node [anchor=south west, fill=red!50, minimum width=1.5cm, minimum height=2.3cm] (mt) at (1,0) {{\color{white} \textbf{机器}}};
\node [anchor=south west, fill=ugreen, minimum width=1.5cm, minimum height=2.7cm] (human) at ([xshift=0.5cm]mt.south east) {{\color{white} \textbf{人}}}; \node [anchor=south west, fill=blue!50, minimum width=1.5cm, minimum height=2.7cm] (human) at ([xshift=0.5cm]mt.south east) {{\color{white} \textbf{人}}};
\node [anchor=south] (mtscore) at (mt.north) {3.9}; \node [anchor=south] (mtscore) at (mt.north) {3.9};
\node [anchor=south] (humanscore) at (human.north) {4.7}; \node [anchor=south] (humanscore) at (human.north) {4.7};
\draw [->,thick] ([xshift=-0.5cm]mt.south west) -- ([xshift=0.5cm]human.south east); \draw [->,thick] ([xshift=-0.5cm]mt.south west) -- ([xshift=0.5cm]human.south east);
...@@ -321,8 +321,8 @@ ...@@ -321,8 +321,8 @@
\visible<4->{ \visible<4->{
\begin{center} \begin{center}
\begin{tikzpicture} \begin{tikzpicture}
\node [anchor=south west, fill=red, minimum width=1.5cm, minimum height=1.5cm] (mt) at (1,0) {{\color{white} \textbf{机器}}}; \node [anchor=south west, fill=red!50, minimum width=1.5cm, minimum height=1.5cm] (mt) at (1,0) {{\color{white} \textbf{机器}}};
\node [anchor=south west, fill=ugreen, minimum width=1.5cm, minimum height=2.7cm] (human) at ([xshift=0.5cm]mt.south east) {{\color{white} \textbf{人}}}; \node [anchor=south west, fill=blue!50, minimum width=1.5cm, minimum height=2.7cm] (human) at ([xshift=0.5cm]mt.south east) {{\color{white} \textbf{人}}};
\node [anchor=south] (mtscore) at (mt.north) {47\%}; \node [anchor=south] (mtscore) at (mt.north) {47\%};
\node [anchor=south] (humanscore) at (human.north) {100\%}; \node [anchor=south] (humanscore) at (human.north) {100\%};
\draw [->,thick] ([xshift=-0.5cm]mt.south west) -- ([xshift=0.5cm]human.south east); \draw [->,thick] ([xshift=-0.5cm]mt.south west) -- ([xshift=0.5cm]human.south east);
......
Section04-Phrasal-and-Syntactic-Models/section04.tex
...@@ -3706,8 +3706,8 @@ d = r_1 \circ r_2 \circ r_3 \circ r_4 ...@@ -3706,8 +3706,8 @@ d = r_1 \circ r_2 \circ r_3 \circ r_4
\subsection{基于chart的解码} \subsection{基于chart的解码}
%%%------------------------------------------------------------------------------------------------------------ %%%------------------------------------------------------------------------------------------------------------
%%% CYK解码 %%% CKY解码
\begin{frame}{CYK解码} \begin{frame}{CKY解码}
% 看NiuTrans Manual % 看NiuTrans Manual
\begin{itemize} \begin{itemize}
\item 基于层次短语的翻译解码与基于短语的模型类似，都是要找到使$\textrm{score}(d)$达到最大的翻译推导$d$ \item 基于层次短语的翻译解码与基于短语的模型类似，都是要找到使$\textrm{score}(d)$达到最大的翻译推导$d$
...@@ -3717,8 +3717,8 @@ d = r_1 \circ r_2 \circ r_3 \circ r_4 ...@@ -3717,8 +3717,8 @@ d = r_1 \circ r_2 \circ r_3 \circ r_4
\end{displaymath} \end{displaymath}
\vspace{-0.8em} \vspace{-0.8em}
\begin{itemize} \begin{itemize}
\item 由于翻译推导由SCFG构成，使用CYK算法进行解码 \item 由于翻译推导由SCFG构成，使用CKY算法进行解码
\item CYK算法解码是一个用来判定任意给定的字符串 是否属于一个上下文无关文法的算法，具体流程如下 \item CKY算法解码是一个用来判定任意给定的字符串 是否属于一个上下文无关文法的算法，具体流程如下
\end{itemize} \end{itemize}
\vspace{0.5em} \vspace{0.5em}
\begin{center} \begin{center}
...@@ -3740,16 +3740,16 @@ d = r_1 \circ r_2 \circ r_3 \circ r_4 ...@@ -3740,16 +3740,16 @@ d = r_1 \circ r_2 \circ r_3 \circ r_4
\end{tikzpicture} \end{tikzpicture}
\end{center} \end{center}
\vspace{0.3em} \vspace{0.3em}
%\item 由于对文法中的非终结符进行了限制，可以直接使用CYK算法进行解码，无需转换成乔姆斯基范式 %\item 由于对文法中的非终结符进行了限制，可以直接使用CKY算法进行解码，无需转换成乔姆斯基范式
\end{itemize} \end{itemize}
\end{frame} \end{frame}
%%%------------------------------------------------------------------------------------------------------------ %%%------------------------------------------------------------------------------------------------------------
%%% CYK解码 %%% CKY解码
\begin{frame}{CYK算法} \begin{frame}{CKY算法}
% 看NiuTrans Manual % 看NiuTrans Manual
\begin{itemize} \begin{itemize}
\item CYK算法通过遍历不同\alert{span}来判断字符串是否符合文法 \item CKY算法通过遍历不同\alert{span}来判断字符串是否符合文法
\begin{itemize} \begin{itemize}
\item 输入：源语串\textbf{s =} $s_1 ... s_J$，以及CNF文法$G$ \item 输入：源语串\textbf{s =} $s_1 ... s_J$，以及CNF文法$G$
\item 输出：判断字符串是否符合G \item 输出：判断字符串是否符合G
...@@ -3762,7 +3762,7 @@ d = r_1 \circ r_2 \circ r_3 \circ r_4 ...@@ -3762,7 +3762,7 @@ d = r_1 \circ r_2 \circ r_3 \circ r_4
\tikzstyle{srcnode} = [anchor=south west] \tikzstyle{srcnode} = [anchor=south west]
\begin{scope}[scale=0.85] \begin{scope}[scale=0.85]
\node[srcnode] (c1) at (0,0) {\small{\textbf{Function} CYK-Algorithm($\textbf{s},G$)}}; \node[srcnode] (c1) at (0,0) {\small{\textbf{Function} CKY-Algorithm($\textbf{s},G$)}};
\node[srcnode,anchor=north west] (c21) at ([xshift=1.5em,yshift=0.4em]c1.south west) {\small{\textbf{fore} $j=0$ to $ J - 1$}}; \node[srcnode,anchor=north west] (c21) at ([xshift=1.5em,yshift=0.4em]c1.south west) {\small{\textbf{fore} $j=0$ to $ J - 1$}};
\node[srcnode,anchor=north west] (c22) at ([xshift=1.5em,yshift=0.4em]c21.south west) {\small{$span[j,j+1 ]$.Add($A \to a \in G$)}}; \node[srcnode,anchor=north west] (c22) at ([xshift=1.5em,yshift=0.4em]c21.south west) {\small{$span[j,j+1 ]$.Add($A \to a \in G$)}};
\node[srcnode,anchor=north west] (c3) at ([xshift=-1.5em,yshift=0.4em]c22.south west) {\small{\textbf{for} $l$ = 1 to $J$}}; \node[srcnode,anchor=north west] (c3) at ([xshift=-1.5em,yshift=0.4em]c22.south west) {\small{\textbf{for} $l$ = 1 to $J$}};
...@@ -3810,11 +3810,11 @@ d = r_1 \circ r_2 \circ r_3 \circ r_4 ...@@ -3810,11 +3810,11 @@ d = r_1 \circ r_2 \circ r_3 \circ r_4
\end{frame} \end{frame}
%%%------------------------------------------------------------------------------------------------------------ %%%------------------------------------------------------------------------------------------------------------
%%% CYK解码 %%% CKY解码
\begin{frame}{CYK算法} \begin{frame}{CKY算法}
% 看NiuTrans Manual % 看NiuTrans Manual
\begin{itemize} \begin{itemize}
\item 我们来看一个CYK算法的具体例子，给定一个上下无关文法以及一个单词\alert{aabbc}，来判断该单词是否属于此文法，解析流程如下 \item 我们来看一个CKY算法的具体例子，给定一个上下无关文法以及一个单词\alert{aabbc}，来判断该单词是否属于此文法，解析流程如下
\vspace{-0.3em} \vspace{-0.3em}
\begin{center} \begin{center}
\begin{tikzpicture} \begin{tikzpicture}
...@@ -3946,11 +3946,11 @@ d = r_1 \circ r_2 \circ r_3 \circ r_4 ...@@ -3946,11 +3946,11 @@ d = r_1 \circ r_2 \circ r_3 \circ r_4
\end{frame} \end{frame}
%%%------------------------------------------------------------------------------------------------------------ %%%------------------------------------------------------------------------------------------------------------
%%% CYK解码 %%% CKY解码
\begin{frame}{CYK解码（续）} \begin{frame}{CKY解码（续）}
% 看NiuTrans Manual % 看NiuTrans Manual
\begin{itemize} \begin{itemize}
\item 实际上，在层次短语解码的时候，不能直接使用CYK算法，需要先转化为乔姆斯基范式，才能进行解码 \item 实际上，在层次短语解码的时候，不能直接使用CKY算法，需要先转化为乔姆斯基范式，才能进行解码
\begin{itemize} \begin{itemize}
\item<2-> 对于每个源语句子，使用短语规则表初始化它的span \item<2-> 对于每个源语句子，使用短语规则表初始化它的span
\item<3-> 自底向上对span中的每个子span进行重新组合（正、反向） \item<3-> 自底向上对span中的每个子span进行重新组合（正、反向）
...@@ -4166,7 +4166,7 @@ d = r_1 \circ r_2 \circ r_3 \circ r_4 ...@@ -4166,7 +4166,7 @@ d = r_1 \circ r_2 \circ r_3 \circ r_4
% 实验结果 % 实验结果
\begin{itemize} \begin{itemize}
\item 从实验结果中可以看出，基于层次短语的翻译模型性能要优于基于短语的翻译模型 \item 从实验结果中可以看出，基于层次短语的翻译模型性能要优于基于短语的翻译模型
\item 选择使用层次短语信息实际上增加了模型的复杂度，但是可以通过借鉴基于短语的翻译模型模型以及CYK解码和立方剪枝等技术来解决 \item 选择使用层次短语信息实际上增加了模型的复杂度，但是可以通过借鉴基于短语的翻译模型模型以及CKY解码和立方剪枝等技术来解决
\item 可以考虑加入更多句法信息来进一步提升模型性能 \item 可以考虑加入更多句法信息来进一步提升模型性能
\end{itemize} \end{itemize}
%\vspace{-1em} %\vspace{-1em}
...@@ -6785,7 +6785,7 @@ NP-BAR(NN$_1$ NP-BAR$_2$) $\to$ NN$_1$ NP-BAR$_2$ ...@@ -6785,7 +6785,7 @@ NP-BAR(NN$_1$ NP-BAR$_2$) $\to$ NN$_1$ NP-BAR$_2$
搜索空间 & 与输入的源语句法树 & 所有推导$D$ \\ 搜索空间 & 与输入的源语句法树 & 所有推导$D$ \\
& 兼容的推导$D_{\textrm{tree}}$ & \\ \hline & 兼容的推导$D_{\textrm{tree}}$ & \\ \hline
适用模型 & 树到串、树到树 & 所有句法模型 \\ \hline 适用模型 & 树到串、树到树 & 所有句法模型 \\ \hline
解码算法 & chart解码 & CYK + 规则二叉化 \\ \hline 解码算法 & chart解码 & CKY + 规则二叉化 \\ \hline
速度 & 快 & 一般较慢 速度 & 快 & 一般较慢
\end{tabular} \end{tabular}
...@@ -7358,7 +7358,7 @@ NP-BAR(NN$_1$ NP-BAR$_2$) $\to$ NN$_1$ NP-BAR$_2$ ...@@ -7358,7 +7358,7 @@ NP-BAR(NN$_1$ NP-BAR$_2$) $\to$ NN$_1$ NP-BAR$_2$
\end{frame} \end{frame}
%%%------------------------------------------------------------------------------------------------------------ %%%------------------------------------------------------------------------------------------------------------
%%% 基于串的解码 - CYK + 规则二叉化 %%% 基于串的解码 - CKY + 规则二叉化
\begin{frame}{基于串的解码 - CKY + 规则二叉化} \begin{frame}{基于串的解码 - CKY + 规则二叉化}
\begin{itemize} \begin{itemize}
......
Section05-Neural-Networks-and-Language-Modeling/section05.tex
...@@ -5031,6 +5031,10 @@ GPT-2 (Transformer) & Radford et al. & 2019 & 35.7 ...@@ -5031,6 +5031,10 @@ GPT-2 (Transformer) & Radford et al. & 2019 & 35.7
\node [anchor=west,draw,inner sep=4pt,fill=ugreen!20!white,minimum width=2em] (e2) at ([xshift=1em]e1.east) {\scriptsize{$\textbf{e}_2$}}; \node [anchor=west,draw,inner sep=4pt,fill=ugreen!20!white,minimum width=2em] (e2) at ([xshift=1em]e1.east) {\scriptsize{$\textbf{e}_2$}};
\node [anchor=west,inner sep=4pt] (sep5) at ([xshift=1em]e2.east) {\scriptsize{...}}; \node [anchor=west,inner sep=4pt] (sep5) at ([xshift=1em]e2.east) {\scriptsize{...}};
\node [anchor=west,draw,inner sep=4pt,fill=ugreen!20!white,minimum width=2em] (e3) at ([xshift=1em]sep5.east) {\scriptsize{$\textbf{e}_m$}}; \node [anchor=west,draw,inner sep=4pt,fill=ugreen!20!white,minimum width=2em] (e3) at ([xshift=1em]sep5.east) {\scriptsize{$\textbf{e}_m$}};
\node [anchor=south] (word1) at ([yshift=-1.5em]e1.south) {\footnotesize {Once}};
\node [anchor=south] (word2) at ([yshift=-1.6em]e2.south) {\footnotesize {upon}};
\node [anchor=south] (wordseq) at ([yshift=-1.5em]sep5.south) {\footnotesize{...}};
\node [anchor=south] (word3) at ([yshift=-1.5em]e3.south) {\footnotesize {island}};
\node [anchor=south,draw,inner sep=4pt,fill=yellow!30,minimum width=2em] (t1) at ([xshift=-2em,yshift=1em]Lstm5.north) {\scriptsize{$\textbf{h}_1$}}; \node [anchor=south,draw,inner sep=4pt,fill=yellow!30,minimum width=2em] (t1) at ([xshift=-2em,yshift=1em]Lstm5.north) {\scriptsize{$\textbf{h}_1$}};
\node [anchor=west,draw,inner sep=4pt,fill=yellow!30,minimum width=2em] (t2) at ([xshift=1em]t1.east) {\scriptsize{$\textbf{h}_2$}}; \node [anchor=west,draw,inner sep=4pt,fill=yellow!30,minimum width=2em] (t2) at ([xshift=1em]t1.east) {\scriptsize{$\textbf{h}_2$}};
...@@ -5130,6 +5134,12 @@ GPT-2 (Transformer) & Radford et al. & 2019 & 35.7 ...@@ -5130,6 +5134,12 @@ GPT-2 (Transformer) & Radford et al. & 2019 & 35.7
\node [anchor=north,draw,inner sep=4pt,fill=ugreen!20!white,minimum width=2em] (e4) at ([yshift=-1em]Trm3.south) {\scriptsize{$\textbf{e}_4$}}; \node [anchor=north,draw,inner sep=4pt,fill=ugreen!20!white,minimum width=2em] (e4) at ([yshift=-1em]Trm3.south) {\scriptsize{$\textbf{e}_4$}};
\node [anchor=north,inner sep=4pt] (sep5) at ([yshift=-1em]sep.south) {\scriptsize{...}}; \node [anchor=north,inner sep=4pt] (sep5) at ([yshift=-1em]sep.south) {\scriptsize{...}};
\node [anchor=north,draw,inner sep=4pt,fill=ugreen!20!white,minimum width=2em] (e5) at ([yshift=-1em]Trm4.south) {\scriptsize{$\textbf{e}_m$}}; \node [anchor=north,draw,inner sep=4pt,fill=ugreen!20!white,minimum width=2em] (e5) at ([yshift=-1em]Trm4.south) {\scriptsize{$\textbf{e}_m$}};
\node [anchor=south] (word1) at ([yshift=-1.5em]e1.south) {\footnotesize {Once}};
\node [anchor=south] (word2) at ([yshift=-1.6em]e2.south) {\footnotesize {upon}};
\node [anchor=south] (word3) at ([yshift=-1.5em]e3.south) {\footnotesize {a}};
\node [anchor=south] (word4) at ([yshift=-1.5em]e4.south) {\footnotesize {time}};
\node [anchor=south] (wordseq) at ([yshift=-2.0em]sep5.south) {\footnotesize{...}};
\node [anchor=south] (word4) at ([yshift=-1.5em]e5.south) {\footnotesize {island}};
\node [anchor=south,draw,inner sep=4pt,fill=yellow!30,minimum width=2em] (t1) at ([yshift=1em]Trm5.north) {\scriptsize{$\textbf{h}_1$}}; \node [anchor=south,draw,inner sep=4pt,fill=yellow!30,minimum width=2em] (t1) at ([yshift=1em]Trm5.north) {\scriptsize{$\textbf{h}_1$}};
\node [anchor=south,draw,inner sep=4pt,fill=yellow!30,minimum width=2em] (t2) at ([yshift=1em]Trm6.north) {\scriptsize{$\textbf{h}_2$}}; \node [anchor=south,draw,inner sep=4pt,fill=yellow!30,minimum width=2em] (t2) at ([yshift=1em]Trm6.north) {\scriptsize{$\textbf{h}_2$}};
...@@ -5214,6 +5224,12 @@ GPT-2 (Transformer) & Radford et al. & 2019 & 35.7 ...@@ -5214,6 +5224,12 @@ GPT-2 (Transformer) & Radford et al. & 2019 & 35.7
\node [anchor=north,draw,inner sep=4pt,fill=ugreen!20!white,minimum width=2em] (e4) at ([yshift=-1em]Trm3.south) {\scriptsize{$\textbf{e}_4$}}; \node [anchor=north,draw,inner sep=4pt,fill=ugreen!20!white,minimum width=2em] (e4) at ([yshift=-1em]Trm3.south) {\scriptsize{$\textbf{e}_4$}};
\node [anchor=north,inner sep=4pt] (sep5) at ([yshift=-1em]sep.south) {\scriptsize{...}}; \node [anchor=north,inner sep=4pt] (sep5) at ([yshift=-1em]sep.south) {\scriptsize{...}};
\node [anchor=north,draw,inner sep=4pt,fill=ugreen!20!white,minimum width=2em] (e5) at ([yshift=-1em]Trm4.south) {\scriptsize{$\textbf{e}_m$}}; \node [anchor=north,draw,inner sep=4pt,fill=ugreen!20!white,minimum width=2em] (e5) at ([yshift=-1em]Trm4.south) {\scriptsize{$\textbf{e}_m$}};
\node [anchor=south] (word1) at ([yshift=-1.5em]e1.south) {\footnotesize {Once}};
\node [anchor=south] (word2) at ([yshift=-1.7em]e2.south) {\footnotesize {[MASK]}};
\node [anchor=south] (word3) at ([yshift=-1.5em]e3.south) {\footnotesize {a}};
\node [anchor=south] (word4) at ([yshift=-1.5em]e4.south) {\footnotesize {time}};
\node [anchor=south] (wordseq) at ([yshift=-2.0em]sep5.south) {\footnotesize{...}};
\node [anchor=south] (word4) at ([yshift=-1.5em]e5.south) {\footnotesize {island}};
\node [anchor=south,draw,inner sep=4pt,fill=yellow!30,minimum width=2em] (t1) at ([yshift=1em]Trm5.north) {\scriptsize{$\textbf{h}_1$}}; \node [anchor=south,draw,inner sep=4pt,fill=yellow!30,minimum width=2em] (t1) at ([yshift=1em]Trm5.north) {\scriptsize{$\textbf{h}_1$}};
\node [anchor=south,draw,inner sep=4pt,fill=yellow!30,minimum width=2em] (t2) at ([yshift=1em]Trm6.north) {\scriptsize{$\textbf{h}_2$}}; \node [anchor=south,draw,inner sep=4pt,fill=yellow!30,minimum width=2em] (t2) at ([yshift=1em]Trm6.north) {\scriptsize{$\textbf{h}_2$}};
......
Markdown 格式
0% 或
您添加了 0 人 到此讨论。请谨慎行事。
请先完成此评论的编辑！
请 注册 或者 后发表评论