合并分支 'caorunzhe' 到 'master'

Caorunzhe

查看合并请求 !386

Chapter12/Figures/figure-dependencies-between-words-of-attention.tex

@@ -10,6 +10,7 @@
 \node [anchor=west,fill=green!20!white] (w5) at ([xshift=0.5em]w4.east) {$w_{m}$};
 \draw [->,thick,red] (w5.north).. controls +(100:0.85) and +(50:0.85) .. (w0.north);
 \draw [->,thick,red] (w5.north).. controls +(110:0.75) and +(50:0.75) .. (w1.north);
+\draw [->,thick,red] (w5.north).. controls +(110:0.75) and +(50:0.75) .. (w2.north);
 \draw [->,thick,red] (w5.north).. controls +(120:0.6) and +(50:0.6) .. ([yshift=0.2em]w3.north);
 \draw [->,thick,red] (w5.north).. controls +(130:0.5) and +(50:0.5) .. (w4.north);
 \draw [->,very thick,red] ([xshift=-5em]w0.west) -- ([xshift=-6.5em]w0.west) node [pos=0,right] {\scriptsize{信息传递}};

Chapter12/Figures/figure-point-product-attention-model.tex

@@ -28,7 +28,7 @@
 {
 \node [anchor=east] (line1) at ([xshift=-4em,yshift=1em]MatMul.west) {\scriptsize{自注意力机制的Query}};
 \node [anchor=north west] (line2) at ([yshift=0.3em]line1.south west) {\scriptsize{Key和Value均来自同一句}};
-\node [anchor=north west] (line3) at ([yshift=0.3em]line2.south west) {\scriptsize{子编码-解码注意力机制}};
+\node [anchor=north west] (line3) at ([yshift=0.3em]line2.south west) {\scriptsize{子，编码-解码注意力机制}};
 \node [anchor=north west] (line4) at ([yshift=0.3em]line3.south west) {\scriptsize{与前面讲的一样}};
 }
 {

Chapter12/chapter12.tex

@@ -25,7 +25,7 @@
 
 循环神经网络和卷积神经网络是两种经典的神经网络结构，在机器翻译中进行应用也是较为自然的想法。但是，这些模型在处理文字序列时也有问题：它们对序列中不同位置之间的依赖关系的建模并不直接。以卷积神经网络为例，如果要对长距离依赖进行描述，需要多层卷积操作，而且不同层之间信息传递也可能有损失，这些都限制了模型的能力。
 
-为了更好地描述文字序列，研究人员提出了一种新的模型Transformer。Transformer并不依赖任何循环单元或者卷积单元，而是使用一种被称作自注意力网络的结构来对序列进行表示。自注意力可以非常高效的描述任意距离之间的依赖关系，因此非常适合处理语言文字序列。Transformer一经提出就受到的了广泛关注，现在已经成为了机器翻译中最先进的架构之一。本章将会对Transformer的基本结构和实现技术进行介绍。这部分知识也会在本书的前沿部分（{\chapterthirteen}$\sim${\chaptereighteen}）中大量使用。
+为了更好地描述文字序列，研究人员提出了一种新的模型Transformer。Transformer并不依赖任何循环单元或者卷积单元，而是使用一种被称作自注意力网络的结构来对序列进行表示。自注意力可以非常高效的描述任意距离之间的依赖关系，因此非常适合处理语言文字序列。Transformer一经提出就受到了广泛关注，现在已经成为了机器翻译中最先进的架构之一。本章将会对Transformer的基本结构和实现技术进行介绍。这部分知识也会在本书的前沿部分（{\chapterthirteen}$\sim${\chaptereighteen}）中大量使用。
 
 %----------------------------------------------------------------------------------------
 %    NEW SECTION  12.1
@@ -56,7 +56,7 @@
 \end{figure}
 %----------------------------------------------
 
-\parinterval 自注意力机制也可以被看做是一个序列表示模型。比如，对于每个目标位置$j$，都生成一个与之对应的源语句子表示，它的形式为：
+\parinterval 自注意力机制也可以被看作是一个序列表示模型。比如，对于每个目标位置$j$，都生成一个与之对应的源语句子表示，它的形式为：
 \begin{eqnarray}
 \mathbi{C}_j & = & \sum_i \alpha_{i,j}\vectorn{\emph{h}}_i
 \label{eq:12-4201}
@@ -118,7 +118,7 @@
 \end{table}
 %----------------------------------------------
 
-\parinterval Transformer在被提出之后，很快就席卷了整个自然语言处理领域。实际上，Transformer也可以当作一种表示模型，因此也被大量地使用在自然语言处理的其他领域，甚至图像处理和语音处理中也能看到它的影子。比如，目前非常流行的BERT等预训练模型就是基于Transformer。表\ref{tab:12-12}展示了Transformer在WMT英德和英法机器翻译任务上的性能。它能用更少的计算量（FLOPS）达到比其他模型更好的翻译品质\footnote{FLOPS = floating-point operations per second，即每秒浮点运算次数。它是度量计算机运算规模的常用单位} 。
+\parinterval Transformer在被提出之后，很快就席卷了整个自然语言处理领域。实际上，也可以把Transformer当作一种表示模型，因此也被大量地使用在自然语言处理的其他领域，甚至图像处理和语音处理中也能看到它的影子。比如，目前非常流行的BERT等预训练模型就是基于Transformer。表\ref{tab:12-12}展示了Transformer在WMT英德和英法机器翻译任务上的性能。它能用更少的计算量（FLOPS）达到比其他模型更好的翻译品质\footnote{FLOPS = floating-point operations per second，即每秒浮点运算次数。它是度量计算机运算规模的常用单位} 。
 
 %----------------------------------------------
 \begin{table}[htp]

Chapter16/Figures/figure-target-side-multi-task-learning.tex

 \begin{tikzpicture}
 \begin{scope}
-
-\node [anchor=center] (node1) at (-2.3,0) {\small{$x,y$：双语数据}};
-\node [anchor=center] (node2) at (-2.1,-0.5) {\small{$z$}：单语数据};
 \node [anchor=center] (node1-1) at (0,0) {\small{$y'$}};
-\node [anchor=center] (node3-1) at ([xshift=5.5em,yshift=-0.1em]node1-1.east) {\small{$z'$}};
 \node[anchor=south,line width=0.6pt,draw,rounded corners,minimum height=1.5em,minimum width=4em,fill=blue!20](node1-2) at ([yshift=-3em]node1-1.south) {\small{softmax}};
-\node[anchor=south,line width=0.6pt,draw,rounded corners,minimum height=1.5em,minimum width=4em,fill=blue!20](node3-2) at ([yshift=-3em]node3-1.south) {\small{softmax}};
-\node[anchor=south,line width=0.6pt,draw,rounded corners,minimum height=2.2em,minimum width=4em,fill=red!20](node1-3) at ([yshift=-4.0em]node1-2.south) {\small{Decoder}};
-\node[anchor=south,line width=0.6pt,draw,rounded corners,minimum height=2.2em,minimum width=4em,fill=yellow!20](node3-3) at ([yshift=-4.0em]node3-2.south) {\small{LM}};
-\node[anchor=south](node1-4) at ([xshift=-0.6em,yshift=-3em]node1-3.south) {\gray{\small{$y$}}};
-\node[anchor=south](node3-41) at ([xshift=-0.6em,yshift=-3em]node3-3.south) {\small{$y$}};
-\node[anchor=south](node3-42) at ([xshift=0.6em,yshift=-2.9em]node3-3.south) {\small{$z$}};
-\node[anchor=west](node2-2) at ([xshift=-4.9em]node1-4.west) {\small{$x$}};
-\node[anchor=north,line width=0.6pt,draw,rounded corners,minimum height=2.2em,minimum width=4em,fill=red!20](node2-1) at ([yshift=4em]node2-2.north) {\small{Encoder}};
+
+\node[anchor=north,line width=0.6pt,draw,rounded corners,minimum height=2.2em,minimum width=4em,fill=red!20](node1-3) at ([yshift=-2.0em]node1-2.south) {\small{Decoder}};
+\node[anchor=north,line width=0.6pt,draw,rounded corners,minimum height=2.2em,minimum width=4em,fill=yellow!20](node3-3) at ([yshift=-2.0em]node1-3.south) {\small{LM}};
+
+
+\node[anchor=west,line width=0.6pt,draw,rounded corners,minimum height=1.5em,minimum width=4em,fill=blue!20](node3-2) at ([xshift=2em]node3-3.east) {\small{softmax}};
+\node [anchor=north] (node3-1) at ([yshift=3.0em]node3-2.north) {\small{$z'$}};
+
+
+\node[anchor=north](node3-41) at ([xshift=-0.6em,yshift=-2em]node3-3.south) {\small{$y$}};
+\node[anchor=north](node3-42) at ([xshift=0.6em,yshift=-2em]node3-3.south) {\small{$z$}};
+
+\node[anchor=east,line width=0.6pt,draw,rounded corners,minimum height=2.2em,minimum width=4em,fill=red!20](node2-1) at ([xshift=-2em]node1-3.west) {\small{Encoder}};
+\node[anchor=north](node2-2) at ([yshift=-2em]node2-1.south) {\small{$x$}};
+
+
 
 \node [rectangle,rounded corners,draw=red,line width=0.2mm,densely dashed,inner sep=0.4em] [fit = (node3-2) (node3-3)] (inputshadow) {};
-\draw [->,thick,draw=gray](node1-4.north)--([xshift=-0.6em]node1-3.south);
 \draw [->,thick](node1-3.north)--(node1-2);
 \draw [->,thick](node1-2.north)--(node1-1);
 \draw [->,thick](node2-2.north)--(node2-1);
@@ -24,11 +27,31 @@
 
 \draw [->,thick](node3-41.north)--([xshift=-0.6em]node3-3.south);
 \draw [->,thick](node3-42.north)--([xshift=0.6em]node3-3.south);
-\draw [->,thick]([xshift=0.6em]node3-3.north)--([xshift=0.6em]node3-2.south);
+\draw [->,thick](node3-3.north)--(node1-3.south);
 \draw [->,thick](node3-2.north)--(node3-1);
-\draw[->,thick]([xshift=-0.6em]node3-3.north)--([xshift=-0.6em,yshift=0.6em]node3-3.north)--([xshift=-3em,yshift=0.6em]node3-3.north)--([xshift=-3em,yshift=-3em]node3-3.north)--([xshift=-5.6em,yshift=-3em]node3-3.north)--([xshift=0.6em]node1-3.south);
+\draw[->,thick](node3-3.east)--(node3-2.west);
+
+
+
+\node [anchor=east] (node2-1-1) at ([xshift=-12.0em,yshift=-4.25em]node1-1.west) {\small{$y'$}};
+\node[anchor=south,line width=0.6pt,draw,rounded corners,minimum height=1.5em,minimum width=4em,fill=blue!20](node2-1-2) at ([yshift=-3em]node2-1-1.south) {\small{softmax}};
+\node[anchor=north,line width=0.6pt,draw,rounded corners,minimum height=2.2em,minimum width=4em,fill=red!20](node2-1-3) at ([yshift=-2.0em]node2-1-2.south) {\small{Decoder}};
+\node[anchor=east,line width=0.6pt,draw,rounded corners,minimum height=2.2em,minimum width=4em,fill=red!20](node2-2-1) at ([xshift=-2em]node2-1-3.west) {\small{Encoder}};
+\node[anchor=north](node2-2-2) at ([yshift=-2em]node2-2-1.south) {\small{$x$}};
+\node[anchor=north](node2-2-3) at ([yshift=-2em]node2-1-3.south) {\small{$y$}};
+
+\draw [->,thick](node2-1-2.north)--(node2-1-1);
+\draw [->,thick](node2-2-2.north)--(node2-2-1);
+\draw[->,thick](node2-2-1.east)--(node2-1-3.west);
+\draw [->,thick](node2-1-3.north)--(node2-1-2.south);
+\draw [->,thick](node2-2-3.north)--(node2-1-3);
+
+\node [anchor=east] (node1) at ([xshift=-2.0em,yshift=4em]node2-1-1.west) {\small{$x,y$：双语数据}};
+\node [anchor=north] (node2) at ([xshift=0.45em]node1.south) {\small{$z$}：单语数据};
+
+\node [anchor=north](pos1) at ([yshift=-3.5em]node3-3.south) {\small{(b)多任务学习}};
+\node [anchor=east](pos2) at ([xshift=-10.0em]pos1.west) {\small{(a)单任务学习}};
 
-%\draw[->](node2-1.north)--([yshift=1em]node2-1.north)--([xshift=2.5em,yshift=1em]node2-1.north)--([xshift=2.5em,yshift=-0.4em]node2-1.north)--(node1-3.west);
 \end{scope}
 
 \end{tikzpicture}
\ No newline at end of file