Skip to content

M
mtbookv2
Commit cccbe612 by 孟霞
Options

合并分支 'caorunzhe' 到 'mengxia' 

Caorunzhe

查看合并请求 !588
parents 30f904ef dba1ccef
正在显示 包含 949 行增加 和 113 行删除
Chapter14/chapter14.tex
...@@ -108,13 +108,13 @@ ...@@ -108,13 +108,13 @@
\parinterval 以上两种推断方式在神经机器翻译中都有应用，对于源语言句子$\seq{x}=\{x_1,x_2,\dots,x_m\}$和目标语句子$\seq{y}=\{y_1,y_2,\dots,y_n\}$，用自左向右的方式可以将翻译概率$\funp{P}(\seq{y}\vert\seq{x})$描述为公式\eqref{eq:14-1}： \parinterval 以上两种推断方式在神经机器翻译中都有应用，对于源语言句子$\seq{x}=\{x_1,x_2,\dots,x_m\}$和目标语句子$\seq{y}=\{y_1,y_2,\dots,y_n\}$，用自左向右的方式可以将翻译概率$\funp{P}(\seq{y}\vert\seq{x})$描述为公式\eqref{eq:14-1}：
\begin{eqnarray} \begin{eqnarray}
\funp{P}(\seq{y}\vert\seq{x}) &=& \prod_{j=1}^n \funp{P}(y_j\vert\seq{y}_{<j},\seq{x}) \funp{P}(\seq{y}\vert\seq{x})=\prod_{j=1}^n \funp{P}(y_j\vert\seq{y}_{<j},\seq{x})
\label{eq:14-1} \label{eq:14-1}
\end{eqnarray} \end{eqnarray}
\parinterval 而用自右向左的方式可以得到公式\eqref{eq:14-2}： \parinterval 而用自右向左的方式可以得到公式\eqref{eq:14-2}：
\begin{eqnarray} \begin{eqnarray}
\funp{P}(\seq{y}\vert\seq{x}) &=& \prod_{j=1}^n \funp{P}(y_{n+1-j}\vert\seq{y}_{>j},\seq{x}) \funp{P}(\seq{y}\vert\seq{x})=\prod_{j=1}^n \funp{P}(y_{n+1-j}\vert\seq{y}_{>j},\seq{x})
\label{eq:14-2} \label{eq:14-2}
\end{eqnarray} \end{eqnarray}
\parinterval 其中，$\seq{y}_{<j}=\{y_1,y_2,\dots,y_{j-1}\}$，$\seq{y}_{>j}=\{y_{j+1},y_{j+2},\dots,y_n\}$。 \parinterval 其中，$\seq{y}_{<j}=\{y_1,y_2,\dots,y_{j-1}\}$，$\seq{y}_{>j}=\{y_{j+1},y_{j+2},\dots,y_n\}$。
...@@ -148,7 +148,7 @@ ...@@ -148,7 +148,7 @@
\item 长度惩罚因子。用译文长度来归一化翻译概率是最常用的方法：对于源语言句子$\seq{x}$和译文句子$\seq{y}$，模型得分$\textrm{score}(\seq{x},\seq{y})$的值会随着译文$\seq{y}$ 的变长而减小，为了避免此现象，可以引入一个长度惩罚函数$\textrm{lp}(\seq{y})$，并定义模型得分如公式\eqref{eq:14-12}所示： \item 长度惩罚因子。用译文长度来归一化翻译概率是最常用的方法：对于源语言句子$\seq{x}$和译文句子$\seq{y}$，模型得分$\textrm{score}(\seq{x},\seq{y})$的值会随着译文$\seq{y}$ 的变长而减小，为了避免此现象，可以引入一个长度惩罚函数$\textrm{lp}(\seq{y})$，并定义模型得分如公式\eqref{eq:14-12}所示：
\begin{eqnarray} \begin{eqnarray}
\textrm{score}(\seq{x},\seq{y}) &=& \frac{\log \funp{P}(\seq{y}\vert\seq{x})}{\textrm{lp}(\seq{y})} \textrm{score}(\seq{x},\seq{y})=\frac{\log \funp{P}(\seq{y}\vert\seq{x})}{\textrm{lp}(\seq{y})}
\label{eq:14-12} \label{eq:14-12}
\end{eqnarray} \end{eqnarray}
...@@ -188,7 +188,7 @@ b &=& \omega_{\textrm{high}}\cdot |\seq{x}| \label{eq:14-4} ...@@ -188,7 +188,7 @@ b &=& \omega_{\textrm{high}}\cdot |\seq{x}| \label{eq:14-4}
\noindent 其中，$\textrm{cp}(\seq{x},\seq{y}) $表示覆盖度模型，它度量了译文对源语言每个单词的覆盖程度。$\textrm{cp}(\seq{x},\seq{y}) $的定义中，$a_{ij}$表示源语言第$i$个位置与目标语第$j$个位置的注意力权重，这样$\sum \limits_{j}^{|\seq{y}|} a_{ij}$就可以用来衡量源语言第$i$个单词被翻译了“多少”，如果它大于1，表明翻译多了；如果小于1，表明翻译少了。公式\eqref{eq:14-6}会惩罚那些欠翻译的翻译假设。覆盖度模型的一种改进形式是\upcite{li-etal-2018-simple}： \noindent 其中，$\textrm{cp}(\seq{x},\seq{y}) $表示覆盖度模型，它度量了译文对源语言每个单词的覆盖程度。$\textrm{cp}(\seq{x},\seq{y}) $的定义中，$a_{ij}$表示源语言第$i$个位置与目标语第$j$个位置的注意力权重，这样$\sum \limits_{j}^{|\seq{y}|} a_{ij}$就可以用来衡量源语言第$i$个单词被翻译了“多少”，如果它大于1，表明翻译多了；如果小于1，表明翻译少了。公式\eqref{eq:14-6}会惩罚那些欠翻译的翻译假设。覆盖度模型的一种改进形式是\upcite{li-etal-2018-simple}：
\begin{eqnarray} \begin{eqnarray}
\textrm{cp}(\seq{x},\seq{y}) &=& \sum_{i=1}^{|\seq{x}|} \log( \textrm{max} ( \sum_{j}^{|\seq{y}|} a_{ij},\beta)) \textrm{cp}(\seq{x},\seq{y}) = \sum_{i=1}^{|\seq{x}|} \log( \textrm{max} ( \sum_{j}^{|\seq{y}|} a_{ij},\beta))
\label{eq:14-7} \label{eq:14-7}
\end{eqnarray} \end{eqnarray}
\noindent 公式\eqref{eq:14-7}将公式\eqref{eq:14-6}中的向下截断方式换为了向上截断。这样，模型可以对过翻译（或重复翻译）有更好的建模能力。不过，这个模型需要在开发集上细致地调整$\beta$，也带来了一定的额外工作量。此外，也可以将这种覆盖度单独建模并进行参数化，与翻译模型一同训练\upcite{Mi2016CoverageEM,TuModeling,Kazimi2017CoverageFC}。这样可以得到更加精细的覆盖度模型。 \noindent 公式\eqref{eq:14-7}将公式\eqref{eq:14-6}中的向下截断方式换为了向上截断。这样，模型可以对过翻译（或重复翻译）有更好的建模能力。不过，这个模型需要在开发集上细致地调整$\beta$，也带来了一定的额外工作量。此外，也可以将这种覆盖度单独建模并进行参数化，与翻译模型一同训练\upcite{Mi2016CoverageEM,TuModeling,Kazimi2017CoverageFC}。这样可以得到更加精细的覆盖度模型。
...@@ -416,7 +416,7 @@ b &=& \omega_{\textrm{high}}\cdot |\seq{x}| \label{eq:14-4} ...@@ -416,7 +416,7 @@ b &=& \omega_{\textrm{high}}\cdot |\seq{x}| \label{eq:14-4}
\parinterval 目前主流的神经机器翻译的推断是一种{\small\sffamily\bfseries{自回归翻译}}\index{自回归翻译}（Autoregressive Translation）\index{Autoregressive Translation}过程。所谓自回归是一种描述时间序列生成的方式。对于目标序列$\seq{y}=\{y_1,\dots,y_n\}$，自回归模型假设$j$时刻状态$y_j$的生成依赖于之前的状态$\{y_1,\dots,y_{j-1}\}$，而且$y_j$与$\{y_1,\dots,y_{j-1}\}$构成线性关系，那么生成$y_j$就是自回归的序列生成过程。神经机器翻译借用了这个概念，但是并不要求线性模型。对于输入的源语言序列$\seq{x}=\{x_1,\dots,x_m\}$，用自回归翻译模型生成译文序列$\seq{y}=\{y_1,\dots,y_n\}$的概率可以被定义为： \parinterval 目前主流的神经机器翻译的推断是一种{\small\sffamily\bfseries{自回归翻译}}\index{自回归翻译}（Autoregressive Translation）\index{Autoregressive Translation}过程。所谓自回归是一种描述时间序列生成的方式。对于目标序列$\seq{y}=\{y_1,\dots,y_n\}$，自回归模型假设$j$时刻状态$y_j$的生成依赖于之前的状态$\{y_1,\dots,y_{j-1}\}$，而且$y_j$与$\{y_1,\dots,y_{j-1}\}$构成线性关系，那么生成$y_j$就是自回归的序列生成过程。神经机器翻译借用了这个概念，但是并不要求线性模型。对于输入的源语言序列$\seq{x}=\{x_1,\dots,x_m\}$，用自回归翻译模型生成译文序列$\seq{y}=\{y_1,\dots,y_n\}$的概率可以被定义为：
\begin{eqnarray} \begin{eqnarray}
\funp{P}(\seq{y}|\seq{x}) &=& \prod_{j=1}^n {\funp{P}(y_j|y_{<j},\seq{x})} \funp{P}(\seq{y}|\seq{x})=\prod_{j=1}^n {\funp{P}(y_j|y_{<j},\seq{x})}
\label{eq:14-8} \label{eq:14-8}
\end{eqnarray} \end{eqnarray}
...@@ -425,7 +425,7 @@ b &=& \omega_{\textrm{high}}\cdot |\seq{x}| \label{eq:14-4} ...@@ -425,7 +425,7 @@ b &=& \omega_{\textrm{high}}\cdot |\seq{x}| \label{eq:14-4}
\parinterval 对于这个问题，研究者也考虑移除翻译的自归回性，进行{\small\sffamily\bfseries{非自回归翻译}}\index{非自回归翻译}（Non-Autoregressive Translation，NAT）\index{Non-Autoregressive Translation}\upcite{Gu2017NonAutoregressiveNM}。一个简单的非自回归翻译模型将问题建模为： \parinterval 对于这个问题，研究者也考虑移除翻译的自归回性，进行{\small\sffamily\bfseries{非自回归翻译}}\index{非自回归翻译}（Non-Autoregressive Translation，NAT）\index{Non-Autoregressive Translation}\upcite{Gu2017NonAutoregressiveNM}。一个简单的非自回归翻译模型将问题建模为：
\begin{eqnarray} \begin{eqnarray}
\funp{P}(\seq{y}|\seq{x}) &=& \prod_{j=1}^n {\funp{P}(y_j|\seq{x})} \funp{P}(\seq{y}|\seq{x})=\prod_{j=1}^n {\funp{P}(y_j|\seq{x})}
\label{eq:14-9} \label{eq:14-9}
\end{eqnarray} \end{eqnarray}
...@@ -485,7 +485,7 @@ b &=& \omega_{\textrm{high}}\cdot |\seq{x}| \label{eq:14-4} ...@@ -485,7 +485,7 @@ b &=& \omega_{\textrm{high}}\cdot |\seq{x}| \label{eq:14-4}
\parinterval 另外，在每个解码器层中还包括额外的位置注意力模块，该模块与Transformer模型的其它部分中使用的多头注意力机制相同，如下： \parinterval 另外，在每个解码器层中还包括额外的位置注意力模块，该模块与Transformer模型的其它部分中使用的多头注意力机制相同，如下：
\begin{eqnarray} \begin{eqnarray}
\textrm{Attention}(\mathbi{Q},\mathbi{K},\mathbi{V}) &=& \textrm{Softmax}(\frac{\mathbi{Q}{\mathbi{K}}^{T}}{\sqrt{d_k}})\cdot \mathbi{V} \textrm{Attention}(\mathbi{Q},\mathbi{K},\mathbi{V})&=&\textrm{Softmax}(\frac{\mathbi{Q}{\mathbi{K}}^{T}}{\sqrt{d_k}})\cdot \mathbi{V}
\label{eq:14-10} \label{eq:14-10}
\end{eqnarray} \end{eqnarray}
...@@ -651,7 +651,7 @@ b &=& \omega_{\textrm{high}}\cdot |\seq{x}| \label{eq:14-4} ...@@ -651,7 +651,7 @@ b &=& \omega_{\textrm{high}}\cdot |\seq{x}| \label{eq:14-4}
\parinterval 神经机器翻译模型对每个目标端位置$j$的单词分布进行预测，即对于目标语言词汇表中的每个单词$y_j$，需要计算$\funp{P}(y_j | \seq{y}_{<j},\seq{x})$。假设有$K$个神经机器翻译系统，那么每个系统$k$都可以独立的计算这个概率，记为$\funp{P}_{k} (y_j | \seq{y}_{<j},\seq{x})$。于是，可以用公式\eqref{eq:14-11}融合这$K$个系统的预测： \parinterval 神经机器翻译模型对每个目标端位置$j$的单词分布进行预测，即对于目标语言词汇表中的每个单词$y_j$，需要计算$\funp{P}(y_j | \seq{y}_{<j},\seq{x})$。假设有$K$个神经机器翻译系统，那么每个系统$k$都可以独立的计算这个概率，记为$\funp{P}_{k} (y_j | \seq{y}_{<j},\seq{x})$。于是，可以用公式\eqref{eq:14-11}融合这$K$个系统的预测：
\begin{eqnarray} \begin{eqnarray}
\funp{P}(y_{j} | \seq{y}_{<j},\seq{x}) &=& \sum_{k=1}^K \gamma_{k} \cdot \funp{P}_{k} (y_j | \seq{y}_{<j},\seq{x}) \funp{P}(y_{j} | \seq{y}_{<j},\seq{x}) = \sum_{k=1}^K \gamma_{k} \cdot \funp{P}_{k} (y_j | \seq{y}_{<j},\seq{x})
\label{eq:14-11} \label{eq:14-11}
\end{eqnarray} \end{eqnarray}
......
Chapter15/Figures/figure-activation-function-swish-structure-diagram.tex 0 → 100644
%%% outline
%-------------------------------------------------------------------------
\begin{tikzpicture}
\tikzstyle{opt}=[draw,minimum height=2em,minimum width=4em,rounded corners=2pt,thick]
\node[circle,minimum size=2em,draw,fill=red!20] (x1) at (0,0) {\small\bfnew{X}};
\node[circle,minimum size=2em,draw,fill=red!20] (x2) at (0,4em) {\small\bfnew{X}};
\node[circle,minimum size=2em,draw,fill=red!20] (x3) at (0,-5em) {\small\bfnew{X}};
\node[anchor=west,opt,fill=yellow!20] (unary1) at ([xshift=3em]x1.east){\small\bfnew{Unary}};
\node[anchor=west,opt,fill=yellow!20] (unary2) at ([xshift=3em]x2.east){\small\bfnew{Unary}};
\node[opt,fill=blue!20] (binary1) at (12em,2em){\small\bfnew{Binary}};
\node[opt,fill=blue!20] (binary2) at (25em,-1.5em){\small\bfnew{Binary}};
\node[anchor=west,opt,fill=yellow!20] (unary3) at ([xshift=3em]binary1.east){\small\bfnew{Unary}};
\node[anchor=west,opt,fill=yellow!20] (unary4) at ([xshift=16em]x3.east){\small\bfnew{Unary}};
\draw[-latex,very thick] (x1.0) -- (unary1.180);
\draw[-latex,very thick] (x2.0) -- (unary2.180);
\draw[-latex,very thick] (x3.0) -- (unary4.180);
\draw[-latex,very thick] (unary1.0) -- ([yshift=-0.2em]binary1.180);
\draw[-latex,very thick] (unary2.0) -- ([yshift=0.2em]binary1.180);
\draw[-latex,very thick] (binary1.0) -- (unary3.180);
\draw[-latex,very thick] (unary3.0) -- ([yshift=0.2em]binary2.180);
\draw[-latex,very thick] (unary4.0) -- ([yshift=-0.2em]binary2.180);
\begin{pgfonlayer}{background}
\node[draw=lightgray,fill=lightgray!50,rounded corners=2pt,inner sep=8pt][fit=(x2)(x1)(binary1)]{};
\end{pgfonlayer}
\node[anchor=south] at ([yshift=1em]binary1.north){\small\bfnew{Core Unit}};
\end{tikzpicture}
Chapter15/Figures/figure-layer-fusion-method.tex 0 → 100644
\begin{tikzpicture}
\begin{scope}
\tikzstyle{encnode}=[rectangle,inner sep=0mm,minimum height=2em,minimum width=4.5em,rounded corners=5pt,thick]
\tikzstyle{decnode}=[rectangle,inner sep=0mm,minimum height=2em,minimum width=4.5em,rounded corners=5pt,thick]
\node [anchor=north,encnode] (n1) at (0, 0) {编码器};
\node [anchor=north,rectangle,minimum height=1.5em,minimum width=2.5em,rounded corners=5pt] (n2) at ([xshift=0em,yshift=-0.2em]n1.south) {$\mathbi{X}$};
\node [anchor=west,encnode,draw=red!60!black!80,fill=red!20] (n3) at ([xshift=1.5em,yshift=0em]n2.east) {$\mathbi{h}_0$};
\node [anchor=west,encnode,draw=red!60!black!80,fill=red!20] (n4) at ([xshift=1.5em,yshift=0em]n3.east) {$\mathbi{h}_1$};
\node [anchor=west,encnode,draw=red!60!black!80,fill=red!20] (n5) at ([xshift=1.5em,yshift=0em]n4.east) {$\mathbi{h}_2$};
\node [anchor=west,rectangle,minimum height=1.5em,minimum width=2.5em,rounded corners=5pt] (n6) at ([xshift=1em,yshift=0em]n5.east) {$\ldots$};
\node [anchor=west,encnode,draw=red!60!black!80,fill=red!20] (n7) at ([xshift=1em,yshift=0em]n6.east) {$\mathbi{h}_{N-1}$};
\node [anchor=north,rectangle,draw=teal!80, inner sep=0mm,minimum height=2em,minimum width=8em,fill=teal!17,rounded corners=5pt,thick] (n8) at ([xshift=3em,yshift=-1.2em]n4.south) {权重聚合$\mathbi{g}$};
\node [anchor=west,decnode] (n9) at ([xshift=0em,yshift=-7.2em]n1.west) {解码器};
\node [anchor=north,rectangle,minimum height=1.5em,minimum width=2.5em,rounded corners=5pt] (n10) at ([xshift=0em,yshift=-0.2em]n9.south) {$\mathbi{y}_{<j}$};
\node [anchor=west,decnode,draw=ublue,fill=blue!10] (n11) at ([xshift=1.5em,yshift=0em]n10.east) {$\mathbi{s}_j^0$};
\node [anchor=west,decnode,draw=ublue,fill=blue!10] (n12) at ([xshift=1.5em,yshift=0em]n11.east) {$\mathbi{s}_j^1$};
\node [anchor=west,decnode,draw=ublue,fill=blue!10] (n13) at ([xshift=1.5em,yshift=0em]n12.east) {$\mathbi{s}_j^2$};
\node [anchor=west,rectangle,minimum height=1.5em,minimum width=2.5em,rounded corners=5pt] (n14) at ([xshift=1em,yshift=0em]n13.east) {$\ldots$};
\node [anchor=west,decnode,draw=ublue,fill=blue!10] (n15) at ([xshift=1em,yshift=0em]n14.east) {$\mathbi{s}_j^{M-1}$};
\node [anchor=west,rectangle,minimum height=1.5em,minimum width=2.5em,rounded corners=5pt] (n16) at ([xshift=1.5em,yshift=0em]n15.east) {$\mathbi{y}_{j}$};
\node [anchor=south,minimum height=1.5em,minimum width=2.5em] (n17) at ([xshift=0em,yshift=6em]n16.north) {};
\node [anchor=north,minimum height=0.5em,minimum width=4em] (n18) at ([xshift=0em,yshift=-0.7em]n4.south) {};
\node [anchor=north,minimum height=0.5em,minimum width=4em] (n19) at ([xshift=0em,yshift=-0.7em]n13.south) {};
\node [anchor=west,minimum height=0.5em,minimum width=4em] (n20) at ([xshift=0em,yshift=0.7em]n8.east) {};
\node [anchor=west,minimum height=0.5em,minimum width=4em] (n21) at ([xshift=0em,yshift=-0.7em]n8.east) {};
\begin{pgfonlayer}{background}
{
\node[rectangle,inner sep=2pt,fill=blue!7] [fit = (n1) (n7) (n17) (n18) (n20)] (bg1) {};
\node[rectangle,inner sep=2pt,fill=red!7] [fit = (n9) (n16) (n19) (n21)] (bg2) {};
}
\end{pgfonlayer}
\draw [->,thick] ([xshift=0em,yshift=0em]n2.east) -- ([xshift=0em,yshift=0em]n3.west);
\draw [->,thick] ([xshift=0em,yshift=0em]n3.east) -- ([xshift=0em,yshift=0em]n4.west);
\draw [->,thick] ([xshift=0em,yshift=0em]n4.east) -- ([xshift=0em,yshift=0em]n5.west);
\draw [->,thick] ([xshift=0em,yshift=0em]n5.east) -- ([xshift=0em,yshift=0em]n6.west);
\draw [->,thick] ([xshift=0em,yshift=0em]n6.east) -- ([xshift=0em,yshift=0em]n7.west);
\draw [->,thick] ([xshift=0em,yshift=0em]n2.south) -- ([xshift=0em,yshift=0em]n8.north);
\draw [->,thick] ([xshift=0em,yshift=0em]n3.south) -- ([xshift=0em,yshift=0em]n8.north);
\draw [->,thick] ([xshift=0em,yshift=0em]n4.south) -- ([xshift=0em,yshift=0em]n8.north);
\draw [->,thick] ([xshift=0em,yshift=0em]n5.south) -- ([xshift=0em,yshift=0em]n8.north);
\draw [->,thick] ([xshift=0em,yshift=0em]n7.south) -- ([xshift=0em,yshift=0em]n8.north);
\draw [->,thick] ([xshift=0em,yshift=0em]n10.east) -- ([xshift=0em,yshift=0em]n11.west);
\draw [->,thick] ([xshift=0em,yshift=0em]n11.east) -- ([xshift=0em,yshift=0em]n12.west);
\draw [->,thick] ([xshift=0em,yshift=0em]n12.east) -- ([xshift=0em,yshift=0em]n13.west);
\draw [->,thick] ([xshift=0em,yshift=0em]n13.east) -- ([xshift=0em,yshift=0em]n14.west);
\draw [->,thick] ([xshift=0em,yshift=0em]n14.east) -- ([xshift=0em,yshift=0em]n15.west);
\draw [->,thick] ([xshift=0em,yshift=0em]n15.east) -- ([xshift=0em,yshift=0em]n16.west);
\draw [->,thick] ([xshift=0em,yshift=0em]n8.south) -- ([xshift=0em,yshift=0em]n11.north);
\draw [->,thick] ([xshift=0em,yshift=0em]n8.south) -- ([xshift=0em,yshift=0em]n12.north);
\draw [->,thick] ([xshift=0em,yshift=0em]n8.south) -- ([xshift=0em,yshift=0em]n13.north);
\draw [->,thick] ([xshift=0em,yshift=0em]n8.south) -- ([xshift=0em,yshift=0em]n15.north);
\end{scope}
\end{tikzpicture}
\ No newline at end of file
Chapter15/Figures/figure-learning-of-local-structure-combination.tex 0 → 100644
%%% outline
%-------------------------------------------------------------------------
\begin{tikzpicture}
\tikzstyle{opt}=[minimum height=1em,minimum width=5em,rounded corners=2pt,thick]
\node[opt] (opt1_0) at (0,0) {固定操作};
\node[draw,anchor=south,opt,fill=green!20] (opt1_1) at ([yshift=0.8em]opt1_0.north) {操作3};
\node[draw,anchor=south,opt,fill=cyan!20] (opt1_2) at ([yshift=0.8em]opt1_1.north) {操作2};
\node[anchor=south,opt] (opt1_3) at ([yshift=0.8em]opt1_2.north) {固定操作};
\node[draw,anchor=south,opt,fill=yellow!20] (opt1_4) at ([yshift=0.8em]opt1_3.north) {操作4};
\node[anchor=south,opt] (opt1_5) at ([yshift=0.8em]opt1_4.north) {$\cdots$};
\node[draw,anchor=south,opt,fill=blue!10] (opt1_6) at ([yshift=0.8em]opt1_5.north) {操作1};
\node[anchor=south,opt] (opt1_7) at ([yshift=0.8em]opt1_6.north) {};
\node[opt] (opt2_0) at (3,0) {固定操作};
\node[draw,anchor=south,opt,fill=blue!10] (opt2_1) at ([yshift=0.8em]opt2_0.north) {操作1};
\node[draw,anchor=south,opt,fill=green!20] (opt2_2) at ([yshift=0.8em]opt2_1.north) {操作3};
\node[anchor=south,opt] (opt2_3) at ([yshift=0.8em]opt2_2.north) {固定操作};
\node[draw,anchor=south,opt,fill=red!20] (opt2_4) at ([yshift=0.8em]opt2_3.north) {操作5};
\node[anchor=south,opt] (opt2_5) at ([yshift=0.8em]opt2_4.north) {$\cdots$};
\node[draw,anchor=south,opt,fill=cyan!20] (opt2_6) at ([yshift=0.8em]opt2_5.north) {操作2};
\node[anchor=south,opt] (opt2_7) at ([yshift=0.8em]opt2_6.north) {};
\node[opt] (opt3_0) at (6,0) {固定操作};
\node[draw,anchor=south,opt,fill=yellow!20] (opt3_1) at ([yshift=0.8em]opt3_0.north) {操作4};
\node[draw,anchor=south,opt,fill=cyan!20] (opt3_2) at ([yshift=0.8em]opt3_1.north) {操作2};
\node[anchor=south,opt] (opt3_3) at ([yshift=0.8em]opt3_2.north) {固定操作};
\node[draw,anchor=south,opt,fill=yellow!20] (opt3_4) at ([yshift=0.8em]opt3_3.north) {操作4};
\node[anchor=south,opt] (opt3_5) at ([yshift=0.8em]opt3_4.north) {$\cdots$};
\node[draw,anchor=south,opt,fill=red!20] (opt3_6) at ([yshift=0.8em]opt3_5.north) {操作5};
\node[anchor=south,opt] (opt3_7) at ([yshift=0.8em]opt3_6.north) {};
\begin{pgfonlayer}{background}
\node[draw,fill=yellow!20,rounded corners=6pt,inner ysep=2.6em,inner xsep=2.6em] [fit=(opt1_0) (opt3_7)](box4){};
\node[draw,fill=gray!10,rounded corners=2pt,inner sep=8pt] [fit=(opt1_0) (opt1_7)](box1){};
\node[draw,fill=gray!10,rounded corners=2pt,inner sep=8pt] [fit=(opt2_0) (opt2_7)](box2){};
\node[draw,fill=gray!10,rounded corners=2pt,inner sep=8pt] [fit=(opt3_0) (opt3_7)](box3){};
\end{pgfonlayer}
\draw[->,thick] (opt1_0) -- (opt1_1);
\draw[->,thick] (opt1_1) -- (opt1_2);
\draw[->,thick] (opt1_2) -- (opt1_3);
\draw[->,thick] (opt1_3) -- (opt1_4);
\draw[->,thick] (opt1_4) -- (opt1_5);
\draw[->,thick] (opt1_5) -- (opt1_6);
\draw[->,thick] (opt2_0) -- (opt2_1);
\draw[->,thick] (opt2_1) -- (opt2_2);
\draw[->,thick] (opt2_2) -- (opt2_3);
\draw[->,thick] (opt2_3) -- (opt2_4);
\draw[->,thick] (opt2_4) -- (opt2_5);
\draw[->,thick] (opt2_5) -- (opt2_6);
\draw[->,thick] (opt3_0) -- (opt3_1);
\draw[->,thick] (opt3_1) -- (opt3_2);
\draw[->,thick] (opt3_2) -- (opt3_3);
\draw[->,thick] (opt3_3) -- (opt3_4);
\draw[->,thick] (opt3_4) -- (opt3_5);
\draw[->,thick] (opt3_5) -- (opt3_6);
\node[] at ([xshift=-1.2em,yshift=0.2em]opt1_7){\small\bfnew{分支1}};
\node[] at ([xshift=-1.2em,yshift=0.2em]opt2_7){\small\bfnew{分支2}};
\node[] at ([xshift=-1.2em,yshift=0.2em]opt3_7){\small\bfnew{分支3}};
\node[] (input) at ([yshift=-5em]opt2_0){\small\bfnew{输入}};
\node[] (output) at ([yshift=5em]opt2_7){\small\bfnew{输出}};
\draw[->,thick,out=140,in=-30] (box4.-90) to (box1.-90);
\draw[->,thick,out=40,in=-150] (box4.-90) to (box3.-90);
\draw[->,thick] (box4.-90) -- (box2.-90);
\draw[->,thick,out=50,in=-130] (box1.90) to (box4.90);
\draw[->,thick,out=130,in=-50] (box3.90) to (box4.90);
\draw[->,thick] (box2.90) -- (box4.90);
\draw[->,thick] (input.90) -- (box4.-90);
\draw[->,thick] (box4.90) -- (output.-90);
\node[] at ([xshift=-2.8em,yshift=1.1em]box1.90){\small\bfnew{模型结构}};
\node[] at ([xshift=-0.8em]box4.0){$\cdots$};
\end{tikzpicture}
Chapter15/Figures/figure-structure-search-based-on-evolutionary-algorithm.tex 0 → 100644
%%% outline
%-------------------------------------------------------------------------
\begin{tikzpicture}
\tikzstyle{node}=[minimum height=2em,minimum width=4.6em,rounded corners=2pt,thick,font=\footnotesize]
\node[node,draw,fill=green!20] (n11) at (0,0) {模型结构1};
\node[anchor=west,draw,node,fill=red!20] (n12) at ([xshift=0.6em]n11.east){模型结构6};
\node[anchor=north,draw,node,fill=orange!20] (n13) at ([yshift=-1em]n11.south){模型结构2};
\node[anchor=west,draw,node,fill=blue!15] (n14) at ([xshift=0.6em]n13.east){模型结构5};
\node[anchor=north,draw,node,fill=cyan!30] (n15) at ([yshift=-1em]n13.south){模型结构1};
\node[anchor=west,draw,node,fill=yellow!20] (n16) at ([xshift=0.6em]n15.east){模型结构4};
\node[inner sep=0pt] (kind) at (-0.7,0.8){\small\bfnew{种群}};
\node[node,draw,fill=green!20] (n21) at (5,0){模型结构1};
\node[anchor=west,draw,node,fill=red!20] (n22) at ([xshift=0.6em]n21.east){模型结构4};
\node[anchor=north,node] (n23) at ([yshift=-1em]n21.south){};
\node[anchor=west,node] (n24) at ([xshift=0.6em]n23.east){};
\node[anchor=north,node] (n25) at ([yshift=-1em]n23.south){};
\node[anchor=west,node] (n26) at ([xshift=0.6em]n25.east){};
\node[inner sep=0pt] (choice) at (4.8,0.8){\small\bfnew{选中的亲本}};
\node[node,draw,fill=green!20,dotted] (n31) at (10,0){模型结构1};
\node[anchor=west,draw,node,fill=red!20,dotted] (n32) at ([xshift=0.6em]n31.east){模型结构4};
\node[anchor=north,node] (n33) at ([yshift=-1em]n31.south){};
\node[anchor=west,node] (n34) at ([xshift=0.6em]n33.east){};
\node[anchor=north,draw,node,fill=green!40] (n35) at ([yshift=-1em]n33.south){模型结构1};
\node[anchor=west,draw,node,fill=red!40] (n36) at ([xshift=0.6em]n35.east){模型结构1};
\node[inner sep=0pt] (change) at (9.7,0.8){\small\bfnew{亲本变异}};
\begin{pgfonlayer}{background}
\node[rounded corners=4pt,draw,thick,fill=yellow!10,inner sep=4pt,drop shadow][fit=(kind)(n16)](box1){};
\node[rounded corners=4pt,draw,thick,fill=yellow!10,inner sep=4pt,drop shadow][fit=(choice)(n26)](box2){};
\node[rounded corners=4pt,draw,thick,fill=yellow!10,inner sep=4pt,drop shadow][fit=(change)(n36)](box3){};
\end{pgfonlayer}
\draw[->,very thick] (box1.0) -- (box2.180);
\draw[->,very thick] (box2.0) -- (box3.180);
\draw[->,very thick] (n31.-90) -- (n35.90);
\draw[->,very thick] (n32.-90) -- (n36.90);
\draw[->,very thick] (box3.-90) .. controls ([yshift=-2em,xshift=-1em]box3.-90) and ([yshift=-2em,xshift=1em]box1.-90) .. node[font=\scriptsize,below]{对变异后的结构进行性能评估，选择优秀的结构加入原始种群} (box1.-90);
\end{tikzpicture}
Chapter15/Figures/figure-structure-search-based-on-gradient-method.tex 0 → 100644
%%% outline
%-------------------------------------------------------------------------
\begin{tikzpicture}
\tikzstyle{node}=[minimum height=6em,inner sep=4pt,align=left,draw,font=\footnotesize,rounded corners=4pt,thick,drop shadow]
\node[node,fill=red!20] (n1) at (0,0){\scriptsize\bfnew{超网络}： \\ [1ex] 模型结构参数 \\[0.4ex] 网络参数};
\node[anchor=west,node,fill=yellow!20] (n2) at ([xshift=4em]n1.east){\scriptsize\bfnew{优化后的超网络}： \\ [1ex]模型{\color{red}结构参数}（已优化） \\ [0.4ex]网络参数（已优化）};
\node[anchor=west,node,fill=blue!20] (n3) at ([xshift=6em]n2.east){\scriptsize\bfnew{找到的模型结构}};
\draw[-latex,thick] (n1.0) -- node[above,align=center,font=\scriptsize]{优化后的\\超网络}(n2.180);
\draw[-latex,thick] (n2.0) -- node[above,align=center,font=\scriptsize]{根据结构参数\\离散化结构}(n3.180);
\draw[-latex,out=90,in=100,thick] ([xshift=-2em]n1.90) to node[above,font=\scriptsize]{参数优化}([xshift=2em]n1.90);
\end{tikzpicture}
Chapter15/Figures/figure-structure-search-based-on-reinforcement-learning.tex 0 → 100644
%%% outline
%-------------------------------------------------------------------------
\begin{tikzpicture}
\tikzstyle{node}=[minimum height=2em,minimum width=5em,draw,rounded corners=2pt,thick,drop shadow]
\node[node,fill=red!20] (n1) at (0,0){\small\bfnew{环境}};
\node[anchor=south,node,fill=blue!20] (n2) at ([yshift=5em]n1.north){\small\bfnew{主体}};
\node[anchor=north,font=\footnotesize] at ([yshift=-0.2em]n1.south){（结构所应用于的任务）};
\node[anchor=south,font=\footnotesize] at ([yshift=0.2em]n2.north){（结构生成器）};
\draw[-latex,thick] ([yshift=.4em]n1.180) .. controls ([xshift=-3.4em,yshift=.4em]n1.180) and ([xshift=-3.4em,yshift=-.4em]n2.180) .. node[right,font=\scriptsize,align=left]{\scriptsize\bfnew{奖励} \\ （对输出结果的评价）}([yshift=-.4em]n2.180);
\draw[-latex,thick] ([yshift=-.4em]n1.180) .. controls ([xshift=-4.4em,yshift=-.4em]n1.180) and ([xshift=-4.4em,yshift=.4em]n2.180) .. node[left,font=\scriptsize,align=right]{\scriptsize\bfnew{状态} \\ （这个结构在任务中应用 \\ 后得到的输出结果）}([yshift=.4em]n2.180);
\draw[-latex,thick] (n2.0) .. controls ([xshift=4em]n2.0) and ([xshift=4em]n1.0) .. node[right,font=\scriptsize,align=left]{\scriptsize\bfnew{动作} \\ （生成一个结构）}(n1.0);
\end{tikzpicture}
Chapter15/Figures/figure-swish-function-image.tex 0 → 100644
%%% outline
%-------------------------------------------------------------------------
\begin{tikzpicture}
\begin{axis}[
width=10cm,height=6cm,
legend style={legend pos=north west,font=\footnotesize,align=center},
xtick={-5,-4,-3,-2,-1,0,1,2,3},
ytick={-2,-1,0,1,2,3},
x tick label style={font=\scriptsize},
y tick label style={font=\scriptsize},
xmin=-5,xmax=3,
ymin=-2,ymax=3,
y tick style={opacity=0},
x tick style={opacity=0},
grid=major,
title={\small\bfnew{Swish}},
]
\addplot[cyan,smooth,thick]{x*(1+exp(-10*x))^-1};
\addplot[red,smooth,thick]{x*(1+exp(-1*x))^-1};
\addplot[brown,smooth,thick]{x*(1+exp(-0.1*x))^-1};
\addplot[gray!60, very thick] coordinates {(0,-2) (0,3)};
\addplot[gray!60, very thick] coordinates {(0,0) (3,0)};
\legend{$\beta=0.1$,$\beta=1.0$,$ \ \ \beta=10.0$}
\end{axis}
\end{tikzpicture}
Chapter15/Figures/figure-whole-structure-and-internal-structure-in-rnn.tex 0 → 100644
%%% outline
%-------------------------------------------------------------------------
\begin{tikzpicture}[scale=0.8]
\tikzstyle{every node}=[scale=0.8]
\tikzstyle{node}=[draw,minimum height=1.4em,minimum width=2em,rounded corners=3pt,thick]
\node[node] (n1) at (0,0){};
\node[node] (n2) at (1.5,0){};
\node[node] (n3) at (3,0){};
\node[node] (n4) at (4.5,0){};
\node[node] (n5) at (1.5,-1.3){};
\node[node] (n6) at (3,-1.3){};
\node[node] (n7) at (2.25,-2.4){};
\node[node] (n8) at (3,1.3){};
\draw[->,thick] (n1.0) -- (n2.180);
\draw[->,thick] (n2.0) -- (n3.180);
\draw[->,thick] (n3.0) -- (n4.180);
\draw[->,thick,out=60,in=180] (n1.90) to (n8.180);
\draw[->,thick,out=-10,in=90] (n8.0) to (n4.90);
\draw[->,thick,out=90,in=-90] (n5.90) to (n3.-90);
\draw[->,thick,out=90,in=-90] (n6.90) to (n4.-90);
\draw[->,thick,out=90,in=-90] (n7.90) to (n5.-90);
\draw[->,thick,out=90,in=-90] (n7.90) to (n6.-90);
\node[font=\huge] (ht) at (-0.2,2.2){$\mathbi h_t$};
\node[draw,font=\huge,inner sep=0pt,minimum width=4em,minimum height=4em,very thick,rounded corners=2pt] (ht-1) at (-3,0) {$\mathbi h_{t-1}$};
\node[draw,font=\huge,inner sep=0pt,minimum width=4em,minimum height=4em,very thick,rounded corners=2pt] (ht+1) at (7.5,0) {$\mathbi h_{t+1}$};
\node[font=\huge] (xt) at (2.25,-4.2){$x_t$};
\node[font=\Huge] at (9,0){$\cdots$};
\node[font=\Huge] at (-4.5,0){$\cdots$};
\node[minimum width=3em,minimum height=2em,fill=red!20,rounded corners=6pt] (b1) at (6,-3.8){};
\node[] (w1) at (7.8,-3.8){\Large 整体框架};
\node[minimum width=3em,minimum height=2em,fill=yellow!30,rounded corners=6pt] (b2) at (6,-4.8){};
\node[] (w2) at (7.8,-4.8){\Large 内部结构};
\begin{pgfonlayer}{background}
\node[draw,rounded corners=6pt,very thick,fill=yellow!30,minimum width=16em,minimum height=15em] (box1) at (2.25,0){};
\node[draw=ublue,very thick,drop shadow,inner sep=1.2em,fill=white,xshift=-0.1em] [fit=(b1)(w2)]{};
\draw[fill=red!20,red!20,rounded corners=6pt] ([yshift=2.4em,xshift=1em]ht-1.east) -- ([yshift=2.4em,xshift=-0.1em]box1.west) -- ([xshift=-8em,xshift=-0.1em]box1.south) -- ([xshift=2em]box1.south) -- ([xshift=2em,yshift=-5em]box1.south) -- ([xshift=0em,yshift=-5em]box1.south) .. controls ([xshift=-12em,yshift=-5em]box1.south) and ([yshift=-2em]ht-1.east) ..([yshift=2.4em]ht-1.east) -- ([yshift=2.4em,xshift=1em]ht-1.east) ;
\end{pgfonlayer}
\draw[->,very thick] (ht-1.0) -- (box1.180);
\draw[->,very thick] (box1.0) -- (ht+1.180);
\draw[->,very thick] (ht-1.90) -- ([yshift=2em]ht-1.90);
\draw[->,very thick] (ht+1.90) -- ([yshift=2em]ht+1.90);
\draw[->,very thick] (box1.90) -- ([yshift=2em]box1.90);
\draw[->,very thick] ([yshift=-2em]ht-1.-90) -- (ht-1.-90);
\draw[->,very thick] ([yshift=-2em]ht+1.-90) -- (ht+1.-90);
\draw[->,very thick] ([yshift=-2em]box1.-90) -- (box1.-90);
\end{tikzpicture}
Chapter15/chapter15.tex
This source diff could not be displayed because it is too large. You can view the blob instead.
Chapter16/Figures/figure-application-process-of-back-translation.tex
...@@ -4,9 +4,9 @@ ...@@ -4,9 +4,9 @@
\tikzstyle{bignode} = [,inner sep=0.3em,draw=black,line width=0.6pt,rounded corners=2pt,minimum width=3.0em] \tikzstyle{bignode} = [,inner sep=0.3em,draw=black,line width=0.6pt,rounded corners=2pt,minimum width=3.0em]
\node [anchor=center] (node1-1) at (0,0) {{汉语}}; \node [anchor=center] (node1-1) at (0,0) {\small{汉语}};
\node [anchor=west] (node1-2) at ([xshift=0.8em]node1-1.east) {{英语}}; \node [anchor=west] (node1-2) at ([xshift=0.8em]node1-1.east) {\small{英语}};
\node [anchor=north] (node1-3) at ([xshift=1.75em]node1-1.south) {{反向翻译模型}}; \node [anchor=north] (node1-3) at ([xshift=1.75em]node1-1.south) {\small{反向翻译模型}};
\draw [->,thick](node1-1.east)--(node1-2.west); \draw [->,thick](node1-1.east)--(node1-2.west);
\begin{pgfonlayer}{background} \begin{pgfonlayer}{background}
...@@ -16,16 +16,16 @@ ...@@ -16,16 +16,16 @@
\end{pgfonlayer} \end{pgfonlayer}
\node [anchor=north,fill=green!20,bignode](node2-1) at ([yshift=-3em]node1-3.south){{汉语}}; \node [anchor=north,fill=green!20,bignode](node2-1) at ([yshift=-3em]node1-3.south){\small{汉语}};
\node [anchor=north,fill=green!20,bignode](node2-2) at (node2-1.south){{英语}}; \node [anchor=north,fill=green!20,bignode](node2-2) at (node2-1.south){\small{英语}};
\draw [->,thick](node2-1.north)--(remark1.south) node [pos=0.5,right] (pos1) {{训练}}; \draw [->,thick](node2-1.north)--(remark1.south) node [pos=0.5,right] (pos1) {\small{训练}};
\node [anchor=west,fill=yellow!20,bignode](node3-1) at ([xshift=6.5em,yshift=0.0em]node1-2.east){{汉语}}; \node [anchor=west,fill=yellow!20,bignode](node3-1) at ([xshift=6.5em,yshift=0.0em]node1-2.east){\small{汉语}};
\node [anchor=north,fill=red!20,bignode](node3-2) at ([yshift=-2.5em]node3-1.south){{英语}}; \node [anchor=north,fill=red!20,bignode](node3-2) at ([yshift=-2.5em]node3-1.south){\small{英语}};
\node [anchor=center](node3-3) at ([xshift=0.4em]node3-2.east){}; \node [anchor=center](node3-3) at ([xshift=0.4em]node3-2.east){};
\draw [->,thick](node3-1.south)--(node3-2.north) node [pos=0.5,right] (pos2) {{翻译}}; \draw [->,thick](node3-1.south)--(node3-2.north) node [pos=0.5,right] (pos2) {\small{翻译}};
\begin{pgfonlayer}{background} \begin{pgfonlayer}{background}
{ {
...@@ -33,21 +33,21 @@ ...@@ -33,21 +33,21 @@
} }
\end{pgfonlayer} \end{pgfonlayer}
\draw [->,thick](remark1.east)--([xshift=5.5em]remark1.east) node [pos=0.5,above] (pos2) {{模型翻译}}; \draw [->,thick](remark1.east)--([xshift=5.5em]remark1.east) node [pos=0.5,above] (pos2) {\small{模型翻译}};
\node [anchor=south](pos2-2) at ([yshift=-0.5em]pos2.north){{使用反向}}; \node [anchor=south](pos2-2) at ([yshift=-0.5em]pos2.north){\small{使用反向}};
\draw[decorate,thick,decoration={brace,amplitude=5pt}] ([yshift=1.5em,xshift=1.5em]node3-1.east) -- ([yshift=-8.6em,xshift=1.5em]node3-1.east) node [pos=0.1,right,xshift=0.0em,yshift=0.0em] (label1) {{{混合}}}; \draw[decorate,thick,decoration={brace,amplitude=5pt}] ([yshift=1.5em,xshift=1.5em]node3-1.east) -- ([yshift=-8.6em,xshift=1.5em]node3-1.east) node [pos=0.1,right,xshift=0.0em,yshift=0.0em] (label1) {\small{{混合}}};
\node [anchor=west,fill=red!20,bignode](node4-1) at ([xshift=2.5em,yshift=1.3em]node3-2.east){{英语}}; \node [anchor=west,fill=red!20,bignode](node4-1) at ([xshift=2.5em,yshift=1.3em]node3-2.east){\small{英语}};
\node [anchor=north,fill=yellow!20,bignode](node4-2) at (node4-1.south){{汉语}}; \node [anchor=north,fill=yellow!20,bignode](node4-2) at (node4-1.south){\small{汉语}};
\node [anchor=west,fill=green!20,bignode](node4-3) at (node4-1.east){{英语}}; \node [anchor=west,fill=green!20,bignode](node4-3) at (node4-1.east){\small{英语}};
\node [anchor=north,fill=green!20,bignode](node4-4) at (node4-3.south){{汉语}}; \node [anchor=north,fill=green!20,bignode](node4-4) at (node4-3.south){\small{汉语}};
\node [anchor=center] (node5-1) at ([xshift=5em,yshift=0.02em]node4-3.east) {{英语}}; \node [anchor=center] (node5-1) at ([xshift=5em,yshift=0.02em]node4-3.east) {\small{英语}};
\node [anchor=west] (node5-2) at ([xshift=0.8em]node5-1.east) {{汉语}}; \node [anchor=west] (node5-2) at ([xshift=0.8em]node5-1.east) {\small{汉语}};
\node [anchor=north] (node5-3) at ([xshift=1.65em]node5-1.south) {{正向翻译模型}}; \node [anchor=north] (node5-3) at ([xshift=1.65em]node5-1.south) {\small{正向翻译模型}};
\draw [->,thick](node5-1.east)--(node5-2.west); \draw [->,thick](node5-1.east)--(node5-2.west);
\begin{pgfonlayer}{background} \begin{pgfonlayer}{background}
...@@ -56,11 +56,11 @@ ...@@ -56,11 +56,11 @@
} }
\end{pgfonlayer} \end{pgfonlayer}
\draw [->,thick]([xshift=-3.2em]remark3.west)--(remark3.west) node [pos=0.5,above] (pos3) {{训练}}; \draw [->,thick]([xshift=-3.2em]remark3.west)--(remark3.west) node [pos=0.5,above] (pos3) {\small{训练}};
\node [anchor=south](d1) at ([xshift=-1.5em,yshift=1em]remark1.north){{真实数据：}}; \node [anchor=south](d1) at ([xshift=-1.5em,yshift=1em]remark1.north){\small{真实数据：}};
\node [anchor=west](d2) at ([xshift=2.0em]d1.east){{伪数据：}}; \node [anchor=west](d2) at ([xshift=2.0em]d1.east){\small{伪数据：}};
\node [anchor=west](d3) at ([xshift=2.0em]d2.east){{额外数据：}}; \node [anchor=west](d3) at ([xshift=2.0em]d2.east){\small{额外数据：}};
\node [anchor=west,fill=green!20,minimum width=1.5em](d1-1) at ([xshift=-0.0em]d1.east){}; \node [anchor=west,fill=green!20,minimum width=1.5em](d1-1) at ([xshift=-0.0em]d1.east){};
\node [anchor=west,fill=red!20,minimum width=1.5em](d2-1) at ([xshift=-0.0em]d2.east){}; \node [anchor=west,fill=red!20,minimum width=1.5em](d2-1) at ([xshift=-0.0em]d2.east){};
\node [anchor=west,fill=yellow!20,minimum width=1.5em](d3-1) at ([xshift=-0.0em]d3.east){}; \node [anchor=west,fill=yellow!20,minimum width=1.5em](d3-1) at ([xshift=-0.0em]d3.east){};
......
Chapter16/Figures/figure-comparison-of-structure-between-gpt-and-bert-model.tex 0 → 100644
\begin{tikzpicture}
\tikzstyle{embedding} = [line width=0.6pt,draw=black,minimum width=2.5em,minimum height=1.6em,fill=green!20]
\tikzstyle{model} = [line width=0.6pt,draw=black,minimum width=3.0em,minimum height=1.6em,fill=blue!20,rounded corners=2pt]
\node [anchor=center,model] (node1-1) at (0,0) {\footnotesize{TRM}};
\node [anchor=west,model] (node1-2) at ([xshift=1.8em]node1-1.east) {\footnotesize{TRM}};
\node [anchor=west,scale=1.8] (node1-3) at ([xshift=1.0em]node1-2.east) {...};
\node [anchor=west,model] (node1-4) at ([xshift=1.0em]node1-3.east) {\footnotesize{TRM}};
\node [anchor=west,model] (node1-5) at ([xshift=2.0em]node1-4.east) {\footnotesize{TRM}};
\node [anchor=west,model] (node1-6) at ([xshift=1.8em]node1-5.east) {\footnotesize{TRM}};
\node [anchor=west,scale=1.8] (node1-7) at ([xshift=1.0em]node1-6.east) {...};
\node [anchor=west,model] (node1-8) at ([xshift=1.0em]node1-7.east) {\footnotesize{TRM}};
\node [anchor=north,embedding] (node0-1) at ([yshift=-2em]node1-1.south){\footnotesize{$\mathbi{e}_1$}};
\node [anchor=north,embedding] (node0-2) at ([yshift=-2em]node1-2.south){\footnotesize{$\mathbi{e}_2$}};
\node [anchor=west,scale=1.8] (node0-3) at ([xshift=1.25em]node0-2.east){...};
\node [anchor=north,embedding] (node0-4) at ([yshift=-2em]node1-4.south){\footnotesize{$\mathbi{e}_n$}};
\node [anchor=south,model](node2-1) at ([yshift=1.8em]node1-1.north){\footnotesize{TRM}};
\node [anchor=south,model](node2-2) at ([yshift=1.8em]node1-2.north){\footnotesize{TRM}};
\node [anchor=west,scale=1.8](node2-3) at ([xshift=1.0em]node2-2.east){...};
\node [anchor=south,model](node2-4) at ([yshift=1.8em]node1-4.north){\footnotesize{TRM}};
\node [anchor=south,model](node2-5) at ([yshift=1.8em]node1-5.north){\footnotesize{TRM}};
\node [anchor=south,model](node2-6) at ([yshift=1.8em]node1-6.north){\footnotesize{TRM}};
\node [anchor=west,scale=1.8](node2-7) at ([xshift=1.0em]node2-6.east){...};
\node [anchor=south,model](node2-8) at ([yshift=1.8em]node1-8.north){\footnotesize{TRM}};
\draw [->,thick](node1-1.north)--(node2-1.south);
\draw [->,thick](node1-2.north)--(node2-2.south);
\draw [->,thick](node1-4.north)--(node2-4.south);
\begin{pgfonlayer}{background}
{
\node[fill=white,inner sep=0.5em,draw=black,line width=0.6pt,minimum width=6.0em,rounded corners=2pt,dashed] [fit =(node1-1)(node1-2)(node1-3)(node1-4)(node2-1)] (remark1) {};
}
\end{pgfonlayer}
\begin{pgfonlayer}{background}
{
\node[fill=white,inner sep=0.5em,draw=black,line width=0.6pt,minimum width=6.0em,rounded corners=2pt,dashed] [fit =(node1-5)(node1-6)(node1-7)(node1-8)(node2-8)] (remark2) {};
}
\end{pgfonlayer}
\draw [->,thick](node0-1.north)--(node1-1.south);
\draw [->,thick](node0-1.north)--(node1-2.south);
\draw [->,thick](node0-1.north)--(node1-4.south);
\draw [->,thick](node0-2.north)--(node1-2.south);
\draw [->,thick](node0-2.north)--(node1-4.south);
\draw [->,thick](node0-4.north)--(node1-4.south);
\draw [->,thick](node1-1.north)--(node2-1.south);
\draw [->,thick](node1-1.north)--(node2-2.south);
\draw [->,thick](node1-1.north)--(node2-4.south);
\draw [->,thick](node1-2.north)--(node2-2.south);
\draw [->,thick](node1-2.north)--(node2-4.south);
\draw [->,thick](node1-4.north)--(node2-4.south);
\node [anchor=south,embedding,fill=yellow!20](node3-1) at ([yshift=2em]node2-1.north){\footnotesize{$\seq{P}_1$}};
\node [anchor=south,embedding,fill=yellow!20] (node3-2) at ([yshift=2em]node2-2.north){\footnotesize{$\seq{P}_2$}};
\node [anchor=west,scale=1.8] (node3-3) at ([xshift=1.25em]node3-2.east){...};
\node [anchor=south,embedding,fill=yellow!20](node3-4) at ([yshift=2em]node2-4.north){\footnotesize{$\seq{P}_n$}};
\draw [<-,thick](node3-1.south)--(node2-1.north);
\draw [<-,thick](node3-2.south)--(node2-2.north);
\draw [<-,thick](node3-4.south)--(node2-4.north);
%%%%%%%%%bert
\node [anchor=north,embedding] (node0-5) at ([yshift=-2em]node1-5.south){\footnotesize{$\mathbi{e}_1$}};
\node [anchor=north,embedding] (node0-6) at ([yshift=-2em]node1-6.south){\footnotesize{$\mathbi{e}_2$}};
\node [anchor=west,scale=1.8] (node0-7) at ([xshift=1.25em]node0-6.east){...};
\node [anchor=north,embedding] (node0-8) at ([yshift=-2em]node1-8.south){\footnotesize{$\mathbi{e}_n$}};
\node [anchor=south,embedding,fill=yellow!20](node3-5) at ([yshift=2em]node2-5.north){\footnotesize{$\seq{P}_1$}};
\node [anchor=south,embedding,fill=yellow!20] (node3-6) at ([yshift=2em]node2-6.north){\footnotesize{$\seq{P}_2$}};
\node [anchor=west,scale=1.8] (node3-7) at ([xshift=1.25em]node3-6.east){...};
\node [anchor=south,embedding,fill=yellow!20](node3-8) at ([yshift=2em]node2-8.north){\footnotesize{$\seq{P}_n$}};
\draw [->,thick](node0-5.north)--(node1-5.south);
\draw [->,thick](node0-5.north)--(node1-6.south);
\draw [->,thick](node0-5.north)--(node1-8.south);
\draw [->,thick](node0-6.north)--(node1-5.south);
\draw [->,thick](node0-6.north)--(node1-6.south);
\draw [->,thick](node0-6.north)--(node1-8.south);
\draw [->,thick](node0-8.north)--(node1-5.south);
\draw [->,thick](node0-8.north)--(node1-6.south);
\draw [->,thick](node0-8.north)--(node1-8.south);
\draw [->,thick](node1-5.north)--(node2-5.south);
\draw [->,thick](node1-5.north)--(node2-6.south);
\draw [->,thick](node1-5.north)--(node2-8.south);
\draw [->,thick](node1-6.north)--(node2-5.south);
\draw [->,thick](node1-6.north)--(node2-6.south);
\draw [->,thick](node1-6.north)--(node2-8.south);
\draw [->,thick](node1-8.north)--(node2-5.south);
\draw [->,thick](node1-8.north)--(node2-6.south);
\draw [->,thick](node1-8.north)--(node2-8.south);
\draw [<-,thick](node3-5.south)--(node2-5.north);
\draw [<-,thick](node3-6.south)--(node2-6.north);
\draw [<-,thick](node3-8.south)--(node2-8.north);
\node [anchor=north] (pos1) at ([xshift=1.5em,yshift=-1.0em]node0-2.south) {\small{(a) GPT模型结构}};
\node [anchor=north] (pos2) at ([xshift=1.5em,yshift=-1.0em]node0-6.south) {\small{(b) BERT模型结构}};
\node [anchor=south] (ex) at ([xshift=2.1em,yshift=0.5em]node3-1.north) {\small{TRM：transformer}};
\end{tikzpicture}
\ No newline at end of file
Chapter16/Figures/figure-contrast-of-traditional-machine-learning&transfer-learning.tex
...@@ -37,6 +37,10 @@ ...@@ -37,6 +37,10 @@
\draw[->,thick] ([yshift=-0.2em]n5.-90) -- ([yshift=0.2em]sys.90); \draw[->,thick] ([yshift=-0.2em]n5.-90) -- ([yshift=0.2em]sys.90);
\draw[->,thick] ([yshift=0.3em,xshift=0.2em]kd.0) -- ([yshift=-0.2em,xshift=-0.2em]sys.180); \draw[->,thick] ([yshift=0.3em,xshift=0.2em]kd.0) -- ([yshift=-0.2em,xshift=-0.2em]sys.180);
\node [anchor=north] (re1) at ([yshift=-1em]nmt2.south) {\small{(a) 传统机器学习}};
\node [anchor=west] (re2) at ([xshift=11.0em]re1.east) {\small{(b) 迁移学习}};
\end{tikzpicture} \end{tikzpicture}
......
Chapter16/Figures/figure-elmo-model-structure.tex 0 → 100644
\begin{tikzpicture}
\tikzstyle{embedding} = [line width=0.6pt,draw=black,minimum width=2.5em,minimum height=1.6em,fill=green!20]
\tikzstyle{model} = [line width=0.6pt,draw=black,minimum width=3.0em,minimum height=1.6em,fill=blue!20,rounded corners=2pt]
\node [anchor=center,model] (node1-1) at (0,0) {\footnotesize{LSTM}};
\node [anchor=west,model] (node1-2) at ([xshift=1.8em]node1-1.east) {\footnotesize{LSTM}};
\node [anchor=west,scale=1.8] (node1-3) at ([xshift=1.0em]node1-2.east) {...};
\node [anchor=west,model] (node1-4) at ([xshift=1.0em]node1-3.east) {\footnotesize{LSTM}};
\node [anchor=west,model] (node1-5) at ([xshift=2.0em]node1-4.east) {\footnotesize{LSTM}};
\node [anchor=west,model] (node1-6) at ([xshift=1.8em]node1-5.east) {\footnotesize{LSTM}};
\node [anchor=west,scale=1.8] (node1-7) at ([xshift=1.0em]node1-6.east) {...};
\node [anchor=west,model] (node1-8) at ([xshift=1.0em]node1-7.east) {\footnotesize{LSTM}};
\node [anchor=south,model](node2-1) at ([yshift=1.8em]node1-1.north){\footnotesize{LSTM}};
\node [anchor=south,model](node2-2) at ([yshift=1.8em]node1-2.north){\footnotesize{LSTM}};
\node [anchor=west,scale=1.8](node2-3) at ([xshift=1.0em]node2-2.east){...};
\node [anchor=south,model](node2-4) at ([yshift=1.8em]node1-4.north){\footnotesize{LSTM}};
\node [anchor=south,model](node2-5) at ([yshift=1.8em]node1-5.north){\footnotesize{LSTM}};
\node [anchor=south,model](node2-6) at ([yshift=1.8em]node1-6.north){\footnotesize{LSTM}};
\node [anchor=west,scale=1.8](node2-7) at ([xshift=1.0em]node2-6.east){...};
\node [anchor=south,model](node2-8) at ([yshift=1.8em]node1-8.north){\footnotesize{LSTM}};
\draw [->,thick](node1-1.east)--(node1-2.west);
\draw [->,thick](node1-2.east)--([xshift=0.5em]node1-3.west);
\draw [->,thick]([xshift=-0.5em]node1-3.east)--(node1-4.west);
\draw [<-,thick](node1-5.east)--(node1-6.west);
\draw [<-,thick](node1-6.east)--([xshift=0.5em]node1-7.west);
\draw [<-,thick]([xshift=-0.5em]node1-7.east)--(node1-8.west);
\draw [->,thick](node1-1.north)--(node2-1.south);
\draw [->,thick](node1-2.north)--(node2-2.south);
\draw [->,thick](node1-4.north)--(node2-4.south);
\draw [->,thick](node1-5.north)--(node2-5.south);
\draw [->,thick](node1-6.north)--(node2-6.south);
\draw [->,thick](node1-8.north)--(node2-8.south);
\draw [->,thick](node2-1.east)--(node2-2.west);
\draw [->,thick](node2-2.east)--([xshift=0.5em]node2-3.west);
\draw [->,thick]([xshift=-0.5em]node2-3.east)--(node2-4.west);
\draw [<-,thick](node2-5.east)--(node2-6.west);
\draw [<-,thick](node2-6.east)--([xshift=0.5em]node2-7.west);
\draw [<-,thick]([xshift=-0.5em]node2-7.east)--(node2-8.west);
\begin{pgfonlayer}{background}
{
\node[fill=white,inner sep=0.5em,draw=black,line width=0.6pt,minimum width=6.0em,rounded corners=2pt,dashed] [fit =(node1-1)(node1-2)(node1-3)(node1-4)(node2-1)] (remark1) {};
}
\end{pgfonlayer}
\begin{pgfonlayer}{background}
{
\node[fill=white,inner sep=0.5em,draw=black,line width=0.6pt,minimum width=6.0em,rounded corners=2pt,dashed] [fit =(node1-5)(node1-6)(node1-7)(node1-8)(node2-8)] (remark1) {};
}
\end{pgfonlayer}
\node [anchor=north,embedding] (node0-2) at ([yshift=-2em]node1-4.south){\footnotesize{$\mathbi{e}_2$}};
\node [anchor=east,embedding] (node0-1) at ([xshift=-1.4em]node0-2.west){\footnotesize{$\mathbi{e}_1$}};
\node [anchor=north,scale=1.8] (node0-3) at ([yshift=-2em]node1-5.south){...};
\node [anchor=north,embedding] (node0-4) at ([yshift=-2em]node1-6.south){\footnotesize{$\mathbi{e}_n$}};
\draw [->,thick](node0-1.north)--(node1-1.south);
\draw [->,thick](node0-1.north)--(node1-5.south);
\draw [->,thick](node0-2.north)--(node1-2.south);
\draw [->,thick](node0-2.north)--(node1-6.south);
\draw [->,thick](node0-4.north)--(node1-4.south);
\draw [->,thick](node0-4.north)--(node1-8.south);
\node [anchor=south,embedding,fill=yellow!20](node3-2) at ([yshift=2em]node2-4.north){\footnotesize{$\seq{P}_2$}};
\node [anchor=east,embedding,fill=yellow!20] (node3-1) at ([xshift=-1.4em]node3-2.west){\footnotesize{$\seq{P}_1$}};
\node [anchor=south,scale=1.8] (node3-3) at ([yshift=2em]node2-5.north){...};
\node [anchor=south,embedding,fill=yellow!20](node3-4) at ([yshift=2em]node2-6.north){\footnotesize{$\seq{P}_n$}};
\draw [<-,thick](node3-1.south)--(node2-1.north);
\draw [<-,thick](node3-1.south)--(node2-5.north);
\draw [<-,thick](node3-2.south)--(node2-2.north);
\draw [<-,thick](node3-2.south)--(node2-6.north);
\draw [<-,thick](node3-4.south)--(node2-4.north);
\draw [<-,thick](node3-4.south)--(node2-8.north);
\end{tikzpicture}
Chapter16/Figures/figure-example-of-iterative-back-translation.tex
\begin{tikzpicture} \begin{tikzpicture}
\tikzstyle{rec} = [inner sep=0.3em,minimum width=4em,draw=black,line width=0.6pt,rounded corners=2pt] \tikzstyle{rec} = [inner sep=0.3em,minimum width=4em,draw=black,line width=0.6pt,rounded corners=2pt]
\node [anchor=north,fill=green!20,rec](node1-1) at (0,0){{汉语}}; \node [anchor=north,fill=green!20,rec](node1-1) at (0,0){\small{汉语}};
\node [anchor=north,fill=green!20,rec](node1-2) at (node1-1.south){{英语}}; \node [anchor=north,fill=green!20,rec](node1-2) at (node1-1.south){\small{英语}};
\node [anchor=north,fill=yellow!20,rec](node2-1) at ([yshift=-5.0em]node1-1.south){{汉语}}; \node [anchor=north,fill=yellow!20,rec](node2-1) at ([yshift=-5.0em]node1-1.south){\small{汉语}};
\node [anchor=north,fill=red!20,rec](node2-2) at (node2-1.south){{英语}}; \node [anchor=north,fill=red!20,rec](node2-2) at (node2-1.south){\small{英语}};
\node [anchor=east] (node3-1) at ([xshift=-4.0em,yshift=-3.5em]node1-1.west) {{正向}}; \node [anchor=east] (node3-1) at ([xshift=-4.0em,yshift=-3.5em]node1-1.west) {\small{正向}};
\node [anchor=north] (node3-2) at ([yshift=0.5em]node3-1.south) {{翻译模型}}; \node [anchor=north] (node3-2) at ([yshift=0.5em]node3-1.south) {\small{翻译模型}};
\begin{pgfonlayer}{background} \begin{pgfonlayer}{background}
{ {
\node[fill=blue!20,inner sep=0.3em,draw=black,line width=0.6pt,minimum width=3.0em,drop shadow,rounded corners=2pt] [fit =(node3-1)(node3-2)] (remark1) {}; \node[fill=blue!20,inner sep=0.3em,draw=black,line width=0.6pt,minimum width=3.0em,drop shadow,rounded corners=2pt] [fit =(node3-1)(node3-2)] (remark1) {};
...@@ -14,8 +14,8 @@ ...@@ -14,8 +14,8 @@
\draw [->,thick]([yshift=-0.75em]node1-1.west)--(remark1.north east); \draw [->,thick]([yshift=-0.75em]node1-1.west)--(remark1.north east);
\draw [->,thick,dashed](remark1.south east)--([yshift=-0.75em]node2-1.west); \draw [->,thick,dashed](remark1.south east)--([yshift=-0.75em]node2-1.west);
\node [anchor=west] (node4-1) at ([xshift=4.0em,yshift=-3.5em]node1-1.east) {{反向}}; \node [anchor=west] (node4-1) at ([xshift=4.0em,yshift=-3.5em]node1-1.east) {\small{反向}};
\node [anchor=north] (node4-2) at ([yshift=0.5em]node4-1.south) {{翻译模型}}; \node [anchor=north] (node4-2) at ([yshift=0.5em]node4-1.south) {\small{翻译模型}};
\begin{pgfonlayer}{background} \begin{pgfonlayer}{background}
{ {
\node[fill=blue!20,inner sep=0.3em,draw=black,line width=0.6pt,minimum width=3.0em,drop shadow,rounded corners=2pt] [fit =(node4-1)(node4-2)] (remark2) {}; \node[fill=blue!20,inner sep=0.3em,draw=black,line width=0.6pt,minimum width=3.0em,drop shadow,rounded corners=2pt] [fit =(node4-1)(node4-2)] (remark2) {};
...@@ -24,15 +24,15 @@ ...@@ -24,15 +24,15 @@
\draw [->,thick]([yshift=-0.75em]node1-1.east)--(remark2.north west); \draw [->,thick]([yshift=-0.75em]node1-1.east)--(remark2.north west);
\draw [->,thick]([yshift=-0.75em]node2-1.east)--(remark2.south west); \draw [->,thick]([yshift=-0.75em]node2-1.east)--(remark2.south west);
\node [anchor=west,fill=green!20,rec](node5-1) at ([xshift=4.0em,yshift=3.48em]node4-1.east){{英语}}; \node [anchor=west,fill=green!20,rec](node5-1) at ([xshift=4.0em,yshift=3.48em]node4-1.east){\small{英语}};
\node [anchor=north,fill=green!20,rec](node5-2) at (node5-1.south){{汉语 }}; \node [anchor=north,fill=green!20,rec](node5-2) at (node5-1.south){\small{汉语}};
\node [anchor=north,fill=yellow!20,rec](node6-1) at ([yshift=-5.0em]node5-1.south){{英语}}; \node [anchor=north,fill=yellow!20,rec](node6-1) at ([yshift=-5.0em]node5-1.south){\small{英语}};
\node [anchor=north,fill=red!20,rec](node6-2) at (node6-1.south){{汉语}}; \node [anchor=north,fill=red!20,rec](node6-2) at (node6-1.south){\small{汉语}};
\draw [->,thick,dashed](remark2.south east)--([yshift=-0.75em]node6-1.west); \draw [->,thick,dashed](remark2.south east)--([yshift=-0.75em]node6-1.west);
\node [anchor=west] (node7-1) at ([xshift=4.0em,yshift=-3.5em]node5-1.east) {{正向}}; \node [anchor=west] (node7-1) at ([xshift=4.0em,yshift=-3.5em]node5-1.east) {\small{正向}};
\node [anchor=north] (node7-2) at ([yshift=0.5em]node7-1.south) {{翻译模型}}; \node [anchor=north] (node7-2) at ([yshift=0.5em]node7-1.south) {\small{翻译模型}};
\begin{pgfonlayer}{background} \begin{pgfonlayer}{background}
{ {
\node[fill=blue!20,inner sep=0.3em,draw=black,line width=0.6pt,minimum width=3.0em,drop shadow,rounded corners=2pt] [fit =(node7-1)(node7-2)] (remark3) {}; \node[fill=blue!20,inner sep=0.3em,draw=black,line width=0.6pt,minimum width=3.0em,drop shadow,rounded corners=2pt] [fit =(node7-1)(node7-2)] (remark3) {};
...@@ -42,14 +42,14 @@ ...@@ -42,14 +42,14 @@
\draw [->,thick]([yshift=-0.75em]node5-1.east)--(remark3.north west); \draw [->,thick]([yshift=-0.75em]node5-1.east)--(remark3.north west);
\draw [->,thick]([yshift=-0.75em]node6-1.east)--(remark3.south west); \draw [->,thick]([yshift=-0.75em]node6-1.east)--(remark3.south west);
\node [anchor=south](d1) at ([xshift=-0.7em,yshift=4em]remark1.north){{真实数据：}}; \node [anchor=south](d1) at ([xshift=-0.7em,yshift=5.5em]remark1.north){\small{真实数据：}};
\node [anchor=west](d2) at ([xshift=2.0em]d1.east){{伪数据：}}; \node [anchor=west](d2) at ([xshift=2.0em]d1.east){\small{伪数据：}};
\node [anchor=west](d3) at ([xshift=2.0em]d2.east){{额外数据：}}; \node [anchor=west](d3) at ([xshift=2.0em]d2.east){\small{额外数据：}};
\node [anchor=west,fill=green!20,minimum width=1.5em](d1-1) at ([xshift=-0.0em]d1.east){}; \node [anchor=west,fill=green!20,minimum width=1.5em](d1-1) at ([xshift=-0.0em]d1.east){};
\node [anchor=west,fill=red!20,minimum width=1.5em](d2-1) at ([xshift=-0.0em]d2.east){}; \node [anchor=west,fill=red!20,minimum width=1.5em](d2-1) at ([xshift=-0.0em]d2.east){};
\node [anchor=west,fill=yellow!20,minimum width=1.5em](d3-1) at ([xshift=-0.0em]d3.east){}; \node [anchor=west,fill=yellow!20,minimum width=1.5em](d3-1) at ([xshift=-0.0em]d3.east){};
\node [anchor=south] (d4) at ([xshift=1em]d1.north) {{训练：}}; \node [anchor=north] (d4) at ([xshift=1em]d1.south) {\small{训练：}};
\node [anchor=south] (d5) at ([xshift=0.5em]d2.north) {{推理：}}; \node [anchor=north] (d5) at ([xshift=0.5em]d2.south) {\small{推理：}};
\draw [->,thick] ([xshift=0em]d4.east)--([xshift=1.5em]d4.east); \draw [->,thick] ([xshift=0em]d4.east)--([xshift=1.5em]d4.east);
\draw [->,thick,dashed] ([xshift=0em]d5.east)--([xshift=1.5em]d5.east); \draw [->,thick,dashed] ([xshift=0em]d5.east)--([xshift=1.5em]d5.east);
......
Chapter16/Figures/figure-mass.tex
\begin{tikzpicture} \begin{tikzpicture}
\begin{scope} \tikzstyle{word} = [font=\scriptsize,minimum height=1.4em]
\tikzstyle{word} = [font=\scriptsize] \tikzstyle{model} = [rectangle,line width=0.7pt,draw,minimum height=3em,minimum width=13em,rounded corners=4pt,fill=red!20]
\tikzstyle{model} = [rectangle,draw,minimum height=3em,minimum width=6em,rounded corners=4pt,fill=red!15!white]
\node [anchor=center] (ate) at (0,0) {}; \node [anchor=center] (ate) at (0,0) {};
%decoder
\node [anchor=center,model,fill=blue!20] (decoder) at ([xshift=7.5em]ate.east) {解码器};
\node [anchor=north,word] (w1) at ([yshift=-1.5em,xshift=0em]decoder.south) {\small{$x_3$}};
\node [anchor=west,word] (w2) at ([xshift=0em]w1.east) {\small{$x_4$}};
\node [anchor=west,word] (w3) at ([xshift=0em]w2.east) {[M]};
\node [model,minimum width=10.5em,line width=0.7pt] (decoder) at ([xshift=6em]ate.east) {Decoder}; \node [anchor=east,word] (w4) at ([xshift=0em]w1.west) {[M]};
\node [word] (w1) at ([yshift=-2em,xshift=1em]decoder.south) {\small{$x_3$}}; \node [anchor=east,word] (w5) at ([xshift=0em]w4.west) {[M]};
\node [word] (w2) at ([xshift=-1em]w1.west) {\#}; \node [anchor=east,word] (w6) at ([xshift=0em]w5.west) {[M]};
\node [word] (w3) at ([xshift=-1em]w2.west) {\#}; \node [anchor=west,word] (w7) at ([xshift=0em]w3.east) {[M]};
\node [word] (w4) at ([xshift=-1em]w3.west) {\#};
\node [word] (w5) at ([xshift=1em]w1.east) {\small{$x_4$}};
\node [word] (w6) at ([xshift=1em]w5.east) {\#};
\node [word] (w7) at ([yshift=2em,xshift=1em]decoder.north) {\small{$x_4$}}; \node [anchor=south,word] (w8) at ([yshift=1.5em,xshift=0em]decoder.north) {\small{$x_4$}};
\node [word] (w8) at ([yshift=0em,xshift=-1em]w7.west) {\small{$x_3$}}; \node [anchor=east,word] (w9) at (w8.west) {\small{$x_3$}};
\node [word] (w9) at ([yshift=0em,xshift=1em]w7.east) {\small{$x_5$}}; \node [anchor=west,word] (w10) at (w8.east) {\small{$x_5$}};
\draw [->,thick] (w1.north) -- ([yshift=1.35em]w1.north); \draw [->,thick] (w1.north) -- ([yshift=1.35em]w1.north);
\draw [->,thick] (w2.north) -- ([yshift=1.35em]w2.north); \draw [->,thick] (w2.north) -- ([yshift=1.35em]w2.north);
...@@ -23,19 +24,23 @@ ...@@ -23,19 +24,23 @@
\draw [->,thick] (w4.north) -- ([yshift=1.35em]w4.north); \draw [->,thick] (w4.north) -- ([yshift=1.35em]w4.north);
\draw [->,thick] (w5.north) -- ([yshift=1.35em]w5.north); \draw [->,thick] (w5.north) -- ([yshift=1.35em]w5.north);
\draw [->,thick] (w6.north) -- ([yshift=1.35em]w6.north); \draw [->,thick] (w6.north) -- ([yshift=1.35em]w6.north);
\draw [->,thick] (w7.north) -- ([yshift=1.35em]w7.north);
\draw [->,thick] ([yshift=-1.4em]w7.south) -- (w7.south);
\draw [->,thick] ([yshift=-1.4em]w8.south) -- (w8.south); \draw [->,thick] ([yshift=-1.4em]w8.south) -- (w8.south);
\draw [->,thick] ([yshift=-1.4em]w9.south) -- (w9.south); \draw [->,thick] ([yshift=-1.4em]w9.south) -- (w9.south);
\draw [->,thick] ([yshift=-1.4em]w10.south) -- (w10.south);
%encoder %encoder
\node [model,minimum width=10.5em,line width=0.7pt] (encoder) at ([xshift=-6em]ate.west) {Encoder}; \node [model] (encoder) at ([xshift=-7.5em]ate.west) {编码器};
\node [word] (we1) at ([yshift=-2em,xshift=1em]encoder.south) {\#};
\node [word] (we2) at ([xshift=-1em]we1.west) {\#}; \node [anchor=north,word] (we1) at ([yshift=-1.5em,xshift=0em]encoder.south) {[M]};
\node [word] (we3) at ([xshift=-1em]we2.west) {\small{$x_2$}}; \node [anchor=west,word] (we2) at ([xshift=0em]we1.east) {[M]};
\node [word] (we4) at ([xshift=-1em]we3.west) {\small{$x_3$}}; \node [anchor=west,word] (we3) at ([xshift=0em]we2.east) {\small{$x_6$}};
\node [word] (we5) at ([xshift=1em]we1.east) {\#};
\node [word] (we6) at ([xshift=1em]we5.east) {\small{$x_6$}}; \node [anchor=east,word] (we4) at ([xshift=0em]we1.west) {[M]};
\node [anchor=east,word] (we5) at ([xshift=0em]we4.west) {\small{$x_2$}};
\node [anchor=east,word] (we6) at ([xshift=0em]we5.west) {\small{$x_1$}};
\node [anchor=west,word] (we7) at ([xshift=0em]we3.east) {\small{$x_7$}};
\draw [->,thick] (we1.north) -- ([yshift=1.35em]we1.north); \draw [->,thick] (we1.north) -- ([yshift=1.35em]we1.north);
\draw [->,thick] (we2.north) -- ([yshift=1.35em]we2.north); \draw [->,thick] (we2.north) -- ([yshift=1.35em]we2.north);
...@@ -43,7 +48,8 @@ ...@@ -43,7 +48,8 @@
\draw [->,thick] (we4.north) -- ([yshift=1.35em]we4.north); \draw [->,thick] (we4.north) -- ([yshift=1.35em]we4.north);
\draw [->,thick] (we5.north) -- ([yshift=1.35em]we5.north); \draw [->,thick] (we5.north) -- ([yshift=1.35em]we5.north);
\draw [->,thick] (we6.north) -- ([yshift=1.35em]we6.north); \draw [->,thick] (we6.north) -- ([yshift=1.35em]we6.north);
\draw [->,thick] (we7.north) -- ([yshift=1.35em]we7.north);
\draw [->,line width=2pt] ([xshift=0.5em]encoder)--([xshift=-0.5em]decoder); \draw [->,very thick] ([xshift=0.5em]encoder)--([xshift=-0.3em]decoder);
\end{scope} \node [anchor=south] (ex) at ([xshift=-4.0em,yshift=1.0em]encoder.north) {\small{[M]：Mask}};
\end{tikzpicture} \end{tikzpicture}
\ No newline at end of file
Chapter16/Figures/figure-optimization-of-the-model-initialization-method.tex
...@@ -6,15 +6,15 @@ ...@@ -6,15 +6,15 @@
\tikzstyle{circle} = [draw,black,line width=0.6pt,inner sep=3.5pt,rounded corners=4pt,minimum width=2em] \tikzstyle{circle} = [draw,black,line width=0.6pt,inner sep=3.5pt,rounded corners=4pt,minimum width=2em]
\tikzstyle{word} = [inner sep=3.5pt] \tikzstyle{word} = [inner sep=3.5pt]
\node[circle,fill=red!20](data) at (0,0) {数据}; \node[circle,fill=red!20](data) at (0,0) {\small{数据}};
\node[circle,fill=blue!20](model) at ([xshift=5em]data.east) {模型}; \node[circle,fill=blue!20](model) at ([xshift=5em]data.east) {\small{模型}};
\node[word] (init) at ([xshift=-5em]data.west){初始化}; \node[word] (init) at ([xshift=-5em]data.west){\small{初始化}};
\draw[->,thick] (init.east) -- ([xshift=-0.2em]data.west); \draw[->,thick] (init.east) -- ([xshift=-0.2em]data.west);
\draw [->,thick] ([yshift=1pt]data.north) .. controls +(90:2em) and +(90:2em) .. ([yshift=1pt]model.north) node[above,midway] {参数优化}; \draw [->,thick] ([yshift=1pt]data.north) .. controls +(90:2em) and +(90:2em) .. ([yshift=1pt]model.north) node[above,midway] {\small{参数优化}};
\draw [->,thick] ([yshift=1pt]model.south) .. controls +(-90:2em) and +(-90:2em) .. ([yshift=1pt]data.south) node[below,midway] {数据优化}; \draw [->,thick] ([yshift=1pt]model.south) .. controls +(-90:2em) and +(-90:2em) .. ([yshift=1pt]data.south) node[below,midway] {\small{数据优化}};
\node[word] at ([xshift=-0.5em,yshift=-5em]data.south){（a）思路1}; \node[word] at ([xshift=-0.5em,yshift=-4em]data.south){\small{(a) 思路1}};
\end{scope} \end{scope}
\end{tikzpicture} \end{tikzpicture}
...@@ -25,15 +25,15 @@ ...@@ -25,15 +25,15 @@
\tikzstyle{circle} = [draw,black,line width=0.6pt,inner sep=3.5pt,rounded corners=4pt,minimum width=2em] \tikzstyle{circle} = [draw,black,line width=0.6pt,inner sep=3.5pt,rounded corners=4pt,minimum width=2em]
\tikzstyle{word} = [inner sep=3.5pt] \tikzstyle{word} = [inner sep=3.5pt]
\node[circle,fill=red!20](data) at (0,0) {数据}; \node[circle,fill=red!20](data) at (0,0) {\small{数据}};
\node[circle,fill=blue!20](model) at ([xshift=5em]data.east) {模型}; \node[circle,fill=blue!20](model) at ([xshift=5em]data.east) {\small{模型}};
\node[word] (init) at ([xshift=5em]model.east){初始化}; \node[word] (init) at ([xshift=5em]model.east){\small{初始化}};
\draw[->,thick] (init.west) -- ([xshift=0.2em]model.east); \draw[->,thick] (init.west) -- ([xshift=0.2em]model.east);
\draw [->,thick] ([yshift=1pt]data.north) .. controls +(90:2em) and +(90:2em) .. ([yshift=1pt]model.north) node[above,midway] {参数优化}; \draw [->,thick] ([yshift=1pt]data.north) .. controls +(90:2em) and +(90:2em) .. ([yshift=1pt]model.north) node[above,midway] {\small{参数优化}};
\draw [->,thick] ([yshift=1pt]model.south) .. controls +(-90:2em) and +(-90:2em) .. ([yshift=1pt]data.south) node[below,midway] {数据优化}; \draw [->,thick] ([yshift=1pt]model.south) .. controls +(-90:2em) and +(-90:2em) .. ([yshift=1pt]data.south) node[below,midway] {\small{数据优化}};
\node[word] at ([xshift=-0.5em,yshift=-5em]model.south){（b）思路2}; \node[word] at ([xshift=-0.5em,yshift=-4em]model.south){\small{(b) 思路2}};
\end{scope} \end{scope}
\end{tikzpicture} \end{tikzpicture}
......
Chapter16/Figures/figure-parameter-initialization-method-diagram.tex
...@@ -6,11 +6,11 @@ ...@@ -6,11 +6,11 @@
\node[node,minimum width=6em,minimum height=2.4em,fill=red!20,line width=0.6pt] (encoder1) at (0,0){\small 编码器}; \node[node,minimum width=6em,minimum height=2.4em,fill=red!20,line width=0.6pt] (encoder1) at (0,0){\small 编码器};
\node[node,anchor=west,minimum width=6em,minimum height=2.4em,fill=red!20,line width=0.6pt] (encoder2) at ([xshift=4em,yshift=0em]encoder1.east){\small 编码器}; \node[node,anchor=west,minimum width=6em,minimum height=2.4em,fill=red!20,line width=0.6pt] (encoder2) at ([xshift=4em,yshift=0em]encoder1.east){\small 编码器};
\node[node,anchor=west,minimum width=6em,minimum height=2.4em,fill=red!40,line width=0.6pt] (encoder3) at ([xshift=3em]encoder2.east){\small 编码器}; \node[node,anchor=west,minimum width=6em,minimum height=2.4em,fill=red!30,line width=0.6pt] (encoder3) at ([xshift=3em]encoder2.east){\small 编码器};
\node[node,anchor=north,minimum width=6em,minimum height=2.4em,fill=blue!20,line width=0.6pt] (decoder1) at ([yshift=-3em]encoder1.south){\small 解码器}; \node[node,anchor=north,minimum width=6em,minimum height=2.4em,fill=blue!20,line width=0.6pt] (decoder1) at ([yshift=-3em]encoder1.south){\small 解码器};
\node[node,anchor=west,minimum width=6em,minimum height=2.4em,fill=blue!20,line width=0.6pt] (decoder2) at ([xshift=4em,yshift=0em]decoder1.east){\small 解码器}; \node[node,anchor=west,minimum width=6em,minimum height=2.4em,fill=blue!20,line width=0.6pt] (decoder2) at ([xshift=4em,yshift=0em]decoder1.east){\small 解码器};
\node[node,anchor=west,minimum width=6em,minimum height=2.4em,fill=blue!40,line width=0.6pt] (decoder3) at ([xshift=3em]decoder2.east){\small 解码器}; \node[node,anchor=west,minimum width=6em,minimum height=2.4em,fill=blue!30,line width=0.6pt] (decoder3) at ([xshift=3em]decoder2.east){\small 解码器};
\node[anchor=north,font=\scriptsize,fill=yellow!20] (w1) at ([yshift=-1.6em]decoder1.south){知识 \ 就是 \ 力量 \ 。 \ <EOS>}; \node[anchor=north,font=\scriptsize,fill=yellow!20] (w1) at ([yshift=-1.6em]decoder1.south){知识 \ 就是 \ 力量 \ 。 \ <EOS>};
\node[anchor=north,font=\scriptsize,fill=green!20] (w3) at ([yshift=-1.6em]decoder3.south){Wissen \ ist \ Machit \ . \ <EOS>}; \node[anchor=north,font=\scriptsize,fill=green!20] (w3) at ([yshift=-1.6em]decoder3.south){Wissen \ ist \ Machit \ . \ <EOS>};
......
Chapter16/Figures/figure-schematic-of-the-domain-discriminator.tex
\begin{tikzpicture} \begin{tikzpicture}
\tikzstyle{rec} = [,line width=0.6pt,draw,rounded corners,minimum height=2.2em,minimum width=4.3em,fill=blue!20] \tikzstyle{rec} = [,line width=0.6pt,draw,rounded corners,minimum height=2.2em,minimum width=4.3em,fill=blue!20]
\node [anchor=center](node1) at (0,0) {源语言}; \node [anchor=center](node1) at (0,0) {\small{源语言}};
\node [anchor=west,rec,fill=red!20](node2) at ([xshift=2.0em]node1.east){编码器}; \node [anchor=west,rec,fill=red!20](node2) at ([xshift=2.0em]node1.east){\small{编码器}};
\node [anchor=west,rec](node3) at ([xshift=3.0em,yshift=2.0em]node2.east){解码器}; \node [anchor=west,rec](node3) at ([xshift=3.0em,yshift=2.0em]node2.east){\small{解码器}};
\node [anchor=west,rec,fill=yellow!20](node4) at ([xshift=3.0em,yshift=-2.0em]node2.east){鉴别器}; \node [anchor=west,rec,fill=yellow!20](node4) at ([xshift=3.0em,yshift=-2.0em]node2.east){\small{判别器}};
\draw [->,thick](node1.east)--(node2.west); \draw [->,thick](node1.east)--(node2.west);
\draw [->,thick](node2.east)--([xshift=1.5em]node2.east)--([xshift=1.5em,yshift=2.0em]node2.east)--(node3.west); \draw [->,thick](node2.east)--([xshift=1.5em]node2.east)--([xshift=1.5em,yshift=2.0em]node2.east)--(node3.west);
\draw [->,thick](node2.east)--([xshift=1.5em]node2.east)--([xshift=1.5em,yshift=-2.0em]node2.east)--(node4.west); \draw [->,thick](node2.east)--([xshift=1.5em]node2.east)--([xshift=1.5em,yshift=-2.0em]node2.east)--(node4.west);
\node [anchor=west,minimum width=5.0em](node5) at ([xshift=2.0em]node3.east) {目标语言}; \node [anchor=west,minimum width=5.0em](node5) at ([xshift=2.0em]node3.east) {\small{目标语言}};
\node [anchor=west,minimum width=5.0em](node6) at ([xshift=2.0em]node4.east) {< 领域 >}; \node [anchor=west,minimum width=5.0em](node6) at ([xshift=2.0em]node4.east) {\small{< 领域 >}};
\draw [->,thick](node3.east)--(node5.west); \draw [->,thick](node3.east)--(node5.west);
\draw [->,thick](node4.east)--(node6.west); \draw [->,thick](node4.east)--(node6.west);
\end{tikzpicture} \end{tikzpicture}
\ No newline at end of file
Chapter16/Figures/figure-the-iterative-process-of-bidirectional-training.tex
...@@ -9,7 +9,7 @@ ...@@ -9,7 +9,7 @@
\node(bilingual_D_shadow)[data_shadow, right of = monolingual_X_shadow, xshift=5cm]{}; \node(bilingual_D_shadow)[data_shadow, right of = monolingual_X_shadow, xshift=5cm]{};
\node(monolingual_Y_shadow)[data_shadow, right of = bilingual_D_shadow, xshift=5cm]{}; \node(monolingual_Y_shadow)[data_shadow, right of = bilingual_D_shadow, xshift=5cm]{};
\node(monolingual_X)[data,right of = monolingual_X_shadow,xshift=-0.08cm,yshift=0.08cm]{单语语料X}; \node(monolingual_X)[data,right of = monolingual_X_shadow,xshift=-0.08cm,yshift=0.08cm]{单语语料X};
\node(bilingual_D)[data, right of = monolingual_X, xshift=5cm, fill=ugreen!25]{双语语料D}; \node(bilingual_D)[data, right of = monolingual_X, xshift=5cm, fill=green!30]{双语语料D};
\node(monolingual_Y)[data, right of = bilingual_D, xshift=5cm, fill=blue!25]{单语语料Y}; \node(monolingual_Y)[data, right of = bilingual_D, xshift=5cm, fill=blue!25]{单语语料Y};
\node(process_1_1)[process, right of = monolingual_X, xshift=2.5cm, yshift=-1.5cm]{\textbf{$M^0_{x\to y}$}}; \node(process_1_1)[process, right of = monolingual_X, xshift=2.5cm, yshift=-1.5cm]{\textbf{$M^0_{x\to y}$}};
...@@ -35,7 +35,7 @@ ...@@ -35,7 +35,7 @@
\node(text_2)[left of = process_5_1, xshift=-4cm,scale=0.8]{第1轮迭代}; \node(text_2)[left of = process_5_1, xshift=-4cm,scale=0.8]{第1轮迭代};
\node(text_3)[left of = ellipsis_2, xshift=-4cm, scale=0.8]{第2轮迭代}; \node(text_3)[left of = ellipsis_2, xshift=-4cm, scale=0.8]{第2轮迭代};
\draw[->, very thick, color=color1!40](monolingual_X.south)--(ellipsis_1.north); \draw[->, very thick, color=color1!40](monolingual_X.south)--(ellipsis_1.north);
\draw[->, very thick, color=ugreen!55](bilingual_D.south)--(ellipsis_3.north); \draw[->, very thick, color=green!30](bilingual_D.south)--(ellipsis_3.north);
\draw[->, very thick, color=blue!55](monolingual_Y.south)--(ellipsis_5.north); \draw[->, very thick, color=blue!55](monolingual_Y.south)--(ellipsis_5.north);
\draw[->, very thick, color=color1!40]([xshift=-1.5cm]process_2_1.west)--(process_2_1.west); \draw[->, very thick, color=color1!40]([xshift=-1.5cm]process_2_1.west)--(process_2_1.west);
\draw[->, very thick, color=color1!40]([xshift=-1.5cm]process_5_1.west)--(process_5_1.west); \draw[->, very thick, color=color1!40]([xshift=-1.5cm]process_5_1.west)--(process_5_1.west);
...@@ -55,13 +55,13 @@ ...@@ -55,13 +55,13 @@
\draw[->, very thick, color=color1!40](process_3_2.west)--([yshift=0.35cm]process_4_1.east); \draw[->, very thick, color=color1!40](process_3_2.west)--([yshift=0.35cm]process_4_1.east);
\draw[->, very thick, color=color1!40](process_6_1.east)--([yshift=0.35cm]process_7_2.west); \draw[->, very thick, color=color1!40](process_6_1.east)--([yshift=0.35cm]process_7_2.west);
\draw[->, very thick, color=color1!40](process_6_2.west)--([yshift=0.35cm]process_7_1.east); \draw[->, very thick, color=color1!40](process_6_2.west)--([yshift=0.35cm]process_7_1.east);
\draw[->, very thick, color=ugreen!55,in=0,out=270]([xshift=-0.3cm]bilingual_D.south)to(process_1_1.east); \draw[->, very thick, color=green!30,in=0,out=270]([xshift=-0.3cm]bilingual_D.south)to(process_1_1.east);
\draw[->, very thick, color=ugreen!55,in=180,out=270]([xshift=0.3cm]bilingual_D.south)to(process_1_2.west); \draw[->, very thick, color=green!30,in=180,out=270]([xshift=0.3cm]bilingual_D.south)to(process_1_2.west);
\draw[->, very thick, color=ugreen!55,in=0,out=270]([yshift=-3.7cm]bilingual_D.south)to(process_4_1.east); \draw[->, very thick, color=green!30,in=0,out=270]([yshift=-3.7cm]bilingual_D.south)to(process_4_1.east);
\draw[->, very thick, color=ugreen!55,in=180,out=270]([yshift=-3.7cm]bilingual_D.south)to(process_4_2.west); \draw[->, very thick, color=green!30,in=180,out=270]([yshift=-3.7cm]bilingual_D.south)to(process_4_2.west);
\draw[->, very thick, color=ugreen!55,in=0,out=270]([yshift=-7.3cm]bilingual_D.south)to(process_7_1.east); \draw[->, very thick, color=green!30,in=0,out=270]([yshift=-7.3cm]bilingual_D.south)to(process_7_1.east);
\draw[->, very thick, color=ugreen!55,in=180,out=270]([yshift=-7.3cm]bilingual_D.south)to(process_7_2.west); \draw[->, very thick, color=green!30,in=180,out=270]([yshift=-7.3cm]bilingual_D.south)to(process_7_2.west);
\draw[->, very thick, color=ugreen!55,in=180,out=270]([yshift=-7.3cm]bilingual_D.south)to(process_7_2.west); \draw[->, very thick, color=green!30,in=180,out=270]([yshift=-7.3cm]bilingual_D.south)to(process_7_2.west);
\draw[-, very thick, dashed, color=blue!55]([xshift=-1cm,yshift=-0.35cm]text_1.south)--([xshift=12.7cm,yshift=-0.35cm]text_1.south); \draw[-, very thick, dashed, color=blue!55]([xshift=-1cm,yshift=-0.35cm]text_1.south)--([xshift=12.7cm,yshift=-0.35cm]text_1.south);
\draw[-, very thick, dashed, color=blue!55]([xshift=-1cm,yshift=-0.35cm]text_2.south)--([xshift=12.7cm,yshift=-0.35cm]text_2.south); \draw[-, very thick, dashed, color=blue!55]([xshift=-1cm,yshift=-0.35cm]text_2.south)--([xshift=12.7cm,yshift=-0.35cm]text_2.south);
......
Chapter16/Figures/figure-unmt-process.tex
...@@ -7,24 +7,24 @@ ...@@ -7,24 +7,24 @@
\node[circle](center) at (0,0) { \node[circle](center) at (0,0) {
\begin{tabular}{c | c} \begin{tabular}{c | c}
$x\rightarrow y$ & $y\rightarrow x$ \\ $x\rightarrow y$ & $y\rightarrow x$ \\
模型 & 模型 \small{模型} & \small{模型}
\end{tabular} \end{tabular}
}; };
\node[circle,fill=red!20] (left) at ([xshift=-9em]center.west) {$x\rightarrow y$ \\ 数据}; \node[circle,fill=red!20] (left) at ([xshift=-9em]center.west) {$x\rightarrow y$ \\ \small{数据}};
\node[circle,fill=red!20] (right) at ([xshift=9em]center.east) {$y\rightarrow x$ \\ 数据}; \node[circle,fill=red!20] (right) at ([xshift=9em]center.east) {$y\rightarrow x$ \\ \small{数据}};
\node[word] (init) at ([yshift=6em]center.north){初始化}; \node[word] (init) at ([yshift=6em]center.north){\small{初始化}};
\node[circle,fill=red!20] (down) at ([yshift=-8em]center.south) {$x,y$ \\ 数据}; \node[circle,fill=red!20] (down) at ([yshift=-8em]center.south) {$x,y$ \\ \small{数据}};
\draw[->,thick] (init.south) -- ([yshift=0.2em]center.north); \draw[->,thick] (init.south) -- ([yshift=0.2em]center.north);
\draw[->,thick] ([yshift=0.2em]down.north) -- ([yshift=-0.2em]center.south) node[pos=0.6,midway,align=left,xshift=-2.5em,yshift=0.5em] {语言模型\\目标函数}; \draw[->,thick] ([yshift=0.2em]down.north) -- ([yshift=-0.2em]center.south) node[pos=0.6,midway,align=left,xshift=-2.5em,yshift=0.5em] {\small{语言模型}\\\small{目标函数}};
\node [anchor=center] at ([yshift=2.0em,xshift=-2.5em]down.north){（模型优化）}; \node [anchor=center] at ([yshift=2.0em,xshift=-2.5em]down.north){\small{（模型优化）}};
\draw[->,thick] ([yshift=1pt]left.north) .. controls +(90:2em) and +(90:2em) .. ([yshift=1pt,xshift=-2.2em]center.north) node[above,midway,align=center] {翻译模型目标函数\\（模型优化）}; \draw[->,thick] ([yshift=1pt]left.north) .. controls +(90:2em) and +(90:2em) .. ([yshift=1pt,xshift=-2.2em]center.north) node[above,midway,align=center] {\small{翻译模型目标函数}\\\small{（模型优化）}};
\draw[->,thick] ([yshift=1pt,xshift=-1.8em]center.north) .. controls +(90:2em) and +(90:2em) .. ([yshift=1pt]right.north) node[above,pos=0.6,align=center] {回译\\（数据优化）}; \draw[->,thick] ([yshift=1pt,xshift=-1.8em]center.north) .. controls +(90:2em) and +(90:2em) .. ([yshift=1pt]right.north) node[above,pos=0.6,align=center] {\small{回译}\\\small{（数据优化）}};
\draw [->,thick] ([yshift=1pt]right.south) .. controls +(-90:2em) and +(-90:2em) .. ([yshift=1pt,xshift=2.2em]center.south) node[below,midway,align=center] {翻译模型目标函数\\（模型优化）}; \draw [->,thick] ([yshift=1pt]right.south) .. controls +(-90:2em) and +(-90:2em) .. ([yshift=1pt,xshift=2.2em]center.south) node[below,midway,align=center] {\small{翻译模型目标函数}\\\small{（模型优化）}};
\draw [->,thick] ([yshift=1pt,xshift=1.8em]center.south) .. controls +(-90:2em) and +(-90:2em) .. ([yshift=1pt]left.south) node[below,pos=0.6,align=center] {回译\\（数据优化）}; \draw [->,thick] ([yshift=1pt,xshift=1.8em]center.south) .. controls +(-90:2em) and +(-90:2em) .. ([yshift=1pt]left.south) node[below,pos=0.6,align=center] {\small{回译}\\\small{（数据优化）}};
\end{scope} \end{scope}
\end{tikzpicture} \end{tikzpicture}
Chapter16/chapter16.tex
This source diff could not be displayed because it is too large. You can view the blob instead.
Chapter18/Figures/figure-comparison-of-incremental-model-optimization-methods.tex 0 → 100644
\addtolength{\tabcolsep}{-4pt}
\begin{tabular}{c c c}
\begin{tikzpicture}
\begin{scope}
% ,minimum height =1em,minimum width=2em
\tikzstyle{model} = [draw,black,very thick,inner sep=3.5pt,rounded corners=4pt,fill=blue!20,minimum width=4em,minimum height=1.5em,font=\footnotesize]
\tikzstyle{data} = [draw,black,very thick,inner sep=3.5pt,rounded corners=4pt,fill=green!20,minimum width=4em,minimum height=1.5em,font=\footnotesize]
\tikzstyle{word} = [inner sep=3.5pt,font=\footnotesize]
\node[data] (old) at (0,0) {旧数据};
\node[data] (new) at ([xshift=3em]old.east) {新数据};
\node[data] (all) at ([xshift=2.55em,yshift=-4em]old.south) {最终数据};
\node[model] (final_model) at ([xshift=0em,yshift=-4em]all.south) {最终模型};
\draw [->,thick] ([yshift=-0.2em]old.south) .. controls +(south:2.5em) and +(north:2.5em) .. ([xshift=-0.2em,yshift=0.2em]all.north);
\draw [->,thick] ([yshift=-0.2em]new.south) .. controls +(south:2.5em) and +(north:2.5em) .. ([xshift=0.2em,yshift=0.2em]all.north);
\draw [->,thick] ([yshift=-0.2em]all.south) -- ([yshift=0.2em]final_model.north)node[pos=0.5,right,align=center,font=\footnotesize] {训练};
\node[word] at ([yshift=-2em]final_model.south){（a）数据混合};
\end{scope}
\end{tikzpicture}
&
\begin{tikzpicture}
\begin{scope}
\tikzstyle{model} = [draw,black,very thick,inner sep=3.5pt,rounded corners=4pt,fill=blue!20,minimum width=4em,minimum height=1.5em,font=\footnotesize]
\tikzstyle{data} = [draw,black,very thick,inner sep=3.5pt,rounded corners=4pt,fill=green!20,minimum width=4em,minimum height=1.5em,font=\footnotesize]
\tikzstyle{word} = [inner sep=3.5pt,font=\footnotesize]
\node[data] (old) at (0,0) {旧数据};
\node[data] (new) at ([xshift=3em]old.east) {新数据};
\node[model] (old_model) at ([yshift=-4em]old.south) {旧模型};
\node[model] (new_model) at ([yshift=-4em]new.south) {新模型};
\node[model] (final_model) at ([xshift=2.55em,yshift=-4em]old_model.south) {最终模型};
\draw [->,thick] ([yshift=-0.2em]old.south) -- ([yshift=0.2em]old_model.north) node[pos=0.5,left,align=center,font=\footnotesize] {训练};
\draw [->,thick] ([yshift=-0.2em]new.south) -- ([yshift=0.2em]new_model.north) node[pos=0.5,right,align=center,font=\footnotesize] {训练};
\draw [->,thick] ([yshift=-0.2em]old_model.south) .. controls +(south:2.5em) and +(north:2.5em) .. ([xshift=-0.2em,yshift=0.2em]final_model.north);
\draw [->,thick] ([yshift=-0.2em]new_model.south) .. controls +(south:2.5em) and +(north:2.5em) .. ([xshift=0.2em,yshift=0.2em]final_model.north);
\node[word] at ([yshift=2em]final_model.north) {插值};
\node[word] at ([yshift=-2em]final_model.south){（b）模型插值};
%空白占位
\node[word] at ([xshift=-3em]old.west) {};
\node[word] at ([xshift=3em]new.east) {};
\end{scope}
\end{tikzpicture}
&
\begin{tikzpicture}
\begin{scope}
\tikzstyle{model} = [draw,black,very thick,inner sep=3.5pt,rounded corners=4pt,fill=blue!20,minimum width=4em,minimum height=1.5em,font=\footnotesize]
\tikzstyle{data} = [draw,black,very thick,inner sep=3.5pt,rounded corners=4pt,fill=green!20,minimum width=4em,minimum height=1.5em,font=\footnotesize]
\tikzstyle{word} = [inner sep=3.5pt,font=\footnotesize]
\node[data] (old) at (0,0) {旧数据};
\node[data] (new) at ([xshift=3em]old.east) {新数据};
\node[model] (final_model) at ([yshift=-8.8em]new.south) {最终模型};
\draw [->,thick] ([yshift=-0.2em]new.south) -- ([xshift=0.2em,yshift=0.2em]final_model.north) node[pos=0.5,right,align=center,font=\footnotesize] {目标\\函数1};
\draw [->,thick,dashed] ([yshift=-0.2em]old.south) .. controls +(south:4.5em) and +(north:4.5em) .. ([xshift=-0.2em,yshift=0.2em]final_model.north) node[align=center,font=\footnotesize] at ([xshift=-0.1em,yshift=-4em]old.south) {目标\\函数2};
\node[word] at ([yshift=-2em,xshift=-2.55em]final_model.south){（c）多目标训练};
\end{scope}
\end{tikzpicture}
\end{tabular}
\addtolength{\tabcolsep}{4pt}
\ No newline at end of file
Chapter18/Figures/figure-memory-multi-use.tex 0 → 100644
\begin{tabular}{c c}
\begin{tikzpicture}
\begin{scope}
% ,minimum height =1em,minimum width=2em
\tikzstyle{memory} = [draw,black,very thick,inner sep=2pt,rounded corners=0pt,fill=blue!20,minimum width=2em,minimum height=1.5em,anchor=west]
\tikzstyle{thread} = [very thick,inner sep=3.5pt,rounded corners=0pt,minimum width=3em,minimum height=1.5em]
\tikzstyle{word} = [inner sep=3.5pt,font=\scriptsize]
\node[thread] (one) at (0,0) {};
\node [word] at (one.north) {\scriptsize 数据1};
\draw[|-|,very thick] (-1.5em,0em) -- (1.6em,0em);
\node[thread,minimum width=5em] (two) at ([yshift=-1em,xshift=2.6em]one.south east) {};
\node [word] at (two.north) {\scriptsize 数据2};
\draw[|-|,very thick] (1.8em,-1.8em) -- (6.5em,-1.8em);
\node[thread,minimum width=4em] (three) at ([yshift=-1em,xshift=0.3em]two.south east) {};
\node [word] at (three.north) {\scriptsize 数据3};
\draw[|-|,very thick] (5em,-1.8*2em) -- (9em,-1.8*2em);
\node[thread,minimum width=2em] (four) at ([yshift=-1em,xshift=1.2em]three.south east) {};
\node [word] at (four.north) {\scriptsize 数据4};
\draw[|-|,very thick] (9.3em,-1.8*3em) -- (11em,-1.8*3em);
\node [memory] (mone) at ([yshift=4em,xshift=1em]one.north) {};
\node [memory] (mtwo) at ([xshift=0em]mone.east) {};
\node [memory] (mthree) at ([xshift=0em]mtwo.east) {};
\node [memory] (mfour) at ([xshift=0em]mthree.east) {};
\draw[->,very thick] (-0.8,-2.5) -- (4.7,-2.5);
\draw[->,very thick] (-0.8,-2.5) -- (-0.8,1);
\node [word] (time) at ([yshift=-1.5em,xshift=0.3em]four.south) {\scriptsize 时间线};
\node [word] (time) at ([yshift=1.5em,xshift=-2.2em]one.west) {\scriptsize 数据};
\draw [->,dashed,line width=0.7pt] ([yshift=0.5em]one.north) .. controls +(north:1.5em) and +(south:1.5em) .. ([yshift=-0.2em]mone.south);
\draw [->,dashed,line width=0.7pt] ([yshift=0.5em]two.north) -- ([yshift=-0.2em]mtwo.south);
\draw [->,dashed,line width=0.7pt] ([yshift=0.5em,xshift=0.5em]three.north) .. controls +(north:3.5em) and +(south:4.5em) .. ([yshift=-0.2em]mthree.south);
\draw [->,dashed,line width=0.7pt] ([yshift=0.5em]four.north) .. controls +(north:4.5em) and +(south:4.5em) .. ([yshift=-0.2em]mfour.south);
\node [word] at ([yshift=-6em]two.south) {（a）显存不复用};
%占位
\node[word] at ([xshift=1em]four.east) {};
\node [word] at ([xshift=1.5em,yshift=5.6em]one.north) {\scriptsize 显存};
\begin{pgfonlayer}{background}
\node [rectangle,inner sep=0.5em,rounded corners=1pt,minimum width=10em,minimum height=3.6em,fill=gray!10,drop shadow] at ([yshift=6.6em,xshift=1em]two.north) {};
\end{pgfonlayer}
\end{scope}
\end{tikzpicture}
&
\begin{tikzpicture}
\begin{scope}
%\tikzstyle{memory} = [draw,black,very thick,inner sep=2pt,rounded corners=0pt,fill=blue!20,minimum width=2em,minimum height=1.5em,anchor=west]
%\tikzstyle{thread} = [draw,black,very thick,inner sep=3.5pt,rounded corners=0pt,fill=green!20,minimum width=3em,minimum height=1.5em]
\tikzstyle{memory} = [draw,black,very thick,inner sep=2pt,rounded corners=0pt,fill=blue!20,minimum width=2em,minimum height=1.5em,anchor=west]
\tikzstyle{thread} = [very thick,inner sep=3.5pt,rounded corners=0pt,minimum width=3em,minimum height=1.5em]
\tikzstyle{word} = [inner sep=3.5pt,font=\scriptsize]
\node[thread] (one) at (0,0) {};
\node [word] at (one.north) {\scriptsize 数据1};
\draw[|-|,very thick] (-1.5em,0em) -- (1.6em,0em);
\node[thread,minimum width=5em] (two) at ([yshift=-1em,xshift=2.6em]one.south east) {};
\node [word] at (two.north) {\scriptsize 数据2};
\draw[|-|,very thick] (1.8em,-1.8em) -- (6.5em,-1.8em);
\node[thread,minimum width=4em] (three) at ([yshift=-1em,xshift=0.3em]two.south east) {};
\node [word] at (three.north) {\scriptsize 数据3};
\draw[|-|,very thick] (5em,-1.8*2em) -- (9em,-1.8*2em);
\node[thread,minimum width=2em] (four) at ([yshift=-1em,xshift=1.2em]three.south east) {};
\node [word] at (four.north) {\scriptsize 数据4};
\draw[|-|,very thick] (9.3em,-1.8*3em) -- (11em,-1.8*3em);
\node [memory] (mone) at ([yshift=4em,xshift=1em]one.north) {};
\node [memory] (mtwo) at ([xshift=0em]mone.east) {};
\node [memory,fill=white,minimum width=4em] (mthree) at ([xshift=0em]mtwo.east) {};
%\node [memory,fill=white] (mfour) at ([xshift=0em]mthree.east) {};
\draw[->,very thick] (-0.8,-2.5) -- (4.7,-2.5);
\draw[->,very thick] (-0.8,-2.5) -- (-0.8,1);
\node [word] (time) at ([yshift=-1.5em,xshift=0.3em]four.south) {\scriptsize 时间线};
\node [word] (time) at ([yshift=1.5em,xshift=-2.2em]one.west) {\scriptsize 数据};
\draw [->,dashed,line width=0.7pt] ([yshift=0.5em]one.north) .. controls +(north:1.5em) and +(south:1.5em) .. ([yshift=-0.2em,xshift=-0.4em]mone.south);
\draw [->,dashed,line width=0.7pt] ([yshift=0.5em]two.north) .. controls +(north:3.5em) and +(south:3.5em) .. ([yshift=-0.2em,xshift=0.4em]mone.south);
\draw [->,dashed,line width=0.7pt] ([yshift=0.5em,xshift=0.5em]three.north) .. controls +(north:3.5em) and +(south:3.5em) .. ([yshift=-0.2em,xshift=-0.4em]mtwo.south);
\draw [->,dashed,line width=0.7pt] ([yshift=0.5em]four.north) .. controls +(north:4.5em) and +(south:3.5em) .. ([yshift=-0.2em,xshift=0.4em]mtwo.south);
\node [word] at ([xshift=1.5em,yshift=5.6em]one.north) {\scriptsize 显存};
\node [word] at ([yshift=-6em]two.south) {（b）显存复用};
\begin{pgfonlayer}{background}
\node [rectangle,inner sep=0.5em,rounded corners=1pt,minimum width=10em,minimum height=3.6em,fill=gray!10,drop shadow] at ([yshift=6.6em,xshift=1em]two.north) {};
\end{pgfonlayer}
\end{scope}
\end{tikzpicture}
\end{tabular}
\ No newline at end of file
Chapter18/Figures/figure-translation-interfered.tex 0 → 100644
%%% outline
%-------------------------------------------------------------------------
\begin{tikzpicture}[scale=0.8]
\tikzstyle{every node}=[scale=0.8]
\tikzstyle{node}=[rounded corners=4pt, draw,minimum width=3em, minimum height=2em, drop shadow={shadow xshift=0.14em, shadow yshift=-0.14em}]
\begin{scope}
%\draw[fill=yellow!20] (-5em, 0) -- (-6em, 1em) -- (5em, 1em) -- (6em, 0em) -- (5em, -1em) -- (-6em, -1em) -- (-5em, 0em);
%\draw[fill=yellow!20] (-5em, 10em) -- (-6em, 11.2em) -- (5em, 11.2em) -- (6em, 10em) -- (5em,8.8em) -- (-6em, 8.8em) -- (-5em, 10em);
\node[] (n1) at (0,0){小牛翻译的总部在哪里？};
\node[node,fill=blue!20] (c1) at (0, 5em){\scriptsize\bfnew{机器翻译}};
\node[align=left] (n2) at (0,10em){Where is the headquarters \\ of {\color{red} Mavericks Translation}?};
\node [draw,single arrow,inner ysep=0.3em, minimum height=2.4em, rotate=90,fill=cyan!40,very thin] (arrow1) at (0, 2.4em) {};
\node [draw,single arrow,inner ysep=0.3em, minimum height=2em, rotate=90,fill=cyan!40,very thin] (arrow1) at (0, 7.2em) {};
\node[font=\Large,text=red] at (0, -2em){\ding{56}};
\end{scope}
\begin{scope}[xshift=14em]
%\draw[fill=yellow!20] (-5em, 0) -- (-6em, 1em) -- (5em, 1em) -- (6em, 0em) -- (5em, -1em) -- (-6em, -1em) -- (-5em, 0em);
%\draw[fill=yellow!20] (-5em, 10em) -- (-6em, 11.2em) -- (5em, 11.2em) -- (6em, 10em) -- (5em,8.8em) -- (-6em, 8.8em) -- (-5em, 10em);
\node[] (n3) at (0,0){小牛翻译的总部在哪里？};
\node[node,fill=blue!20] (c2) at (-3em, 5em){\scriptsize\bfnew{机器翻译}};
\node[node,fill=red!20] (c3) at (3em, 5em){\scriptsize\bfnew{术语词典}};
\node[font=\scriptsize,draw,inner sep=3pt,fill=red!20,minimum height=1em] (w1) at (9em, 6.5em){小牛翻译};
\node[font=\scriptsize,draw,inner sep=3pt,fill=red!20,minimum height=1em] (w2) at (9em, 3.5em){NiuTrans};
\node[font=\Large] (add) at (0em, 5em){+};
\node[align=left] (n4) at (0,10em){Where is the headquarters \\ of {\color{red} NiuTrans}?};
\node [draw,single arrow,inner ysep=0.3em, minimum height=2.4em, rotate=90,fill=cyan!40,very thin] (arrow1) at (0, 2.4em) {};
\node [draw,single arrow,inner ysep=0.3em, minimum height=2em, rotate=90,fill=cyan!40,very thin] (arrow1) at (0, 7.2em) {};
\draw[dash pattern=on 1pt off 0.5pt,black,line width=1.2pt,->, out=180, in=45] ([xshift=-0.2em]w1.180) to ([xshift=0.2em]c3.20);
\draw[dash pattern=on 1pt off 0.5pt,black,line width=1.2pt,->,out=180,in=-45] ([xshift=-0.2em]w2.180) to ([xshift=0.2em]c3.-20);
\node[font=\Large,text=ugreen] at (0, -2em){\ding{52}};
\end{scope}
\end{tikzpicture}
Markdown 格式
0% 或
您添加了 0 人 到此讨论。请谨慎行事。
请先完成此评论的编辑！
请 注册 或者 后发表评论