Skip to content

M
mtbookv2
Commit 3b34f35c by zengxin
Options

合并分支 'caorunzhe' 到 'zengxin' 

Caorunzhe

查看合并请求 !971
parents db55bbab 30f143b7
正在显示 包含 626 行增加 和 423 行删除
Chapter1/Figures/figure-example-smt.tex
......@@ -52,7 +52,7 @@
}
{
\begin{scope}[xshift=1.7in]
\begin{scope}[xshift=2in]
{\scriptsize
\node [anchor=north west] (phrase1) at (0,0) {$\textrm{Pr}(\textrm{我} \to \textrm{I}) = 0.7$};
\node [anchor=north west] (phrase2) at ([yshift=0.1em]phrase1.south west) {$\textrm{Pr}(\textrm{我} \to \textrm{me}) = 0.3$};
......@@ -76,7 +76,7 @@
}
{
\begin{scope}[xshift=1.7in,yshift=-1.55in]
\begin{scope}[xshift=2in,yshift=-1.55in]
{\scriptsize
\node [anchor=north west] (ngram1) at (0,0) {$\textrm{Pr}(\textrm{I}) = 0.0001$};
\node [anchor=north west] (ngram2) at ([yshift=0.0em]ngram1.south west) {$\textrm{Pr}(\textrm{I}\ \textrm{am}) = 0.623$};
......@@ -96,14 +96,14 @@
}
{
\draw[->,thick,ublue] (bidata.east)--([xshift=2.2em]bidata.east) node[pos=0.5,above] (simexample) {\color{red}{\scriptsize{\scriptsize{学习}}}};
\draw[->,thick,ublue] (bidata.east)--([xshift=4.2em]bidata.east) node[pos=0.5,above] (simexample) {\color{red}{\scriptsize{\scriptsize{学习}}}};
}
{
\draw[->,thick,ublue] (monodata.east)--([xshift=2.2em]monodata.east) node[pos=0.5,above] (simexample) {\color{red}{\scriptsize{\scriptsize{学习}}}};
\draw[->,thick,ublue] (monodata.east)--([xshift=4.2em]monodata.east) node[pos=0.5,above] (simexample) {\color{red}{\scriptsize{\scriptsize{学习}}}};
}
\begin{scope}[xshift=3.6in]
\begin{scope}[xshift=4in]
{\footnotesize
{
\node[anchor=center] (srcsentence) at (0,0) {我 对 你 感到 满意};
......
Chapter1/chapter1.tex
......@@ -54,14 +54,6 @@
\parinterval 从机器翻译系统的组成上来看，通常可以抽象为两个部分，如图\ref{fig:1-2}所示：
\begin{itemize}
\vspace{0.5em}
\item {\small\bfnew{资源}}：如果把机器翻译系统比作一辆汽车，资源就好比是可以使汽车运行的“汽油”，它包括很多内容，如翻译规则、双（单）语数据、知识库等翻译知识，且这些“知识”都是计算机可读的。值得一提的是,如果没有翻译资源的支持，任何机器翻译系统都无法运行起来。
\vspace{0.5em}
\item {\small\bfnew{系统}}：机器翻译算法的程序实现被称作系统，也就是机器翻译研究人员开发的软件。无论是翻译规则、翻译模板还是统计模型中的参数都需要通过机器翻译系统进行读取和使用。
\vspace{0.5em}
\end{itemize}
%----------------------------------------------
\begin{figure}[htp]
\centering
......@@ -71,6 +63,14 @@
\end{figure}
%-------------------------------------------
\begin{itemize}
\vspace{0.5em}
\item {\small\bfnew{资源}}：如果把机器翻译系统比作一辆汽车，资源就好比是可以使汽车运行的“汽油”，它包括很多内容，如翻译规则、双（单）语数据、知识库等翻译知识，且这些“知识”都是计算机可读的。值得一提的是,如果没有翻译资源的支持，任何机器翻译系统都无法运行起来。
\vspace{0.5em}
\item {\small\bfnew{系统}}：机器翻译算法的程序实现被称作系统，也就是机器翻译研究人员开发的软件。无论是翻译规则、翻译模板还是统计模型中的参数都需要通过机器翻译系统进行读取和使用。
\vspace{0.5em}
\end{itemize}
\parinterval 构建一个强大的机器翻译系统需要“资源”和“系统”两方面共同作用。在资源方面，随着语料库语言学的发展，已经有大量的高质量的双语和单语数据（称为语料）被整理并且被电子化存储，因此可以说具备了研发机器翻译系统所需要的语料基础。特别是像英语、汉语等世界主流语种，相关语料资源已经非常丰富，这也大大加速了相关研究的进展。当然，对于一些稀缺资源语种或者特殊的领域，语料库中的语料仍然匮乏，但是这些并不影响机器翻译领域整体的发展速度。因此在现有语料库的基础上，很多研究者把精力集中在“系统”研发上。
%----------------------------------------------------------------------------------------
......@@ -129,7 +129,7 @@
\parinterval 1957年，Noam Chomsky在\emph{Syntactic Structures}中描述了转换生成语法\upcite{chomsky1957syntactic}，并使用数学方法来研究自然语言，建立了包括上下文有关语法、上下文无关语法等4种类型的语法。这些工作最终为今天计算机中广泛使用的“形式语言”奠定了基础。而他的思想也深深地影响了同时期的语言学和自然语言处理领域的学者。特别的是，早期基于规则的机器翻译中也大量使用了这些思想。
\parinterval 虽然在这段时间，使用机器进行翻译的议题越加火热，但是事情并不总是一帆风顺，怀疑论者对机器翻译一直存有质疑，并很容易找出一些机器翻译无法解决的问题。自然地，人们也期望能够客观地评估一下机器翻译的可行性。当时美国基金资助组织委任自动语言处理咨询会承担了这项任务。经过近两年的调查与分析，该委员会于1966年11月公布了一个题为\emph{LANGUAGE AND MACHINES}的报告（图\ref{fig:1-5}），即ALPAC报告。该报告全面否定了机器翻译的可行性，为机器翻译的研究泼了一盆冷水。
\parinterval 虽然在这段时间，使用机器进行翻译的议题越加火热，但是事情并不总是一帆风顺，怀疑论者对机器翻译一直存有质疑，并很容易找出一些机器翻译无法解决的问题。自然地，人们也期望能够客观地评估一下机器翻译的可行性。当时美国基金资助组织委任自动语言处理咨询会承担了这项任务。经过近两年的调查与分析，该委员会于1966年11月公布了一个题为\emph{LANGUAGE AND MACHINES}的报告（图\ref{fig:1-4}），即ALPAC报告。该报告全面否定了机器翻译的可行性，为机器翻译的研究泼了一盆冷水。
%----------------------------------------------
\begin{figure}[htp]
......@@ -258,8 +258,6 @@
\parinterval 规则就像语言中的“If-then”语句，如果满足条件，则执行相应的语义动作。比如，可以将待翻译句子中的某个词，使用目标语言单词进行替换，但是这种替换并非随意的，而是在语言学知识的指导下进行的。
\parinterval 图\ref{fig:1-8}展示了一个使用转换法进行翻译的实例。这里，利用一个简单的汉译英规则库完成对句子“我对你感到满意”的翻译。当翻译“我”时，从规则库中找到规则1，该规则表示遇到单词“我”就翻译为“I”；类似地，也可以从规则库中找到规则4，该规则表示翻译调序，即将单词“you”放到“be satisfied with”后面。这种通过规则表示单词之间的对应关系也为统计机器翻译方法提供了思路。如统计机器翻译中，基于短语的翻译模型使用短语对对原文进行替换，详细描述可以参考{\chapterseven}。
%----------------------------------------------
\begin{figure}[htp]
\centering
......@@ -269,6 +267,8 @@
\end{figure}
%-------------------------------------------
\parinterval 图\ref{fig:1-8}展示了一个使用转换法进行翻译的实例。这里，利用一个简单的汉译英规则库完成对句子“我对你感到满意”的翻译。当翻译“我”时，从规则库中找到规则1，该规则表示遇到单词“我”就翻译为“I”；类似地，也可以从规则库中找到规则4，该规则表示翻译调序，即将单词“you”放到“be satisfied with”后面。这种通过规则表示单词之间的对应关系也为统计机器翻译方法提供了思路。如统计机器翻译中，基于短语的翻译模型使用短语对对原文进行替换，详细描述可以参考{\chapterseven}。
\parinterval 在上述例子中可以发现，规则不仅仅可以翻译句子之间单词的对应，如规则1，还可以表示句法甚至语法之间的对应，如规则6。因此基于规则的方法可以分成多个层次，如图\ref{fig:1-9}所示。图中不同的层次表示采用不同的知识来书写规则，进而完成机器翻译过程。对于翻译问题，可以构建不同层次的基于规则的机器翻译系统。这里包括四个层次，分别为：词汇转换、句法转换、语义转换和中间语言层。其中，上层可以继承下层的翻译知识，比如说句法转换层会利用词汇转换层知识。早期基于规则的方法属于词汇转换层。
%----------------------------------------------
......
Chapter13/Figures/figure-bpe.tex
......@@ -11,11 +11,11 @@
\node [anchor=west,tnode] (n3) at ([xshift=5em,yshift=0em]n2.east) {};
\node [anchor=north west,align=left,font=\footnotesize] (n31) at ([xshift=0.2em,yshift=-0.2em]n3.north west) {{\small 词表}\\low\\lower\\newest\\widest\\$\ldots$};
\node [anchor=north east,align=center,font=\footnotesize] (n32) at ([xshift=-0.2em,yshift=-0.2em]n3.north east) {{\small 频率}\\6\\2\\6\\3\\$\ldots$};
\node [anchor=north east,align=center,font=\footnotesize] (n32) at ([xshift=-0.2em,yshift=-0.2em]n3.north east) {{\small 频次}\\6\\2\\6\\3\\$\ldots$};
\node [anchor=west,tnode] (n4) at ([xshift=5em,yshift=0em]n3.east) {};
\node [anchor=north west,align=left,font=\footnotesize] (n41) at ([xshift=0.2em,yshift=-0.2em]n4.north west) {{\small 词表}\\l/o/w\\l/o/w/e/r\\n/e/w/e/s/t\\w/i/d/e/s/t\\$\ldots$};
\node [anchor=north east,align=center,font=\footnotesize] (n42) at ([xshift=-0.2em,yshift=-0.2em]n4.north east) {{\small 频率}\\6\\2\\6\\3\\$\ldots$};
\node [anchor=north east,align=center,font=\footnotesize] (n42) at ([xshift=-0.2em,yshift=-0.2em]n4.north east) {{\small 频次}\\6\\2\\6\\3\\$\ldots$};
\begin{pgfonlayer}{background}
......@@ -32,13 +32,13 @@
\node [anchor=north west,tnode] (n6) at ([xshift=0em,yshift=-0.5em]n5.south west) {};
\node [anchor=north west,align=left,font=\footnotesize] (n61) at ([xshift=0.2em,yshift=-0.2em]n6.north west) {{\small 词表}\\l/o/w\\l/o/w/e/r\\n/e/w/e/s/t\\w/i/d/e/s/t\\$\ldots$};
\node [anchor=north east,align=center,font=\footnotesize] (n62) at ([xshift=-0.2em,yshift=-0.2em]n6.north east) {{\small 频率}\\6\\2\\6\\3\\$\ldots$};
\node [anchor=north east,align=center,font=\footnotesize] (n62) at ([xshift=-0.2em,yshift=-0.2em]n6.north east) {{\small 频次}\\6\\2\\6\\3\\$\ldots$};
\draw [->,thick,ublue] ([xshift=-0em,yshift=-0em]n4.south) .. controls +(south:4em) and +(north:4em) .. ([xshift=1em,yshift=-0em]n6.north);
\node [anchor=west,pnode] (n7) at ([xshift=5em,yshift=0em]n6.east) {};
\node [anchor=north west,align=left,font=\footnotesize] (n71) at ([xshift=0.2em,yshift=-0.2em]n7.north west) {{\small 二元组}\\(e,s)\\(s,t)\\(l,o)\\(o,w)\\$\ldots$};
\node [anchor=north east,align=center,font=\footnotesize] (n72) at ([xshift=-0.2em,yshift=-0.2em]n7.north east) {{\small 频率}\\9\\9\\8\\8\\$\ldots$};
\node [anchor=north east,align=center,font=\footnotesize] (n72) at ([xshift=-0.2em,yshift=-0.2em]n7.north east) {{\small 频次}\\9\\9\\8\\8\\$\ldots$};
\node [anchor=west,mnode] (n8) at ([xshift=5em,yshift=0em]n7.east) {};
\node [anchor=north,align=center,font=\footnotesize] (n81) at ([xshift=0em,yshift=-0.2em]n8.north) {{\small 符号合并表}\\(e,s)};
......@@ -51,14 +51,14 @@
%第三排
\node [anchor=north,tnode] (n9) at ([xshift=0em,yshift=-2.5em]n6.south) {};
\node [anchor=north west,align=left,font=\footnotesize] (n91) at ([xshift=0.2em,yshift=-0.2em]n9.north west) {{\small 词表}\\l/o/w\\l/o/w/e/r\\n/e/w/{\red es}/t\\w/i/d/{\red es}/t\\$\ldots$};
\node [anchor=north east,align=center,font=\footnotesize] (n92) at ([xshift=-0.2em,yshift=-0.2em]n9.north east) {{\small 频率}\\6\\2\\6\\3\\$\ldots$};
\node [anchor=north east,align=center,font=\footnotesize] (n92) at ([xshift=-0.2em,yshift=-0.2em]n9.north east) {{\small 频次}\\6\\2\\6\\3\\$\ldots$};
\draw [->,thick,ublue] ([xshift=-0em,yshift=-0em]n8.south) .. controls +(south:3em) and +(north:3em) .. ([xshift=1em,yshift=-0em]n9.north);
\node [anchor=north west,ublue,font=\footnotesize,align=left] (l1) at ([xshift=1em,yshift=-0em]n7.south east) {在词表中\\[0.8ex]合并(e,s)};
\node [anchor=west,pnode] (n10) at ([xshift=5em,yshift=0em]n9.east) {};
\node [anchor=north west,align=left,font=\footnotesize] (n101) at ([xshift=0.2em,yshift=-0.2em]n10.north west) {{\small 二元组}\\(es,t)\\(l,o)\\(o,w)\\(n,e)\\$\ldots$};
\node [anchor=north east,align=center,font=\footnotesize] (n102) at ([xshift=-0.2em,yshift=-0.2em]n10.north east) {{\small 频率}\\9\\8\\8\\6\\$\ldots$};
\node [anchor=north east,align=center,font=\footnotesize] (n102) at ([xshift=-0.2em,yshift=-0.2em]n10.north east) {{\small 频次}\\9\\8\\8\\6\\$\ldots$};
\node [anchor=west,mnode] (n11) at ([xshift=5em,yshift=0em]n10.east) {};
\node [anchor=north,align=center,font=\footnotesize] (n111) at ([xshift=0em,yshift=-0.2em]n11.north) {{\small 符号合并表}\\(e,s)\\(es,t)};
......
Chapter13/Figures/figure-difference-between-word-level-and-sequence-level-in-knowledge-distillation.tex
......@@ -13,6 +13,7 @@
\coordinate (topright) at ([shift={(0.25cm,0.25cm)}]prob11.center);
\node [draw,fit=(prob11) (prob21) (prob31) (prob41) (prob51) (topright) (bottomleft)] (prob1) {};
\end{pgfonlayer}
% \node [anchor=center] (word11) at ([yshift=0.7cm]prob11.center) {I};
% Column 2
\node [prob,minimum size=0.1cm,anchor=center] (prob12) at ([xshift=1cm]prob11.center) {};
......@@ -25,18 +26,20 @@
\coordinate (topright) at ([shift={(0.25cm,0.25cm)}]prob12.center);
\node [draw,fit=(prob12) (prob22) (prob32) (prob42) (prob52) (topright) (bottomleft)] (prob2) {};
\end{pgfonlayer}
% \node [anchor=center] (word12) at ([yshift=0.68cm]prob12.center) {am};
% Column 3
\node [prob,minimum size=0.1cm,anchor=center] (prob13) at ([xshift=1cm]prob12.center) {};
\node [prob,minimum size=0.1cm,anchor=center] (prob23) at ([yshift=-0.5cm]prob13.center) {};
\node [prob,minimum size=0.1cm,anchor=center] (prob33) at ([yshift=-0.5cm]prob23.center) {};
\node [prob,minimum size=0.4cm,anchor=center] (prob43) at ([yshift=-0.5cm]prob33.center) {$.6$};
\node [prob,minimum size=0.1cm,anchor=center] (prob53) at ([yshift=-0.5cm]prob43.center) {};
\node [prob,minimum size=0.1cm,anchor=center] (prob43) at ([yshift=-0.5cm]prob33.center) {};
\node [prob,minimum size=0.4cm,anchor=center] (prob53) at ([yshift=-0.5cm]prob43.center) {$.6$};
\begin{pgfonlayer}{background}
\coordinate (bottomleft) at ([shift={(-0.25cm,-0.25cm)}]prob53.center);
\coordinate (topright) at ([shift={(0.25cm,0.25cm)}]prob13.center);
\node [draw,fit=(prob13) (prob23) (prob33) (prob43) (prob53) (topright) (bottomleft)] (prob3) {};
\end{pgfonlayer}
% \node [anchor=center] (word13) at ([yshift=0.7cm]prob13.center) {fine};
% Column 4
\node [prob,minimum size=0.5cm,anchor=center] (prob14) at ([xshift=1cm]prob13.center) {$.8$};
......@@ -49,9 +52,10 @@
\coordinate (topright) at ([shift={(0.25cm,0.25cm)}]prob14.center);
\node [draw,fit=(prob14) (prob24) (prob34) (prob44) (prob54) (topright) (bottomleft)] (prob4) {};
\end{pgfonlayer}
% \node [anchor=center] (word14) at ([yshift=0.68cm]prob14.center) {$\langle$eos$\rangle$};
% Label
\draw [decorate,decoration={brace}] ([yshift=0.1cm]prob1.north west) to node [midway,above,font=\small] {学习目标(Teacher输出)} ([yshift=0.1cm]prob4.north east);
\draw [decorate,decoration={brace}] ([xshift=0.1cm]prob4.north east) to node [midway,right,font=\small,align=center] {教师\\模型\\输出\\分布} ([xshift=0.1cm]prob4.south east);
% Vocab
\node [word,anchor=center] () at ([xshift=-0.9cm]prob11.center) {$\langle$eos$\rangle$};
......@@ -72,19 +76,20 @@
\draw [-latex,thick] (prob\i.south) to ([yshift=-0.5cm]prob\i.south);
% Input
\node [word,anchor=south] (input1) at ([yshift=-1.8cm]prob1.south) {$\langle$eos$\rangle$};
\node [word,anchor=south] (input1) at ([yshift=-1.87cm]prob1.south) {$\langle$eos$\rangle$};
\node [word,anchor=south] (input2) at ([yshift=-1.8cm]prob2.south) {I};
\node [word,anchor=south] (input3) at ([yshift=-1.8cm]prob3.south) {am};
\node [word,anchor=south] (input4) at ([yshift=-1.8cm]prob4.south) {fine};
\foreach \i in {1,2,...,4}
\foreach \i in {2,3,4}
\draw [-latex,thick] ([yshift=0.3cm]input\i.south) to ([yshift=0.8cm]input\i.south);
\draw [-latex,thick] ([yshift=0.36cm]input1.south) to ([yshift=0.86cm]input1.south);
\node [word,anchor=south] (ns) at ([xshift=-1cm]input1.south) {输入:};
\node [word,anchor=south] (ns) at ([xshift=-1.2cm]input1.south) {真实数据:};
\node [word,anchor=north] () at ([xshift=2.1cm,yshift=-0.5cm]ns.south) {(a)\ Word-level};
\end{scope}
\begin{scope}[xshift=2.5in]
\begin{scope}[xshift=2.7in]
% Column 1
\node [prob,minimum size=0.1cm] (prob11) at (0,0) {};
\node [prob,minimum size=0.5cm,anchor=center] (prob21) at ([yshift=-0.5cm]prob11.center) {$1.$};
......@@ -96,6 +101,7 @@
\coordinate (topright) at ([shift={(0.25cm,0.25cm)}]prob11.center);
\node [draw,fit=(prob11) (prob21) (prob31) (prob41) (prob51) (topright) (bottomleft)] (prob1) {};
\end{pgfonlayer}
\node [anchor=center] (word11) at ([yshift=0.7cm]prob11.center) {I};
% Column 2
\node [prob,minimum size=0.1cm,anchor=center] (prob12) at ([xshift=1cm]prob11.center) {};
......@@ -108,6 +114,7 @@
\coordinate (topright) at ([shift={(0.25cm,0.25cm)}]prob12.center);
\node [draw,fit=(prob12) (prob22) (prob32) (prob42) (prob52) (topright) (bottomleft)] (prob2) {};
\end{pgfonlayer}
\node [anchor=center] (word12) at ([yshift=0.68cm]prob12.center) {am};
% Column 3
\node [prob,minimum size=0.1cm,anchor=center] (prob13) at ([xshift=1cm]prob12.center) {};
......@@ -120,6 +127,7 @@
\coordinate (topright) at ([shift={(0.25cm,0.25cm)}]prob13.center);
\node [draw,fit=(prob13) (prob23) (prob33) (prob43) (prob53) (topright) (bottomleft)] (prob3) {};
\end{pgfonlayer}
\node [anchor=center] (word13) at ([yshift=0.68cm]prob13.center) {good};
% Column 4
\node [prob,minimum size=0.5cm,anchor=center] (prob14) at ([xshift=1cm]prob13.center) {$1.$};
......@@ -132,9 +140,10 @@
\coordinate (topright) at ([shift={(0.25cm,0.25cm)}]prob14.center);
\node [draw,fit=(prob14) (prob24) (prob34) (prob44) (prob54) (topright) (bottomleft)] (prob4) {};
\end{pgfonlayer}
\node [anchor=center] (word14) at ([yshift=0.68cm]prob14.center) {$\langle$eos$\rangle$};
% Label
\draw [decorate,decoration={brace}] ([yshift=0.1cm]prob1.north west) to node [midway,above,font=\small] {学习目标(Teacher输出)} ([yshift=0.1cm]prob4.north east);
\draw [decorate,decoration={brace}] ([xshift=0.1cm]prob4.north east) to node [midway,right,font=\small,align=center] {教师\\模型\\输出\\译文} ([xshift=0.1cm]prob4.south east);
% Vocab
\node [word,anchor=center] () at ([xshift=-0.9cm]prob11.center) {$\langle$eos$\rangle$};
......@@ -155,16 +164,17 @@
\draw [-latex,thick] (prob\i.south) to ([yshift=-0.5cm]prob\i.south);
% Input
\node [word,anchor=south] (input1) at ([yshift=-1.8cm]prob1.south) {$\langle$eos$\rangle$};
\node [word,anchor=south] (input1) at ([yshift=-1.85cm]prob1.south) {$\langle$eos$\rangle$};
\node [word,anchor=south] (input2) at ([yshift=-1.8cm]prob2.south) {I};
\node [word,anchor=south] (input3) at ([yshift=-1.8cm]prob3.south) {am};
\node [word,anchor=center] (input4) at ([xshift=1cm]input3.center) {good};
\foreach \i in {1,2,3}
\foreach \i in {2,3}
\draw [-latex,thick] ([yshift=0.3cm]input\i.south) to ([yshift=0.8cm]input\i.south);
\draw [-latex,thick] ([yshift=0.36cm]input4.south) to ([yshift=0.86cm]input4.south);
\draw [-latex,thick] ([yshift=0.36cm]input1.south) to ([yshift=0.86cm]input1.south);
\node [word,anchor=south] (ns) at ([xshift=-1cm]input1.south) {输入:};
\node [word,anchor=south] (ns) at ([xshift=-1.2cm]input1.south) {教师译文:};
\node [word,anchor=north] () at ([xshift=2.1cm,yshift=-0.5cm]ns.south) {(b)\ Sequence-level };
\end{scope}
\end{tikzpicture}
Chapter13/Figures/figure-unk-of-bpe.tex
......@@ -3,24 +3,28 @@
\node[rounded corners=3pt,minimum width=1.0em,minimum height=2.0em,font=\scriptsize,fill=red!10,drop shadow,thick](top) at (0,0) {
\begin{tabular}{lllllll}
\multicolumn{7}{c}{符号合并表} \\
(r,$<$e$>$), & (e,s), & (l,o), & (es,t), & (lo,w), & (est,$<$e$>$), & (e,r$<$e$>$),
(r,$<$e$>$), & (e,s), & (l,o), & (es,t), & (lo,w), & (est,$<$e$>$), & (e,r$<$e$>$)
\end{tabular}
};
\node[font=\footnotesize,anchor=north] (l1) at ([xshift=0em,yshift=-1em]top.south) {(a) 符号合并表};
\node[font=\scriptsize,anchor=west] (n1) at ([xshift=-6em,yshift=-6em]top.west) {l\ o\ w\ e\ r\ $<$e$>$};
\node[font=\scriptsize,anchor=west] (n1) at ([xshift=-4.5em,yshift=-6em]top.west) {l\ o\ w\ e\ r\ $<$e$>$};
\node[font=\scriptsize,anchor=west] (n2) at ([xshift=2.6em]n1.east) {l\ o\ w\ e\ {\red r$<$e$>$}};
\node[font=\scriptsize,anchor=west] (n3) at ([xshift=2.6em]n2.east) {{\red lo}\ w\ e\ r$<$e$>$};
\node[font=\scriptsize,anchor=west] (n4) at ([xshift=2.6em]n3.east) {{\red low}\ e\ r$<$e$>$};
\node[font=\scriptsize,anchor=west] (n5) at ([xshift=2.6em]n4.east) {low\ {\red er$<$e$>$}};
\node[font=\scriptsize,anchor=west] (t1) at ([yshift=-1.5em]n1.south west) {l\ o\ w\ e\ s\ t\ $<$e$>$};
\node[font=\scriptsize,anchor=west] (t1) at ([yshift=-2.5em]n1.south west) {l\ o\ w\ e\ s\ t\ $<$e$>$};
\node[font=\scriptsize,anchor=west] (t2) at ([xshift=0.8em]t1.east) {l\ o\ w\ {\red es}\ t\ $<$e$>$};
\node[font=\scriptsize,anchor=west] (t3) at ([xshift=0.8em]t2.east) {{\red lo}\ w\ es\ t\ $<$e$>$};
\node[font=\scriptsize,anchor=west] (t4) at ([xshift=0.8em]t3.east) {lo\ w\ {\red est}\ $<$e$>$};
\node[font=\scriptsize,anchor=west] (t5) at ([xshift=0.8em]t4.east) {{\red low}\ est\ $<$e$>$};
\node[font=\scriptsize,anchor=west] (t6) at ([xshift=0.8em]t5.east) {low\ {\red est$<$e$>$}};
\node[font=\footnotesize,anchor=north] (l2) at ([xshift=1.5em,yshift=-1em]t3.south) {(b) 合并样例};
\node[font=\scriptsize,anchor=north east] (s1) at ([yshift=0.1em]n1.north west) {样例1:};
\node[font=\scriptsize,anchor=north east] (s1) at ([yshift=0.1em]t1.north west) {样例2:};
\node[font=\footnotesize,anchor=north] (l2) at ([xshift=2em,yshift=-1em]t3.south) {(b) 合并样例};
\draw[->,thick](n1.east) -- (n2.west);
\draw[->,thick](n2.east) -- (n3.west);
......
Chapter14/Figures/figure-3vs.tex
......@@ -38,7 +38,7 @@
\draw [->,thick]([yshift=-1.2em]eos.south) to (eos.south);
\end{scope}
\begin{scope}[yshift=-1.55in]
\begin{scope}[yshift=-1.45in]
\node (encoder) at (0,0) {来自编码器的信息};
\node (aa)[decoder,anchor=east] at ([xshift=5.5cm]encoder.east) {};
\node (y1y2b)[rectangle,anchor=south,inner sep=0.25em,densely dashed,draw] at ([yshift=-2.6em]aa.south) {$y_1\;y_2$};
......@@ -66,13 +66,13 @@
\draw [->,thick]([yshift=-1.4em]eos.south) to ([yshift=-0.05em]eos.south);
\end{scope}
\begin{scope}[yshift=-3.1in]
\begin{scope}[yshift=-2.9in]
\node (encoder) at (0,0) {来自编码器的信息};
\node (aa)[decoder,anchor=east] at ([xshift=5.5cm]encoder.east) {非自回归解码器};
%\node (encoder)[decoder,anchor=west,fill=red!20] at ([xshift=-2cm]aa.west) {编码器};
\draw [->,thick](encoder.east) to (aa.west);
\node (label)[anchor=south] at ([xshift=-4em,yshift=-4.3em]aa.south) {\small{(c) 非自回归解码}};
\node (label)[anchor=south] at ([xshift=-4em,yshift=-2.3em]aa.south) {\small{(c) 非自回归解码}};
\node (y2a)[anchor=north] at ([xshift=-1.5em,yshift=2.5em]aa.north) {$y_2$};
\node (y1a)[anchor=east] at ([xshift=-3em]y2a.east) {$y_1$};
......
Chapter15/Figures/figure-dynamic-linear-aggregation-network-structure.tex
......@@ -6,21 +6,21 @@
\node [anchor=west,rectangle, inner sep=0mm,minimum height=1.2em,minimum width=0em,rounded corners=5pt,thick] (n2) at ([xshift=3.5em,yshift=-0.5em]n1.east) {$\mathbi{X}$};
\node [anchor=west,rectangle,draw, inner sep=0mm,minimum height=1.2em,minimum width=3em,fill=orange!20,rounded corners=5pt,thick] (n3) at ([xshift=3.5em,yshift=0em]n2.east) {$\mathbi{x}_1$};
\node [anchor=west,rectangle,draw, inner sep=0mm,minimum height=1.2em,minimum width=3em,fill=orange!30,rounded corners=5pt,thick] (n3) at ([xshift=3.5em,yshift=0em]n2.east) {$\mathbi{x}_1$};
\node [anchor=west,rectangle,draw, inner sep=0mm,minimum height=1.2em,minimum width=3em,fill=orange!20,rounded corners=5pt,thick] (n4) at ([xshift=3.5em,yshift=0em]n3.east) {$\mathbi{x}_2$};
\node [anchor=west,rectangle,draw, inner sep=0mm,minimum height=1.2em,minimum width=3em,fill=orange!30,rounded corners=5pt,thick] (n4) at ([xshift=3.5em,yshift=0em]n3.east) {$\mathbi{x}_2$};
\node [anchor=west,rectangle, inner sep=0mm,minimum height=1.2em,minimum width=1em,rounded corners=5pt,thick] (n6) at ([xshift=1.5em,yshift=0em]n4.east) {$\ldots$};
\node [anchor=west,rectangle,draw, inner sep=0mm,minimum height=1.2em,minimum width=3em,fill=orange!20,rounded corners=5pt,thick] (n5) at ([xshift=3.5em,yshift=0em]n6.east) {$\mathbi{x}_l$};
\node [anchor=west,rectangle,draw, inner sep=0mm,minimum height=1.2em,minimum width=3em,fill=orange!30,rounded corners=5pt,thick] (n5) at ([xshift=3.5em,yshift=0em]n6.east) {$\mathbi{x}_l$};
\node [anchor=west,rectangle,draw, inner sep=0mm,minimum height=1.2em,minimum width=3em,fill=orange!20,rounded corners=5pt,thick] (n7) at ([xshift=1.5em,yshift=0em]n5.east) {$\mathbi{x}_{l+1}$};
\node [anchor=west,rectangle,draw, inner sep=0mm,minimum height=1.2em,minimum width=3em,fill=orange!30,rounded corners=5pt,thick] (n7) at ([xshift=1.5em,yshift=0em]n5.east) {$\mathbi{x}_{l+1}$};
\node [anchor=north,rectangle,draw, inner sep=0mm,minimum height=1.2em,minimum width=15em,fill=teal!17,rounded corners=5pt,thick] (n8) at ([xshift=0em,yshift=-3em]n4.south) {层正则化};
\node [anchor=north,rectangle,draw, inner sep=0mm,minimum height=1.2em,minimum width=15em,fill=teal!30,rounded corners=5pt,thick] (n8) at ([xshift=0em,yshift=-3em]n4.south) {层正则化};
\node [anchor=north,rectangle,draw, inner sep=0mm,minimum height=1.2em,minimum width=15em,fill=purple!17,rounded corners=5pt,thick] (n9) at ([xshift=0em,yshift=-1em]n8.south) {$\mathbi{X}\ \quad \mathbi{h}^1\ \quad \mathbi{h}^2\quad \ldots \quad\ \mathbi{h}^l$};
\node [anchor=north,rectangle,draw, inner sep=0mm,minimum height=1.2em,minimum width=15em,fill=purple!30,rounded corners=5pt,thick] (n9) at ([xshift=0em,yshift=-1em]n8.south) {$\mathbi{X}\ \quad \mathbi{h}^1\ \quad \mathbi{h}^2\quad \ldots \quad\ \mathbi{h}^l$};
\node [anchor=north,rectangle,draw, inner sep=0mm,minimum height=1.2em,minimum width=15em,fill=teal!17,rounded corners=5pt,thick] (n10) at ([xshift=0em,yshift=-2em]n9.south) {权重累加};
\node [anchor=north,rectangle,draw, inner sep=0mm,minimum height=1.2em,minimum width=15em,fill=teal!30,rounded corners=5pt,thick] (n10) at ([xshift=0em,yshift=-2em]n9.south) {权重累加};
\node [anchor=west,rectangle, inner sep=0mm,minimum height=1.2em, rounded corners=5pt,thick] (n11) at ([xshift=0em,yshift=-4.5em]n1.west) {聚合网络};
......
Chapter15/Figures/figure-encoder-of-bidirectional-tree-structure.tex
......@@ -2,10 +2,10 @@
\begin{tikzpicture}
\begin{scope}
\tikzstyle{hnode}=[rectangle,inner sep=0mm,minimum height=2em,minimum width=3em,rounded corners=5pt,fill=green!20]
\tikzstyle{tnode}=[rectangle,inner sep=0mm,minimum height=2em,minimum width=3em,rounded corners=5pt,fill=red!20]
\tikzstyle{fnoder}=[rectangle,inner sep=0mm,minimum height=2.4em,minimum width=6.8em,draw,dashed,very thick,rounded corners=5pt,red!40]
\tikzstyle{fnodeg}=[rectangle,inner sep=0mm,minimum height=2.4em,minimum width=6.8em,draw,dashed,very thick,rounded corners=5pt,green!40]
\tikzstyle{hnode}=[rectangle,inner sep=0mm,minimum height=2em,minimum width=3em,rounded corners=5pt,fill=green!30,draw,thick]
\tikzstyle{tnode}=[rectangle,inner sep=0mm,minimum height=2em,minimum width=3em,rounded corners=5pt,fill=red!30,draw,thick]
\tikzstyle{fnoder}=[rectangle,inner sep=0em,minimum height=2.6em,minimum width=6.8em,draw,dashed,very thick,rounded corners=5pt,red!50]
\tikzstyle{fnodeg}=[rectangle,inner sep=0em,minimum height=2.6em,minimum width=6.8em,draw,dashed,very thick,rounded corners=5pt,green!50]
\node [anchor=south west,fnodeg] (f1) at (0,0) {};
\node [anchor=west,hnode] (n1) at ([xshift=0.2em,yshift=0em]f1.west) {$\mathbi{h}_1^{\textrm{up}}$};
......
Chapter15/Figures/figure-encoder-structure-of-transformer-model-optimized-by-nas.tex
\begin{tikzpicture}
\tikzstyle{unit}=[draw,rounded corners=2pt,drop shadow,font=\tiny]
\tikzstyle{unit}=[draw,rounded corners=2pt,drop shadow,font=\tiny,thick]
%left
\begin{scope}
\foreach \x/\d in {1/2em, 2/8em}
\node[unit,fill=yellow!20] at (0,\d) (ln_\x) {层正则化};
\node[unit,fill=yellow!30] at (0,\d) (ln_\x) {层正则化};
\foreach \x/\d in {1/4em}
\node[unit,fill=green!20] at (0,\d) (sa_\x) {8头自注意力：512};
\node[unit,fill=green!30] at (0,\d) (sa_\x) {8头自注意力：512};
\foreach \x/\d in {1/6em, 2/16em}
\node[draw,circle,minimum size=1em,inner sep=1pt] at (0,\d) (add_\x) {\scriptsize\bfnew{+}};
\foreach \x/\d in {2/14em}
\node[unit,fill=red!20] at (0,\d) (conv_\x) {卷积$1 \times 1$：512};
\node[unit,fill=red!30] at (0,\d) (conv_\x) {卷积$1 \times 1$：512};
\foreach \x/\d in {1/10em}
\node[unit,fill=red!20] at (0,\d) (conv_\x) {卷积$1 \times 1$：2048};
\node[unit,fill=red!30] at (0,\d) (conv_\x) {卷积$1 \times 1$：2048};
\foreach \x/\d in {1/12em}
\node[unit,fill=blue!20] at (0,\d) (relu_\x) {RELU};
\node[unit,fill=blue!30] at (0,\d) (relu_\x) {RELU};
\draw[->,thick] ([yshift=-1.4em]ln_1.-90) -- ([yshift=-0.1em]ln_1.-90);
\draw[->,thick] ([yshift=0.1em]ln_1.90) -- ([yshift=-0.1em]sa_1.-90);
......@@ -44,17 +44,17 @@
\foreach \x/\d in {1/2em, 2/8em, 3/16em}
\node[unit,fill=yellow!20] at (0,\d) (ln_\x) {层正则化};
\node[unit,fill=yellow!30] at (0,\d) (ln_\x) {层正则化};
\foreach \x/\d in {1/6em, 2/14em, 3/20em}
\node[draw,circle,minimum size=1em,inner sep=1pt] at (0,\d) (add_\x) {\scriptsize\bfnew{+}};
\node[unit,fill=red!20] at (0,4em) (glu_1) {门控线性单元：512};
\node[unit,fill=red!20] at (-3em,10em) (conv_1) {卷积$1 \times 1$：2048};
\node[unit,fill=cyan!20] at (3em,10em) (conv_2) {卷积$3 \times 1$：256};
\node[unit,fill=blue!20] at (-3em,12em) (relu_1) {RELU};
\node[unit,fill=blue!20] at (3em,12em) (relu_2) {RELU};
\node[unit,fill=cyan!20] at (0em,18em) (conv_3) {Sep卷积$9 \times 1$：256};
\node[unit,fill=red!30] at (0,4em) (glu_1) {门控线性单元：512};
\node[unit,fill=red!30] at (-3em,10em) (conv_1) {卷积$1 \times 1$：2048};
\node[unit,fill=cyan!30] at (3em,10em) (conv_2) {卷积$3 \times 1$：256};
\node[unit,fill=blue!30] at (-3em,12em) (relu_1) {RELU};
\node[unit,fill=blue!30] at (3em,12em) (relu_2) {RELU};
\node[unit,fill=cyan!30] at (0em,18em) (conv_3) {Sep卷积$9 \times 1$：256};
\draw[->,thick] ([yshift=-1.4em]ln_1.-90) -- ([yshift=-0.1em]ln_1.-90);
......@@ -81,15 +81,15 @@
\node[font=\scriptsize,align=center] at (0em, -1.5em){(b) 使用结构搜索方法优化后的 \\ Transformer编码器中若干块的结构};
\node[minimum size=0.8em,inner sep=0pt,rounded corners=1pt,draw,fill=blue!20] (act) at (8em, 20em){};
\node[minimum size=0.8em,inner sep=0pt,rounded corners=1pt,draw,fill=blue!30] (act) at (8em, 20em){};
\node[anchor=west,font=\footnotesize] at ([xshift=0.1em]act.east){激活函数};
\node[anchor=north,minimum size=0.8em,inner sep=0pt,rounded corners=1pt,draw,fill=yellow!20] (nor) at ([yshift=-0.6em]act.south){};
\node[anchor=north,minimum size=0.8em,inner sep=0pt,rounded corners=1pt,draw,fill=yellow!30] (nor) at ([yshift=-0.6em]act.south){};
\node[anchor=west,font=\footnotesize] at ([xshift=0.1em]nor.east){层正则化};
\node[anchor=north,minimum size=0.8em,inner sep=0pt,rounded corners=1pt,draw,fill=cyan!20] (wc) at ([yshift=-0.6em]nor.south){};
\node[anchor=north,minimum size=0.8em,inner sep=0pt,rounded corners=1pt,draw,fill=cyan!30] (wc) at ([yshift=-0.6em]nor.south){};
\node[anchor=west,font=\footnotesize] at ([xshift=0.1em]wc.east){宽卷积};
\node[anchor=north,minimum size=0.8em,inner sep=0pt,rounded corners=1pt,draw,fill=green!20] (at) at ([yshift=-0.6em]wc.south){};
\node[anchor=north,minimum size=0.8em,inner sep=0pt,rounded corners=1pt,draw,fill=green!30] (at) at ([yshift=-0.6em]wc.south){};
\node[anchor=west,font=\footnotesize] (tag) at ([xshift=0.1em]at.east){注意力机制};
\node[anchor=north,minimum size=0.8em,inner sep=0pt,rounded corners=1pt,draw,fill=red!20] (nsl) at ([yshift=-0.6em]at.south){};
\node[anchor=north,minimum size=0.8em,inner sep=0pt,rounded corners=1pt,draw,fill=red!30] (nsl) at ([yshift=-0.6em]at.south){};
\node[anchor=west,font=\footnotesize] at ([xshift=0.1em]nsl.east){非空间层};
\begin{pgfonlayer}{background}
......
Chapter15/Figures/figure-encoder-tree-structure-modeling.tex
......@@ -2,20 +2,20 @@
\begin{tikzpicture}
\begin{scope}
\tikzstyle{hnode}=[rectangle,inner sep=0mm,minimum height=2em,minimum width=4.5em,rounded corners=5pt,fill=green!30]
\tikzstyle{tnode}=[rectangle,inner sep=0mm,minimum height=2em,minimum width=4.5em,rounded corners=5pt,fill=red!30]
\tikzstyle{hnode}=[rectangle,inner sep=0mm,minimum height=2em,minimum width=4.5em,rounded corners=5pt,fill=green!30,draw,thick]
\tikzstyle{tnode}=[rectangle,inner sep=0mm,minimum height=2em,minimum width=4.5em,rounded corners=5pt,fill=red!30,draw,thick]
\tikzstyle{wnode}=[inner sep=0mm,minimum height=1.4em,minimum width=4.4em]
\node [anchor=south west,hnode] (n1) at (0,0) {$\mathbi{h}_1$};
\node [anchor=west,hnode] (n2) at ([xshift=1em,yshift=0em]n1.east) {$\mathbi{h}_2$};
\node [anchor=west,hnode] (n3) at ([xshift=1em,yshift=0em]n2.east) {$\mathbi{h}_3$};
\node [anchor=west,hnode] (n4) at ([xshift=1em,yshift=0em]n3.east) {$\cdots$};
\node [anchor=west,hnode] (n5) at ([xshift=1em,yshift=0em]n4.east) {$\mathbi{h}_m$};
\node [anchor=south,tnode] (t1) at ([xshift=2.8em,yshift=1em]n1.north) {$\mathbi{h}_{m+1}$};
\node [anchor=south,tnode] (t2) at ([xshift=2.8em,yshift=1em]t1.north) {$\mathbi{h}_{m+2}$};
\node [anchor=south,tnode] (t3) at ([xshift=2.8em,yshift=1em]t2.north) {$\cdots$};
\node [anchor=south,tnode] (t4) at ([xshift=2.8em,yshift=1em]t3.north) {$\mathbi{h}_{2m-1}$};
\node [anchor=west,hnode] (n2) at ([xshift=2em,yshift=0em]n1.east) {$\mathbi{h}_2$};
\node [anchor=west,hnode] (n3) at ([xshift=2em,yshift=0em]n2.east) {$\mathbi{h}_3$};
\node [anchor=west,hnode] (n4) at ([xshift=2em,yshift=0em]n3.east) {$\cdots$};
\node [anchor=west,hnode] (n5) at ([xshift=2em,yshift=0em]n4.east) {$\mathbi{h}_m$};
\node [anchor=south,tnode] (t1) at ([xshift=3.3em,yshift=1em]n1.north) {$\mathbi{h}_{m+1}$};
\node [anchor=south,tnode] (t2) at ([xshift=3.3em,yshift=1em]t1.north) {$\mathbi{h}_{m+2}$};
\node [anchor=south,tnode] (t3) at ([xshift=3.3em,yshift=1em]t2.north) {$\cdots$};
\node [anchor=south,tnode] (t4) at ([xshift=3.3em,yshift=1em]t3.north) {$\mathbi{h}_{2m-1}$};
\draw [->,thick] ([xshift=0em,yshift=0em]n1.east) -- ([xshift=0em,yshift=0em]n2.west);
\draw [->,thick] ([xshift=0em,yshift=0em]n2.east) -- ([xshift=0em,yshift=0em]n3.west);
......
Chapter15/Figures/figure-evolution-and-change-of-ml-methods.tex
\begin{tikzpicture}
\tikzstyle{opnode}=[rectangle,inner sep=0mm,minimum height=2em,minimum width=4em,rounded corners=5pt,fill=teal!17]
\tikzstyle{opnode}=[rectangle,inner sep=0mm,minimum height=2em,minimum width=4em,rounded corners=5pt,fill=teal!30]
\tikzstyle{cnode}=[circle,draw,minimum size=1.2em]
\tikzstyle{mnode}=[rectangle,inner sep=0mm,minimum height=5em,minimum width=11em,rounded corners=5pt,fill=yellow!20]
\tikzstyle{mnode}=[rectangle,inner sep=0mm,minimum height=5em,minimum width=11em,rounded corners=5pt,fill=yellow!30]
\tikzstyle{wnode}=[inner sep=0mm,minimum height=1.5em]
\begin{pgfonlayer}{background}
......@@ -10,14 +10,14 @@
\node[anchor=west,mnode] (m2) at ([xshift=1em,yshift=0em]m1.east){};
\node[anchor=west,mnode] (m3) at ([xshift=1em,yshift=0em]m2.east){};
\node[anchor=north west,rectangle,inner sep=0mm,minimum height=2.6em,minimum width=3.5em,rounded corners=5pt,fill=blue!20] (ml1) at ([xshift=0em,yshift=-0.5em]m1.south west){};
\node[anchor=west,rectangle,inner sep=0mm,minimum height=2.6em,minimum width=3.5em,rounded corners=5pt,fill=ugreen!20] (ml2) at ([xshift=0.25em,yshift=0em]ml1.east){};
\node[anchor=north east,rectangle,inner sep=0mm,minimum height=2.6em,minimum width=3.5em,rounded corners=5pt,fill=red!20] (ml3) at ([xshift=0em,yshift=-0.5em]m1.south east){};
\node[anchor=north west,rectangle,inner sep=0mm,minimum height=2.6em,minimum width=3.5em,rounded corners=5pt,fill=blue!30] (ml1) at ([xshift=0em,yshift=-0.5em]m1.south west){};
\node[anchor=west,rectangle,inner sep=0mm,minimum height=2.6em,minimum width=3.5em,rounded corners=5pt,fill=green!30] (ml2) at ([xshift=0.25em,yshift=0em]ml1.east){};
\node[anchor=north east,rectangle,inner sep=0mm,minimum height=2.6em,minimum width=3.5em,rounded corners=5pt,fill=red!30] (ml3) at ([xshift=0em,yshift=-0.5em]m1.south east){};
\node[anchor=north west,rectangle,inner sep=0mm,minimum height=2.6em,minimum width=5.25em,rounded corners=5pt,fill=blue!20] (mc1) at ([xshift=0em,yshift=-0.5em]m2.south west){};
\node[anchor=north east,rectangle,inner sep=0mm,minimum height=2.6em,minimum width=5.25em,rounded corners=5pt,fill=red!20] (mc2) at ([xshift=0em,yshift=-0.5em]m2.south east){};
\node[anchor=north west,rectangle,inner sep=0mm,minimum height=2.6em,minimum width=5.25em,rounded corners=5pt,fill=blue!30] (mc1) at ([xshift=0em,yshift=-0.5em]m2.south west){};
\node[anchor=north east,rectangle,inner sep=0mm,minimum height=2.6em,minimum width=5.25em,rounded corners=5pt,fill=red!30] (mc2) at ([xshift=0em,yshift=-0.5em]m2.south east){};
\node[anchor=north,rectangle,inner sep=0mm,minimum height=2.6em,minimum width=11em,rounded corners=5pt,fill=blue!20] (mr1) at ([xshift=0em,yshift=-0.5em]m3.south){};
\node[anchor=north,rectangle,inner sep=0mm,minimum height=2.6em,minimum width=11em,rounded corners=5pt,fill=blue!30] (mr1) at ([xshift=0em,yshift=-0.5em]m3.south){};
\end{pgfonlayer}
......
Chapter15/Figures/figure-introducing-rnn-mechanism-into-transformer.tex 0 → 100644
%%%------------------------------------------------------------------------------------------------------------
%%% 调序模型1：基于距离的调序
\begin{center}
\begin{tikzpicture}
\tikzstyle{tnode}=[rectangle,inner sep=0mm,minimum height=1.8em,minimum width=4em,rounded corners=5pt,thick,draw,fill=teal!20]
\tikzstyle{stnode}=[rectangle,inner sep=0mm,minimum height=1.8em,minimum width=4em,rounded corners=5pt,thick,fill=red!20,draw]
\tikzstyle{rnnnode}=[rectangle,inner sep=0mm,minimum height=1.8em,minimum width=5em,rounded corners=5pt,thick,fill=green!20,draw]
\tikzstyle{wnode}=[inner sep=0mm,minimum height=2em,minimum width=4em]
\begin{scope}[]
\node [anchor=west,rnnnode] (r1) at (0, 0) {循环单元};
\node [anchor=south,rnnnode] (r2) at ([xshift=0em,yshift=1em]r1.north){循环单元};
\node [anchor=south] (r3) at ([xshift=0em,yshift=1em]r2.north){$\cdots$};
\node [anchor=south,rnnnode] (r4) at ([xshift=0em,yshift=1em]r3.north){循环单元};
\node [anchor=east,wnode,font=\footnotesize,align=left,minimum width=3em] (wr1) at ([xshift=-1em,yshift=0em]r1.west) {1时刻\\ 输入};
\node [anchor=east,wnode,font=\footnotesize,align=left,minimum width=3em] (wr2) at ([xshift=-1em,yshift=0em]r2.west) {2时刻\\ 输入};
\node [anchor=east,wnode,font=\footnotesize,align=left,minimum width=3em] (wr3) at ([xshift=-1em,yshift=0em]r4.west) {$n$时刻\\ 输入};
\node [anchor=north,wnode] (input) at ([xshift=0em,yshift=-1em]r1.south) {$\mathbi{h}_0$};
\node [anchor=south,wnode] (output) at ([xshift=0em,yshift=1em]r4.north) {输出};
\draw[->,very thick] ([xshift=-0.3em,yshift=-1em]wr1.west)--([xshift=-0.3em,yshift=1em]wr3.west);
\draw[->,thick] ([xshift=0em,yshift=0em]input.north)--([xshift=0em,yshift=0em]r1.south);
\draw[->,thick] ([xshift=0em,yshift=0em]r1.north)--([xshift=0em,yshift=0em]r2.south);
\draw[->,thick] ([xshift=0em,yshift=0em]r2.north)--([xshift=0em,yshift=0em]r3.south);
\draw[->,thick] ([xshift=0em,yshift=0em]r3.north)--([xshift=0em,yshift=0em]r4.south);
\draw[->,thick] ([xshift=0em,yshift=0em]r4.north)--([xshift=0em,yshift=0em]output.south);
\draw[->,thick] ([xshift=0em,yshift=0em]wr1.east)--([xshift=0em,yshift=0em]r1.west);
\draw[->,thick] ([xshift=0em,yshift=0em]wr2.east)--([xshift=0em,yshift=0em]r2.west);
\draw[->,thick] ([xshift=0em,yshift=0em]wr3.east)--([xshift=0em,yshift=0em]r4.west);
\node [anchor=north,font=\small] (label) at ([xshift=-3em,yshift=-0.7em]input.south) {(a)RNN};
\end{scope}
\begin{scope}[xshift=1.85in]
\node [anchor=west,stnode] (r1) at (0, 0) {第1层};
\node [anchor=south,tnode] (r2) at ([xshift=0em,yshift=1em]r1.north){第2层};
\node [anchor=south] (r3) at ([xshift=0em,yshift=1em]r2.north){$\cdots$};
\node [anchor=south,tnode] (r4) at ([xshift=0em,yshift=1em]r3.north){第$n$层};
\node [anchor=east,wnode,font=\footnotesize,minimum width=3em] (wr1) at ([xshift=-0.3em,yshift=0em]r1.west) {1时刻};
\node [anchor=east,wnode,font=\footnotesize,minimum width=3em] (wr2) at ([xshift=-0.3em,yshift=0em]r2.west) {2时刻};
\node [anchor=east,wnode,font=\footnotesize,minimum width=3em] (wr3) at ([xshift=-0.3em,yshift=0em]r4.west) {$n$时刻};
\node [anchor=north,wnode] (input) at ([xshift=0em,yshift=-1em]r1.south) {$\mathbi{h}_0$};
\node [anchor=south,wnode] (output) at ([xshift=0em,yshift=1em]r4.north) {输出};
\draw[->,very thick] ([xshift=-0.3em,yshift=-1em]wr1.west)--([xshift=-0.3em,yshift=1em]wr3.west);
\draw[->,thick] ([xshift=0em,yshift=0em]input.north)--([xshift=0em,yshift=0em]r1.south);
\draw[->,thick] ([xshift=0em,yshift=0em]r1.north)--([xshift=0em,yshift=0em]r2.south);
\draw[->,thick] ([xshift=0em,yshift=0em]r2.north)--([xshift=0em,yshift=0em]r3.south);
\draw[->,thick] ([xshift=0em,yshift=0em]r3.north)--([xshift=0em,yshift=0em]r4.south);
\draw[->,thick] ([xshift=0em,yshift=0em]r4.north)--([xshift=0em,yshift=0em]output.south);
\node [anchor=north,font=\small] (label) at ([xshift=-1.5em,yshift=-0.7em]input.south) {(b)原始Transformer模型};
\end{scope}
\begin{scope}[xshift=3.9in]
\node [anchor=west,stnode] (r1) at (0, 0) {第1层};
\node [anchor=south,stnode] (r2) at ([xshift=0em,yshift=1em]r1.north){第2层};
\node [anchor=south] (r3) at ([xshift=0em,yshift=1em]r2.north){$\cdots$};
\node [anchor=south,stnode] (r4) at ([xshift=0em,yshift=1em]r3.north){第$n$层};
\node [anchor=east,wnode,font=\footnotesize,align=left] (wr1) at ([xshift=-1em,yshift=0em]r1.west) {1时刻\\ 编码向量};
\node [anchor=east,wnode,font=\footnotesize,align=left] (wr2) at ([xshift=-1em,yshift=0em]r2.west) {2时刻\\ 编码向量};
\node [anchor=east,wnode,font=\footnotesize,align=left] (wr3) at ([xshift=-1em,yshift=0em]r4.west) {$n$时刻\\ 编码向量};
\node [anchor=north,wnode] (input) at ([xshift=0em,yshift=-1em]r1.south) {$\mathbi{h}_0$};
\node [anchor=south,wnode] (output) at ([xshift=0em,yshift=1em]r4.north) {输出};
\draw[->,very thick] ([xshift=-0.3em,yshift=-1em]wr1.west)--([xshift=-0.3em,yshift=1em]wr3.west);
\draw[->,thick] ([xshift=0em,yshift=0em]input.north)--([xshift=0em,yshift=0em]r1.south);
\draw[->,thick] ([xshift=0em,yshift=0em]r1.north)--([xshift=0em,yshift=0em]r2.south);
\draw[->,thick] ([xshift=0em,yshift=0em]r2.north)--([xshift=0em,yshift=0em]r3.south);
\draw[->,thick] ([xshift=0em,yshift=0em]r3.north)--([xshift=0em,yshift=0em]r4.south);
\draw[->,thick] ([xshift=0em,yshift=0em]r4.north)--([xshift=0em,yshift=0em]output.south);
\draw[->,thick] ([xshift=0em,yshift=0em]wr1.east)--([xshift=0em,yshift=0em]r1.west);
\draw[->,thick] ([xshift=0em,yshift=0em]wr2.east)--([xshift=0em,yshift=0em]r2.west);
\draw[->,thick] ([xshift=0em,yshift=0em]wr3.east)--([xshift=0em,yshift=0em]r4.west);
\node [anchor=north,font=\small,align=left] (label) at ([xshift=-3em,yshift=-0.7em]input.south) {(c)共享权重的\\ Transformer模型};
\end{scope}
\end{tikzpicture}
\end{center}
\ No newline at end of file
Chapter15/Figures/figure-layer-fusion-method-2d.tex
......@@ -2,9 +2,9 @@
\begin{tikzpicture}
\begin{scope}
\tikzstyle{lnode}=[rectangle,inner sep=0mm,minimum height=1.5em,minimum width=3.5em,rounded corners=2pt,draw]
\tikzstyle{snode}=[rectangle,inner sep=0mm,minimum height=1.5em,minimum width=0.8em,rounded corners=2pt,draw]
\tikzstyle{vlnode}=[rectangle,inner sep=0mm,minimum height=1em,minimum width=5em,rounded corners=2pt,draw]
\tikzstyle{lnode}=[rectangle,inner sep=0mm,minimum height=1.5em,minimum width=3em,rounded corners=2pt,draw,thick]
\tikzstyle{snode}=[rectangle,inner sep=0mm,minimum height=1.5em,minimum width=0.8em,rounded corners=2pt,draw,thick]
\tikzstyle{vlnode}=[rectangle,inner sep=0mm,minimum height=1em,minimum width=4em,rounded corners=2pt,draw,thick]
\node [anchor=west,lnode] (n1) at (0, 0) {$\mathbi{h}^3$};
......@@ -13,53 +13,53 @@
\node [anchor=south] (d1) at ([xshift=0em,yshift=0.2em]n1.north) {1D};
\node [anchor=west,lnode] (n4) at ([xshift=1.2em,yshift=0em]n1.east) {};
\node [anchor=west,lnode] (n5) at ([xshift=1.2em,yshift=0em]n2.east) {};
\node [anchor=west,lnode] (n6) at ([xshift=1.2em,yshift=0em]n3.east) {};
\node [anchor=west,lnode] (n4) at ([xshift=1em,yshift=0em]n1.east) {};
\node [anchor=west,lnode] (n5) at ([xshift=1em,yshift=0em]n2.east) {};
\node [anchor=west,lnode] (n6) at ([xshift=1em,yshift=0em]n3.east) {};
\node [anchor=south,lnode] (n7) at ([xshift=0em,yshift=1em]n4.north) {$\mathbi{W}_1$};
\node [anchor=west] (sig) at ([xshift=0em,yshift=0.4em]n5.east) {$\sigma$};
\node [anchor=west,snode,fill=purple!30] (nc11) at ([xshift=1.2em,yshift=0em]n4.east) {};
\node [anchor=west,snode,fill=yellow!30] (nc12) at ([xshift=0em,yshift=0em]nc11.east) {};
\node [anchor=west,snode,fill=red!30] (nc13) at ([xshift=0em,yshift=0em]nc12.east) {};
\node [anchor=west,snode,fill=blue!30] (nc14) at ([xshift=0em,yshift=0em]nc13.east) {};
\node [anchor=west,snode,font=\footnotesize,fill=ugreen!30] (nc15) at ([xshift=0em,yshift=0em]nc14.east) {$\mathbi{o}_5^3$};
\node [anchor=west,snode,fill=yellow!30] (nc12) at ([xshift=-0.06em,yshift=0em]nc11.east) {};
\node [anchor=west,snode,fill=red!30] (nc13) at ([xshift=-0.06em,yshift=0em]nc12.east) {};
\node [anchor=west,snode,fill=blue!30] (nc14) at ([xshift=-0.06em,yshift=0em]nc13.east) {};
\node [anchor=west,snode,font=\footnotesize,fill=ugreen!30] (nc15) at ([xshift=-0.06em,yshift=0em]nc14.east) {$\mathbi{o}_5^3$};
\node [anchor=west,snode,fill=purple!30] (nc21) at ([xshift=1.2em,yshift=0em]n5.east) {};
\node [anchor=west,snode,fill=yellow!30] (nc22) at ([xshift=0em,yshift=0em]nc21.east) {};
\node [anchor=west,snode,fill=red!30] (nc23) at ([xshift=0em,yshift=0em]nc22.east) {};
\node [anchor=west,snode,fill=blue!30] (nc24) at ([xshift=0em,yshift=0em]nc23.east) {};
\node [anchor=west,snode,font=\footnotesize,fill=ugreen!30] (nc25) at ([xshift=0em,yshift=0em]nc24.east) {$\mathbi{o}_5^2$};
\node [anchor=west,snode,fill=yellow!30] (nc22) at ([xshift=-0.06em,yshift=0em]nc21.east) {};
\node [anchor=west,snode,fill=red!30] (nc23) at ([xshift=-0.06em,yshift=0em]nc22.east) {};
\node [anchor=west,snode,fill=blue!30] (nc24) at ([xshift=-0.06em,yshift=0em]nc23.east) {};
\node [anchor=west,snode,font=\footnotesize,fill=ugreen!30] (nc25) at ([xshift=-0.06em,yshift=0em]nc24.east) {$\mathbi{o}_5^2$};
\node [anchor=west,snode,fill=purple!30] (nc31) at ([xshift=1.2em,yshift=0em]n6.east) {};
\node [anchor=west,snode,fill=yellow!30] (nc32) at ([xshift=0em,yshift=0em]nc31.east) {};
\node [anchor=west,snode,fill=red!30] (nc33) at ([xshift=0em,yshift=0em]nc32.east) {};
\node [anchor=west,snode,fill=blue!30] (nc34) at ([xshift=0em,yshift=0em]nc33.east) {};
\node [anchor=west,snode,font=\footnotesize,fill=ugreen!30] (nc35) at ([xshift=0em,yshift=0em]nc34.east) {$\mathbi{o}_5^1$};
\node [anchor=west,snode,fill=yellow!30] (nc32) at ([xshift=-0.06em,yshift=0em]nc31.east) {};
\node [anchor=west,snode,fill=red!30] (nc33) at ([xshift=-0.06em,yshift=0em]nc32.east) {};
\node [anchor=west,snode,fill=blue!30] (nc34) at ([xshift=-0.06em,yshift=0em]nc33.east) {};
\node [anchor=west,snode,font=\footnotesize,fill=ugreen!30] (nc35) at ([xshift=-0.06em,yshift=0em]nc34.east) {$\mathbi{o}_5^1$};
\node [anchor=south,lnode] (n8) at ([xshift=0em,yshift=1em]nc13.north) {$\mathbi{W}_2$};
\node [anchor=west,font=\footnotesize] (n9) at ([xshift=0.1em,yshift=0.5em]nc25.east) {Softmax};
\node [anchor=west,snode,fill=purple!30] (ns11) at ([xshift=3.5em,yshift=0em]nc15.east) {};
\node [anchor=west,snode,fill=yellow!30] (ns12) at ([xshift=0em,yshift=0em]ns11.east) {};
\node [anchor=west,snode,fill=red!30] (ns13) at ([xshift=0em,yshift=0em]ns12.east) {};
\node [anchor=west,snode,fill=blue!30] (ns14) at ([xshift=0em,yshift=0em]ns13.east) {};
\node [anchor=west,snode,font=\tiny,fill=ugreen!30] (ns15) at ([xshift=0em,yshift=0em]ns14.east) {0.3};
\node [anchor=west,snode,fill=yellow!30] (ns12) at ([xshift=-0.06em,yshift=0em]ns11.east) {};
\node [anchor=west,snode,fill=red!30] (ns13) at ([xshift=-0.06em,yshift=0em]ns12.east) {};
\node [anchor=west,snode,fill=blue!30] (ns14) at ([xshift=-0.06em,yshift=0em]ns13.east) {};
\node [anchor=west,snode,font=\tiny,fill=ugreen!30] (ns15) at ([xshift=-0.06em,yshift=0em]ns14.east) {0.3};
\node [anchor=west,snode,fill=purple!30] (ns21) at ([xshift=3.5em,yshift=0em]nc25.east) {};
\node [anchor=west,snode,fill=yellow!30] (ns22) at ([xshift=0em,yshift=0em]ns21.east) {};
\node [anchor=west,snode,fill=red!30] (ns23) at ([xshift=0em,yshift=0em]ns22.east) {};
\node [anchor=west,snode,fill=blue!30] (ns24) at ([xshift=0em,yshift=0em]ns23.east) {};
\node [anchor=west,snode,font=\tiny,fill=ugreen!30] (ns25) at ([xshift=0em,yshift=0em]ns24.east) {0.2};
\node [anchor=west,snode,fill=yellow!30] (ns22) at ([xshift=-0.06em,yshift=0em]ns21.east) {};
\node [anchor=west,snode,fill=red!30] (ns23) at ([xshift=-0.06em,yshift=0em]ns22.east) {};
\node [anchor=west,snode,fill=blue!30] (ns24) at ([xshift=-0.06em,yshift=0em]ns23.east) {};
\node [anchor=west,snode,font=\tiny,fill=ugreen!30] (ns25) at ([xshift=-0.06em,yshift=0em]ns24.east) {0.2};
\node [anchor=west,snode,fill=purple!30] (ns31) at ([xshift=3.5em,yshift=0em]nc35.east) {};
\node [anchor=west,snode,fill=yellow!30] (ns32) at ([xshift=0em,yshift=0em]ns31.east) {};
\node [anchor=west,snode,fill=red!30] (ns33) at ([xshift=0em,yshift=0em]ns32.east) {};
\node [anchor=west,snode,fill=blue!30] (ns34) at ([xshift=0em,yshift=0em]ns33.east) {};
\node [anchor=west,snode,font=\tiny,fill=ugreen!30] (ns35) at ([xshift=0em,yshift=0em]ns34.east) {0.5};
\node [anchor=west,snode,fill=yellow!30] (ns32) at ([xshift=-0.06em,yshift=0em]ns31.east) {};
\node [anchor=west,snode,fill=red!30] (ns33) at ([xshift=-0.06em,yshift=0em]ns32.east) {};
\node [anchor=west,snode,fill=blue!30] (ns34) at ([xshift=-0.06em,yshift=0em]ns33.east) {};
\node [anchor=west,snode,font=\tiny,fill=ugreen!30] (ns35) at ([xshift=-0.06em,yshift=0em]ns34.east) {0.5};
\node [anchor=west,vlnode,fill=purple!30] (ln1) at ([xshift=3.5em,yshift=-1.5em]ns15.east) {};
\node [anchor=north west,vlnode,fill=yellow!30] (ln2) at ([xshift=-0.4em,yshift=-0.4em]ln1.north west) {};
......@@ -71,7 +71,7 @@
\node [anchor=south,vlnode,rotate=-90] (ffn) at ([xshift=2em,yshift=0em]ln3.east) {FFN};
\node [anchor=west,rectangle,inner sep=0mm,minimum height=3.5em,minimum width=0.8em,rounded corners=2pt,draw] (fn) at ([xshift=1.5em,yshift=0em]ffn.north) {$\mathbi{g}$};
\node [anchor=west,rectangle,inner sep=0mm,minimum height=3.5em,minimum width=0.8em,rounded corners=2pt,draw,thick] (fn) at ([xshift=1em,yshift=0em]ffn.north) {$\mathbi{g}$};
\draw [->,thick] ([xshift=0em,yshift=0em]n1.east) -- ([xshift=0em,yshift=0em]n4.west);
......@@ -99,7 +99,7 @@
\draw [decorate,decoration={brace,mirror}] ([xshift=0em]n3.south west) to node [midway,font=\small,align=center,xshift=0em,yshift=-0.8em] {$d_{\textrm{model}}$} ([xshift=0em]n3.south east);
\draw [decorate,decoration={brace,mirror}] ([xshift=0em]n6.south west) to node [midway,font=\small,align=center,xshift=0em,yshift=-0.8em] {$d_{\rm a}$} ([xshift=0em]n6.south east);
\draw [decorate,decoration={brace,mirror}] ([xshift=0em]n7.north west) to node [midway,font=\small,align=center,xshift=-1.4em,yshift=-0em] {$d_{\textrm{model}}$} ([xshift=0em]n7.south west);
\draw [decorate,decoration={brace}] ([xshift=0em]n7.north west) to node [midway,font=\small,align=center,xshift=0em,yshift=0.7em] {$d_{\textrm{model}}$} ([xshift=0em]n7.north east);
\draw [decorate,decoration={brace}] ([xshift=0em]n7.north west) to node [midway,font=\small,align=center,xshift=0em,yshift=0.7em] {$d_{\rm a}$} ([xshift=0em]n7.north east);
\draw [decorate,decoration={brace,mirror}] ([xshift=0em]n8.north west) to node [midway,font=\small,align=center,xshift=-0.8em,yshift=-0em] {$d_{\rm a}$} ([xshift=0em]n8.south west);
\draw [decorate,decoration={brace}] ([xshift=0em]n8.north west) to node [midway,font=\small,align=center,xshift=0em,yshift=0.8em] {$n_{\rm hop}$} ([xshift=0em]n8.north east);
\draw [decorate,decoration={brace,mirror}] ([xshift=0em]nc31.south west) to node [midway,font=\small,align=center,xshift=0em,yshift=-0.8em] {$n_{\rm hop}$} ([xshift=0em]nc35.south east);
......
Chapter15/Figures/figure-learning-rate.tex
......@@ -15,7 +15,7 @@
]
\addplot[red,line width=1.25pt] coordinates {(0,0) (1.6,2) (1.8,1.888) (2,1.787) (2.5,1.606) (3,1.462) (3.5,1.3549) (4,1.266) (4.5,1.193) (5,1.131)};
\addlegendentry{\scriptsize 原始学习率}
\addlegendentry{\scriptsize 原始的学习率}
%\addplot[red,line width=1.25pt] coordinates {(0,0) (8000,0.002) (10000,0.00179) (12000,0.00163) (12950,0.001572)};
\addplot[blue,line width=1.25pt] coordinates {(0,0) (0.8,2) (0.9906,1.7983)};
%\addplot[red,line width=1.25pt] coordinates {(0,0) (8000,0.002) (9906,0.0017983)};
......
Chapter15/Figures/figure-light-weight-transformer-module.tex
......@@ -3,9 +3,9 @@
\begin{center}
\begin{tikzpicture}
\tikzstyle{manode}=[rectangle,inner sep=0mm,minimum height=4em,minimum width=4em,rounded corners=5pt,thick,draw,fill=blue!15]
\tikzstyle{ffnnode}=[rectangle,inner sep=0mm,minimum height=1.8em,minimum width=6em,rounded corners=5pt,thick,fill=red!15,draw]
\tikzstyle{ebnode}=[rectangle,inner sep=0mm,minimum height=1.8em,minimum width=10em,rounded corners=5pt,thick,fill=ugreen!15,draw]
\tikzstyle{manode}=[rectangle,inner sep=0mm,minimum height=4em,minimum width=4em,rounded corners=5pt,thick,draw,fill=blue!30]
\tikzstyle{ffnnode}=[rectangle,inner sep=0mm,minimum height=1.8em,minimum width=6em,rounded corners=5pt,thick,fill=red!30,draw]
\tikzstyle{ebnode}=[rectangle,inner sep=0mm,minimum height=1.8em,minimum width=10em,rounded corners=5pt,thick,fill=green!30,draw]
\begin{scope}[]
......@@ -15,7 +15,8 @@
\node [anchor=south east,manode] (c1) at ([xshift=0em,yshift=1em]e1.north east){Conv};
\node [anchor=south west,ebnode] (e2) at ([xshift=0em,yshift=1em]a1.north west){Embedding};
\node [anchor=south,draw,circle,inner sep=4pt] (add1) at ([xshift=0em,yshift=0.5em]e2.north){};
\node [anchor=south,inner sep=0mm,minimum height=1.8em] (f2) at ([xshift=0em,yshift=0.5em]add1.north){output};
\node [anchor=south,ffnnode] (f2) at ([xshift=0em,yshift=0.5em]add1.north){FFN};
\node [anchor=south,inner sep=0mm,minimum height=1.8em] (op) at ([xshift=0em,yshift=0.5em]f2.north){output};
\draw[->,thick] ([xshift=0em,yshift=-0.3em]f1.north)--([xshift=0em,yshift=0em]e1.south);
......@@ -24,7 +25,9 @@
\draw[->,thick] ([xshift=0em,yshift=0em]a1.north)--([xshift=0em,yshift=1em]a1.north);
\draw[->,thick] ([xshift=0em,yshift=0em]c1.north)--([xshift=0em,yshift=1em]c1.north);
\draw[-,thick] ([xshift=0em,yshift=0em]e2.north)--([xshift=0em,yshift=0em]add1.south);
\draw[->,thick] ([xshift=0em,yshift=0em]add1.north)--([xshift=0em,yshift=0.3em]f2.south);
\draw[->,thick] ([xshift=0em,yshift=0em]add1.north)--([xshift=0em,yshift=0em]f2.south);
\draw[->,thick] ([xshift=0em,yshift=0em]f2.north)--([xshift=0em,yshift=0.3em]op.south);
\draw[-] ([xshift=0em,yshift=0em]add1.west)--([xshift=-0em,yshift=0em]add1.east);
\draw[-] ([xshift=0em,yshift=0em]add1.south)--([xshift=-0em,yshift=-0em]add1.north);
......
Chapter15/Figures/figure-multi-branch-attention-model.tex
......@@ -3,9 +3,9 @@
\begin{center}
\begin{tikzpicture}
\tikzstyle{manode}=[rectangle,inner sep=0mm,minimum height=1.8em,minimum width=10em,rounded corners=5pt,thick,draw,fill=teal!20]
\tikzstyle{ffnnode}=[rectangle,inner sep=0mm,minimum height=1.8em,minimum width=3em,rounded corners=5pt,thick,fill=red!20,draw]
\tikzstyle{lnnode}=[rectangle,inner sep=0mm,minimum height=2em,minimum width=2.5em,rounded corners=5pt,thick,fill=green!20,draw]
\tikzstyle{manode}=[rectangle,inner sep=0mm,minimum height=1.8em,minimum width=10em,rounded corners=5pt,thick,draw,fill=teal!30]
\tikzstyle{ffnnode}=[rectangle,inner sep=0mm,minimum height=1.8em,minimum width=3em,rounded corners=5pt,thick,fill=red!30,draw]
\tikzstyle{lnnode}=[rectangle,inner sep=0mm,minimum height=2em,minimum width=2.5em,rounded corners=5pt,thick,fill=green!30,draw]
\begin{scope}[]
......@@ -42,11 +42,11 @@
\node [anchor=west,ublue,font=\footnotesize,align=left] (w1) at ([xshift=5em,yshift=-0.5em]a2.east){以概率\\$p$丢弃};
\node [anchor=west,ublue,font=\footnotesize,align=left] (w2) at ([xshift=0.5em,yshift=0em]f1.east){以概率\\$p$丢弃};
\draw[-] ([xshift=0em,yshift=0em]n2.west)--([xshift=-0em,yshift=0em]n2.east);
\draw[-] ([xshift=0em,yshift=0em]n2.south)--([xshift=-0em,yshift=-0em]n2.north);
\draw[-,thick] ([xshift=0em,yshift=0em]n2.west)--([xshift=-0em,yshift=0em]n2.east);
\draw[-,thick] ([xshift=0em,yshift=0em]n2.south)--([xshift=-0em,yshift=-0em]n2.north);
\draw[-] ([xshift=0em,yshift=0em]n5.west)--([xshift=-0em,yshift=0em]n5.east);
\draw[-] ([xshift=0em,yshift=0em]n5.south)--([xshift=-0em,yshift=-0em]n5.north);
\draw[-,thick] ([xshift=0em,yshift=0em]n5.west)--([xshift=-0em,yshift=0em]n5.east);
\draw[-,thick] ([xshift=0em,yshift=0em]n5.south)--([xshift=-0em,yshift=-0em]n5.north);
\end{scope}
\end{tikzpicture}
......
Chapter15/Figures/figure-multi-scale-local-modeling.tex
\begin{tikzpicture}
\begin{scope}
\tikzstyle{cirnode}=[rectangle,rounded corners=2pt,minimum height=2em,minimum width=3em,font=\footnotesize,draw=blue!50!white!50,fill=blue!20,thick]
\tikzstyle{cirnode}=[rectangle,rounded corners=2pt,minimum height=2em,minimum width=3em,font=\footnotesize,draw,fill=blue!20,thick]
\tikzstyle{recnode}=[rectangle,rounded corners=2pt,inner sep=0mm,minimum height=1.8em,minimum width=6em]
\node [anchor=west,cirnode] (n1) at (0, 0) {$\mathbi{h}_{i-2}^l$};
......@@ -16,9 +16,9 @@
\node [anchor=center,orange!30,minimum height=2.5em,minimum width=4em,very thick,draw] (c1) at ([xshift=0em,yshift=0em]n3.center) {};
\end{pgfonlayer}
\node [anchor=south,recnode,fill=red!20] (r1) at ([xshift=-3.5em,yshift=2.5em]n2.north) {$\textrm{head}_1$};
\node [anchor=south,recnode,fill=orange!20] (r2) at ([xshift=0em,yshift=2.5em]n3.north) {$\textrm{head}_2$};
\node [anchor=south,recnode,fill=ugreen!20] (r3) at ([xshift=3.5em,yshift=2.5em]n4.north) {$\textrm{head}_3$};
\node [anchor=south,recnode,fill=red!30] (r1) at ([xshift=-3.5em,yshift=2.5em]n2.north) {$\textrm{head}_1$};
\node [anchor=south,recnode,fill=orange!30] (r2) at ([xshift=0em,yshift=2.5em]n3.north) {$\textrm{head}_2$};
\node [anchor=south,recnode,fill=ugreen!30] (r3) at ([xshift=3.5em,yshift=2.5em]n4.north) {$\textrm{head}_3$};
\node [anchor=south,cirnode] (n6) at ([xshift=0em,yshift=1.5em]r2.north) {$\mathbi{h}_{i}^{l+1}$};
......
Chapter15/Figures/figure-multi-task-structure.tex
......@@ -2,8 +2,8 @@
\begin{tikzpicture}
\begin{scope}
\tikzstyle{enode}=[rectangle,inner sep=0mm,minimum height=5em,minimum width=5em,rounded corners=7pt,fill=green!30]
\tikzstyle{dnode}=[rectangle,inner sep=0mm,minimum height=2em,minimum width=6.5em,rounded corners=5pt,fill=red!30]
\tikzstyle{enode}=[rectangle,inner sep=0mm,minimum height=5em,minimum width=5em,rounded corners=7pt,fill=green!30,draw,thick]
\tikzstyle{dnode}=[rectangle,inner sep=0mm,minimum height=2em,minimum width=6.5em,rounded corners=5pt,fill=red!30,draw,thick]
\tikzstyle{wnode}=[inner sep=0mm,minimum height=2em,minimum width=4em]
\node [anchor=south west] (w1) at (0,0) {汉语（源语言）};
......
Chapter15/Figures/figure-parallel-RNN-structure.tex
......@@ -3,8 +3,8 @@
\begin{center}
\begin{tikzpicture}
\tikzstyle{wrnode}=[rectangle,inner sep=0mm,minimum height=1.8em,minimum width=4em,rounded corners=5pt,fill=blue!30]
\tikzstyle{arnode}=[rectangle,inner sep=0mm,minimum height=1.8em,minimum width=4em,rounded corners=5pt,fill=red!30]
\tikzstyle{wrnode}=[rectangle,inner sep=0mm,minimum height=1.8em,minimum width=4em,rounded corners=5pt,fill=blue!30,draw,thick]
\tikzstyle{arnode}=[rectangle,inner sep=0mm,minimum height=1.8em,minimum width=4em,rounded corners=5pt,fill=red!30,draw,thick]
\tikzstyle{dotnode}=[inner sep=0mm,minimum height=0.5em,minimum width=1.5em]
\tikzstyle{wnode}=[inner sep=0mm,minimum height=1.8em]
{\small
......
Chapter15/Figures/figure-post-norm-vs-pre-norm.tex
......@@ -6,14 +6,14 @@
\begin{scope}[minimum height = 20pt]
\node [anchor=east] (x1) at (-0.5em, 0) {$\mathbi{x}_l$};
\node [anchor=west,draw,fill=red!20,inner xsep=5pt,rounded corners=2pt] (F1) at ([xshift=4em]x1.east){\small{$F$}};
\node [anchor=west,draw,fill=red!20,inner xsep=5pt,rounded corners=2pt,thick] (F1) at ([xshift=4em]x1.east){\small{$F$}};
\node [anchor=west,circle,draw,minimum size=1em] (n1) at ([xshift=4em]F1.east) {};
\node [anchor=west,draw,fill=green!20,inner xsep=5pt,rounded corners=2pt] (ln1) at ([xshift=4em]n1.east){\small{\textrm{LN}}};
\node [anchor=west,draw,fill=green!20,inner xsep=5pt,rounded corners=2pt,thick] (ln1) at ([xshift=4em]n1.east){\small{\textrm{LN}}};
\node [anchor=west] (x2) at ([xshift=4em]ln1.east) {$\mathbi{x}_{l+1}$};
\node [anchor=north] (x3) at ([yshift=-5em]x1.south) {$\mathbi{x}_l$};
\node [anchor=west,draw,fill=green!20,inner xsep=5pt,rounded corners=2pt] (F2) at ([xshift=4em]x3.east){\small{\textrm{LN}}};
\node [anchor=west,draw,fill=red!20,inner xsep=5pt,rounded corners=2pt] (ln2) at ([xshift=4em]F2.east){\small{$F$}};
\node [anchor=west,draw,fill=green!20,inner xsep=5pt,rounded corners=2pt,thick] (F2) at ([xshift=4em]x3.east){\small{\textrm{LN}}};
\node [anchor=west,draw,fill=red!20,inner xsep=5pt,rounded corners=2pt,thick] (ln2) at ([xshift=4em]F2.east){\small{$F$}};
\node [anchor=west,circle,draw,,minimum size=1em] (n2) at ([xshift=4em]ln2.east){};
\node [anchor=west] (x4) at ([xshift=4em]n2.east) {$\mathbi{x}_{l+1}$};
......
Chapter15/Figures/figure-relative-position-weight.tex
\begin{tikzpicture}
\tikzstyle{node1} = [anchor=center,draw,minimum height=2em,minimum width=2em,inner sep=0pt,fill=green!80]
\tikzstyle{node2} = [anchor=center,draw,minimum height=2em,minimum width=2em,inner sep=0pt,fill=green!40]
\tikzstyle{node3} = [anchor=center,draw,minimum height=2em,minimum width=2em,inner sep=0pt,fill=green!20]
\tikzstyle{node1} = [anchor=center,draw,minimum height=2em,minimum width=2em,inner sep=0pt,fill=ugreen!80]
\tikzstyle{node2} = [anchor=center,draw,minimum height=2em,minimum width=2em,inner sep=0pt,fill=ugreen!50]
\tikzstyle{node3} = [anchor=center,draw,minimum height=2em,minimum width=2em,inner sep=0pt,fill=ugreen!20]
\tikzstyle{node4} = [anchor=center,draw,minimum height=2em,minimum width=2em,inner sep=0pt]
\tikzstyle{node5} = [anchor=center,draw,minimum height=2em,minimum width=2em,inner sep=0pt,fill=red!20]
\tikzstyle{node6} = [anchor=center,draw,minimum height=2em,minimum width=2em,inner sep=0pt,fill=red!40]
\tikzstyle{node7} = [anchor=center,draw,minimum height=2em,minimum width=2em,inner sep=0pt,fill=red!80]
\tikzstyle{node5} = [anchor=center,draw,minimum height=2em,minimum width=2em,inner sep=0pt,fill=teal!20]
\tikzstyle{node6} = [anchor=center,draw,minimum height=2em,minimum width=2em,inner sep=0pt,fill=teal!50]
\tikzstyle{node7} = [anchor=center,draw,minimum height=2em,minimum width=2em,inner sep=0pt,fill=teal!80]
\begin{scope}[scale=1.0]
\foreach \i / \j / \k / \z in
......@@ -27,6 +27,9 @@
{0/0/8, 1/0/9, 2/0/10, 3/0/11, 4/0/12, 5/0/13, 6/0/14}
\node[anchor=north] (n\k) at ([xshift=-0em,yshift=-0.5em]a\i\j.south) {\i};
\node [anchor=east] (l1) at ([xshift=-0.3em,yshift=0em]n8.west) {$i$};
\node [anchor=north] (l2) at ([xshift=0em,yshift=-0em]n7.south) {$j$};
%\node [anchor=north] (n1) at ([xshift=0em,yshift=0em]a00.south west) {};
\end{scope}
......
Chapter15/Figures/figure-structure-search-based-on-gradient-method.tex
......@@ -4,9 +4,9 @@
\begin{tikzpicture}
\tikzstyle{node}=[minimum height=6em,inner sep=4pt,align=left,draw,font=\footnotesize,rounded corners=4pt,thick,drop shadow]
\node[node,fill=red!20] (n1) at (0,0){\scriptsize\bfnew{超网络}： \\ [1ex] 模型结构参数 \\[0.4ex] 网络参数};
\node[anchor=west,node,fill=yellow!20] (n2) at ([xshift=4em]n1.east){\scriptsize\bfnew{优化后的超网络}： \\ [1ex]模型{\color{red}结构参数}（已优化） \\ [0.4ex]网络参数（已优化）};
\node[anchor=west,node,fill=green!20] (n3) at ([xshift=6em]n2.east){\scriptsize\bfnew{找到的模型结构}};
\node[node,fill=red!30] (n1) at (0,0){\scriptsize\bfnew{超网络}： \\ [1ex] 模型结构参数 \\[0.4ex] 网络参数};
\node[anchor=west,node,fill=yellow!30] (n2) at ([xshift=4em]n1.east){\scriptsize\bfnew{优化后的超网络}： \\ [1ex]模型{\color{red}结构参数}（已优化） \\ [0.4ex]网络参数（已优化）};
\node[anchor=west,node,fill=green!30] (n3) at ([xshift=6em]n2.east){\scriptsize\bfnew{找到的模型结构}};
\draw[-latex,thick] (n1.0) -- node[above,align=center,font=\scriptsize]{优化后的\\超网络}(n2.180);
\draw[-latex,thick] (n2.0) -- node[above,align=center,font=\scriptsize]{根据结构参数\\离散化结构}(n3.180);
......
Chapter15/Figures/figure-structure-search-based-on-reinforcement-learning.tex
......@@ -2,15 +2,15 @@
%%% outline
%-------------------------------------------------------------------------
\begin{tikzpicture}
\tikzstyle{node}=[minimum height=2em,minimum width=5em,draw,rounded corners=2pt,thick,drop shadow]
\tikzstyle{node}=[minimum height=2.5em,minimum width=8em,draw,rounded corners=2pt,thick,drop shadow]
\node[node,fill=red!20] (n1) at (0,0){\small\bfnew{环境}};
\node[anchor=south,node,fill=green!20] (n2) at ([yshift=5em]n1.north){\small\bfnew{智能体}};
\node[node,fill=red!30] (n1) at (0,0){\small\bfnew{环境}};
\node[anchor=south,node,fill=green!30] (n2) at ([yshift=5em]n1.north){\small\bfnew{智能体}};
\node[anchor=north,font=\footnotesize] at ([yshift=-0.2em]n1.south){（结构所应用于的任务）};
\node[anchor=south,font=\footnotesize] at ([yshift=0.2em]n2.north){（结构生成器）};
\draw[-latex,thick] ([yshift=.4em]n1.180) .. controls ([xshift=-3.4em,yshift=.4em]n1.180) and ([xshift=-3.4em,yshift=-.4em]n2.180) .. node[right,font=\scriptsize,align=left]{\scriptsize\bfnew{奖励} \\ （对输出结果的评价）}([yshift=-.4em]n2.180);
\draw[-latex,thick] ([yshift=-.4em]n1.180) .. controls ([xshift=-4.4em,yshift=-.4em]n1.180) and ([xshift=-4.4em,yshift=.4em]n2.180) .. node[left,font=\scriptsize,align=right]{\scriptsize\bfnew{状态} \\ （这个结构在任务中应用 \\ 后得到的输出结果）}([yshift=.4em]n2.180);
\draw[-latex,thick] ([yshift=-.4em]n1.180) .. controls ([xshift=-4.4em,yshift=-.4em]n1.180) and ([xshift=-4.4em,yshift=.4em]n2.180) .. node[left,font=\scriptsize,align=right]{\scriptsize\bfnew{状态} \\ （这个结构在任务中应 \\ 用后得到的输出结果）}([yshift=.4em]n2.180);
\draw[-latex,thick] (n2.0) .. controls ([xshift=4em]n2.0) and ([xshift=4em]n1.0) .. node[right,font=\scriptsize,align=left]{\scriptsize\bfnew{动作} \\ （生成一个结构）}(n1.0);
\end{tikzpicture}
......
Chapter15/Figures/figure-sublayer-skip.tex
......@@ -6,24 +6,24 @@
\begin{scope}[minimum height = 20pt]
\node [anchor=east] (x1) at (-0.5em, 0) {$\mathbi{x}_l$};
\node [anchor=west,draw,fill=red!20,inner xsep=5pt,rounded corners=2pt] (ln1) at ([xshift=1em]x1.east){\small{\textrm{LN}}};
\node [anchor=west,draw,fill=green!20,inner xsep=5pt,rounded corners=2pt] (f1) at ([xshift=0.6em]ln1.east){\small{$F$}};
\node [anchor=west,draw,fill=red!30,inner xsep=5pt,rounded corners=2pt,draw,thick] (ln1) at ([xshift=1em]x1.east){\small{\textrm{LN}}};
\node [anchor=west,draw,fill=green!30,inner xsep=5pt,rounded corners=2pt,draw,thick] (f1) at ([xshift=0.6em]ln1.east){\small{$F$}};
\node [anchor=west,circle,draw,,minimum size=1em] (n1) at ([xshift=3em]f1.east){};
\node [anchor=west] (x2) at ([xshift=1em]n1.east) {$\mathbi{x}_{l+1}$};
\node [anchor=west,draw,fill=red!20,inner xsep=5pt,rounded corners=2pt] (ln12) at ([xshift=1em]x2.east){\small{\textrm{LN}}};
\node [anchor=west,draw,fill=green!20,inner xsep=5pt,rounded corners=2pt] (f12) at ([xshift=0.6em]ln12.east){\small{$F$}};
\node [anchor=west,draw,fill=red!30,inner xsep=5pt,rounded corners=2pt,draw,thick] (ln12) at ([xshift=1em]x2.east){\small{\textrm{LN}}};
\node [anchor=west,draw,fill=green!30,inner xsep=5pt,rounded corners=2pt,draw,thick] (f12) at ([xshift=0.6em]ln12.east){\small{$F$}};
\node [anchor=west,circle,draw,,minimum size=1em] (n12) at ([xshift=3em]f12.east){};
\node [anchor=west] (x22) at ([xshift=1em]n12.east) {$\mathbi{x}_{l+2}$};
\node [anchor=north] (x3) at ([yshift=-5em]x1.south) {$\mathbi{x}_l$};
\node [anchor=west,draw,fill=red!20,inner xsep=5pt,rounded corners=2pt] (ln2) at ([xshift=1em]x3.east){\small{\textrm{LN}}};
\node [anchor=west,draw,fill=green!20,inner xsep=5pt,rounded corners=2pt] (f2) at ([xshift=0.6em]ln2.east){\small{$F$}};
\node [anchor=west,draw,fill=red!30,inner xsep=5pt,rounded corners=2pt,draw,thick] (ln2) at ([xshift=1em]x3.east){\small{\textrm{LN}}};
\node [anchor=west,draw,fill=green!30,inner xsep=5pt,rounded corners=2pt,draw,thick] (f2) at ([xshift=0.6em]ln2.east){\small{$F$}};
\node [anchor=west,minimum size=1em] (p1) at ([xshift=1em]f2.east){};
\node [anchor=north] (m1) at ([yshift=0.6em]p1.south){\footnotesize{\red{Mask=1}}};
\node [anchor=west,circle,draw,,minimum size=1em] (n2) at ([xshift=3em]f2.east){};
\node [anchor=west] (x4) at ([xshift=1em]n2.east) {$\mathbi{x}_{l+1}$};
\node [anchor=west,draw,fill=red!20,inner xsep=5pt,rounded corners=2pt] (ln22) at ([xshift=1em]x4.east){\small{\textrm{LN}}};
\node [anchor=west,draw,fill=green!20,inner xsep=5pt,rounded corners=2pt] (f22) at ([xshift=0.6em]ln22.east){\small{$F$}};
\node [anchor=west,draw,fill=red!30,inner xsep=5pt,rounded corners=2pt,draw,thick] (ln22) at ([xshift=1em]x4.east){\small{\textrm{LN}}};
\node [anchor=west,draw,fill=green!30,inner xsep=5pt,rounded corners=2pt,draw,thick] (f22) at ([xshift=0.6em]ln22.east){\small{$F$}};
\node [anchor=west,minimum size=1em] (p2) at ([xshift=1em]f22.east){};
\node [anchor=north] (m2) at ([yshift=0.6em]p2.south){\footnotesize{\red{Mask=0}}};
\node [anchor=west,circle,draw,,minimum size=1em] (n22) at ([xshift=3em]f22.east){};
......
Chapter15/Figures/figure-three-fusion-methods-of-tree-structure-information-1.tex
......@@ -3,8 +3,8 @@
\begin{center}
\begin{tikzpicture}
\tikzstyle{wrnode}=[rectangle,inner sep=0mm,minimum height=1.6em,minimum width=3em,rounded corners=5pt,fill=blue!30]
\tikzstyle{srnode}=[rectangle,inner sep=0mm,minimum height=1.6em,minimum width=3em,rounded corners=5pt,fill=orange!30]
\tikzstyle{wrnode}=[rectangle,inner sep=0mm,minimum height=1.6em,minimum width=3em,rounded corners=5pt,fill=blue!30,draw,thick]
\tikzstyle{srnode}=[rectangle,inner sep=0mm,minimum height=1.6em,minimum width=3em,rounded corners=5pt,fill=orange!30,draw,thick]
\tikzstyle{dotnode}=[inner sep=0mm,minimum height=0.5em,minimum width=1.5em]
\tikzstyle{wnode}=[inner sep=0mm,minimum height=1.6em]
{\small
......
Chapter15/Figures/figure-three-fusion-methods-of-tree-structure-information-2.tex
......@@ -3,8 +3,8 @@
\begin{center}
\begin{tikzpicture}
\tikzstyle{wrnode}=[rectangle,inner sep=0mm,minimum height=1.8em,minimum width=3em,rounded corners=5pt,fill=blue!30]
\tikzstyle{srnode}=[rectangle,inner sep=0mm,minimum height=1.8em,minimum width=3em,rounded corners=5pt,fill=orange!30]
\tikzstyle{wrnode}=[rectangle,inner sep=0mm,minimum height=1.8em,minimum width=3em,rounded corners=5pt,fill=blue!30,draw,thick]
\tikzstyle{srnode}=[rectangle,inner sep=0mm,minimum height=1.8em,minimum width=3em,rounded corners=5pt,fill=orange!30,draw,thick]
\tikzstyle{dotnode}=[inner sep=0mm,minimum height=0.5em,minimum width=1.5em]
\tikzstyle{wnode}=[inner sep=0mm,minimum height=1.8em]
......@@ -48,9 +48,9 @@
\node [anchor=south,wnode] (w10) at ([xshift=0em,yshift=0.5em]c3.north) {$\mathbi{e}_{w_2}$};
\begin{pgfonlayer}{background}
\node [rectangle,minimum height=5em,inner sep=0.6em,fill=green!20,rounded corners=8pt] [fit = (c1) (w8)] (box6) {};
\node [rectangle,minimum height=5em,inner sep=0.6em,fill=green!20,rounded corners=8pt] [fit = (c2) (w9)] (box7) {};
\node [rectangle,minimum height=5em,inner sep=0.6em,fill=green!20,rounded corners=8pt] [fit = (c3) (w10)] (box8) {};
\node [rectangle,minimum height=5em,inner sep=0.6em,fill=green!30,rounded corners=8pt,draw,thick] [fit = (c1) (w8)] (box6) {};
\node [rectangle,minimum height=5em,inner sep=0.6em,fill=green!30,rounded corners=8pt,draw,thick] [fit = (c2) (w9)] (box7) {};
\node [rectangle,minimum height=5em,inner sep=0.6em,fill=green!30,rounded corners=8pt,draw,thick] [fit = (c3) (w10)] (box8) {};
\end{pgfonlayer}
\node [anchor=south,wrnode] (wr1) at ([xshift=0em,yshift=1em]box6.north) {$\mathbi{h}_{w_1}$};
......
Chapter15/Figures/figure-three-fusion-methods-of-tree-structure-information-3.tex
......@@ -3,7 +3,7 @@
\begin{center}
\begin{tikzpicture}
\tikzstyle{hnode}=[rectangle,inner sep=0mm,minimum height=1.6em,minimum width=3em,rounded corners=5pt,fill=red!30]
\tikzstyle{hnode}=[rectangle,inner sep=0mm,minimum height=1.6em,minimum width=3em,rounded corners=5pt,fill=red!30,draw,thick]
\tikzstyle{dotnode}=[inner sep=0mm,minimum height=0.5em,minimum width=1.5em]
\tikzstyle{wnode}=[inner sep=0mm,minimum height=1.6em]
{\small
......
Chapter15/Figures/figure-weight-visualization-of-convergence-DLCL-network.tex
\begin{tikzpicture}[node distance = 0,scale = 1]
\tikzstyle{every node}=[scale=1]
\node[draw=white] (input) at (0,0){\includegraphics[width=0.62\textwidth]{./Chapter15/Figures/DLCL-picture.png}};(1.9,-1.4);
\node[scale = 2] at (4.5,3.6){4};
\node[scale = 2] at (4.5,1.8){2};
\node[scale = 2] at (4.5,0){0};
\node[scale = 2] at (4.5,-1.8){-2};
\node[scale = 2] at (4.5,-3.6){-4};
\node[scale = 1.5] at (-4.5,3.75){$\rm x_{1}$};
\node[scale = 1.5] at (-4.5,2.5){$\rm x_{6}$};
\node[scale = 1.5] at (-4.5,1.4){$\rm x_{11}$};
\node[scale = 1.5] at (-4.5,0.1){$\rm x_{16}$};
\node[scale = 1.5] at (-4.5,-1.1){$\rm x_{21}$};
\node[scale = 1.5] at (-4.5,-2.3){$\rm x_{26}$};
\node[scale = 1.5] at (-4.5,-3.4){$\rm x_{31}$};
\node[scale = 1.5] at (-3.8,-4){$\rm y_{0}$};
\node[scale = 1.5] at (-2.7,-4){$\rm y_{5}$};
\node[scale = 1.5] at (-1.5,-4){$\rm y_{10}$};
\node[scale = 1.5] at (-0.3,-4){$\rm y_{15}$};
\node[scale = 1.5] at (0.9,-4){$\rm y_{20}$};
\node[scale = 1.5] at (2.1,-4){$\rm y_{25}$};
\node[scale = 1.5] at (3.3,-4){$\rm y_{30}$};
\node[draw=white] (input) at (0,0){\includegraphics[width=0.62\textwidth]{./Chapter15/Figures/DLCL-picture.png}};
{\footnotesize
\node[anchor=center] at (4.1,3){4};
\node[anchor=center] at (4.1,1.5){2};
\node[anchor=center] at (4.1,0){0};
\node[anchor=center] at (4.1,-1.5){-2};
\node[anchor=center] at (4.1,-3){-4};
\node[anchor=center] at (-4.2,3.6){$ x_{1}$};
\node[anchor=center] at (-4.2,2.45){$ x_{6}$};
\node[anchor=center] at (-4.2,1.3){$ x_{11}$};
\node[anchor=center] at (-4.2,0.15){$ x_{16}$};
\node[anchor=center] at (-4.2,-1){$ x_{21}$};
\node[anchor=center] at (-4.2,-2.15){$ x_{26}$};
\node[anchor=center] at (-4.2,-3.3){$ x_{31}$};
\node[anchor=center] at (-3.75,-3.8){$ y_{0}$};
\node[anchor=center] at (-2.6,-3.8){$ y_{5}$};
\node[anchor=center] at (-1.45,-3.8){$ y_{10}$};
\node[anchor=center] at (-0.3,-3.8){$ y_{15}$};
\node[anchor=center] at (0.85,-3.8){$ y_{20}$};
\node[anchor=center] at (2,-3.8){$ y_{25}$};
\node[anchor=center] at (3.15,-3.8){$ y_{30}$};
}
\end{tikzpicture}
\ No newline at end of file
Chapter17/Figures/figure-an-end-to-end-voice-translation-model-based-on-transformer.tex
\begin{tikzpicture}
\tikzstyle{layer}=[draw,rounded corners=2pt,font=\scriptsize,align=center,minimum width=5em]
\tikzstyle{layer}=[draw,rounded corners=2pt,font=\scriptsize,align=center,minimum width=7.1em]
\tikzstyle{word}=[font=\scriptsize]
%%%%encoder
\node[layer,fill=red!20] (en_sa) at (0,0){Multi-Head \\ Attention};
\node[layer,anchor=south,fill=green!20] (en_ffn) at ([yshift=1.4em]en_sa.north){Feed Forward \\ Network};
\node[draw,circle,inner sep=0pt, minimum size=1em,anchor=north] (en_add) at ([yshift=-1.4em]en_sa.south){};
\draw[] (en_add.90) -- (en_add.-90);
\draw[] (en_add.0) -- (en_add.180);
\node[layer,anchor=north,fill=yellow!20] (en_cnn) at ([yshift=-1.4em]en_add.south){CNN};
\node[anchor=south,layer,fill=yellow!20](en_add1) at ([yshift=1.0em]en_sa.north) {Add \& LayerNorm};
\node[layer,anchor=south,fill=green!20] (en_ffn) at ([yshift=1.0em]en_add1.north){Feed Forward \\ Network};
\node[anchor=south,layer,fill=yellow!20](en_add2) at ([yshift=1.0em]en_ffn.north) {Add \& LayerNorm};
\node[draw,circle,inner sep=0pt, minimum size=1em,anchor=north,thick] (en_add) at ([yshift=-1.4em]en_sa.south){};
\draw[thick] (en_add.90) -- (en_add.-90);
\draw[thick] (en_add.0) -- (en_add.180);
\node[layer,anchor=north,fill=yellow!20] (en_cnn) at ([yshift=-1.0em]en_add.south){CNN};
\node[anchor=east,font=\scriptsize,align=center] (en_pos) at ([xshift=-2em]en_add.west){位置编码};
\node[anchor=north,font=\scriptsize,align=center] (en_input) at ([yshift=-1em]en_cnn.south){源语言语音特征\\（FBank/MFCC）};
\draw[->,thick] (en_input.90) -- ([yshift=-0.1em]en_cnn.-90);
\draw[->,thick] ([yshift=0.1em]en_cnn.90) -- ([yshift=-0.1em]en_add.-90);
\draw[->,thick] ([yshift=0.1em]en_add.90) -- ([yshift=-0.1em]en_sa.-90);
\draw[->,thick] ([yshift=0.1em]en_sa.90) -- ([yshift=-0.1em]en_add1.-90);
\draw[->,thick] ([yshift=0.1em]en_add1.90) -- ([yshift=-0.1em]en_ffn.-90);
\draw[->,thick] ([yshift=0.1em]en_ffn.90) --([yshift=-0.1em]en_add2.-90);
\draw[->,rounded corners=2pt,thick] ([yshift=-0.6em]en_sa.south)--([yshift=-0.6em,xshift=-4.0em]en_sa.south)--([xshift=-0.43em]en_add1.west)--(en_add1.west);
\draw[->,rounded corners=2pt,thick] ([yshift=-0.6em]en_ffn.south)--([yshift=-0.6em,xshift=-4.0em]en_ffn.south)--([xshift=-0.43em]en_add2.west)--(en_add2.west);
\node[draw,circle,inner sep=0pt, minimum size=1em,anchor=west] (de_add) at ([xshift=7em]en_add.east){};
\draw[] (de_add.90) -- (de_add.-90);
\draw[] (de_add.0) -- (de_add.180);
%%%%decoder
\node[draw,circle,inner sep=0pt, minimum size=1em,anchor=west,thick] (de_add) at ([xshift=9em]en_add.east){};
\draw[thick] (de_add.90) -- (de_add.-90);
\draw[thick] (de_add.0) -- (de_add.180);
\node[layer,anchor=south,fill=red!20] (de_sa) at ([yshift=1.4em]de_add.north){Masked \\Multi-Head\\Attention};
\node[layer,anchor=south,fill=red!20] (de_ca) at ([yshift=1.4em]de_sa.north){Multi-Head \\ Attention};
\node[layer,anchor=south,fill=green!20] (de_ffn) at ([yshift=1.4em]de_ca.north){Feed Forward \\ Network};
\node[anchor=south,layer,fill=yellow!20](de_add1) at ([yshift=1.0em]de_sa.north) {Add \& LayerNorm};
\node[layer,anchor=south,fill=red!20] (de_ca) at ([yshift=1.0em]de_add1.north){Multi-Head \\ Attention};
\node[anchor=south,layer,fill=yellow!20](de_add2) at ([yshift=1.0em]de_ca.north) {Add \& LayerNorm};
\node[layer,anchor=south,fill=green!20] (de_ffn) at ([yshift=1.0em]de_add2.north){Feed Forward \\ Network};
\node[anchor=south,layer,fill=yellow!20](de_add3) at ([yshift=1.0em]de_ffn.north) {Add \& LayerNorm};
\node[layer,anchor=south,fill=blue!20] (sf) at ([yshift=1.2em]de_add3.north){Softmax};
\node[anchor=north,font=\scriptsize,align=center] (de_input) at ([yshift=-1.1em]de_add.south){目标语言文本\\编码表示};
\node[anchor=west,font=\scriptsize,align=center] (de_pos) at ([xshift=2em]de_add.east){位置编码};
\node[layer,anchor=south,fill=blue!20] (sf) at ([yshift=1.6em]de_ffn.north){Softmax};
%\node[layer,anchor=south,fill=orange!20] (output) at ([yshift=1.4em]sf.north){STLoss};
\draw[->,thick] (de_input.90) -- ([yshift=-0.1em]de_add.-90);
\draw[->,thick] ([yshift=0.1em]de_add.90) -- ([yshift=-0.1em]de_sa.-90);
\draw[->,thick] ([yshift=0.1em]de_sa.90) -- ([yshift=-0.1em]de_add1.-90);
\draw[->,thick] ([yshift=0.1em]de_add1.90) -- ([yshift=-0.1em]de_ca.-90);
\draw[->,thick] ([yshift=0.1em]de_ca.90) -- ([yshift=-0.1em]de_add2.-90);
\draw[->,thick] ([yshift=0.1em]de_add2.90) -- ([yshift=-0.1em]de_ffn.-90);
\draw[->,thick] ([yshift=0.1em]de_ffn.90) -- ([yshift=-0.1em]de_add3.-90);
\draw[->,thick] ([yshift=0.1em]de_add3.90) -- ([yshift=-0.1em]sf.-90);
\draw[->,thick] ([yshift=0.1em]sf.90) -- ([yshift=1.0em]sf.90);
\draw[->,thick] ([xshift=0.1em]en_pos.0) -- ([xshift=-0.1em]en_add.180);
\draw[->,thick] ([xshift=-0.1em]de_pos.180) -- ([xshift=0.1em]de_add.0);
\draw[->,rounded corners=2pt,thick] ([yshift=-0.6em]de_sa.south)--([yshift=-0.6em,xshift=4.0em]de_sa.south)--([xshift=0.43em]de_add1.east)--(de_add1.east);
\draw[->,rounded corners=2pt,thick] ([yshift=-0.6em]de_ca.south)--([yshift=-0.6em,xshift=4.0em]de_ca.south)--([xshift=0.43em]de_add2.east)--(de_add2.east);
\draw[->,rounded corners=2pt,thick] ([yshift=-0.6em]de_ffn.south)--([yshift=-0.6em,xshift=4.0em]de_ffn.south)--([xshift=0.43em]de_add3.east)--(de_add3.east);
\node[anchor=north,font=\scriptsize,align=center] (en_input) at ([yshift=-1em]en_cnn.south){语音特征\\(FBank/MFCC)};
\node[anchor=north,font=\scriptsize,align=center] (de_input) at ([yshift=-1.1em]de_add.south){标注文本\\编码表示};
\draw[->,rounded corners=2pt,thick] ([yshift=0.1em]en_add2.90) -- ([yshift=1.5em]en_add2.90) -- ([xshift=5.0em,yshift=1.5em]en_add2.90) -- ([xshift=-1.5em]de_ca.west) -- ([xshift=-0.1em]de_ca.west);
\node[anchor=east,font=\scriptsize,align=center] (en_pos) at ([xshift=-2em]en_add.west){位置编码};
\node[anchor=west,font=\scriptsize,align=center] (de_pos) at ([xshift=2em]de_add.east){位置编码};
\draw[->] (en_input.90) -- ([yshift=-0.1em]en_cnn.-90);
\draw[->] ([yshift=0.1em]en_cnn.90) -- ([yshift=-0.1em]en_add.-90);
\draw[->] ([yshift=0.1em]en_add.90) -- ([yshift=-0.1em]en_sa.-90);
\draw[->] ([yshift=0.1em]en_sa.90) -- ([yshift=-0.1em]en_ffn.-90);
\draw[->] (de_input.90) -- ([yshift=-0.1em]de_add.-90);
\draw[->] ([yshift=0.1em]de_add.90) -- ([yshift=-0.1em]de_sa.-90);
\draw[->] ([yshift=0.1em]de_sa.90) -- ([yshift=-0.1em]de_ca.-90);
\draw[->] ([yshift=0.1em]de_ca.90) -- ([yshift=-0.1em]de_ffn.-90);
\draw[->] ([yshift=0.1em]de_ffn.90) -- ([yshift=-0.1em]sf.-90);
\draw[->] ([yshift=0.1em]sf.90) -- ([yshift=1.5em]sf.90);
\draw[->] ([xshift=0.1em]en_pos.0) -- ([xshift=-0.1em]en_add.180);
\draw[->] ([xshift=-0.1em]de_pos.180) -- ([xshift=0.1em]de_add.0);
\draw[->,rounded corners=2pt] ([yshift=0.1em]en_ffn.90) -- ([yshift=2em]en_ffn.90) -- ([xshift=4em,yshift=2em]en_ffn.90) -- ([xshift=-1.5em]de_ca.west) -- ([xshift=-0.1em]de_ca.west);
\begin{pgfonlayer}{background}
\node[draw=ugreen,rounded corners=2pt,inner xsep=6pt,inner ysep=8pt,dashed,thick][fit=(en_sa)(en_ffn)]{};
\node[draw=red,rounded corners=2pt,inner xsep=6pt,inner ysep=8pt,dashed,thick][fit=(de_sa)(de_ca)(de_ffn)]{};
\node[draw=ugreen,rounded corners=2pt,inner xsep=6pt,inner ysep=8pt,dashed,thick,xshift=-0.2em,yshift=-0.2em][fit=(en_add1)(en_add2)(en_sa)(en_ffn)](box1){};
\node[draw=red,rounded corners=2pt,inner xsep=6pt,inner ysep=8pt,dashed,thick,xshift=0.2em,yshift=-0.2em][fit=(de_sa)(de_ca)(de_ffn)(de_add3)](box2){};
\end{pgfonlayer}
\node[anchor=east,font=\scriptsize,text=ugreen] at ([xshift=-0.1em]box1.west){$N \times$};
\node[anchor=west,font=\scriptsize,text=red] at ([xshift=0.1em]box2.east){$\times N$};
\node[anchor=east,font=\scriptsize] at ([xshift=-0.1em]en_cnn.west){$2 \times$};
\node[anchor=east,font=\scriptsize,align=center,text=ugreen] at ([xshift=-0.1em,yshift=3em]box1.west){ST\\ 编码器};
\node[anchor=west,font=\scriptsize,align=center,text=red] at ([xshift=0.1em,yshift=5em]box2.east){ST\\解码器};
\node[anchor=east,font=\scriptsize,align=center,text=ugreen] at ([xshift=-0.1em,yshift=3em]box1.west){ST \\ 编码器};
\node[anchor=west,font=\scriptsize,align=center,text=red] at ([xshift=0.1em,yshift=5em]box2.east){ST \\ 解码器};
\end{tikzpicture}
\ No newline at end of file
Chapter17/Figures/figure-application-of-multimodal-machine-translation-to-multitask-learning.tex
\tikzstyle{coder} = [rectangle,rounded corners,minimum height=2.2em,minimum width=4.3em,text centered,draw=black,fill=red!25]
\tikzstyle{coder} = [rectangle,thick,rounded corners,minimum height=2.2em,minimum width=4.3em,text centered,draw=black,fill=red!20]
\begin{tikzpicture}[node distance = 0,scale = 0.75]
\tikzstyle{every node}=[scale=0.75]
\node(x)[]{$x$};
\node(encoder)[coder, above of = x,yshift=4em]{{编码器}};
\node(decoder_left)[coder, above of = encoder, yshift=6em,fill=blue!25]{{解码器}};
\node(encoder)[coder, above of = x,yshift=4em]{\large{编码器}};
\node(decoder_left)[coder, above of = encoder, yshift=6em,fill=blue!20]{\large{解码器}};
\node(y_hat)[above of = decoder_left, yshift=4em]{{$y$}};
\node(y)[above of = decoder_left, xshift=-6em]{{$y_{<}$}};
\node(decoder_right)[coder, above of = encoder, xshift=11em,fill=yellow!25]{{解码器}};
\node(decoder_right)[coder, above of = encoder, xshift=11em,fill=yellow!20]{\large{解码器}};
\node(figure)[draw=white,above of = decoder_right,yshift=6.5em,scale=0.25] {\includegraphics[width=0.62\textwidth]{./Chapter17/Figures/figure-bank-without-attention.jpg}};
\node [anchor=south,scale=1.2] (node1) at ([xshift=-2.5em,yshift=4.5em]y.north) {\small{$x$：源语言文本数据}};
\node [anchor=north,scale=1.2] (node2) at ([xshift=0.57em]node1.south){\small{$y$：目标语言文本数据}};
\node [anchor=south,scale=1.2] (node1) at ([xshift=-2.5em,yshift=4.5em]y.north) {{$x$：源语言文本数据}};
\node [anchor=north,scale=1.2] (node2) at ([xshift=0.57em]node1.south){{$y$：目标语言文本数据}};
\draw[->,thick](x)to(encoder);
\draw[->,thick](encoder)to(decoder_left)node[right,xshift=-0.1cm,yshift=-1.25cm,scale=1.2]{\small{翻译}};
\draw[->,thick](encoder)to(decoder_left)node[right,xshift=-0.1cm,yshift=-1.25cm,scale=1.2]{{翻译}};
\draw[->,thick](decoder_left)to(y_hat);
\draw[->,thick](y)to(decoder_left);
\draw[->,thick](encoder)to(decoder_right)node[left,xshift=-3.1em,yshift=0.25cm,scale=1.2]{\small{生成图片}};
\draw[->,thick](encoder)to(decoder_right)node[left,xshift=-3.1em,yshift=0.25cm,scale=1.2]{{生成图片}};
\draw[->,thick](decoder_right)to(figure);
\end{tikzpicture}
\ No newline at end of file
Chapter17/Figures/figure-audio-processing.tex
......@@ -36,6 +36,7 @@
\draw[->,very thick](process_2.east)to([xshift=1.8cm]process_2.east);
%%%%音频
\node(signal)[right of = process_2,xshift=5.5cm]{};
\node(text_3)[below of = signal,yshift=-1.98cm,scale=1.3]{语音信号};
\draw[-,thick,]([xshift=-1.2cm]signal.center)--([xshift=1.2cm]signal.center);
\draw[-,thick]([xshift=-1cm,yshift=-0.8cm]signal.center)--([xshift=-0.9cm,yshift=0.4cm]signal.center)--([xshift=-0.8cm,yshift=-0.3cm]signal.center)--([xshift=-0.7cm,yshift=0.7cm]signal.center)--([xshift=-0.6cm,yshift=-0.1cm]signal.center)--([xshift=-0.5cm,yshift=0.3cm]signal.center)--([xshift=-0.4cm,yshift=-0.5cm]signal.center)--([xshift=-0.3cm,yshift=0.7cm]signal.center)--([xshift=-0.2cm,yshift=-0.2cm]signal.center)--([xshift=-0.1cm,yshift=0.4cm]signal.center)--([xshift=0cm,yshift=-0.9cm]signal.center)--([xshift=0.1cm,yshift=0.5cm]signal.center)--([xshift=0.2cm,yshift=-0.4cm]signal.center)--([xshift=0.3cm,yshift=0.3cm]signal.center)--([xshift=0.4cm,yshift=-0.2cm]signal.center)--([xshift=0.5cm,yshift=0.1cm]signal.center)--([xshift=0.6cm,yshift=-0.8cm]signal.center)--([xshift=0.7cm,yshift=0.4cm]signal.center)--([xshift=0.8cm,yshift=-0.6cm]signal.center)--([xshift=0.9cm,yshift=0.7cm]signal.center)--([xshift=1cm,yshift=-0.2cm]signal.center);
\end{tikzpicture}
\ No newline at end of file
Chapter17/Figures/figure-cache.tex
......@@ -13,8 +13,8 @@
\draw[] ([yshift=-0.5em]value.180) -- ([yshift=-0.5em]value.0);
\draw[] ([yshift=-1.5em]value.180) -- ([yshift=-1.5em]value.0);
\node[anchor=south,font=\footnotesize,inner sep=0pt] at ([yshift=0.1em]key.north){key};
\node[anchor=south,font=\footnotesize,inner sep=0pt] at ([yshift=0.2em]value.north){value};
\node[anchor=south,font=\footnotesize,inner sep=0pt] at ([yshift=0.1em]key.north){键};
\node[anchor=south,font=\footnotesize,inner sep=0pt] at ([yshift=0.1em]value.north){值};
\node[anchor=south,font=\footnotesize,inner sep=0pt] (cache)at ([yshift=2em,xshift=1.5em]key.north){\small\bfnew{缓存}};
\node[draw,anchor=east,thick,minimum size=1.8em,fill=orange!30] (dt) at ([yshift=2.1em,xshift=-4em]key.west){${\mathbi{d}}_{t}$};
......
Chapter17/Figures/figure-cascading-speech-translation.tex
......@@ -34,7 +34,7 @@
\node(text)[below of = process_3,yshift=-1.8cm,scale=1.8]{你是谁};
\node(process_4)[process,fill=ugreen!20,right of = process_3,xshift=8.2cm,text width=4cm,align=center]{\Large\textbf{Who are you?}};
\node(text_4)[below of = process_4,yshift=-2cm,scale=1.5]{翻译译文};
\node(text_4)[below of = process_4,yshift=-2cm,scale=1.5]{翻译结果};
\draw[->,very thick](process_1.east)to(process_2.west);
\draw[->,very thick](process_2.east)to(process_3.west);
......
Chapter17/Figures/figure-multiencoder.tex
......@@ -6,7 +6,7 @@
\tikzstyle{every node}=[scale=0.7]
\node(encoder_c)[coder]{\large{编码器}};
\node(encoder_s)[coder, right of = encoder_c, xshift=3.5cm, fill=red!30]{\large{编码器}};
\node(h_pre)[above of = encoder_c, yshift=1.3cm,scale=1.3]{${\mathbi{h}}_{\rm pre}$};
\node(h_pre)[above of = encoder_c, yshift=1.3cm,scale=1.3]{${\mathbi{h}}^{\rm pre}$};
\node(h)[above of = encoder_s, yshift=1.3cm,scale=1.3]{$\mathbi{h}$};
\node(cir)[circle,very thick, right of = h, draw=black!90,minimum width=0.5cm,xshift=1.1cm]{};
\draw[-,very thick,draw=black!90]([xshift=0.04cm]cir.west)--([xshift=-0.04cm]cir.east);
......@@ -23,20 +23,20 @@
\node(point_above)[above of = attention_right, yshift=1.8cm]{\Huge{...}};
\node(target_above)[above of = attention_right, yshift=3.3cm]{\large{目标语言句子}};
\draw[->, very thick](last)to([yshift=-0.05cm]encoder_c.south);
\draw[->, very thick](current)to([yshift=-0.05cm]encoder_s.south);
\draw[->, very thick](target.north)to([yshift=-0.05cm]point_below.south);
\draw[->, very thick]([yshift=0.05cm]encoder_c.north)to([yshift=0.03cm]h_pre.south);
\draw[->, very thick]([yshift=0.05cm]encoder_s.north)to(h.south);
\draw[->, very thick]([yshift=0cm]h.north)to([yshift=0.95cm]h.north);
\draw[->, very thick,in=270,out=90]([yshift=-0.15cm]h_pre.north)to([xshift=1.25cm,yshift=0.9cm]h_pre.north);
\draw[->, very thick,in=270,out=80]([yshift=-0.15cm]h_pre.north)to([xshift=2.4cm,yshift=0.9cm]h_pre.north);
\draw[->, very thick]([yshift=0.03cm]attention_left.north)to([yshift=0.1cm]d.south);
\draw[->, very thick]([xshift=-0.03cm]h.east)to([xshift=-0.03cm]cir.west);
\draw[->, very thick](point_below.north)to([yshift=2.03cm]point_below.north);
\draw[->, very thick](attention_right.north)to([yshift=-0.03cm]point_above.south);
\draw[->, very thick](point_above.north)to([yshift=0.83cm]point_above.north);
\draw[->, very thick, in=270,out=0]([xshift=0.2cm]cir.east)to([xshift=3cm,yshift=0.88cm]cir.east);
\draw[->, very thick, in=270,out=0]([xshift=0.2cm]cir.east)to([xshift=2cm,yshift=0.88cm]cir.east);
\draw[->,thick](last)to([yshift=-0.05cm]encoder_c.south);
\draw[->,thick](current)to([yshift=-0.05cm]encoder_s.south);
\draw[->,thick](target.north)to([yshift=-0.05cm]point_below.south);
\draw[->,thick]([yshift=0.05cm]encoder_c.north)to([yshift=0.03cm]h_pre.south);
\draw[->,thick]([yshift=0.05cm]encoder_s.north)to(h.south);
\draw[->,thick]([yshift=0cm]h.north)to([yshift=0.95cm]h.north);
\draw[->,thick,in=270,out=90]([yshift=-0.15cm]h_pre.north)to([xshift=1.25cm,yshift=0.9cm]h_pre.north);
\draw[->,thick,in=270,out=80]([yshift=-0.15cm]h_pre.north)to([xshift=2.4cm,yshift=0.9cm]h_pre.north);
\draw[->,thick]([yshift=0.03cm]attention_left.north)to([yshift=0.1cm]d.south);
\draw[->,thick]([xshift=-0.03cm]h.east)to([xshift=-0.03cm]cir.west);
\draw[->,thick](point_below.north)to([yshift=2.03cm]point_below.north);
\draw[->,thick](attention_right.north)to([yshift=-0.03cm]point_above.south);
\draw[->,thick](point_above.north)to([yshift=0.83cm]point_above.north);
\draw[->,thick, in=270,out=0]([xshift=0.2cm]cir.east)to([xshift=3cm,yshift=0.88cm]cir.east);
\draw[->,thick, in=270,out=0]([xshift=0.2cm]cir.east)to([xshift=2cm,yshift=0.88cm]cir.east);
\draw[->,very thick,]([xshift=0.1cm]d.east)to([xshift=1.92cm]d.east)to([yshift=0.03cm]cir.north);
\end{tikzpicture}
\ No newline at end of file
Chapter17/Figures/figure-picture-translation.tex
......@@ -3,16 +3,16 @@
\begin {scope}
\node[draw=white,scale=0.6] (input) at (0,0){\includegraphics[width=0.62\textwidth]{./Chapter17/Figures/figure-bank-without-attention.jpg}};(1.9,-1.4);
\node[anchor=west] (label1) at ([xshift=-3.5em]input.west) {\begin{tabular}{l}{\normalsize{图片：}}\end{tabular}};
\node[anchor=south] (label2) at ([yshift=-6em]label1.south) {\begin{tabular}{l}{\normalsize{源文：}}\end{tabular}};
\node[anchor=south] (english1) at ([xshift=-0.35em,yshift=-2.3em]input.south) {\begin{tabular}{l}{\large{A\; girl\; jumps\; off\; a\; {\red{\underline{bank}}}\quad .}}\end{tabular}};
\node[anchor=south] (label2) at ([yshift=-6em]label1.south) {\begin{tabular}{l}{\normalsize{源语言：}}\end{tabular}};
\node[anchor=south] (english1) at ([xshift=-0.28em,yshift=-2.3em]input.south) {\begin{tabular}{l}{\large{A\; girl\; jumps\; off\; a\; {\red{\underline{bank}}}\quad .}}\end{tabular}};
\draw[decorate,decoration={brace,amplitude=4mm},very thick] ([xshift=6.3em]input.90) -- ([xshift=0.3em,yshift=-0em]english1.east);
\draw[decorate,decoration={brace,amplitude=4mm},thick] ([xshift=6.3em]input.90) -- ([xshift=0.3em,yshift=-0em]english1.east);
\node[anchor=east,rectangle,thick,rounded corners,minimum width=3.5em,minimum height=2.5em,text centered,draw=black!70,fill=red!25](trans)at ([xshift=7.4em,yshift=5.27em]english1.east){\normalsize{翻译模型}};
\draw[->,very thick]([xshift=-1.4em]trans.west) to (trans.west);
\draw[->,very thick](trans.east) to ([xshift=1.4em]trans.east);
\node[anchor=east] (de1) at ([xshift=4.5cm,yshift=-0.1em]trans.east) {\begin{tabular}{l}{\normalsize{译文：}}{\normalsize{一个/女孩/从/{\red{河床}}/}}\end{tabular}};
\node[anchor=south] (de2) at ([xshift=-0em,yshift=-1.5em]de1.south) {\begin{tabular}{l}{\normalsize{上/跳下来/。}} \end{tabular}};
\draw[->,thick]([xshift=-1.4em]trans.west) to (trans.west);
\node[anchor=east] (de1) at ([xshift=5.2cm,yshift=-0.1em]trans.east) {\begin{tabular}{l}{\normalsize{目标语言：}}{\normalsize{一个/女孩/从/{\red{河床}}/}}\end{tabular}};
\node[anchor=south] (de2) at ([xshift=1.1em,yshift=-1.5em]de1.south) {\begin{tabular}{l}{\normalsize{上/跳下来/。}} \end{tabular}};
\draw[->,thick](trans.east) to ([xshift=0.5em,yshift=0.1em]de1.west);
\end {scope}
\end{tikzpicture}
\ No newline at end of file
Chapter17/Figures/figure-speech-recognition-model-based-on-transformer.tex
\begin{tikzpicture}
\tikzstyle{layer}=[draw,rounded corners=2pt,font=\scriptsize,align=center,minimum width=5em]
\tikzstyle{layer}=[draw,rounded corners=2pt,font=\scriptsize,align=center,minimum width=7.1em]
\tikzstyle{word}=[font=\scriptsize]
%%%%encoder
\node[layer,fill=red!20] (en_sa) at (0,0){Multi-Head \\ Attention};
\node[layer,anchor=south,fill=green!20] (en_ffn) at ([yshift=1.4em]en_sa.north){Feed Forward \\ Network};
\node[draw,circle,inner sep=0pt, minimum size=1em,anchor=north] (en_add) at ([yshift=-1.4em]en_sa.south){};
\draw[] (en_add.90) -- (en_add.-90);
\draw[] (en_add.0) -- (en_add.180);
\node[layer,anchor=north,fill=yellow!20] (en_cnn) at ([yshift=-1.4em]en_add.south){CNN};
\node[anchor=south,layer,fill=yellow!20](en_add1) at ([yshift=1.0em]en_sa.north) {Add \& LayerNorm};
\node[layer,anchor=south,fill=green!20] (en_ffn) at ([yshift=1.0em]en_add1.north){Feed Forward \\ Network};
\node[anchor=south,layer,fill=yellow!20](en_add2) at ([yshift=1.0em]en_ffn.north) {Add \& LayerNorm};
\node[draw,circle,inner sep=0pt, minimum size=1em,anchor=north,thick] (en_add) at ([yshift=-1.4em]en_sa.south){};
\draw[thick] (en_add.90) -- (en_add.-90);
\draw[thick] (en_add.0) -- (en_add.180);
\node[layer,anchor=north,fill=yellow!20] (en_cnn) at ([yshift=-1.0em]en_add.south){CNN};
\node[anchor=east,font=\scriptsize,align=center] (en_pos) at ([xshift=-2em]en_add.west){位置编码};
\node[anchor=north,font=\scriptsize,align=center] (en_input) at ([yshift=-1em]en_cnn.south){语音特征\\（FBank/MFCC）};
\draw[->,thick] (en_input.90) -- ([yshift=-0.1em]en_cnn.-90);
\draw[->,thick] ([yshift=0.1em]en_cnn.90) -- ([yshift=-0.1em]en_add.-90);
\draw[->,thick] ([yshift=0.1em]en_add.90) -- ([yshift=-0.1em]en_sa.-90);
\draw[->,thick] ([yshift=0.1em]en_sa.90) -- ([yshift=-0.1em]en_add1.-90);
\draw[->,thick] ([yshift=0.1em]en_add1.90) -- ([yshift=-0.1em]en_ffn.-90);
\draw[->,thick] ([yshift=0.1em]en_ffn.90) --([yshift=-0.1em]en_add2.-90);
\draw[->,rounded corners=2pt,thick] ([yshift=-0.6em]en_sa.south)--([yshift=-0.6em,xshift=-4.0em]en_sa.south)--([xshift=-0.43em]en_add1.west)--(en_add1.west);
\draw[->,rounded corners=2pt,thick] ([yshift=-0.6em]en_ffn.south)--([yshift=-0.6em,xshift=-4.0em]en_ffn.south)--([xshift=-0.43em]en_add2.west)--(en_add2.west);
\node[draw,circle,inner sep=0pt, minimum size=1em,anchor=west] (de_add) at ([xshift=7em]en_add.east){};
\draw[] (de_add.90) -- (de_add.-90);
\draw[] (de_add.0) -- (de_add.180);
%%%%decoder
\node[draw,circle,inner sep=0pt, minimum size=1em,anchor=west,thick] (de_add) at ([xshift=9em]en_add.east){};
\draw[thick] (de_add.90) -- (de_add.-90);
\draw[thick] (de_add.0) -- (de_add.180);
\node[layer,anchor=south,fill=red!20] (de_sa) at ([yshift=1.4em]de_add.north){Masked \\Multi-Head\\Attention};
\node[layer,anchor=south,fill=red!20] (de_ca) at ([yshift=1.4em]de_sa.north){Multi-Head \\ Attention};
\node[layer,anchor=south,fill=green!20] (de_ffn) at ([yshift=1.4em]de_ca.north){Feed Forward \\ Network};
\node[layer,anchor=south,fill=blue!20] (sf) at ([yshift=1.6em]de_ffn.north){Softmax};
%\node[layer,anchor=south,fill=orange!20] (output) at ([yshift=1.4em]sf.north){Output Probabilities};
\node[anchor=north,font=\scriptsize,align=center] (en_input) at ([yshift=-1em]en_cnn.south){语音特征\\(FBank/MFCC)};
\node[anchor=south,layer,fill=yellow!20](de_add1) at ([yshift=1.0em]de_sa.north) {Add \& LayerNorm};
\node[layer,anchor=south,fill=red!20] (de_ca) at ([yshift=1.0em]de_add1.north){Multi-Head \\ Attention};
\node[anchor=south,layer,fill=yellow!20](de_add2) at ([yshift=1.0em]de_ca.north) {Add \& LayerNorm};
\node[layer,anchor=south,fill=green!20] (de_ffn) at ([yshift=1.0em]de_add2.north){Feed Forward \\ Network};
\node[anchor=south,layer,fill=yellow!20](de_add3) at ([yshift=1.0em]de_ffn.north) {Add \& LayerNorm};
\node[layer,anchor=south,fill=blue!20] (sf) at ([yshift=1.2em]de_add3.north){Softmax};
\node[anchor=north,font=\scriptsize,align=center] (de_input) at ([yshift=-1.1em]de_add.south){标注文本\\编码表示};
\node[anchor=east,font=\scriptsize,align=center] (en_pos) at ([xshift=-2em]en_add.west){位置编码};
\node[anchor=west,font=\scriptsize,align=center] (de_pos) at ([xshift=2em]de_add.east){位置编码};
\draw[->] (en_input.90) -- ([yshift=-0.1em]en_cnn.-90);
\draw[->] ([yshift=0.1em]en_cnn.90) -- ([yshift=-0.1em]en_add.-90);
\draw[->] ([yshift=0.1em]en_add.90) -- ([yshift=-0.1em]en_sa.-90);
\draw[->] ([yshift=0.1em]en_sa.90) -- ([yshift=-0.1em]en_ffn.-90);
\draw[->] (de_input.90) -- ([yshift=-0.1em]de_add.-90);
\draw[->] ([yshift=0.1em]de_add.90) -- ([yshift=-0.1em]de_sa.-90);
\draw[->] ([yshift=0.1em]de_sa.90) -- ([yshift=-0.1em]de_ca.-90);
\draw[->] ([yshift=0.1em]de_ca.90) -- ([yshift=-0.1em]de_ffn.-90);
\draw[->] ([yshift=0.1em]de_ffn.90) -- ([yshift=-0.1em]sf.-90);
\draw[->] ([yshift=0.1em]sf.90) -- ([yshift=1.5em]sf.90);
\draw[->] ([xshift=0.1em]en_pos.0) -- ([xshift=-0.1em]en_add.180);
\draw[->] ([xshift=-0.1em]de_pos.180) -- ([xshift=0.1em]de_add.0);
\draw[->,rounded corners=2pt] ([yshift=0.1em]en_ffn.90) -- ([yshift=2em]en_ffn.90) -- ([xshift=4em,yshift=2em]en_ffn.90) -- ([xshift=-1.5em]de_ca.west) -- ([xshift=-0.1em]de_ca.west);
\draw[->,thick] (de_input.90) -- ([yshift=-0.1em]de_add.-90);
\draw[->,thick] ([yshift=0.1em]de_add.90) -- ([yshift=-0.1em]de_sa.-90);
\draw[->,thick] ([yshift=0.1em]de_sa.90) -- ([yshift=-0.1em]de_add1.-90);
\draw[->,thick] ([yshift=0.1em]de_add1.90) -- ([yshift=-0.1em]de_ca.-90);
\draw[->,thick] ([yshift=0.1em]de_ca.90) -- ([yshift=-0.1em]de_add2.-90);
\draw[->,thick] ([yshift=0.1em]de_add2.90) -- ([yshift=-0.1em]de_ffn.-90);
\draw[->,thick] ([yshift=0.1em]de_ffn.90) -- ([yshift=-0.1em]de_add3.-90);
\draw[->,thick] ([yshift=0.1em]de_add3.90) -- ([yshift=-0.1em]sf.-90);
\draw[->,thick] ([yshift=0.1em]sf.90) -- ([yshift=1.0em]sf.90);
\draw[->,thick] ([xshift=0.1em]en_pos.0) -- ([xshift=-0.1em]en_add.180);
\draw[->,thick] ([xshift=-0.1em]de_pos.180) -- ([xshift=0.1em]de_add.0);
\draw[->,rounded corners=2pt,thick] ([yshift=-0.6em]de_sa.south)--([yshift=-0.6em,xshift=4.0em]de_sa.south)--([xshift=0.43em]de_add1.east)--(de_add1.east);
\draw[->,rounded corners=2pt,thick] ([yshift=-0.6em]de_ca.south)--([yshift=-0.6em,xshift=4.0em]de_ca.south)--([xshift=0.43em]de_add2.east)--(de_add2.east);
\draw[->,rounded corners=2pt,thick] ([yshift=-0.6em]de_ffn.south)--([yshift=-0.6em,xshift=4.0em]de_ffn.south)--([xshift=0.43em]de_add3.east)--(de_add3.east);
\draw[->,rounded corners=2pt,thick] ([yshift=0.1em]en_add2.90) -- ([yshift=1.5em]en_add2.90) -- ([xshift=5.0em,yshift=1.5em]en_add2.90) -- ([xshift=-1.5em]de_ca.west) -- ([xshift=-0.1em]de_ca.west);
\begin{pgfonlayer}{background}
\node[draw=ugreen,rounded corners=2pt,inner xsep=6pt,inner ysep=8pt,dashed,thick][fit=(en_sa)(en_ffn)](box1){};
\node[draw=red,rounded corners=2pt,inner xsep=6pt,inner ysep=8pt,dashed,thick][fit=(de_sa)(de_ca)(de_ffn)](box2){};
\node[draw=ugreen,rounded corners=2pt,inner xsep=6pt,inner ysep=8pt,dashed,thick,xshift=-0.2em,yshift=-0.2em][fit=(en_add1)(en_add2)(en_sa)(en_ffn)](box1){};
\node[draw=red,rounded corners=2pt,inner xsep=6pt,inner ysep=8pt,dashed,thick,xshift=0.2em,yshift=-0.2em][fit=(de_sa)(de_ca)(de_ffn)(de_add3)](box2){};
\end{pgfonlayer}
\node[anchor=east,font=\scriptsize,text=ugreen] at ([xshift=-0.1em]box1.west){$N \times$};
......
Chapter17/Figures/figure-speech-translation-model-based-on-CTC.tex
\begin{tikzpicture}
\tikzstyle{layer}=[draw,rounded corners=2pt,font=\scriptsize,align=center,minimum width=5em]
\tikzstyle{layer}=[draw,rounded corners=2pt,font=\scriptsize,align=center,minimum width=7.1em]
\tikzstyle{word}=[font=\scriptsize]
%%%%encoder
\node[layer,fill=red!20] (en_sa) at (0,0){Multi-Head \\ Attention};
\node[layer,anchor=south,fill=green!20] (en_ffn) at ([yshift=1.4em]en_sa.north){Feed Forward \\ Network};
\node[draw,circle,inner sep=0pt, minimum size=1em,anchor=north] (en_add) at ([yshift=-1.4em]en_sa.south){};
\draw[] (en_add.90) -- (en_add.-90);
\draw[] (en_add.0) -- (en_add.180);
\node[layer,anchor=north,fill=yellow!20] (en_cnn) at ([yshift=-1.4em]en_add.south){CNN};
\node[anchor=south,layer,fill=yellow!20](en_add1) at ([yshift=1.0em]en_sa.north) {Add \& LayerNorm};
\node[layer,anchor=south,fill=green!20] (en_ffn) at ([yshift=1.0em]en_add1.north){Feed Forward \\ Network};
\node[anchor=south,layer,fill=yellow!20](en_add2) at ([yshift=1.0em]en_ffn.north) {Add \& LayerNorm};
\node[layer,anchor=south,fill=blue!20] (en_sf) at ([yshift=2.4em]en_add2.north){Softmax};
\node[draw,circle,inner sep=0pt, minimum size=1em,anchor=north,thick] (en_add) at ([yshift=-1.4em]en_sa.south){};
\draw[thick] (en_add.90) -- (en_add.-90);
\draw[thick] (en_add.0) -- (en_add.180);
\node[layer,anchor=north,fill=yellow!20] (en_cnn) at ([yshift=-1.0em]en_add.south){CNN};
\node[anchor=east,font=\scriptsize,align=center] (en_pos) at ([xshift=-2em]en_add.west){位置编码};
\node[anchor=north,font=\scriptsize,align=center] (en_input) at ([yshift=-1em]en_cnn.south){源语言语音特征\\（FBank/MFCC）};
\draw[->,thick] (en_input.90) -- ([yshift=-0.1em]en_cnn.-90);
\draw[->,thick] ([yshift=0.1em]en_cnn.90) -- ([yshift=-0.1em]en_add.-90);
\draw[->,thick] ([yshift=0.1em]en_add.90) -- ([yshift=-0.1em]en_sa.-90);
\draw[->,thick] ([yshift=0.1em]en_sa.90) -- ([yshift=-0.1em]en_add1.-90);
\draw[->,thick] ([yshift=0.1em]en_add1.90) -- ([yshift=-0.1em]en_ffn.-90);
\draw[->,thick] ([yshift=0.1em]en_ffn.90) --([yshift=-0.1em]en_add2.-90);
\draw[->,thick] ([yshift=0.1em]en_add2.90) -- ([yshift=-0.1em]en_sf.-90);
\draw[->,thick] ([yshift=0.1em]en_sf.90) -- ([yshift=1.5em]en_sf.90)node[left,pos=0.5]{\scriptsize{CTC Loss}};
\draw[->,rounded corners=2pt,thick] ([yshift=-0.6em]en_sa.south)--([yshift=-0.6em,xshift=-4.0em]en_sa.south)--([xshift=-0.43em]en_add1.west)--(en_add1.west);
\draw[->,rounded corners=2pt,thick] ([yshift=-0.6em]en_ffn.south)--([yshift=-0.6em,xshift=-4.0em]en_ffn.south)--([xshift=-0.43em]en_add2.west)--(en_add2.west);
\node[draw,circle,inner sep=0pt, minimum size=1em,anchor=west] (de_add) at ([xshift=7em]en_add.east){};
\draw[] (de_add.90) -- (de_add.-90);
\draw[] (de_add.0) -- (de_add.180);
%%%%decoder
\node[draw,circle,inner sep=0pt, minimum size=1em,anchor=west,thick] (de_add) at ([xshift=9em]en_add.east){};
\draw[thick] (de_add.90) -- (de_add.-90);
\draw[thick] (de_add.0) -- (de_add.180);
\node[layer,anchor=south,fill=red!20] (de_sa) at ([yshift=1.4em]de_add.north){Masked \\Multi-Head\\Attention};
\node[layer,anchor=south,fill=red!20] (de_ca) at ([yshift=1.4em]de_sa.north){Multi-Head \\ Attention};
\node[layer,anchor=south,fill=green!20] (de_ffn) at ([yshift=1.4em]de_ca.north){Feed Forward \\ Network};
\node[anchor=south,layer,fill=yellow!20](de_add1) at ([yshift=1.0em]de_sa.north) {Add \& LayerNorm};
\node[layer,anchor=south,fill=red!20] (de_ca) at ([yshift=1.0em]de_add1.north){Multi-Head \\ Attention};
\node[anchor=south,layer,fill=yellow!20](de_add2) at ([yshift=1.0em]de_ca.north) {Add \& LayerNorm};
\node[layer,anchor=south,fill=green!20] (de_ffn) at ([yshift=1.0em]de_add2.north){Feed Forward \\ Network};
\node[anchor=south,layer,fill=yellow!20](de_add3) at ([yshift=1.0em]de_ffn.north) {Add \& LayerNorm};
\node[layer,anchor=south,fill=blue!20] (sf) at ([yshift=1.2em]de_add3.north){Softmax};
\node[anchor=north,font=\scriptsize,align=center] (de_input) at ([yshift=-1.1em]de_add.south){目标语言文本\\编码表示};
\node[anchor=west,font=\scriptsize,align=center] (de_pos) at ([xshift=2em]de_add.east){位置编码};
\node[layer,anchor=south,fill=blue!20] (en_sf) at ([yshift=3em]en_ffn.north){Softmax};
\node[layer,anchor=south,fill=blue!20] (sf) at ([yshift=2em]de_ffn.north){Softmax};
\node[layer,anchor=south,fill=orange!20] (en_output) at ([yshift=1.4em]en_sf.north){CTC Output};
%\node[layer,anchor=south,fill=orange!20] (output) at ([yshift=1.4em]sf.north){ST Output};
\draw[->,thick] (de_input.90) -- ([yshift=-0.1em]de_add.-90);
\draw[->,thick] ([yshift=0.1em]de_add.90) -- ([yshift=-0.1em]de_sa.-90);
\draw[->,thick] ([yshift=0.1em]de_sa.90) -- ([yshift=-0.1em]de_add1.-90);
\draw[->,thick] ([yshift=0.1em]de_add1.90) -- ([yshift=-0.1em]de_ca.-90);
\draw[->,thick] ([yshift=0.1em]de_ca.90) -- ([yshift=-0.1em]de_add2.-90);
\draw[->,thick] ([yshift=0.1em]de_add2.90) -- ([yshift=-0.1em]de_ffn.-90);
\draw[->,thick] ([yshift=0.1em]de_ffn.90) -- ([yshift=-0.1em]de_add3.-90);
\draw[->,thick] ([yshift=0.1em]de_add3.90) -- ([yshift=-0.1em]sf.-90);
\draw[->,thick] ([yshift=0.1em]sf.90) -- ([yshift=1.0em]sf.90);
\draw[->,thick] ([xshift=0.1em]en_pos.0) -- ([xshift=-0.1em]en_add.180);
\draw[->,thick] ([xshift=-0.1em]de_pos.180) -- ([xshift=0.1em]de_add.0);
\draw[->,rounded corners=2pt,thick] ([yshift=-0.6em]de_sa.south)--([yshift=-0.6em,xshift=4.0em]de_sa.south)--([xshift=0.43em]de_add1.east)--(de_add1.east);
\draw[->,rounded corners=2pt,thick] ([yshift=-0.6em]de_ca.south)--([yshift=-0.6em,xshift=4.0em]de_ca.south)--([xshift=0.43em]de_add2.east)--(de_add2.east);
\draw[->,rounded corners=2pt,thick] ([yshift=-0.6em]de_ffn.south)--([yshift=-0.6em,xshift=4.0em]de_ffn.south)--([xshift=0.43em]de_add3.east)--(de_add3.east);
\node[anchor=north,font=\scriptsize,align=center] (en_input) at ([yshift=-1em]en_cnn.south){语音特征\\(FBank/MFCC)};
\node[anchor=north,font=\scriptsize,align=center] (de_input) at ([yshift=-1em]de_add.south){标注文本\\编码表示};
\draw[->,rounded corners=2pt,thick] ([yshift=0.1em]en_add2.90) -- ([yshift=1.5em]en_add2.90) -- ([xshift=5.0em,yshift=1.5em]en_add2.90) -- ([xshift=-1.5em]de_ca.west) -- ([xshift=-0.1em]de_ca.west);
\node[anchor=east,font=\scriptsize,align=center] (en_pos) at ([xshift=-2em]en_add.west){位置编码};
\node[anchor=west,font=\scriptsize,align=center] (de_pos) at ([xshift=2em]de_add.east){位置编码};
\draw[->] (en_input.90) -- ([yshift=-0.1em]en_cnn.-90);
\draw[->] ([yshift=0.1em]en_cnn.90) -- ([yshift=-0.1em]en_add.-90);
\draw[->] ([yshift=0.1em]en_add.90) -- ([yshift=-0.1em]en_sa.-90);
\draw[->] ([yshift=0.1em]en_sa.90) -- ([yshift=-0.1em]en_ffn.-90);
\draw[->] (de_input.90) -- ([yshift=-0.1em]de_add.-90);
\draw[->] ([yshift=0.1em]de_add.90) -- ([yshift=-0.1em]de_sa.-90);
\draw[->] ([yshift=0.1em]de_sa.90) -- ([yshift=-0.1em]de_ca.-90);
\draw[->] ([yshift=0.1em]de_ca.90) -- ([yshift=-0.1em]de_ffn.-90);
\draw[->] ([yshift=0.1em]en_ffn.90) -- ([yshift=-0.1em]en_sf.-90);
\draw[->] ([yshift=0.1em]en_sf.90) -- ([yshift=-0.1em]en_output.-90);
\draw[->] ([yshift=0.1em]de_ffn.90) -- ([yshift=-0.1em]sf.-90);
\draw[->] ([yshift=0.1em]sf.90) -- ([yshift=1.5em]sf.90);
\draw[->] ([xshift=0.1em]en_pos.0) -- ([xshift=-0.1em]en_add.180);
\draw[->] ([xshift=-0.1em]de_pos.180) -- ([xshift=0.1em]de_add.0);
\draw[->,rounded corners=2pt] ([yshift=2em]en_ffn.90) -- ([xshift=4em,yshift=2em]en_ffn.90) -- ([xshift=-1.5em]de_ca.west) -- ([xshift=-0.1em]de_ca.west);
\begin{pgfonlayer}{background}
\node[draw=ugreen,rounded corners=2pt,inner xsep=6pt,inner ysep=8pt,dashed,thick][fit=(en_sa)(en_ffn)]{};
\node[draw=red,rounded corners=2pt,inner xsep=6pt,inner ysep=8pt,dashed,thick][fit=(de_sa)(de_ca)(de_ffn)]{};
\node[draw=ugreen,rounded corners=2pt,inner xsep=6pt,inner ysep=8pt,dashed,thick,xshift=-0.2em,yshift=-0.2em][fit=(en_add1)(en_add2)(en_sa)(en_ffn)](box1){};
\node[draw=red,rounded corners=2pt,inner xsep=6pt,inner ysep=8pt,dashed,thick,xshift=0.2em,yshift=-0.2em][fit=(de_sa)(de_ca)(de_ffn)(de_add3)](box2){};
\end{pgfonlayer}
\node[anchor=east,font=\scriptsize,text=ugreen] at ([xshift=-0.1em]box1.west){$N \times$};
\node[anchor=west,font=\scriptsize,text=red] at ([xshift=0.1em]box2.east){$\times N$};
\node[anchor=east,font=\scriptsize] at ([xshift=-0.1em]en_cnn.west){$2 \times$};
\node[anchor=east,font=\scriptsize,align=center,text=ugreen] at ([xshift=-0.1em,yshift=3em]box1.west){ST\\ 编码器};
\node[anchor=west,font=\scriptsize,align=center,text=red] at ([xshift=0.1em,yshift=5em]box2.east){ST\\解码器};
\node[anchor=east,font=\scriptsize,align=center,text=ugreen] at ([xshift=-0.1em,yshift=3em]box1.west){ST \\ 编码器};
\node[anchor=west,font=\scriptsize,align=center,text=red] at ([xshift=0.1em,yshift=5em]box2.east){ST \\ 解码器};
\end{tikzpicture}
\ No newline at end of file
Chapter17/Figures/figure-three-ways-of-dual-decoder-speech-translation.tex
\tikzstyle{coder} = [rectangle,thick,rounded corners,minimum height=2.2em,minimum width=4.3em,text centered,draw=black!70,fill=red!20]
\tikzstyle{coder} = [rectangle,thick,rounded corners,minimum height=2.2em,minimum width=4.3em,text centered,draw=black,fill=red!20]
\begin{tikzpicture}[node distance = 0,scale = 0.75]
\tikzstyle{every node}=[scale=0.75]
......
Chapter17/Figures/figure-traditional-methods-of-image-description.tex
......@@ -2,31 +2,31 @@
\definecolor{color_green}{rgb}{0.663,0.82,0.557}
\definecolor{color_orange}{rgb}{0.957,0.694,0.514}
\definecolor{color_blue}{rgb}{0.335,0.708,0.735}
\tikzstyle{description} = [rectangle,rounded corners=1mm, minimum width=3cm,minimum height=0.6cm,text centered]
\tikzstyle{description} = [rectangle,rounded corners=1mm, minimum width=3cm,minimum height=0.6cm,text centered,draw,thick]
\begin{tikzpicture}[node distance = 0,scale = 0.8]
\tikzstyle{every node}=[scale=0.8]
\node(figure-1)[draw=white,scale=0.25] at (0,0){\includegraphics[width=0.62\textwidth]{./Chapter17/Figures/figure-dog-with-hat.png}};
\node(ground-1)[rectangle,rounded corners, minimum width=5cm, minimum height=3.5cm,right of = figure-1, xshift=5cm,fill=blue!20]{};
\node(text-1)[right of = figure-1, xshift=3.6cm,yshift=2cm,scale=1.2]{\textcolor{color_gray}{描述候选池}};
\node(text_1-1)[description, right of = figure-1, xshift=4.2cm,yshift=1.2cm,fill=color_gray!50]{\textcolor{white}{天空中有很多鸟。}};
\node(text_2-1)[description, right of = figure-1, xshift=5.3cm,yshift=0.5cm,fill=color_green]{\textcolor{white}{孩子从河岸上跳下来。}};
\node(text_3-1)[description, right of = figure-1, xshift=4.5cm,yshift=-0.2cm,fill=color_orange]{\textcolor{white}{狗在吐舌头。}};
\node(ground-1)[rectangle,rounded corners, minimum width=5cm, minimum height=3.5cm,right of = figure-1, xshift=5cm,fill=gray!10,draw,thick,drop shadow]{};
\node(text-1)[right of = figure-1, xshift=3.6cm,yshift=2.1cm,scale=1.2]{{描述候选池}};
\node(text_1-1)[description, right of = figure-1, xshift=4.2cm,yshift=1.2cm,fill=gray!20]{{天空中有很多鸟。}};
\node(text_2-1)[description, right of = figure-1, xshift=5.3cm,yshift=0.5cm,fill=green!20]{{孩子从河岸上跳下来。}};
\node(text_3-1)[description, right of = figure-1, xshift=4.5cm,yshift=-0.2cm,fill=orange!20]{{狗在吐舌头。}};
\node(surd-1)[right of = text_3-1, xshift=2cm,scale=1.5]{\textcolor{red}{$\surd$}};
\node(text_4-1)[description, right of = figure-1, xshift=5.2cm,yshift=-0.9cm,fill=color_blue]{\textcolor{white}{男人戴着眼镜。}};
\node(text_4-1)[description, right of = figure-1, xshift=5.2cm,yshift=-0.9cm,fill=blue!20]{{男人戴着眼镜。}};
\node(point-1)[right of = figure-1, xshift=5cm,yshift=-1.4cm,scale=1.5]{...};
\draw[->,thick](figure-1)to([xshift=-0.1cm]ground-1.west);
\node(figure)[draw=white,scale=0.25]at ([xshift=20.0em]figure-1.east){\includegraphics[width=0.62\textwidth]{./Chapter17/Figures/figure-dog-with-hat.png}};
\node(ground)[rectangle,rounded corners, minimum width=5cm, minimum height=1.5cm,right of = figure, xshift=5cm,yshift=-2.6em,fill=blue!20]{\large{图片中有\underline{\textcolor{red}{狗}}，\underline{\textcolor{red}{帽子}}，\underline{\quad\ }。}};
\node(dog)[rectangle,rounded corners, minimum width=1cm, minimum height=0.7cm,right of = figure, xshift=3cm,yshift=1.5cm,thick, draw=color_orange,fill=color_orange!50]{狗};
\node(hat)[rectangle,rounded corners, minimum width=1.5cm, minimum height=0.7cm,right of = figure, xshift=4.5cm,yshift=1.5cm,thick, draw=color_green,fill=color_green!50]{帽子};
\draw[->, thick,color=black!60](figure.east)to([xshift=-0.1cm]dog.west)node[left,xshift=-0.2cm,yshift=-0.1cm,color=black]{图片检测};
\draw[->, thick,color=black!60]([yshift=-0.1cm]hat.south)to([yshift=0.1cm]ground.north)node[right,xshift=-0.2cm,yshift=0.5cm,color=black]{模板填充};
\node(ground)[rectangle,rounded corners, minimum width=5cm, minimum height=1.5cm,right of = figure, xshift=5cm,yshift=-2.6em,fill=gray!10,draw,thick,drop shadow]{\large{图片中有\underline{\textcolor{red}{狗}}，\underline{\textcolor{red}{帽子}}，\underline{\quad\ }。}};
\node(dog)[rectangle,rounded corners, minimum width=1cm, minimum height=0.7cm,right of = figure, xshift=3cm,yshift=1.5cm,thick, draw,fill=orange!20,thick]{狗};
\node(hat)[rectangle,rounded corners, minimum width=1.5cm, minimum height=0.7cm,right of = figure, xshift=4.5cm,yshift=1.5cm,thick, draw,fill=green!20,thick]{帽子};
\draw[->, thick](figure.east)to([xshift=-0.1cm]dog.west)node[left,xshift=-0.2cm,yshift=-0.1cm,color=black]{图片检测};
\draw[->, thick]([yshift=-0.1cm]hat.south)to([yshift=0.1cm]ground.north)node[right,xshift=-0.2cm,yshift=0.5cm,color=black]{模板填充};
\node [anchor=north](pos1)at ([xshift=-3.8em,yshift=-0.5em]ground-1.south){(a) 基于检索的图像描述生成};
\node [anchor=north](pos2)at ([xshift=-3.8em,yshift=-0.5em]ground.south){(b) 基于模板的图像描述生成};
\node [anchor=north](pos1)at ([xshift=-3.8em,yshift=-1em]ground-1.south){(a) 基于检索的图像描述生成};
\node [anchor=north](pos2)at ([xshift=-3.8em,yshift=-1em]ground.south){(b) 基于模板的图像描述生成};
\end{tikzpicture}
\ No newline at end of file
Chapter2/Figures/figure-word-frequency-distribution.tex
\definecolor{ublue}{rgb}{0.152,0.250,0.545}
\begin{tikzpicture}
\begin{axis}[
width=11cm,
width=13cm,
height=5.5cm,
xlabel={WikiText-103上的词表},
ylabel={词汇出现总次数},
......
Chapter2/chapter2.tex
......@@ -862,9 +862,6 @@ c(\cdot) & \textrm{当计算最高阶模型时} \\
\parinterval 当任务对单词序列长度没有限制时，上述两种方法枚举出的单词序列也是无穷无尽的。因此这两种枚举策略并不具备完备性而且会导致枚举过程无法停止。由于日常生活中通常不会见到特别长的句子，因此可以通过限制单词序列的最大长度来避免这个问题。一旦单词序列的最大长度被确定，以上两种枚举策略就可以在一定时间内枚举出所有可能的单词序列，因而一定可以找到最优的单词序列，即具备最优性。
\parinterval 此时上述生成策略虽然可以满足完备性和最优性，但其仍然算不上是优秀的生成策略，因为这两种算法在时间复杂度和空间复杂度上的表现很差，如表\ref{tab:2-4}所示。其中$|V|$为词表大小，$m$ 为序列长度。值得注意的是，在之前的遍历过程中，除了在序列开头一定会挑选<sos>之外，其他位置每次可挑选的单词并不只有词表中的单词，还有结束符号<eos>，因此实际上生成过程中每个位置的单词候选数量为$|V|+1$。
\vspace{0.5em}
%------------------------------------------------------
\begin{table}[htp]{
\begin{center}
......@@ -881,6 +878,8 @@ c(\cdot) & \textrm{当计算最高阶模型时} \\
}\end{table}
%------------------------------------------------------
\parinterval 此时上述生成策略虽然可以满足完备性和最优性，但其仍然算不上是优秀的生成策略，因为这两种算法在时间复杂度和空间复杂度上的表现很差，如表\ref{tab:2-4}所示。其中$|V|$为词表大小，$m$ 为序列长度。值得注意的是，在之前的遍历过程中，除了在序列开头一定会挑选<sos>之外，其他位置每次可挑选的单词并不只有词表中的单词，还有结束符号<eos>，因此实际上生成过程中每个位置的单词候选数量为$|V|+1$。
\parinterval 那么是否有比枚举策略更高效的方法呢？答案是肯定的。一种直观的方法是将搜索的过程表示成树型结构，称为解空间树。它包含了搜索过程中可生成的全部序列。该树的根节点恒为<sos>，代表序列均从<sos> 开始。该树结构中非叶子节点的兄弟节点有$|V|+1$个，由词表和结束符号<eos>构成。从图\ref{fig:2-13}可以看到，对于一个最大长度为4的序列的搜索过程，生成某个单词序列的过程实际上就是访问解空间树中从根节点<sos> 开始一直到叶子节点<eos>结束的某条路径，而这条的路径上节点按顺序组成了一段独特的单词序列。此时对所有可能单词序列的枚举就变成了对解空间树的遍历。并且枚举的过程与语言模型打分的过程也是一致的，每枚举一个词$i$也就是在图\ref{fig:2-13}选择$w_i$一列的一个节点，语言模型就可以为当前的树节点$w_i$给出一个分值，即$\funp{P}(w_i | w_1 w_2 \ldots w_{i-1})$。对于$n$-gram语言模型，这个分值可以表示为$\funp{P}(w_i | w_1 w_2 \ldots w_{i-1})=\funp{P}(w_i | w_{i-n+1} \ldots w_{i-1})$
%----------------------------------------------
......
Chapter4/Figures/figure-schematic-diagram-of-phrase-level-quality-assessment-task.tex
......@@ -50,10 +50,10 @@
\node[bad_tag,anchor=north] at ([yshift=-2pt]box2_3.south){BAD};
\node[ok_tag,anchor=north] (tag_1) at ([yshift=-2pt]box2_4.south){OK};
\node[ok_tag,anchor=north] (gap_1) at ([xshift=-5.8em,yshift=-3em]box2_1.south){OK};
\node[bad_tag,anchor=north] (gap_2) at ([xshift=5.6em,yshift=-3em]box2_1.south){BAD\_omission};
\node[ok_tag,anchor=north] (gap_3) at ([xshift=5.6em,yshift=-3em]box2_2.south){OK};
\node[ok_tag,anchor=north] (gap_4) at ([xshift=6.4em,yshift=-3em]box2_3.south){OK};
\node[ok_tag,anchor=north] (gap_1) at ([xshift=-7em,yshift=-3em]box2_1.south){OK};
\node[bad_tag,anchor=north] (gap_2) at ([xshift=7em,yshift=-3em]box2_1.south){BAD\_omission};
\node[ok_tag,anchor=north] (gap_3) at ([xshift=6.8em,yshift=-3em]box2_2.south){OK};
\node[ok_tag,anchor=north] (gap_4) at ([xshift=7.5em,yshift=-3em]box2_3.south){OK};
\node[ok_tag,anchor=north] (tag_2) at ([xshift=2.5em,yshift=-3em]box2_4.south){OK};
......
Chapter4/Figures/figure-schematic-diagram-of-word-level-quality-assessment-task.tex
......@@ -56,14 +56,14 @@
\node[ok_tag,anchor=north] at ([yshift=-3pt]n3_8.south){OK};
\node[ok_tag,anchor=north] (tag2) at ([yshift=-3pt]n3_9.south){OK};
\node[ok_tag,anchor=north] (gap_1)at ([xshift=-2.6em,yshift=-2em]n3_2.south){OK};
\node[bad_tag,anchor=north] (gap_2)at ([xshift=2.55em,yshift=-2em]n3_2.south){BAD};
\node[ok_tag,anchor=north] (gap_3)at ([xshift=1.85em,yshift=-2em]n3_3.south){OK};
\node[ok_tag,anchor=north] (gap_4)at ([xshift=2.3em,yshift=-2em]n3_4.south){OK};
\node[ok_tag,anchor=north] (gap_5)at ([xshift=1.5em,yshift=-2em]n3_5.south){OK};
\node[ok_tag,anchor=north] (gap_6)at ([xshift=1.8em,yshift=-2em]n3_6.south){OK};
\node[ok_tag,anchor=north] (gap_7)at ([xshift=2.0em,yshift=-2em]n3_7.south){OK};
\node[ok_tag,anchor=north] (gap_8)at ([xshift=1.60em,yshift=-2em]n3_8.south){OK};
\node[ok_tag,anchor=north] (gap_1)at ([xshift=-3.2em,yshift=-2em]n3_2.south){OK};
\node[bad_tag,anchor=north] (gap_2)at ([xshift=3.5em,yshift=-2em]n3_2.south){BAD};
\node[ok_tag,anchor=north] (gap_3)at ([xshift=2.1em,yshift=-2em]n3_3.south){OK};
\node[ok_tag,anchor=north] (gap_4)at ([xshift=3em,yshift=-2em]n3_4.south){OK};
\node[ok_tag,anchor=north] (gap_5)at ([xshift=1.85em,yshift=-2em]n3_5.south){OK};
\node[ok_tag,anchor=north] (gap_6)at ([xshift=2em,yshift=-2em]n3_6.south){OK};
\node[ok_tag,anchor=north] (gap_7)at ([xshift=2.25em,yshift=-2em]n3_7.south){OK};
\node[ok_tag,anchor=north] (gap_8)at ([xshift=1.75em,yshift=-2em]n3_8.south){OK};
\node[ok_tag,anchor=north] (tag3) at ([xshift=1.7em,yshift=-2em]n3_9.south){OK};
\draw[dash pattern=on 2pt off 1pt,gray,line width=1pt](gap_1.north) -- ([yshift=2em]gap_1.north);
......
Chapter8/Figures/figure-combination-of-translation-with-different-rules.tex
......@@ -5,7 +5,7 @@
\begin{scope}%[scale=0.2]
\node[anchor=north] (q1) at (0,0) {\scriptsize\sffamily\bfseries{输入字符串：}};
\node[anchor=west] (q2) at ([xshift=0em,yshift=-2em]q1.west) {\footnotesize{进口$\quad$和$\quad$出口$\quad$大幅度$\quad$下降$\quad$了}};
\node[anchor=west] (q2) at ([xshift=0em,yshift=-1.5em]q1.west) {\footnotesize{进口$\quad$和$\quad$出口$\quad$大幅度$\quad$下降$\quad$了}};
\node[anchor=north,fill=blue!20,minimum height=4em,minimum width=1em] (f1) at ([xshift=2.7em,yshift=-0.7em]q2.south) {};
......
Chapter8/Figures/figure-hierarchical-phrase-rule-match-generate.tex
......@@ -5,7 +5,7 @@
\begin{scope}%[scale=0.2]
\node[anchor=north] (q1) at (0,0) {\scriptsize\sffamily\bfseries{输入字符串：}};
\node[anchor=west] (q2) at ([xshift=0em,yshift=-2em]q1.west) {\footnotesize{进口$\quad$和$\quad$出口$\quad$大幅度$\quad$下降$\quad$了}};
\node[anchor=west] (q2) at ([xshift=0em,yshift=-1.5em]q1.west) {\footnotesize{进口$\quad$和$\quad$出口$\quad$大幅度$\quad$下降$\quad$了}};
\node[anchor=north,fill=blue!20,minimum height=1em,minimum width=1em] (f1) at ([xshift=-4.2em,yshift=-0.8em]q2.south) {};
......
Chapter8/Figures/figure-one-best-node-alignment-and-alignment-matrix.tex
......@@ -107,7 +107,7 @@
\vspace{-1em}
\footnotesize{(a)节点对齐矩阵（1-best vs Matrix）}
\end{center}
\vspace{-3em}
\begin{center}
\begin{tabular}[t]{C{0.48\linewidth} C{0.48\linewidth} }
......
Chapter9/Figures/figure-activate-2.tex 0 → 100644
%%%------------------------------------------------------------------------------------------------------------
\begin{tikzpicture}
\tikzstyle{every node}=[scale=1.2]
\begin{scope}
\draw[->, line width=1pt](-1.4,0)--(1.4,0)node[left,below,font=\scriptsize]{$x$};
\draw[->, line width=1pt](0,-1.2)--(0,1.4)node[right,font=\scriptsize]{$y$};
\foreach \x in {-1.0,-0.5,0.5,1.0}{\draw(\x,0)--(\x,0.05)node[below,outer sep=2pt,font=\scriptsize,,scale=0.8]at(\x,0.1){\x};}
\node[below,outer sep=2pt,font=\scriptsize,scale=0.8]at(0.1,0.1){0};
\foreach \y in {0.5,1.0}{\draw(0,\y)--(0.05,\y)node[left,outer sep=2pt,font=\scriptsize,,scale=0.8]at(0.1,\y){\y};}
\draw[color=red ,domain=-1.4:1.2, line width=1pt]plot(\x,{max(\x,0)});
\node[black,anchor=south,scale=0.8] at (0,1.6) {\small $y =\max (0, x)$};
\node [anchor=south east,inner sep=1pt,scale=0.8] (labeld) at (0.8,-2) {\small{(a) ReLU}};
\end{scope}
%%%------------------------------------------------------------------------------------------------------------
\begin{scope}[xshift=1.7in]
\draw[->, line width=1pt](-1.4,0)--(1.4,0)node[left,below,font=\scriptsize]{$x$};
\draw[->, line width=1pt](0,-1.2)--(0,1.4)node[right,font=\scriptsize]{$y$};
\foreach \x in {-1.0,-0.5,0.5,1.0}{\draw(\x,0)--(\x,0.05)node[below,outer sep=2pt,font=\scriptsize,scale=0.8]at(\x,0.1){\x};}
\node[below,outer sep=2pt,font=\scriptsize,scale=0.8]at(0.1,0.1){0};
\foreach \y in {0.5,1.0}{\draw(0,\y)--(0.05,\y)node[left,outer sep=2pt,font=\scriptsize,scale=0.8]at(0.1,\y){};}
\node[left,outer sep=2pt,font=\scriptsize,scale=0.8]at(0.1,0.5){0.5};
\node[left,outer sep=2pt,font=\scriptsize,scale=0.8]at(0.1,1.1){1.0};
\draw[color=red ,domain=-1.4:1.4, line width=1pt]plot(\x,{exp(-1*((\x)^2))});
\node[black,anchor=south,scale=0.8] at (0,1.6) {\small $y ={\textrm e}^{-x^2}$};
\node [anchor=south east,inner sep=1pt,scale=0.8] (labele) at (0.8,-2) {\small{(b) Gaussian}};
\end{scope}
%%%------------------------------------------------------------------------------------------------------------
\begin{scope}[xshift=3.4in]
\draw[->, line width=1pt](-1.4,0)--(1.4,0)node[left,below,font=\scriptsize]{$x$};
\draw[->, line width=1pt](0,-1.2)--(0,1.4)node[right,font=\scriptsize]{$y$};
\foreach \x in {-1.0,-0.5,0.5,1.0}{\draw(\x,0)--(\x,0.05)node[below,outer sep=2pt,font=\scriptsize,scale=0.8]at(\x,0.1){\x};}
\node[below,outer sep=2pt,font=\scriptsize,scale=0.8]at(0.1,0.1){0};
\foreach \y in {0.5,1.0}{\draw(0,\y)--(0.05,\y)node[left,outer sep=2pt,font=\scriptsize,scale=0.8]at(0.1,\y){\y};}
\draw[color=red ,domain=-1:1, line width=1pt]plot(\x,\x);
\node[black,anchor=south,scale=0.8] at (0,1.6) {\small $y =x$};
\node [anchor=south east,inner sep=1pt,scale=0.8] (labelf) at (0.8,-2) {\small{(c) Identity}};
\end{scope}
\end{tikzpicture}
%%%------------------------------------------------------------------------------------------------------------
\ No newline at end of file
Chapter9/Figures/figure-activate.tex
%%%------------------------------------------------------------------------------------------------------------
\begin{tikzpicture}
\tikzstyle{every node}=[scale=1.2]
\begin{scope}
\draw[->, line width=1pt](-1.4,0)--(1.4,0)node[left,below,font=\scriptsize]{$x$};
\draw[->, line width=1pt](0,-1.4)--(0,1.4)node[right,font=\scriptsize]{$y$};
\foreach \x in {-1.0,-0.5,0.0,0.5,1.0}{\draw(\x,0)--(\x,0.05)node[below,outer sep=2pt,font=\scriptsize]at(\x,0){\x};}
\foreach \y in {1.0,0.5}{\draw(0,\y)--(0.05,\y)node[left,outer sep=2pt,font=\scriptsize]at(0,\y){\y};}
\draw[color=red ,domain=-1.4:1, line width=1pt]plot(\x,{ln(1+(exp(\x))});
\node[black,anchor=south] at (0,1.6) {\small $y = \ln(1+{\textrm e}^x)$};
\node [anchor=south east,inner sep=1pt] (labela) at (0.8,-2) {\small{(a) Softplus}};
\draw[->, line width=1pt](0,-1.2)--(0,1.4)node[right,font=\scriptsize]{$y$};
\foreach \x in {-1.0,-0.5,0.5,1.0}{\draw(\x,0)--(\x,0.05)node[below,outer sep=2pt,font=\scriptsize,scale=0.8]at(\x,0.1){\x};}
\node[below,outer sep=2pt,font=\scriptsize,scale=0.8]at(0.1,0.1){0};
\foreach \y in {1.0,0.5}{\draw(0,\y)--(0.05,\y)node[left,outer sep=2pt,font=\scriptsize,scale=0.8]at(0.1,\y){};}
\node[left,outer sep=2pt,font=\scriptsize,scale=0.8]at(0.1,0.4){0.5};
\node[left,outer sep=2pt,font=\scriptsize,scale=0.8]at(0.1,1.0){1.0};
\draw[color=red ,domain=-1.4:1.0, line width=1pt]plot(\x,{ln(1+(exp(\x))});
\node[black,anchor=south,scale=0.8] at (0,1.5) {\small $y = \ln(1+{\textrm e}^x)$};
\node [anchor=south east,inner sep=1pt,scale=0.8] (labela) at (0.8,-2) {\small{(a) Softplus}};
\end{scope}
%%%------------------------------------------------------------------------------------------------------------
\begin{scope}[xshift=1.7in]
\draw[->, line width=1pt](-1.4,0)--(1.4,0)node[left,below,font=\scriptsize]{$x$};
\draw[->, line width=1pt](0,-1.4)--(0,1.4)node[right,font=\scriptsize]{$y$};
\draw[->, line width=1pt](0,-1.2)--(0,1.4)node[right,font=\scriptsize,scale=0.8]{$y$};
\draw[dashed](0,1)--(1.4,1);
\foreach \x in {-1,-0.5,0,0.5,1}{\draw(\x,0)--(\x,0.05)node[below,outer sep=2pt,font=\scriptsize]at(\x,0){
\foreach \x in {-1,-0.5,0.5,1}{\draw(\x,0)--(\x,0.05)node[below,outer sep=2pt,font=\scriptsize,scale=0.8]at(\x,0.1){
\pgfmathparse{(\x)*5}
\pgfmathresult};}
\foreach \y in {0.5,1.0}{\draw(0,\y)--(0.05,\y)node[left,outer sep=2pt,font=\scriptsize]at(-0.15,\y){\y};}
\node[below,outer sep=2pt,font=\scriptsize,scale=0.8]at(0.1,0.1){0};
\foreach \y in {0.5,1.0}{\draw(0,\y)--(0.05,\y)node[left,outer sep=2pt,font=\scriptsize,scale=0.8]at(0.1,\y){\y};}
\draw[color=red,domain=-1.4:1.4, line width=1pt]plot(\x,{1/(1+(exp(-5*\x)))});
\node[black,anchor=south] at (0,1.6) {\small $y = \frac{1}{1+{\textrm e}^{-x}}$};
\node [anchor=south east,inner sep=1pt] (labelb) at (0.8,-2) {\small{(b) Sigmoid}};
\node[black,anchor=south,scale=0.8] at (0,1.5) {\small $y = \frac{1}{1+{\textrm {e}}^{-x}}$};
\node [anchor=south east,inner sep=1pt,scale=0.8] (labelb) at (0.8,-2) {\small{(b) Sigmoid}};
\end{scope}
%%%------------------------------------------------------------------------------------------------------------
\begin{scope}[xshift=3.4in]
\draw[->, line width=1pt](-1.4,0)--(1.4,0)node[left,below,font=\scriptsize]{$x$};
\draw[->, line width=1pt](0,-1.4)--(0,1.4)node[right,font=\scriptsize]{$y$};
\draw[->, line width=1pt](0,-1.4)--(0,1.2)node[right,font=\scriptsize]{$y$};
\draw[dashed](0,1)--(1.4,1);
\draw[dashed](-1.4,-1)--(0,-1);
\foreach \x in {-1.0,-0.5,0.0,0.5,1.0}{\draw(\x,0)--(\x,0.05)node[below,outer sep=2pt,font=\scriptsize]at(\x,0){\x};}
\foreach \y in {,-1.0-0.5,0.5,1.0}{\draw(0,\y)--(0.05,\y)node[left,outer sep=2pt,font=\scriptsize]at(0,\y){\y};}
\foreach \x in {-1.0,-0.5,0.5,1.0}{\draw(\x,0)--(\x,0.05)node[below,outer sep=2pt,font=\scriptsize,scale=0.8]at(\x,0.1){\x};}
\node[below,outer sep=2pt,font=\scriptsize,scale=0.8]at(0.1,0.1){0};
\foreach \y in {0.5,1.0}{\draw(0,\y)--(0.05,\y)node[left,outer sep=2pt,font=\scriptsize,scale=0.8]at(0,\y){\y};}
\draw[color=red ,domain=-1.4:1.4, line width=1pt]plot(\x,{tanh(\x)});
\node[black,anchor=south] at (0,1.6) {\small $y = \frac{{\textrm e}^{x}-{\textrm e}^{-x}}{{e}^{x}+e^{-x}}$};
\node [anchor=south east,inner sep=1pt] (labelc) at (0.8,-2) {\small{(c) Tanh}};
\end{scope}
%%%------------------------------------------------------------------------------------------------------------
\begin{scope}[yshift=-1.8in]
\draw[->, line width=1pt](-1.4,0)--(1.4,0)node[left,below,font=\scriptsize]{$x$};
\draw[->, line width=1pt](0,-1.4)--(0,1.4)node[right,font=\scriptsize]{$y$};
\foreach \x in {-1.0,-0.5,0.0,0.5,1.0}{\draw(\x,0)--(\x,0.05)node[below,outer sep=2pt,font=\scriptsize]at(\x,0){\x};}
\foreach \y in {0.5,1.0}{\draw(0,\y)--(0.05,\y)node[left,outer sep=2pt,font=\scriptsize]at(0,\y){\y};}
\draw[color=red ,domain=-1.4:1.4, line width=1pt]plot(\x,{max(\x,0)});
\node[black,anchor=south] at (0,1.6) {\small $y =\max (0, x)$};
\node [anchor=south east,inner sep=1pt] (labeld) at (0.8,-2) {\small{(d) ReLU}};
\end{scope}
%%%------------------------------------------------------------------------------------------------------------
\begin{scope}[yshift=-1.8in,xshift=1.7in]
\draw[->, line width=1pt](-1.4,0)--(1.4,0)node[left,below,font=\scriptsize]{$x$};
\draw[->, line width=1pt](0,-1.4)--(0,1.4)node[right,font=\scriptsize]{$y$};
\foreach \x in {-1.0,-0.5,0.0,0.5,1.0}{\draw(\x,0)--(\x,0.05)node[below,outer sep=2pt,font=\scriptsize]at(\x,0){\x};}
\foreach \y in {0.5,1.0}{\draw(0,\y)--(0.05,\y)node[left,outer sep=2pt,font=\scriptsize]at(-0.15,\y){\y};}
\draw[color=red ,domain=-1.4:1.4, line width=1pt]plot(\x,{exp(-1*((\x)^2))});
\node[black,anchor=south] at (0,1.6) {\small $y =e^{-x^2}$};
\node [anchor=south east,inner sep=1pt] (labele) at (0.8,-2) {\small{(e) Gaussian}};
\end{scope}
%%%------------------------------------------------------------------------------------------------------------
\begin{scope}[yshift=-1.8in,xshift=3.4in]
\draw[->, line width=1pt](-1.4,0)--(1.4,0)node[left,below,font=\scriptsize]{$x$};
\draw[->, line width=1pt](0,-1.4)--(0,1.4)node[right,font=\scriptsize]{$y$};
\foreach \x in {-1.0,-0.5,0.0,0.5,1.0}{\draw(\x,0)--(\x,0.05)node[below,outer sep=2pt,font=\scriptsize]at(\x,0){\x};}
\foreach \y in {0.5,1.0}{\draw(0,\y)--(0.05,\y)node[left,outer sep=2pt,font=\scriptsize]at(0,\y){\y};}
\draw[color=red ,domain=-1:1, line width=1pt]plot(\x,\x);
\node[black,anchor=south] at (0,1.6) {\small $y =x$};
\node [anchor=south east,inner sep=1pt] (labelf) at (0.8,-2) {\small{(f) Identity}};
\node[black,anchor=south,scale=0.8] at (0,1.5) {\small $y = \frac{{\textrm e}^{x}-{\textrm e}^{-x}}{\textrm{e}^{x}+{\textrm e}^{-x}}$};
\node [anchor=south east,inner sep=1pt,scale=0.8] (labelc) at (0.8,-2) {\small{(c) Tanh}};
\end{scope}
\end{tikzpicture}
%%%------------------------------------------------------------------------------------------------------------
\ No newline at end of file
ChapterAcknowledgement/acknowledgement.tex
......@@ -28,13 +28,21 @@
\begin{center}
{\Huge \bfnew{后\ \ \ 记}}
\end{center}
\vspace{2em}
\vspace{5em}
\begin{spacing}{1.18}
\parinterval 我知道这里本应该再写点儿什么，感慨一下蹉跎岁月，最后致敬所有人。
\parinterval 我知道这里本应该再写点什么，感慨一下蹉跎岁月，最后致敬所有人。
\parinterval 不过我还是最想说：谢谢你，我的妻子。没有你的支持与照顾，我应该没有勇气来完成这本书。爱你 $\sim$
\vspace{1em}
\parinterval 不过我还是最想说：
\vspace{1em}
\parinterval 谢谢你，我的妻子。没有你的支持与照顾，我应该没有勇气来完成这本书。爱你 $\sim$
\vspace{8em}
\hfill 肖桐
......
ChapterPreface/chapterpreface.tex
......@@ -50,7 +50,7 @@
本书可以供计算机相关专业高年级本科生及研究生学习之用，也可以作为自然语言处理领域，特别是机器翻译方向相关研究人员的参考资料。此外，本书各章主题明确，内容紧凑。因此，读者也可将每章作为某一专题的学习资料。
{\sffamily\bfseries 用最简单的方式阐述机器翻译的基本思想}是笔者所期望达到的目标。但是，书中不可避免会使用一些形式化定义和算法的抽象描述，因此，笔者尽所能通过图例进行解释（本书共XXX张插图）。不过，本书所包含的内容较为广泛，难免会有疏漏，望读者海涵，并指出不当之处。
{\sffamily\bfseries 用最简单的方式阐述机器翻译的基本思想}是笔者所期望达到的目标。但是，书中不可避免会使用一些形式化定义和算法的抽象描述，因此，笔者尽所能通过图例进行解释（本书共396张插图）。不过，本书所包含的内容较为广泛，难免会有疏漏，望读者海涵，并指出不当之处。
\vspace{1.0em}
......
structure.tex
......@@ -628,6 +628,8 @@ addtohook={%
%\usetikzlibrary{arrows}
%\usetikzlibrary{decorations}
\usetikzlibrary{arrows,shapes}
\usepackage{xeCJK}
\newfontfamily{\yh}{微软雅黑}
%%%%%%%%%%%chapter5图片等---------------------------------------
\usepackage{tikz-3dplot}
......
Markdown 格式
0% 或
您添加了 0 人 到此讨论。请谨慎行事。
请先完成此评论的编辑！
请 注册 或者 后发表评论