Skip to content

T
Toy-MT-Introduction
Commit 934b3f23 by zhoutao
Options

合并分支 'caorunzhe' 到 'zhoutao' 

caorunzhe to zhoutao

查看合并请求 !174
parents 83c358e9 fbcf0a02
正在显示 包含 1939 行增加 和 2172 行删除
Book/Chapter1/Figures/figure-comparison-mt-ht.tex → Book/Chapter1/Figures/figure-comparison-mt-ht-1.tex
......@@ -17,7 +17,6 @@
\node [anchor=north west] (mt1-2) at ([yshift=0.4em]mt1-1.south west) {\scriptsize{-pared the boat and {\color{red}\underline{left the island.}} Only Love decided to stay.She {\color{red}\underline{ wanted to stick}} to it until the last}};
\node [anchor=north west] (mt1-3) at ([yshift=0.4em]mt1-2.south west) {\scriptsize{moment. After a few days, the island was really going to sink and love {\color{red}\underline{ wanted help.}}}};
%人工翻译---------------
\node [anchor=north west] (ht1) at ([xshift=-3.5em,yshift=0.4em]mt1-3.south west) {\scriptsize{人工翻译：Once upon a time, there was a small island {\color{red}\underline{where}} lived all kinds of emotions like JOY,SADNESS,}};
......@@ -37,53 +36,11 @@
\node[anchor=north west] (original4-4) at ([yshift=0.3em]original4-3.south west) {\scriptsize{道。快乐走近爱的身边，但是她太快乐了，竟然没有听见爱在叫她！}};
%机器翻译--------------
\node [anchor=north west] (mt4) at ([xshift=-3.5em,yshift=0.3em]original4-4.south west) {\scriptsize{机器翻译：At this time, Richness {\color{red}\underline{passed by}} in a big ship. Love said, ``Rich, can you take me away?'' Richness}};
\node [anchor=north west] (mt4-1) at ([xshift=3.5em,yshift=0.4em]mt4.south west) {\scriptsize{replied, ``No, {\color{red}\underline{there are many treasures}} of gold and silver in my ship,and there is no place for you.''}};
\node [anchor=north west] (mt4-2) at ([yshift=0.4em]mt4-1.south west) {\scriptsize{ Love saw vanity in a magnificent boat and said, ``Vanity, help me!'' ``I can't help you. You are {\color{red}\underline{soak}}}};
\node [anchor=north west] (mt4-3) at ([yshift=0.4em]mt4-2.south west) {\scriptsize{{\color{red}\underline{-ed to the skin}} and will damage my beautiful boat.'' When sorrow came, love asked him for help: ``}};
\node [anchor=north west] (mt4-4) at ([yshift=0.4em]mt4-3.south west) {\scriptsize{sorrow, let me go with you!'' ``Oh,...love, I am so sad that I want to be alone for a while!'' Sadly rep}};
\node [anchor=north west] (mt4-5) at ([yshift=0.4em]mt4-4.south west) {\scriptsize{-lied. Happiness {\color{red}\underline{approached}} love, but she was too happy to hear love calling her!}};
%人工翻译---------------
\node [anchor=north west] (ht4) at ([xshift=-3.5em,yshift=0.3em]mt4-5.south west) {\scriptsize{人工翻译：At that moment, WEALTH {\color{red}\underline{was passing by}} in a big boat. Love said,``WEALTH, can you take me}};
\node [anchor=north west] (ht4-1) at ([xshift=3.5em,yshift=0.4em]ht4.south west) {\scriptsize{with you?'' WEALTH answered, ``no, {\color{red}\underline{there is a lot}} of gold and silver in my boat. There is no place}};
\node [anchor=north west] (ht4-2) at ([yshift=0.4em]ht4-1.south west) {\scriptsize{for you.'' Love saw VANITY in a beautiful boat and said, ``VANITY, help me!'' ``I can't help you.}};
\node [anchor=north west] (ht4-3) at ([yshift=0.4em]ht4-2.south west) {\scriptsize{You are {\color{red}\underline{all wet,}} and will break my pretty boat.'' Then SADNESS came. Love asked for help,``SAD}};
\node [anchor=north west] (ht4-4) at ([yshift=0.4em]ht4-3.south west) {\scriptsize{-NESS, let me go with you!'' ``Oh,...LOVE, I am so sad that I want to be alone for a while!'' ``Repli}};
\node [anchor=north west] (ht4-5) at ([yshift=0.4em]ht4-4.south west) {\scriptsize{-ed SADNESS. JOY {\color{red}\underline{came close to }} love, but she was so happy that she did not hear him call her!}};
%第三段--------------------------------
\node[anchor=north west] (original8) at ([xshift=-3.5em,yshift=-0.3em]ht4-5.south west) {\scriptsize{源 \qquad 文：突然，一个声音传来：``过来，爱，我带你走。''这是位长者。爱大喜过望，竟忘了问他他}};
\node[anchor=north west] (original8-1) at ([xshift=3.5em,yshift=0.3em]original8.south west) {\scriptsize{的名字。登上陆地后，长者独自走开了。爱对长者感激不尽，问另一位长者知识：``帮我的}};
\node[anchor=north west] (original8-2) at ([yshift=0.3em]original8-1.south west) {\scriptsize{那个人是谁？''``他是时间。''知识老人回答。``时间？''爱问道，``他为什么要帮我？''知识老}};
\node[anchor=north west] (original8-3) at ([yshift=0.3em]original8-2.south west) {\scriptsize{人笑道：``因为只有时间才能理解爱有多么伟大。''}};
%机器翻译--------------
\node [anchor=north west] (mt8) at ([xshift=-3.5em,yshift=0.4em]original8-3.south west) {\scriptsize{机器翻译：Suddenly, a voice {\color{red}\underline{came:}} ``Come here, love, I'll take you away.'' This is an elder. Love was {\color{red}\underline{overjoy}}}};
\node [anchor=north west] (mt8-1) at ([xshift=3.5em,yshift=0.4em]mt8.south west) {\scriptsize{{\color{red}\underline{-ed and}} forgot to ask his name. After landing on land, the elder walked away alone.Love was very}};
\node [anchor=north west] (mt8-2) at ([yshift=0.4em]mt8-1.south west) {\scriptsize{grateful to the elder and asked another elder knowledge, {\color{red}\underline{``Who is the person who helped me?''}} ``He}};
\node [anchor=north west] (mt8-3) at ([yshift=0.4em]mt8-2.south west) {\scriptsize{is time.'' The {\color{red}\underline{old intellectual}} replied. ``Time?'' Love asked,``Why did he help me?'' The old intellec}};
\node [anchor=north west] (mt8-4) at ([yshift=0.4em]mt8-3.south west) {\scriptsize{-tual laughed, ``Because only time canunderstand how great love is.''}};
%人工翻译---------------
\node [anchor=north west] (ht8) at ([xshift=-3.5em,yshift=0.4em]mt8-4.south west) {\scriptsize{人工翻译：Suddenly, a voice {\color{red}\underline{said,}} ``come, LOVE, I'll take you.'' This is an elder. LOVE was {\color{red}\underline{so over that}} she}};
\node [anchor=north west] (ht8-1) at ([xshift=3.5em,yshift=0.4em]ht8.south west) {\scriptsize{forgot to ask his name. After landing on land, the elder walked away}};
\node [anchor=north west] (ht8-2) at ([yshift=0.4em]ht8-1.south west) {\scriptsize{alone.LOVE was so grateful to the elder that she asked KNOWLEDGE, another elder, {\color{red}\underline{``who help}}}};
\node [anchor=north west] (ht8-3) at ([yshift=0.4em]ht8-2.south west) {\scriptsize{{\color{red}\underline{-ed me?''}} ``He is TIME.'' The {\color{red}\underline{old man}} replied. ``TIME?'' LOVE asked. ``why did he help me?'' The}};
\node [anchor=north west] (ht8-4) at ([yshift=0.4em]ht8-3.south west) {\scriptsize{old man smiled and said, ``Because only time can understand how great love is.''}};
%{
%\draw[dotted,thick,ublue] ([xshift=10.3em,yshift=0.3em]mt8.south west)--%([xshift=-5.2em,yshift=-0.3em]ht8.north);
%}
\begin{pgfonlayer}{background}
{
\node[rectangle,draw=ublue, inner sep=0mm] [fit =(original1)(mt4)(ht1)(mt1)(mt1)(ht1)(ht8-4)] {};
\node[rectangle,draw=ublue, inner sep=0mm] [fit =(original1)(ht1)(mt1)(mt4-1)] {};
}
\end{pgfonlayer}
......
Book/Chapter1/Figures/figure-comparison-mt-ht-2.tex 0 → 100644
\definecolor{ublue}{rgb}{0.152,0.250,0.545}
\definecolor{ugreen}{rgb}{0,0.5,0}
%%% outline
%-------------------------------------------------------------------------
\begin{tikzpicture}
\node [pos=0.4,left,xshift=-0.4em,yshift=2.0em,,opacity=0.0] (original1) {\scriptsize{源 \qquad 文：从前有一个小岛，上面住着快乐、悲哀、知识和爱，还有其他各种情感。一天，情感们得知}};
\node [anchor=north west] (mt4-2) at ([xshift=3.5em,yshift=0.3em]original1.south west) {\scriptsize{ Love saw vanity in a magnificent boat and said, ``Vanity, help me!'' ``I can't help you. You are {\color{red}\underline{soak}}}};
\node [anchor=north west] (mt4-3) at ([yshift=0.4em]mt4-2.south west) {\scriptsize{{\color{red}\underline{-ed to the skin}} and will damage my beautiful boat.'' When sorrow came, love asked him for help: ``}};
\node [anchor=north west] (mt4-4) at ([yshift=0.4em]mt4-3.south west) {\scriptsize{sorrow, let me go with you!'' ``Oh,...love, I am so sad that I want to be alone for a while!'' Sadly rep}};
\node [anchor=north west] (mt4-5) at ([yshift=0.4em]mt4-4.south west) {\scriptsize{-lied. Happiness {\color{red}\underline{approached}} love, but she was too happy to hear love calling her!}};
%人工翻译---------------
\node [anchor=north west] (ht4) at ([xshift=-3.5em,yshift=0.3em]mt4-5.south west) {\scriptsize{人工翻译：At that moment, WEALTH {\color{red}\underline{was passing by}} in a big boat. Love said,``WEALTH, can you take me}};
\node [anchor=north west] (ht4-1) at ([xshift=3.5em,yshift=0.4em]ht4.south west) {\scriptsize{with you?'' WEALTH answered, ``no, {\color{red}\underline{there is a lot}} of gold and silver in my boat. There is no place}};
\node [anchor=north west] (ht4-2) at ([yshift=0.4em]ht4-1.south west) {\scriptsize{for you.'' Love saw VANITY in a beautiful boat and said, ``VANITY, help me!'' ``I can't help you.}};
\node [anchor=north west] (ht4-3) at ([yshift=0.4em]ht4-2.south west) {\scriptsize{You are {\color{red}\underline{all wet,}} and will break my pretty boat.'' Then SADNESS came. Love asked for help,``SAD}};
\node [anchor=north west] (ht4-4) at ([yshift=0.4em]ht4-3.south west) {\scriptsize{-NESS, let me go with you!'' ``Oh,...LOVE, I am so sad that I want to be alone for a while!'' ``Repli}};
\node [anchor=north west] (ht4-5) at ([yshift=0.4em]ht4-4.south west) {\scriptsize{-ed SADNESS. JOY {\color{red}\underline{came close to }} love, but she was so happy that she did not hear him call her!}};
%第三段--------------------------------
\node[anchor=north west] (original8) at ([xshift=-3.5em,yshift=-0.3em]ht4-5.south west) {\scriptsize{源 \qquad 文：突然，一个声音传来：``过来，爱，我带你走。''这是位长者。爱大喜过望，竟忘了问他他}};
\node[anchor=north west] (original8-1) at ([xshift=3.5em,yshift=0.3em]original8.south west) {\scriptsize{的名字。登上陆地后，长者独自走开了。爱对长者感激不尽，问另一位长者知识：``帮我的}};
\node[anchor=north west] (original8-2) at ([yshift=0.3em]original8-1.south west) {\scriptsize{那个人是谁？''``他是时间。''知识老人回答。``时间？''爱问道，``他为什么要帮我？''知识老}};
\node[anchor=north west] (original8-3) at ([yshift=0.3em]original8-2.south west) {\scriptsize{人笑道：``因为只有时间才能理解爱有多么伟大。''}};
%机器翻译--------------
\node [anchor=north west] (mt8) at ([xshift=-3.5em,yshift=0.4em]original8-3.south west) {\scriptsize{机器翻译：Suddenly, a voice {\color{red}\underline{came:}} ``Come here, love, I'll take you away.'' This is an elder. Love was {\color{red}\underline{overjoy}}}};
\node [anchor=north west] (mt8-1) at ([xshift=3.5em,yshift=0.4em]mt8.south west) {\scriptsize{{\color{red}\underline{-ed and}} forgot to ask his name. After landing on land, the elder walked away alone.Love was very}};
\node [anchor=north west] (mt8-2) at ([yshift=0.4em]mt8-1.south west) {\scriptsize{grateful to the elder and asked another elder knowledge, {\color{red}\underline{``Who is the person who helped me?''}} ``He}};
\node [anchor=north west] (mt8-3) at ([yshift=0.4em]mt8-2.south west) {\scriptsize{is time.'' The {\color{red}\underline{old intellectual}} replied. ``Time?'' Love asked,``Why did he help me?'' The old intellec}};
\node [anchor=north west] (mt8-4) at ([yshift=0.4em]mt8-3.south west) {\scriptsize{-tual laughed, ``Because only time canunderstand how great love is.''}};
%人工翻译---------------
\node [anchor=north west] (ht8) at ([xshift=-3.5em,yshift=0.4em]mt8-4.south west) {\scriptsize{人工翻译：Suddenly, a voice {\color{red}\underline{said,}} ``come, LOVE, I'll take you.'' This is an elder. LOVE was {\color{red}\underline{so over that}} she}};
\node [anchor=north west] (ht8-1) at ([xshift=3.5em,yshift=0.4em]ht8.south west) {\scriptsize{forgot to ask his name. After landing on land, the elder walked away}};
\node [anchor=north west] (ht8-2) at ([yshift=0.4em]ht8-1.south west) {\scriptsize{alone.LOVE was so grateful to the elder that she asked KNOWLEDGE, another elder, {\color{red}\underline{``who help}}}};
\node [anchor=north west] (ht8-3) at ([yshift=0.4em]ht8-2.south west) {\scriptsize{{\color{red}\underline{-ed me?''}} ``He is TIME.'' The {\color{red}\underline{old man}} replied. ``TIME?'' LOVE asked. ``why did he help me?'' The}};
\node [anchor=north west] (ht8-4) at ([yshift=0.4em]ht8-3.south west) {\scriptsize{old man smiled and said, ``Because only time can understand how great love is.''}};
%{
%\draw[dotted,thick,ublue] ([xshift=10.3em,yshift=0.3em]mt8.south west)--%([xshift=-5.2em,yshift=-0.3em]ht8.north);
%}
\begin{pgfonlayer}{background}
{
\node[rectangle,draw=ublue, inner sep=0mm] [fit = (mt4-2)(ht4)(ht8-4)] {};
}
\end{pgfonlayer}
\end{tikzpicture}
Book/Chapter1/Figures/figure-Example-NMT.tex → Book/Chapter1/Figures/figure-example-NMT.tex
\definecolor{ublue}{rgb}{0.152,0.250,0.545}
\definecolor{ugreen}{rgb}{0,0.5,0}
......
Book/Chapter1/Figures/figure-Example-RBMT.tex → Book/Chapter1/Figures/figure-example-RBMT.tex
\definecolor{ublue}{rgb}{0.152,0.250,0.545}
\definecolor{ugreen}{rgb}{0,0.5,0}
......
Book/Chapter1/Figures/figure-Example-SMT.tex → Book/Chapter1/Figures/figure-example-SMT.tex
\definecolor{ublue}{rgb}{0.152,0.250,0.545}
\definecolor{ugreen}{rgb}{0,0.5,0}
......@@ -39,7 +38,7 @@
\node [anchor=north west] (entry3) at ([yshift=0.0em]entry2.south west) {\textbf{3:} Yes, you are right .};
\node [anchor=north west] (entry4) at ([yshift=0.0em]entry3.south west) {\hspace{1em} ...};
\node [anchor=north west] (entry5) at ([yshift=0.1em]entry4.south west) {\hspace{1em}{\quad}};
\node [anchor=north west] (entry6) at ([xshift=10.97em,yshift=0.65em]entry5.south west) {};
\node [anchor=north west] (entry6) at ([xshift=11.6em,yshift=0.65em]entry5.south west) {};
\node [anchor=south west] (monodatalabel) at (entry1.north west) {{\color{ublue}\small{资源2:单语语料}}};
}
......@@ -96,11 +95,11 @@
}
{
\draw[->,thick,ublue] (bidata.east)--([xshift=2.2em]bidata.east) node[pos=0.5,above] (simexample) {\color{red}{\scriptsize{\scriptsize\sffamily\bfseries{学习}}}};
\draw[->,thick,ublue] (bidata.east)--([xshift=2.2em]bidata.east) node[pos=0.5,above] (simexample) {\color{red}{\scriptsize{\scriptsize{学习}}}};
}
{
\draw[->,thick,ublue] (monodata.east)--([xshift=2.7em]monodata.east) node[pos=0.5,above] (simexample) {\color{red}{\scriptsize{\scriptsize\sffamily\bfseries{学习}}}};
\draw[->,thick,ublue] (monodata.east)--([xshift=2.2em]monodata.east) node[pos=0.5,above] (simexample) {\color{red}{\scriptsize{\scriptsize{学习}}}};
}
\begin{scope}[xshift=3.6in]
......@@ -135,16 +134,16 @@ You and me & {0.02}\\
}
{
\draw[->,thick,double,ublue] (decoder.north) -- ([yshift=2.2em]decoder.north) node[pos=0.5,right] (decodinglabel) {\color{red}{\tiny\sffamily\bfseries{枚举所有可能}}};
\draw[->,thick,double,ublue] (decoder.north) -- ([yshift=2.2em]decoder.north) node[pos=0.5,right] (decodinglabel) {\color{red}{\tiny{枚举所有可能}}};
}
{
\draw[->,thick,double,ublue] (decoder.east) .. controls +(east:3.5em) .. ([xshift=3.5em,yshift=3.0em]decoder.east) node[xshift=0.5em,pos=0.3,below] (decodinglabel) {\color{red}{\tiny\sffamily\bfseries{计算翻译可能性}}};
\draw[->,thick,double,ublue] (decoder.east) .. controls +(east:3.5em) .. ([xshift=3.5em,yshift=3.0em]decoder.east) node[xshift=0.5em,pos=0.3,below] (decodinglabel) {\color{red}{\tiny{计算翻译可能性}}};
}
{
\node[anchor=west,draw,thick,red,minimum width=11.5em,minimum height=1em] (outputlabel) at ([xshift=-0.3em,yshift=-6.1em]srcsentence.south west){};
\node[anchor=west] (outputlabel2) at ([xshift=-0.3em]outputlabel.east) {\color{red}{\tiny\sffamily\bfseries{输出}}};
\node[anchor=west] (outputlabel2) at ([xshift=-0.3em]outputlabel.east) {\color{red}{\tiny{输出}}};
}
}
......
Book/Chapter1/Figures/figure-zh-sentences-into-en-sentences.tex
......@@ -35,12 +35,14 @@
\node [anchor=north west] (entry3) at ([yshift=0.1em]entry2.south west) {\textbf{3:} 满意 \hspace{-0.4em} $\to$ \hspace{-0.4em} satisfy \hspace{-0.12em}$\mid$\hspace{-0.12em} satisfied ... };
\node [anchor=north west] (entry4) at ([yshift=0.1em]entry3.south west) {\hspace{1em} ...};
\node [anchor=south west] (dictionarylabel) at (entry1.north west) {{\color{ublue} 资源2：翻译词典}};
\node [anchor=west,opacity=0.0] (empty) at ([yshift=-0.2em]entry3.west) {\hspace{1em} 译=``I am happy with him''};
}
}
\begin{pgfonlayer}{background}
{
\node[rectangle,draw=ublue, inner sep=0mm] [fit = (entry1) (entry2) (entry3) (entry4) (dictionarylabel)] {};
\node[rectangle,draw=ublue, inner sep=0mm] [fit = (entry1) (entry2) (entry3) (entry4) (dictionarylabel) (empty)] {};
}
\end{pgfonlayer}
......@@ -77,22 +79,22 @@
\end{scope}
{
\draw[<->,dotted,thick,red] (example2.east)..controls +(east:1.3) and +(west:1.3)..(c1.west) node[pos=0.9,left,xshift=0.6em,yshift=0.4em] (simexample) {\color{red}{\tiny\sffamily\bfseries{相似实例}}};
\draw[<->,dotted,thick,red] ([xshift=-0.2em]example2part2.east)--([xshift=4.5em]example2part2.east);
\draw[<->,dotted,thick,red] (example2.east)..controls +(east:0.6) and +(west:0.6)..(c1.west) node[pos=0.9,left,xshift=0.6em,yshift=0.4em] (simexample) {\color{red}{\tiny{相似实例}}};
\draw[<->,dotted,thick,red] ([xshift=-0.2em]example2part2.east)..controls +(east:0.5) and +(west:0.5)..(e1.west);
}
{
\draw[<->,dotted,thick,ublue] (w3.south)--(c3.north) node[pos=0.5,right] (mismatch1) {\color{red}{\tiny\sffamily\bfseries{不匹配}}};
\draw[<->,dotted,thick,ublue] (w3.south)--(c3.north) node[pos=0.5,right] (mismatch1) {\color{red}{\tiny{不匹配}}};
\draw[dotted,thick,ublue] ([xshift=-0.6em,yshift=0.0em]c3.south east)--([xshift=0.8em,yshift=0.0em]e5.north west);
}
{
\draw[<->,dotted,thick,ublue] (w5.south)--(c5.north) node[pos=0.5,right] (mismatch1) {\color{red}{\tiny\sffamily\bfseries{不匹配}}};
\draw[<->,dotted,thick,ublue] (w5.south)--(c5.north) node[pos=0.5,right] (mismatch1) {\color{red}{\tiny{不匹配}}};
\draw[dotted,thick,ublue] ([xshift=0.8em,yshift=0.0em]c5.south west)--([xshift=0.0em,yshift=-0.2em]e3.north);
}
{
\draw[double,->,thick,ublue] (e3.south)--([yshift=-1.2em]e3.south) node[pos=0.5,right,xshift=0.2em,yshift=0.2em] (step1) {\color{red}{\tiny\sffamily\bfseries{用`你'替换`他'}}};
\draw[double,->,thick,ublue] (e3.south)--([yshift=-1.2em]e3.south) node[pos=0.5,right,xshift=0.2em,yshift=0.2em] (step1) {\color{red}{\tiny{用`你'替换`他'}}};
\draw[->,dotted,thick,red] ([xshift=-0.1em]entry2.east)..controls +(east:4) and +(west:4)..([yshift=-0.6em,xshift=-0.5em]e3.south) ;
}
......@@ -101,7 +103,7 @@
{\footnotesize
\node [anchor=north west,inner sep=1mm] (c1) at (0,0) {我};
\node [anchor=north west,inner sep=1mm] (c2) at ([xshift=0.3em]c1.north east) {对};
\node [anchor=north west,inner sep=1mm] (c3) at ([xshift=0.3em]c2.north east) {\footnotesize\sffamily\bfseries{{\color{ublue} 你}}};
\node [anchor=north west,inner sep=1mm] (c3) at ([xshift=0.3em]c2.north east) {\footnotesize{{\color{ublue} 你}}};
\node [anchor=north west,inner sep=1mm] (c4) at ([xshift=0.3em]c3.north east) {感到};
\node [anchor=north west,inner sep=1mm] (c5) at ([xshift=0.3em]c4.north east) {高兴};
}
......@@ -122,7 +124,7 @@
}
{
\draw[double,->,thick,ublue] (e3.south)--([yshift=-1.2em]e3.south) node[pos=0.5,right,xshift=0.2em,yshift=0.2em] (step1) {\color{red}{\tiny\sffamily\bfseries{用`满意'替换`高兴'}}};
\draw[double,->,thick,ublue] (e3.south)--([yshift=-1.2em]e3.south) node[pos=0.5,right,xshift=0.2em,yshift=0.2em] (step1) {\color{red}{\tiny{用`满意'替换`高兴'}}};
\draw[->,dotted,thick,red] ([xshift=-1.2em,yshift=-0.6em]entry3.north east)..controls +(east:2) and +(west:3)..([yshift=-0.6em,xshift=-0.5em]e3.south) ;
}
......@@ -133,7 +135,7 @@
\node [anchor=north west,inner sep=1mm] (c2) at ([xshift=0.3em]c1.north east) {对};
\node [anchor=north west,inner sep=1mm] (c3) at ([xshift=0.3em]c2.north east) {你};
\node [anchor=north west,inner sep=1mm] (c4) at ([xshift=0.3em]c3.north east) {感到};
\node [anchor=north west,inner sep=1mm] (c5) at ([xshift=0.3em]c4.north east) {\footnotesize\sffamily\bfseries{{\color{ublue} 满意}}};
\node [anchor=north west,inner sep=1mm] (c5) at ([xshift=0.3em]c4.north east) {\footnotesize{{\color{ublue} 满意}}};
}
\end{scope}
......@@ -154,7 +156,7 @@
\begin{pgfonlayer}{background}
{\footnotesize
\node[rectangle,draw=red,thick,inner sep=0mm] [fit = (e1) (e2) (e3) (e4) (e5)] {};
\node[anchor=north] (outputlabel) at (e3.south) {\footnotesize\sffamily\bfseries{\color{red}{输出翻译结果}}};
\node[anchor=north] (outputlabel) at (e3.south) {\footnotesize{\color{red}{输出翻译结果}}};
}
\end{pgfonlayer}
......
Book/Chapter1/chapter1.tex
This source diff could not be displayed because it is too large. You can view the blob instead.
Book/Chapter2/Figures/figure-MT-system-as-a-black-box.tex
......@@ -55,16 +55,16 @@
}
{
\node[minimum height=4em,minimum width=4.5em,fill=white] (inputmarking) at (0.85in,-0.39in) {};
\node[minimum height=4em,minimum width=5.2em,fill=white] (outputmarking) at (2.55in,-0.39in) {};
\node[minimum height=4em,minimum width=4.5em,fill=white] (inputmarking) at (0.88in,-0.39in) {};
\node[minimum height=4em,minimum width=5.2em,fill=white] (outputmarking) at (2.57in,-0.39in) {};
}
\node [anchor=south] (inputlabel) at ([yshift=-0.5em]input.north) {{\scriptsize \color{red}{\textbf{输入}}}};
\node [anchor=south] (outputlabel) at ([yshift=-0.5em]output.north) {{\scriptsize \color{red}{\textbf{输出}}}};
{
\node [anchor=west] (mtinputlabel) at ([xshift=0.29in]inputlabel.east) {{\scriptsize \color{red}{\textbf{实际的输入}}}};
\node [anchor=west] (mtoutputlabel) at ([xshift=0.86in]mtinputlabel.east) {{\scriptsize \color{red}{\textbf{实际的输出}}}};
\node [anchor=west] (mtinputlabel) at ([xshift=0.32in]inputlabel.east) {{\scriptsize \color{red}{\textbf{实际的输入}}}};
\node [anchor=west] (mtoutputlabel) at ([xshift=0.88in]mtinputlabel.east) {{\scriptsize \color{red}{\textbf{实际的输出}}}};
\node[rectangle,draw=ublue, inner sep=0mm] [fit = (mtinputlabel) (mtoutputlabel) (inputmarking) (outputmarking)] {};
}
......@@ -73,9 +73,9 @@
}
\begin{scope}[scale=0.9,xshift=1.2in,yshift=-1.2in,level distance=20pt,sibling distance=0pt]
\end{scope}
%\begin{scope}[scale=0.9,xshift=1.2in,yshift=-1.2in,level distance=20pt,sibling distance=0pt]
%
%\end{scope}
\end{scope}
......
Book/Chapter2/Figures/figure-MT=language-analysis+translation-engine.tex
......@@ -63,7 +63,7 @@
{
\node [anchor=west] (mtinputlabel) at ([xshift=0.29in]inputlabel.east) {{\scriptsize \color{red}{\textbf{实际的输入}}}};
\node [anchor=west] (mtoutputlabel) at ([xshift=1.0in]mtinputlabel.east) {{\scriptsize \color{red}{\textbf{实际的输出}}}};
\node [anchor=west] (mtoutputlabel) at ([xshift=0.86in]mtinputlabel.east) {{\scriptsize \color{red}{\textbf{实际的输出}}}};
\node[rectangle,draw=ublue, inner sep=0mm] [fit = (mtinputlabel) (mtoutputlabel) (inputmarking) (outputmarking)] {};
}
......
Book/Chapter2/chapter2.tex
This source diff could not be displayed because it is too large. You can view the blob instead.
Book/Chapter3/Chapter3.tex
This source diff could not be displayed because it is too large. You can view the blob instead.
Book/Chapter3/Figures/figure-a-more-detailed-explanation-of-formula-3.40.tex
......@@ -46,7 +46,7 @@
{
\draw[decorate,thick,decoration={brace,amplitude=5pt,mirror}] ([yshift=-0.2em]eq5.south west) -- ([yshift=-0.2em]eq6.south east) node [pos=0.4,below,xshift=-0.0em,yshift=-0.3em] (expcount1) {\footnotesize{{``$t_v$翻译为$s_u$''这个事件}}};
\node [anchor=north west] (expcount2) at ([yshift=0.5em]expcount1.south west) {\footnotesize{{出现次数的期望的估计}}};
\node [anchor=north west] (expcount3) at ([yshift=0.5em]expcount2.south west) {\footnotesize{{称之为期望频次}}（expected count）};
\node [anchor=north west] (expcount3) at ([yshift=0.5em]expcount2.south west) {\footnotesize{{称之为期望频次}}（Expected Count）};
}
\end{tikzpicture}
......
Book/Chapter3/Figures/figure-calculation-of-the-expected-frequency-1.tex
......@@ -3,5 +3,5 @@
\multicolumn{1}{c|}{$x_1$} & 2 \\
\multicolumn{1}{c|}{$x_2$} & 1 \\
\multicolumn{1}{c|}{$x_3$} & 5 \\ \hline
\multicolumn{2}{c}{$c(X)=8$}
\multicolumn{2}{c}{$c_{\mathbb{E}}(X)=8$}
\end{tabular}
Book/Chapter3/Figures/figure-calculation-of-the-expected-frequency-2.tex
......@@ -4,5 +4,5 @@
\multicolumn{1}{c|}{$x_1$} & 2 & 0.1 & 0.2 \\
\multicolumn{1}{c|}{$x_2$} & 1 & 0.3 & 0.3 \\
\multicolumn{1}{c|}{$x_3$} & 5 & 0.2 & 1.0 \\ \hline
\multicolumn{4}{c}{$c(X)=0.2+0.3+1.0=1.5$}
\multicolumn{4}{c}{$c_{\mathbb{E}}(X)=0.2+0.3+1.0=1.5$}
\end{tabular}
Book/Chapter3/Figures/figure-different-alignment-comparison.tex
......@@ -3,7 +3,7 @@
%%% outline
%-------------------------------------------------------------------------
\begin{tikzpicture}
\begin{scope}
\begin{scope}[xshift=0.5in]
{\small
\node [anchor=west,inner sep=2pt] (s1) at (0,0) {谢谢};
\node [anchor=west,inner sep=2pt] (s2) at ([xshift=2em]s1.east) {你};
......@@ -13,8 +13,9 @@
\draw [-] (s2.south) -- (t2.north);
\node [anchor=center,draw=ublue,circle,thick,fill=white,inner sep=1pt,circular drop shadow={shadow xshift=0.1em,shadow yshift=-0.1em}] (mark) at ([xshift=0.8em,yshift=-0.7em]s2.south east) {{\color{ugreen} \tiny{\textbf{Yes}}}};
}
\node [anchor=center] (labela) at ([xshift=2em,yshift=-1.0em]t1.south) {\scriptsize{(a)}};
\end{scope}
\begin{scope}[xshift=1.8in]
\begin{scope}[xshift=2.3in]
{\small
\node [anchor=west,inner sep=2pt] (s1) at (0,0) {谢谢};
\node [anchor=west,inner sep=2pt] (s2) at ([xshift=2em]s1.east) {你};
......@@ -24,8 +25,9 @@
\draw [-] (s1.south) -- (t2.north);
\node [anchor=center,draw=ublue,circle,thick,fill=white,inner sep=1.5pt,circular drop shadow={shadow xshift=0.1em,shadow yshift=-0.1em}] (mark) at ([xshift=0.8em,yshift=-0.7em]s2.south east) {{\color{red} \tiny{\textbf{No}}}};
}
\node [anchor=center] (labelb) at ([xshift=2em,yshift=-1.0em]t1.south) {\scriptsize{(b)}};
\end{scope}
\begin{scope}[xshift=3.6in]
\begin{scope}[xshift=4.1in]
{\small
\node [anchor=west,inner sep=2pt] (s1) at (0,0) {谢谢};
\node [anchor=west,inner sep=2pt] (s2) at ([xshift=2em]s1.east) {你};
......@@ -35,6 +37,7 @@
\draw [-] (s2.south) -- ([yshift=-0.2em]t1.north);
\node [anchor=center,draw=ublue,circle,thick,fill=white,inner sep=1pt,circular drop shadow={shadow xshift=0.1em,shadow yshift=-0.1em}] (mark) at ([xshift=0.8em,yshift=-0.7em]s2.south east) {{\color{ugreen} \tiny{\textbf{Yes}}}};
}
\node [anchor=center] (labelc) at ([xshift=2em,yshift=-1.0em]t1.south) {\scriptsize{(c)}};
\end{scope}
\end{tikzpicture}
......
Book/Chapter3/Figures/figure-expression.tex
......@@ -12,15 +12,15 @@
\node [anchor=west,inner sep=2pt,minimum height=2.5em] (eq1) at (0,0) {${\textrm{P}(\tau,\pi|\mathbf{t}) = \prod_{i=1}^{l}{\textrm{P}(\varphi_i|\varphi_{1}^{i-1},\mathbf{t})} \times {\textrm{P}(\varphi_0|\varphi_{1}^{l},\mathbf{t})} \times}$};
\node [anchor=north west,inner sep=2pt,minimum height=2.5em] (eq2) at ([xshift=-15.06em,yshift=0.0em]eq1.south east) {${\prod_{i=0}^l{\prod_{k=1}^{\varphi_i}{\textrm{P}(\tau_{ik}|\tau_{i1}^{k-1},\tau_{1}^{i-1},\varphi_{0}^{l},\mathbf{t} )}} \times}$};
\node [anchor=north west,inner sep=2pt,minimum height=2.5em] (eq3) at ([xshift=-15.56em,yshift=0.0em]eq2.south east) {${\prod_{i=1}^l{\prod_{k=1}^{\varphi_i}{\textrm{P}(\pi_{ik}|\pi_{i1}^{k-1},\pi_{1}^{i-1},\tau_{0}^{l},\varphi_{0}^{l},\mathbf{t} )}} \times}$};
\node [anchor=north west,inner sep=2pt,minimum height=2.5em] (eq4) at ([xshift=-17.10em,yshift=0.0em]eq3.south east) {{${\prod_{k=1}^{\varphi_0}{\textrm{P}(\pi_{0k}|\pi_{01}^{k-1},\pi_{1}^{l},\tau_{0}^{l},\varphi_{0}^{l},\mathbf{t} )}}$}};
\node [anchor=west,inner sep=2pt,minimum height=2.0em,fill=red!30] (part1) at ([xshift=-12.3em,yshift=0.0em]eq1.east) {{${\textrm{P}(\varphi_i|\varphi_{1}^{i-1},\mathbf{t})}$}};
\node [anchor=west,inner sep=2pt,minimum height=2.0em,fill=blue!30] (part2) at ([xshift=-5.9em,yshift=0.0em]eq1.east) {{${\textrm{P}(\varphi_0|\varphi_{1}^{l},\mathbf{t})}$}};
\node [anchor=west,inner sep=2pt,minimum height=2.0em,fill=green!30] (part3) at ([xshift=-10.5em,yshift=0.0em]eq2.east) {{${\textrm{P}(\tau_{ik}|\tau_{i1}^{k-1},\tau_{1}^{i-1},\varphi_{0}^{l},\mathbf{t} )}$}};
\node [anchor=west,inner sep=2pt,minimum height=2.0em,fill=yellow!30] (part4) at ([xshift=-12.03em,yshift=0.0em]eq3.east) {{${\textrm{P}(\pi_{ik}|\pi_{i1}^{k-1},\pi_{1}^{i-1},\tau_{0}^{l},\varphi_{0}^{l},\mathbf{t} )}$}};
\node [anchor=west,inner sep=2pt,minimum height=2.5em] (eq1) at (0,0) {${\textrm{P}(\tau,\pi|\mathbf{t}) = \prod_{i=1}^{l}{\textrm{P}(\varphi_i|\varphi_{1}^{i-1},\mathbf{t})} \ \ \times \ \ {\textrm{P}(\varphi_0|\varphi_{1}^{l},\mathbf{t})}\ \ \times}$};
\node [anchor=north west,inner sep=2pt,minimum height=2.5em] (eq2) at ([xshift=-16.06em,yshift=0.0em]eq1.south east) {${\prod_{i=0}^l{\prod_{k=1}^{\varphi_i}{\textrm{P}(\tau_{ik}|\tau_{i1}^{k-1},\tau_{1}^{i-1},\varphi_{0}^{l},\mathbf{t} )}} \ \ \times}$};
\node [anchor=north west,inner sep=2pt,minimum height=2.5em] (eq3) at ([xshift=-15.9em,yshift=0.0em]eq2.south east) {${\prod_{i=1}^l{\prod_{k=1}^{\varphi_i}{\textrm{P}(\pi_{ik}|\pi_{i1}^{k-1},\pi_{1}^{i-1},\tau_{0}^{l},\varphi_{0}^{l},\mathbf{t} )}} \ \ \times}$};
\node [anchor=north west,inner sep=2pt,minimum height=2.5em] (eq4) at ([xshift=-17.40em,yshift=0.0em]eq3.south east) {{${\prod_{k=1}^{\varphi_0}{\textrm{P}(\pi_{0k}|\pi_{01}^{k-1},\pi_{1}^{l},\tau_{0}^{l},\varphi_{0}^{l},\mathbf{t} )}}$}};
\node [anchor=west,inner sep=2pt,minimum height=2.0em,fill=red!30] (part1) at ([xshift=-13.8em,yshift=0.0em]eq1.east) {{${\textrm{P}(\varphi_i|\varphi_{1}^{i-1},\mathbf{t})}$}};
\node [anchor=west,inner sep=2pt,minimum height=2.0em,fill=blue!30] (part2) at ([xshift=-6.3em,yshift=0.0em]eq1.east) {{${\textrm{P}(\varphi_0|\varphi_{1}^{l},\mathbf{t})}$}};
\node [anchor=west,inner sep=2pt,minimum height=2.0em,fill=green!30] (part3) at ([xshift=-11em,yshift=0.0em]eq2.east) {{${\textrm{P}(\tau_{ik}|\tau_{i1}^{k-1},\tau_{1}^{i-1},\varphi_{0}^{l},\mathbf{t} )}$}};
\node [anchor=west,inner sep=2pt,minimum height=2.0em,fill=yellow!30] (part4) at ([xshift=-12.53em,yshift=0.0em]eq3.east) {{${\textrm{P}(\pi_{ik}|\pi_{i1}^{k-1},\pi_{1}^{i-1},\tau_{0}^{l},\varphi_{0}^{l},\mathbf{t} )}$}};
\node [anchor=west,inner sep=2pt,minimum height=2.0em,fill=gray!30] (part5) at ([xshift=-10.4em,yshift=0.0em]eq4.east) {{${\textrm{P}(\pi_{0k}|\pi_{01}^{k-1},\pi_{1}^{l},\tau_{0}^{l},\varphi_{0}^{l},\mathbf{t} )}$}};
......
Book/Chapter3/Figures/figure-greedy-MT-decoding-pseudo-code.tex
......@@ -101,11 +101,11 @@
\node [anchor=north west,inner sep=2pt,align=left] (remark4) at ([xshift=0.31em,yshift=-0.8em]remark3.south west) {\textsc{Join}($a,b$) 返回\\$a$ 和$b$ 的所有组合 };
{\scriptsize
\node [anchor=north west,inner sep=1pt,align=center,draw] (a1) at ([yshift=-0.5em]remark4.north east) {a1\\a2};
\node [anchor=north west,inner sep=1pt,align=center,draw,minimum height=2.3em] (a1) at ([yshift=-0.5em]remark4.north east) {$a_1$\\$a_2$};
\node [anchor=west] (join) at (a1.east) {$\times$};
\node [anchor=north west,inner sep=1pt,align=center,draw] (b1) at ([xshift=1.5em]a1.north east) {b1\\b2};
\node [anchor=north west,inner sep=1pt,align=center,draw,minimum height=2.3em] (b1) at ([xshift=1.5em]a1.north east) {$b_1$\\$b_2$};
\node [anchor=west] (join) at (b1.east) {$=$};
\node [anchor=north west,inner sep=1pt,align=center,draw] (result) at ([xshift=1.5em]b1.north east) {a1b1 a1b2\\a2b1 a2b2};
\node [anchor=north west,inner sep=1pt,align=center,draw] (result) at ([xshift=1.5em]b1.north east) {$a_1b_1$ $a_1b_2$\\$a_2b_1$ $a_2b_2$};
}
\begin{pgfonlayer}{background}
......
Book/Chapter3/Figures/figure-processes-SMT.tex
......@@ -16,20 +16,20 @@
\end{pgfonlayer}
}
\node [anchor=west,ugreen] (P) at ([xshift=4em,yshift=-0.7em]corpus.east){P($\mathbf{t}|\mathbf{s}$)};
\node [anchor=west,ugreen] (P) at ([xshift=5em,yshift=-0.7em]corpus.east){P($\mathbf{t}|\mathbf{s}$)};
\node [anchor=south] (modellabel) at (P.north) {{\color{ublue} {\scriptsize \sffamily\bfseries{翻译模型}}}};
\begin{pgfonlayer}{background}
\node[rectangle,draw=ublue,thick,inner sep=0.2em,fill=white,drop shadow,minimum height=1.6cm] [fit = (P) (modellabel)] (model) {};
\end{pgfonlayer}
\draw [->,very thick,ublue] ([xshift=0.2em]corpus.east) -- ([xshift=3.2em]corpus.east) node [pos=0.5, above] {\color{red}{\scriptsize{模型学习}}};
\draw [->,very thick,ublue] ([xshift=0.2em]corpus.east) -- ([xshift=4.2em]corpus.east) node [pos=0.5, above] {\color{red}{\scriptsize{模型学习}}};
{
\draw [->,very thick,ublue] ([xshift=0.4em]model.east) -- ([xshift=3.4em]model.east) node [inner sep=0pt,pos=0.5, above,yshift=0.3em] (decodingarrow) {\color{red}{\scriptsize{搜索\&计算}}};
\draw [->,very thick,ublue] ([xshift=0.4em]model.east) -- ([xshift=4.4em]model.east) node [inner sep=0pt,pos=0.5, above,yshift=0.3em] (decodingarrow) {\color{red}{\scriptsize{搜索\&计算}}};
{\scriptsize
\node [anchor=north west,inner sep=2pt] (sentlabel) at ([xshift=5.5em,yshift=-0.9em]model.north east) {{\color{ublue} \sffamily\bfseries{机器翻译引擎}}};
\node [anchor=north west,inner sep=2pt] (sentlabel) at ([xshift=6.5em,yshift=-0.9em]model.north east) {{\color{ublue} \sffamily\bfseries{机器翻译引擎}}};
\node [anchor=north west] (sent) at ([yshift=-0.5em]sentlabel.south west) {{对任意句子进行翻译}};
\node [anchor=north west] (sentpart2) at ([yshift=0.3em]sent.south west) {\textbf{\quad}};
}
......
Book/Chapter3/Figures/figure-scores-of-different-translation_model&language_model.tex
......@@ -2,7 +2,7 @@
%-------------------------------------------------------------------------
\begin{tabular}{| l | l |}
\hline
& {\footnotesize{$\prod\limits_{(j,i) \in \hat{A}} \textrm{P}(s_j,t_i)$} \color{red}{{\footnotesize{$\times\textrm{P}_{lm}(\mathbf{t})$}}}} \\ \hline
& {\footnotesize{$\prod\limits_{(j,i) \in \hat{A}} \textrm{P}(s_j,t_i)$} \color{red}{{\footnotesize{$\times\textrm{P}_{\textrm{lm}}(\mathbf{t})$}}}} \\ \hline
\begin{tikzpicture}
......
Book/Chapter3/Figures/figure-word-alignment&probability-distribution-in-IBM-model-3.tex
......@@ -13,7 +13,7 @@
\begin{scope}[scale=0.6]
\node [anchor=west] (eq1) at (0,0) {吃};
\node [anchor=west] (eq2) at ([xshift=3.0em,yshift=0.0em]eq1.east) {早饭};
\node [anchor=north] (eq3) at ([xshift=0.0em,yshift=-2.0em]eq1.south) {Have};
\node [anchor=north] (eq3) at ([xshift=0.0em,yshift=-2.0em]eq1.south) {have};
\node [anchor=north] (eq4) at ([xshift=0.0em,yshift=-2.0em]eq2.south) {breakfast};
\node [anchor=east] (eq5) at ([xshift=-1.0em,yshift=-1.8em]eq1.west) {$\mathbf{a}_{1}$};
\node [anchor=west] (eq6) at ([xshift=1.0em,yshift=-1.8em]eq2.east) {$\textrm{P}(\mathbf{s},\mathbf{a}_{1}|\mathbf{t})=0.5$};
......@@ -21,7 +21,7 @@
\draw [-,very thick](eq2.south) -- (eq4.north);
\node [anchor=west] (eq7) at ([xshift=13.1em,yshift=1.4em]eq2.east) {};
\node [anchor=west] (eq8) at ([xshift=13.1em,yshift=-25.6em]eq2.east) {};
\node [anchor=south] (eq9) at ([xshift=0.0em,yshift=0.0em]eq7.north) {吃早饭 $\Leftrightarrow $ Have breakfast};
\node [anchor=south] (eq9) at ([xshift=0.0em,yshift=0.0em]eq7.north) {吃早饭 $\Leftrightarrow $ have breakfast};
\draw [-,very thick,dotted](eq7.south) -- (eq8.north);
......@@ -32,7 +32,7 @@
\begin{scope}[scale=0.6,xshift=0.0,yshift=-10.0em]
\node [anchor=west] (eq1) at (0,0) {吃};
\node [anchor=west] (eq2) at ([xshift=3.0em,yshift=0.0em]eq1.east) {早饭};
\node [anchor=north] (eq3) at ([xshift=0.0em,yshift=-2.0em]eq1.south) {Have};
\node [anchor=north] (eq3) at ([xshift=0.0em,yshift=-2.0em]eq1.south) {have};
\node [anchor=north] (eq4) at ([xshift=0.0em,yshift=-2.0em]eq2.south) {breakfast};
\node [anchor=east] (eq5) at ([xshift=-1.0em,yshift=-1.8em]eq1.west) {$\mathbf{a}_{2}$};
\node [anchor=west] (eq6) at ([xshift=1.0em,yshift=-1.8em]eq2.east) {$\textrm{P}(\mathbf{s},\mathbf{a}_{2}|\mathbf{t})=0.1$};
......@@ -43,7 +43,7 @@
\begin{scope}[scale=0.6,xshift=0.0,yshift=-20.0em]
\node [anchor=west] (eq1) at (0,0) {吃};
\node [anchor=west] (eq2) at ([xshift=3.0em,yshift=0.0em]eq1.east) {早饭};
\node [anchor=north] (eq3) at ([xshift=0.0em,yshift=-2.0em]eq1.south) {Have};
\node [anchor=north] (eq3) at ([xshift=0.0em,yshift=-2.0em]eq1.south) {have};
\node [anchor=north] (eq4) at ([xshift=0.0em,yshift=-2.0em]eq2.south) {breakfast};
\node [anchor=east] (eq5) at ([xshift=-1.0em,yshift=-1.8em]eq1.west) {$\mathbf{a}_{3}$};
\node [anchor=west] (eq6) at ([xshift=1.0em,yshift=-1.8em]eq2.east) {$\textrm{P}(\mathbf{s},\mathbf{a}_{3}|\mathbf{t})=0.1$};
......@@ -54,7 +54,7 @@
\begin{scope}[scale=0.6,xshift=29.0em,yshift=0.0em]
\node [anchor=west] (eq1) at (0,0) {吃};
\node [anchor=west] (eq2) at ([xshift=3.0em,yshift=0.0em]eq1.east) {早饭};
\node [anchor=north] (eq3) at ([xshift=0.0em,yshift=-2.0em]eq1.south) {Have};
\node [anchor=north] (eq3) at ([xshift=0.0em,yshift=-2.0em]eq1.south) {have};
\node [anchor=north] (eq4) at ([xshift=0.0em,yshift=-2.0em]eq2.south) {breakfast};
\node [anchor=east] (eq5) at ([xshift=-1.0em,yshift=-1.8em]eq1.west) {$\mathbf{a}_{4}$};
\node [anchor=west] (eq6) at ([xshift=1.0em,yshift=-1.8em]eq2.east) {$\textrm{P}(\mathbf{s},\mathbf{a}_{4}|\mathbf{t})=0.1$};
......@@ -65,7 +65,7 @@
\begin{scope}[scale=0.6,xshift=29.0em,yshift=-10.0em]
\node [anchor=west] (eq1) at (0,0) {吃};
\node [anchor=west] (eq2) at ([xshift=3.0em,yshift=0.0em]eq1.east) {早饭};
\node [anchor=north] (eq3) at ([xshift=0.0em,yshift=-2.0em]eq1.south) {Have};
\node [anchor=north] (eq3) at ([xshift=0.0em,yshift=-2.0em]eq1.south) {have};
\node [anchor=north] (eq4) at ([xshift=0.0em,yshift=-2.0em]eq2.south) {breakfast};
\node [anchor=east] (eq5) at ([xshift=-1.0em,yshift=-1.8em]eq1.west) {$\mathbf{a}_{5}$};
\node [anchor=west] (eq6) at ([xshift=1.0em,yshift=-1.8em]eq2.east) {$\textrm{P}(\mathbf{s},\mathbf{a}_{5}|\mathbf{t})=0.05$};
......@@ -80,7 +80,7 @@
\begin{scope}[scale=0.6,xshift=29.0em,yshift=-20.0em]
\node [anchor=west] (eq1) at (0,0) {吃};
\node [anchor=west] (eq2) at ([xshift=3.0em,yshift=0.0em]eq1.east) {早饭};
\node [anchor=north] (eq3) at ([xshift=0.0em,yshift=-2.0em]eq1.south) {Have};
\node [anchor=north] (eq3) at ([xshift=0.0em,yshift=-2.0em]eq1.south) {have};
\node [anchor=north] (eq4) at ([xshift=0.0em,yshift=-2.0em]eq2.south) {breakfast};
\node [anchor=east] (eq5) at ([xshift=-1.0em,yshift=-1.8em]eq1.west) {$\mathbf{a}_{6}$};
\node [anchor=west] (eq6) at ([xshift=1.0em,yshift=-1.8em]eq2.east) {$\textrm{P}(\mathbf{s},\mathbf{a}_{6}|\mathbf{t})=0.05$};
......
Book/Chapter4/Figures/basic-process-of-translation.tex
......@@ -5,9 +5,9 @@
\begin{scope}[minimum height = 18pt]
\node[anchor=east] (s0) at (-0.5em, 0) {$\textbf{s}$:};
\node[anchor=west,fill=green!20] (s1) at (0, 0) {\footnotesize{桌子 上}};
\node[anchor=west,fill=red!20] (s2) at ([xshift=1em]s1.east) {\footnotesize{有}};
\node[anchor=west,fill=blue!20] (s3) at ([xshift=1em]s2.east) {\footnotesize{一个 苹果}};
\node[anchor=west,fill=gray!20] (s1) at (0, 0) {\footnotesize{桌子 上}};
\node[anchor=west,fill=gray!20] (s2) at ([xshift=1em]s1.east) {\footnotesize{有}};
\node[anchor=west,fill=gray!20] (s3) at ([xshift=1em]s2.east) {\footnotesize{一个 苹果}};
\node[anchor=east] (t0) at (-0.5em, -1.5) {$\textbf{t}$:};
......@@ -19,9 +19,9 @@
\begin{scope}[xshift=15em,minimum height = 18pt]
\node[anchor=east] (s0) at (-0.5em, 0) {$\textbf{s}$:};
\node[anchor=west,fill=green!20] (s1) at (0, 0) {\footnotesize{桌子 上}};
\node[anchor=west,fill=gray!20] (s1) at (0, 0) {\footnotesize{桌子 上}};
\node[anchor=west,fill=red!20] (s2) at ([xshift=1em]s1.east) {\footnotesize{有}};
\node[anchor=west,fill=blue!20] (s3) at ([xshift=1em]s2.east) {\footnotesize{一个 苹果}};
\node[anchor=west,fill=gray!20] (s3) at ([xshift=1em]s2.east) {\footnotesize{一个 苹果}};
\node[anchor=east] (t0) at (-0.5em, -1.5) {$\textbf{t}$:};
{
......@@ -37,17 +37,17 @@
\begin{scope}[yshift=-9.5em,minimum height = 18pt]
\node[anchor=east] (s0) at (-0.5em, 0) {$\textbf{s}$:};
\node[anchor=west,fill=green!20] (s1) at (0, 0) {\footnotesize{桌子 上}};
\node[anchor=west,fill=red!20] (s2) at ([xshift=1em]s1.east) {\footnotesize{有}};
\node[anchor=west,fill=blue!20] (s3) at ([xshift=1em]s2.east) {\footnotesize{一个 苹果}};
\node[anchor=west,fill=gray!20] (s1) at (0, 0) {\footnotesize{桌子 上}};
\node[anchor=west,fill=gray!20] (s2) at ([xshift=1em]s1.east) {\footnotesize{有}};
\node[anchor=west,fill=red!20] (s3) at ([xshift=1em]s2.east) {\footnotesize{一个 苹果}};
\node[anchor=east] (t0) at (-0.5em, -1.5) {$\textbf{t}$:};
{
\node[anchor=west,fill=red!20] (t1) at (0, -1.5) {\footnotesize{There is}};
\node[anchor=west,fill=gray!20] (t1) at (0, -1.5) {\footnotesize{There is}};
\path[<->, thick] (s2.south) edge (t1.north);
}
{
\node[anchor=west,fill=blue!20] (t2) at ([xshift=1em]t1.east) {\footnotesize{an apple}};
\node[anchor=west,fill=red!20] (t2) at ([xshift=1em]t1.east) {\footnotesize{an apple}};
\path[<->, thick] (s3.south) edge (t2.north);
}
\node[anchor=north] (l) at ([xshift=7em,yshift=-0.5em]t0.south) {\footnotesize{(c)\ }};
......@@ -59,21 +59,21 @@
\begin{scope}[xshift=15em,yshift=-9.5em,minimum height = 18pt]%[scale=0.5]
\node[anchor=east] (s0) at (-0.5em, 0) {$\textbf{s}$:};
\node[anchor=west,fill=green!20] (s1) at (0, 0) {\footnotesize{桌子 上}};
\node[anchor=west,fill=red!20] (s2) at ([xshift=1em]s1.east) {\footnotesize{有}};
\node[anchor=west,fill=blue!20] (s3) at ([xshift=1em]s2.east) {\footnotesize{一个 苹果}};
\node[anchor=west,fill=red!20] (s1) at (0, 0) {\footnotesize{桌子 上}};
\node[anchor=west,fill=gray!20] (s2) at ([xshift=1em]s1.east) {\footnotesize{有}};
\node[anchor=west,fill=gray!20] (s3) at ([xshift=1em]s2.east) {\footnotesize{一个 苹果}};
\node[anchor=east] (t0) at (-0.5em, -1.5) {$\textbf{t}$:};
{
\node[anchor=west,fill=red!20] (t1) at (0, -1.5) {\footnotesize{There is}};
\node[anchor=west,fill=gray!20] (t1) at (0, -1.5) {\footnotesize{There is}};
\path[<->, thick] (s2.south) edge (t1.north);
}
{
\node[anchor=west,fill=blue!20] (t2) at ([xshift=1em]t1.east) {\footnotesize{an apple}};
\node[anchor=west,fill=gray!20] (t2) at ([xshift=1em]t1.east) {\footnotesize{an apple}};
\path[<->, thick] (s3.south) edge (t2.north);
}
{
\node[anchor=west,fill=green!20] (t3) at ([xshift=1em]t2.east) {\footnotesize{on the table}};
\node[anchor=west,fill=red!20] (t3) at ([xshift=1em]t2.east) {\footnotesize{on the table}};
\path[<->, thick] (s1.south) edge (t3.north);
}
\node[anchor=north] (l) at ([xshift=7em,yshift=-0.5em]t0.south) {\footnotesize{(d)\ }};
......
Book/Chapter4/Figures/cky-algorithm.tex 0 → 100644
%------------------------------------------------------------------------------------------------------------
%%% CKY解码
% 看NiuTrans Manual
\begin{center}
\begin{tikzpicture}
\tikzstyle{alignmentnode} = [rectangle,fill=blue!30,minimum size=0.45em,text=white,inner sep=0.1pt]
\tikzstyle{selectnode} = [rectangle,fill=green!20,minimum height=1.5em,minimum width=1.5em,inner sep=1.2pt]
\tikzstyle{srcnode} = [anchor=south west]
\begin{scope}[scale=0.85]
\node[srcnode] (c1) at (0,0) {\normalsize{\textbf{Function} CKY-Algorithm($\textbf{s},G$)}};
\node[srcnode,anchor=north west] (c21) at ([xshift=1.5em,yshift=0.4em]c1.south west) {\normalsize{\textbf{fore} $j=0$ to $ J - 1$}};
\node[srcnode,anchor=north west] (c22) at ([xshift=1.5em,yshift=0.4em]c21.south west) {\normalsize{$span[j,j+1 ]$.Add($A \to a \in G$)}};
\node[srcnode,anchor=north west] (c3) at ([xshift=-1.5em,yshift=0.4em]c22.south west) {\normalsize{\textbf{for} $l$ = 1 to $J$}};
\node[srcnode,anchor=west] (c31) at ([xshift=6em]c3.east) {\normalsize{// length of span}};
\node[srcnode,anchor=north west] (c4) at ([xshift=1.5em,yshift=0.4em]c3.south west) {\normalsize{\textbf{for} $j$ = 0 to $J-l$}};
\node[srcnode,anchor=north west] (c41) at ([yshift=0.4em]c31.south west) {\normalsize{// beginning of span}};
\node[srcnode,anchor=north west] (c5) at ([xshift=1.5em,yshift=0.4em]c4.south west) {\normalsize{\textbf{for} $k$ = $j$ to $j+l$}};
\node[srcnode,anchor=north west] (c51) at ([yshift=0.4em]c41.south west) {\normalsize{// partition of span}};
\node[srcnode,anchor=north west] (c6) at ([xshift=1.5em,yshift=0.4em]c5.south west) {\normalsize{$hypos$ = Compose($span[j, k], span[k, j+l]$)}};
\node[srcnode,anchor=north west] (c7) at ([yshift=0.4em]c6.south west) {\normalsize{$span[j, j+l]$.Update($hypos$)}};
\node[srcnode,anchor=north west] (c8) at ([xshift=-4.5em,yshift=0.4em]c7.south west) {\normalsize{\textbf{return} $span[0, J]$}};
\node[anchor=west] (c9) at ([xshift=-3.2em,yshift=1.7em]c1.west) {\small{\textrm{参数：}\textbf{s}为输入字符串。$G$为输入CFG。$J$为待分析字符串长度。}};
\node[anchor=west] (c10) at ([xshift=0em,yshift=1.3em]c9.west) {\small{\textrm{输出：字符串全部可能的语法分析结果}}};
\node[anchor=west] (c11) at ([xshift=0em,yshift=1.3em]c10.west) {\small{\textrm{输入：符合乔姆斯基范式的待分析字符串和一个上下文无关文法（CFG）}}};
\begin{pgfonlayer}{background}
\node [rectangle,inner sep=0.2em,rounded corners=1pt,fill=blue!10!white] [fit = (c1) (c21) (c3) (c6) (c7) (c8) (c11)] (gl1) {};
\end{pgfonlayer}
\end{scope}
\end{tikzpicture}
\end{center}
Book/Chapter4/Figures/example-of-cky-algorithm-execution-label.tex 0 → 100644
%------------------------------------------------------------------------------------------------------------
%%% 基于树的解码方法 - chart-based decoding
\begin{center}
\begin{tikzpicture}\footnotesize
\begin{scope}[scale=0.2]
\node[anchor=south east] (g1) at (0,0) {\small{$\textrm{S} \to \textrm{AB}\ \ \ \textrm{A} \to \textrm{CD}\ \vert \ \textrm{CF}\ \ \ \textrm{B} \to \textrm{c}\ \vert \ \textrm{BE}$}};
\node[anchor=north west] (g2) at ([yshift=0.3em]g1.south west) {\small{$\textrm{C} \to \textrm{a}\ \ \ \ \textrm{D} \to \textrm{b}\ \ \ \ \textrm{E} \to \textrm{c}\ \ \ \ \textrm{F} \to \textrm{AD}$}};
\begin{pgfonlayer}{background}
\node [rectangle,inner sep=0.1em,rounded corners=1pt,fill=green!10,drop shadow,draw=ugreen] [fit = (g1) (g2)] (gl1) {};
\end{pgfonlayer}
\end{scope}
\end{tikzpicture}
\end{center}
Book/Chapter4/Figures/example-of-cky-algorithm-execution.tex 0 → 100644
%------------------------------------------------------------------------------------------------------------
%%% CYK解码
\begin{tikzpicture}\scriptsize
\tikzstyle{alignmentnode} = [rectangle,fill=blue!20,minimum size=0.5em,text=white,inner sep=0.1pt]
\tikzstyle{selectnode} = [rectangle,fill=green!20,minimum height=1.5em,minimum width=1.5em,inner sep=1.2pt]
\tikzstyle{srcnode} = [anchor=south west]
\tikzstyle{chartnode}=[rectangle,minimum size=1.3em,draw]
%图1
\begin{scope}[scale=0.5]
\node [anchor=east] (s1) at (0,0) {a};
\node [anchor=north west] (s2) at ([yshift=-2.5em]s1.south west) {a};
\node [anchor=north west] (s3) at ([yshift=-2.4em]s2.south west) {b};
\node [anchor=north west] (s4) at ([yshift=-2.3em]s3.south west) {b};
\node [anchor=north west] (s5) at ([yshift=-2.2em]s4.south west) {c};
\node [alignmentnode,anchor=west] (cell11) at ([xshift=1.9em]s1.east) {};
\node [alignmentnode,anchor=west] (cell21) at ([xshift=1.9em]s2.east) {};
\node [alignmentnode,anchor=west] (cell22) at ([xshift=3.5em]cell21.east) {};
\node [alignmentnode,anchor=west] (cell31) at ([xshift=2em]s3.east) {};
\node [alignmentnode,anchor=west] (cell32) at ([xshift=3.5em]cell31.east) {};
\node [alignmentnode,anchor=west] (cell33) at ([xshift=3.5em]cell32.east) {};
\node [alignmentnode,anchor=west] (cell41) at ([xshift=1.9em]s4.east) {};
\node [alignmentnode,anchor=west] (cell42) at ([xshift=3.5em]cell41.east) {};
\node [alignmentnode,anchor=west] (cell43) at ([xshift=3.5em]cell42.east) {};
\node [alignmentnode,anchor=west] (cell44) at ([xshift=3.5em]cell43.east) {};
\node [alignmentnode,anchor=west] (cell51) at ([xshift=1.9em]s5.east) {};
\node [alignmentnode,anchor=west] (cell52) at ([xshift=3.5em]cell51.east) {};
\node [alignmentnode,anchor=west] (cell53) at ([xshift=3.5em]cell52.east) {};
\node [alignmentnode,anchor=west] (cell54) at ([xshift=3.5em]cell53.east) {};
\node [alignmentnode,anchor=west] (cell55) at ([xshift=3.5em]cell54.east) {};
\node [anchor=north] (l1) at ([yshift=-1em]cell51.south) {\tiny{$l$=1}};
\node [anchor=north] (l2) at ([yshift=-1em]cell52.south) {\tiny{$l$=2}};
\node [anchor=north] (l3) at ([yshift=-1em]cell53.south) {\tiny{$l$=3}};
\node [anchor=north] (l4) at ([yshift=-1em]cell54.south) {\tiny{$l$=4}};
\node [anchor=north] (l5) at ([yshift=-1em]cell55.south) {\tiny{$l$=5}};
\node [anchor=north] (caption1) at ([xshift=0.0em,yshift=0.0em]l5.south) {(a)};
\node [anchor=center] (y1) at ([xshift=-2.1em,yshift=2em]cell11.center) {\tiny{\blue 0}};
\node [anchor=center] (y2) at ([xshift=-2.1em,yshift=2em]cell21.center) {\tiny{\blue 1}};
\node [anchor=center] (y3) at ([xshift=-2.1em,yshift=2em]cell31.center) {\tiny{\blue 2}};
\node [anchor=center] (y4) at ([xshift=-2.1em,yshift=2em]cell41.center) {\tiny{\blue 3}};
\node [anchor=center] (y5) at ([xshift=-2.1em,yshift=2em]cell51.center) {\tiny{\blue 4}};
\node [anchor=center] (y6) at ([xshift=-2.1em,yshift=-2em]cell51.center) {\tiny{\blue 5}};
\node [anchor=west] (num) at ([xshift=22.3em,yshift=2em]s1.east) {\scriptsize{序号}};
\node [anchor=west] (kua) at ([xshift=0em]num.east) {\scriptsize{跨度}};
\node [anchor=west] (tui) at ([xshift=0.4em]kua.east) {\scriptsize{推导}};
\draw[-] ([yshift=-0.1em]num.south west)--([xshift=17em,yshift=-0.1em]num.south west);
{
\node [anchor=west] (n1) at ([xshift=1.4em,yshift=-1.5em]num.south west) {\scriptsize{1}};
\node [anchor=west] (k1) at ([yshift=-1.8em]kua.south west) {\scriptsize{[{\blue 0},{\blue 1}]}};
\node [anchor=west] (t1) at ([xshift=-0.8em,yshift=-1.5em]tui.south west) {\scriptsize{C $\to$ a}};
\node [anchor=center,selectnode,fill=red!20] (alig11) at (cell11.center) {\tiny{C}};
}
\end{scope}
%图2
\begin{scope}[xshift=22.5em,scale=0.5]
\node [anchor=east] (s1) at (0,0) {a};
\node [anchor=north west] (s2) at ([yshift=-2.5em]s1.south west) {a};
\node [anchor=north west] (s3) at ([yshift=-2.4em]s2.south west) {b};
\node [anchor=north west] (s4) at ([yshift=-2.3em]s3.south west) {b};
\node [anchor=north west] (s5) at ([yshift=-2.2em]s4.south west) {c};
\node [alignmentnode,anchor=west] (cell11) at ([xshift=1.9em]s1.east) {};
\node [alignmentnode,anchor=west] (cell21) at ([xshift=1.9em]s2.east) {};
\node [alignmentnode,anchor=west] (cell22) at ([xshift=3.5em]cell21.east) {};
\node [alignmentnode,anchor=west] (cell31) at ([xshift=2em]s3.east) {};
\node [alignmentnode,anchor=west] (cell32) at ([xshift=3.5em]cell31.east) {};
\node [alignmentnode,anchor=west] (cell33) at ([xshift=3.5em]cell32.east) {};
\node [alignmentnode,anchor=west] (cell41) at ([xshift=1.9em]s4.east) {};
\node [alignmentnode,anchor=west] (cell42) at ([xshift=3.5em]cell41.east) {};
\node [alignmentnode,anchor=west] (cell43) at ([xshift=3.5em]cell42.east) {};
\node [alignmentnode,anchor=west] (cell44) at ([xshift=3.5em]cell43.east) {};
\node [alignmentnode,anchor=west] (cell51) at ([xshift=1.9em]s5.east) {};
\node [alignmentnode,anchor=west] (cell52) at ([xshift=3.5em]cell51.east) {};
\node [alignmentnode,anchor=west] (cell53) at ([xshift=3.5em]cell52.east) {};
\node [alignmentnode,anchor=west] (cell54) at ([xshift=3.5em]cell53.east) {};
\node [alignmentnode,anchor=west] (cell55) at ([xshift=3.5em]cell54.east) {};
\node [anchor=north] (l1) at ([yshift=-1em]cell51.south) {\tiny{$l$=1}};
\node [anchor=north] (l2) at ([yshift=-1em]cell52.south) {\tiny{$l$=2}};
\node [anchor=north] (l3) at ([yshift=-1em]cell53.south) {\tiny{$l$=3}};
\node [anchor=north] (l4) at ([yshift=-1em]cell54.south) {\tiny{$l$=4}};
\node [anchor=north] (l5) at ([yshift=-1em]cell55.south) {\tiny{$l$=5}};
\node [anchor=north] (caption2) at ([xshift=0.0em,yshift=0.0em]l5.south) {(b)};
\node [anchor=center] (y1) at ([xshift=-2.1em,yshift=2em]cell11.center) {\tiny{\blue 0}};
\node [anchor=center] (y2) at ([xshift=-2.1em,yshift=2em]cell21.center) {\tiny{\blue 1}};
\node [anchor=center] (y3) at ([xshift=-2.1em,yshift=2em]cell31.center) {\tiny{\blue 2}};
\node [anchor=center] (y4) at ([xshift=-2.1em,yshift=2em]cell41.center) {\tiny{\blue 3}};
\node [anchor=center] (y5) at ([xshift=-2.1em,yshift=2em]cell51.center) {\tiny{\blue 4}};
\node [anchor=center] (y6) at ([xshift=-2.1em,yshift=-2em]cell51.center) {\tiny{\blue 5}};
\node [anchor=west] (num) at ([xshift=22.3em,yshift=2em]s1.east) {\scriptsize{序号}};
\node [anchor=west] (kua) at ([xshift=0em]num.east) {\scriptsize{跨度}};
\node [anchor=west] (tui) at ([xshift=0.4em]kua.east) {\scriptsize{推导}};
\draw[-] ([yshift=-0.1em]num.south west)--([xshift=17em,yshift=-0.1em]num.south west);
{
\node [anchor=west] (n1) at ([xshift=1.4em,yshift=-1.5em]num.south west) {\scriptsize{1}};
\node [anchor=west] (k1) at ([yshift=-1.8em]kua.south west) {\scriptsize{[{\blue 0},{\blue 1}]}};
\node [anchor=west] (t1) at ([xshift=-0.8em,yshift=-1.5em]tui.south west) {\scriptsize{C $\to$ a}};
\node [anchor=center,selectnode,fill=red!20] (alig11) at (cell11.center) {\tiny{C}};
}
{
\node [anchor=center] (n2) at ([yshift=-2.2em]n1.center) {\scriptsize{2}};
\node [anchor=center] (k2) at ([yshift=-2.2em]k1.center) {\scriptsize{[{\blue 1},{\blue 2}]}};
\node [anchor=west] (t2) at ([yshift=-2.2em]t1.west) {\scriptsize{C $\to$ a}};
\node [anchor=center,selectnode,fill=red!20] (alig21) at (cell21.center) {\tiny{C}};
}
{
\node [anchor=center] (n3) at ([yshift=-2.2em]n2.center) {\scriptsize{3}};
\node [anchor=center] (k3) at ([yshift=-2.2em]k2.center) {\scriptsize{[{\blue 2},{\blue 3}]}};
\node [anchor=west] (t3) at ([yshift=-2.2em]t2.west) {\scriptsize{D $\to$ b}};
\node [anchor=center,selectnode,fill=red!20] (alig31) at (cell31.center) {\tiny{D}};
}
{
\node [anchor=center] (n4) at ([yshift=-2.2em]n3.center) {\scriptsize{4}};
\node [anchor=center] (k4) at ([yshift=-2.2em]k3.center) {\scriptsize{[{\blue 3},{\blue 4}]}};
\node [anchor=west] (t4) at ([yshift=-2.2em]t3.west) {\scriptsize{D $\to$ b}};
\node [anchor=center,selectnode,fill=red!20] (alig41) at (cell41.center) {\tiny{D}};
}
{
\node [anchor=center] (n5) at ([yshift=-2.2em]n4.center) {\scriptsize{5}};
\node [anchor=center] (k5) at ([yshift=-2.2em]k4.center) {\scriptsize{[{\blue 4},{\blue 5}]}};
\node [anchor=west] (t5) at ([yshift=-2.2em]t4.west) {\scriptsize{B $\to$ c , }};
\node [anchor=east] (t52) at ([xshift=-1em,yshift=-2em]t5.east) {\scriptsize{E $\to$ c}};
\node [anchor=center,selectnode,fill=red!20] (alig51) at (cell51.center) {\tiny{B,E}};
}
\end{scope}
%图3
\begin{scope}[yshift=-16.0em,scale=0.5]
\node [anchor=east] (s1) at (0,0) {a};
\node [anchor=north west] (s2) at ([yshift=-2.5em]s1.south west) {a};
\node [anchor=north west] (s3) at ([yshift=-2.4em]s2.south west) {b};
\node [anchor=north west] (s4) at ([yshift=-2.3em]s3.south west) {b};
\node [anchor=north west] (s5) at ([yshift=-2.2em]s4.south west) {c};
\node [alignmentnode,anchor=west] (cell11) at ([xshift=1.9em]s1.east) {};
\node [alignmentnode,anchor=west] (cell21) at ([xshift=1.9em]s2.east) {};
\node [alignmentnode,anchor=west] (cell22) at ([xshift=3.5em]cell21.east) {};
\node [alignmentnode,anchor=west] (cell31) at ([xshift=2em]s3.east) {};
\node [alignmentnode,anchor=west] (cell32) at ([xshift=3.5em]cell31.east) {};
\node [alignmentnode,anchor=west] (cell33) at ([xshift=3.5em]cell32.east) {};
\node [alignmentnode,anchor=west] (cell41) at ([xshift=1.9em]s4.east) {};
\node [alignmentnode,anchor=west] (cell42) at ([xshift=3.5em]cell41.east) {};
\node [alignmentnode,anchor=west] (cell43) at ([xshift=3.5em]cell42.east) {};
\node [alignmentnode,anchor=west] (cell44) at ([xshift=3.5em]cell43.east) {};
\node [alignmentnode,anchor=west] (cell51) at ([xshift=1.9em]s5.east) {};
\node [alignmentnode,anchor=west] (cell52) at ([xshift=3.5em]cell51.east) {};
\node [alignmentnode,anchor=west] (cell53) at ([xshift=3.5em]cell52.east) {};
\node [alignmentnode,anchor=west] (cell54) at ([xshift=3.5em]cell53.east) {};
\node [alignmentnode,anchor=west] (cell55) at ([xshift=3.5em]cell54.east) {};
\node [anchor=north] (l1) at ([yshift=-1em]cell51.south) {\tiny{$l$=1}};
\node [anchor=north] (l2) at ([yshift=-1em]cell52.south) {\tiny{$l$=2}};
\node [anchor=north] (l3) at ([yshift=-1em]cell53.south) {\tiny{$l$=3}};
\node [anchor=north] (l4) at ([yshift=-1em]cell54.south) {\tiny{$l$=4}};
\node [anchor=north] (l5) at ([yshift=-1em]cell55.south) {\tiny{$l$=5}};
\node [anchor=north] (caption3) at ([xshift=0.0em,yshift=0.0em]l5.south) {(c)};
\node [anchor=center] (y1) at ([xshift=-2.1em,yshift=2em]cell11.center) {\tiny{\blue 0}};
\node [anchor=center] (y2) at ([xshift=-2.1em,yshift=2em]cell21.center) {\tiny{\blue 1}};
\node [anchor=center] (y3) at ([xshift=-2.1em,yshift=2em]cell31.center) {\tiny{\blue 2}};
\node [anchor=center] (y4) at ([xshift=-2.1em,yshift=2em]cell41.center) {\tiny{\blue 3}};
\node [anchor=center] (y5) at ([xshift=-2.1em,yshift=2em]cell51.center) {\tiny{\blue 4}};
\node [anchor=center] (y6) at ([xshift=-2.1em,yshift=-2em]cell51.center) {\tiny{\blue 5}};
\node [anchor=west] (num) at ([xshift=22.3em,yshift=2em]s1.east) {\scriptsize{序号}};
\node [anchor=west] (kua) at ([xshift=0em]num.east) {\scriptsize{跨度}};
\node [anchor=west] (tui) at ([xshift=0.4em]kua.east) {\scriptsize{推导}};
\draw[-] ([yshift=-0.1em]num.south west)--([xshift=17em,yshift=-0.1em]num.south west);
{
\node [anchor=west] (n1) at ([xshift=1.4em,yshift=-1.5em]num.south west) {\scriptsize{1}};
\node [anchor=west] (k1) at ([yshift=-1.8em]kua.south west) {\scriptsize{[{\blue 0},{\blue 1}]}};
\node [anchor=west] (t1) at ([xshift=-0.8em,yshift=-1.5em]tui.south west) {\scriptsize{C $\to$ a}};
\node [anchor=center,selectnode,fill=red!20] (alig11) at (cell11.center) {\tiny{C}};
}
{
\node [anchor=center] (n2) at ([yshift=-2.2em]n1.center) {\scriptsize{2}};
\node [anchor=center] (k2) at ([yshift=-2.2em]k1.center) {\scriptsize{[{\blue 1},{\blue 2}]}};
\node [anchor=west] (t2) at ([yshift=-2.2em]t1.west) {\scriptsize{C $\to$ a}};
\node [anchor=center,selectnode,fill=red!20] (alig21) at (cell21.center) {\tiny{C}};
}
{
\node [anchor=center] (n3) at ([yshift=-2.2em]n2.center) {\scriptsize{3}};
\node [anchor=center] (k3) at ([yshift=-2.2em]k2.center) {\scriptsize{[{\blue 2},{\blue 3}]}};
\node [anchor=west] (t3) at ([yshift=-2.2em]t2.west) {\scriptsize{D $\to$ b}};
\node [anchor=center,selectnode,fill=red!20] (alig31) at (cell31.center) {\tiny{D}};
}
{
\node [anchor=center] (n4) at ([yshift=-2.2em]n3.center) {\scriptsize{4}};
\node [anchor=center] (k4) at ([yshift=-2.2em]k3.center) {\scriptsize{[{\blue 3},{\blue 4}]}};
\node [anchor=west] (t4) at ([yshift=-2.2em]t3.west) {\scriptsize{D $\to$ b}};
\node [anchor=center,selectnode,fill=red!20] (alig41) at (cell41.center) {\tiny{D}};
}
{
\node [anchor=center] (n5) at ([yshift=-2.2em]n4.center) {\scriptsize{5}};
\node [anchor=center] (k5) at ([yshift=-2.2em]k4.center) {\scriptsize{[{\blue 4},{\blue 5}]}};
\node [anchor=west] (t5) at ([yshift=-2.2em]t4.west) {\scriptsize{B $\to$ c , }};
\node [anchor=east] (t52) at ([xshift=-1em,yshift=-2em]t5.east) {\scriptsize{E $\to$ c}};
\node [anchor=center,selectnode,fill=red!20] (alig51) at (cell51.center) {\tiny{B,E}};
}
{
\node [anchor=center] (n6) at ([yshift=-4em]n5.center) {\scriptsize{6}};
\node [anchor=center] (k6) at ([yshift=-4em]k5.center) {\scriptsize{[{\blue 0},{\blue 2}]}};
\node [anchor=west] (t6) at ([xshift=0.2em,yshift=-4em]t5.west) {\scriptsize{none}};
\node [anchor=center,selectnode,fill=red!20] (alig22) at (cell22.center) {\tiny{}};
}
{
\node [anchor=center] (n7) at ([yshift=-2.2em]n6.center) {\scriptsize{7}};
\node [anchor=center] (k7) at ([yshift=-2.2em]k6.center) {\scriptsize{[{\blue 1},{\blue 3}]}};
\node [anchor=west] (t7) at ([yshift=-2.2em]t6.west) {\scriptsize{A $\to$ CD}};
\node [anchor=center,selectnode,fill=red!20] (alig32) at (cell32.center) {\tiny{A}};
}
\end{scope}
%图4
\begin{scope}[xshift=22.5em,yshift=-16.0em,scale=0.5]
\node [anchor=east] (s1) at (0,0) {a};
\node [anchor=north west] (s2) at ([yshift=-2.5em]s1.south west) {a};
\node [anchor=north west] (s3) at ([yshift=-2.4em]s2.south west) {b};
\node [anchor=north west] (s4) at ([yshift=-2.3em]s3.south west) {b};
\node [anchor=north west] (s5) at ([yshift=-2.2em]s4.south west) {c};
\node [alignmentnode,anchor=west] (cell11) at ([xshift=1.9em]s1.east) {};
\node [alignmentnode,anchor=west] (cell21) at ([xshift=1.9em]s2.east) {};
\node [alignmentnode,anchor=west] (cell22) at ([xshift=3.5em]cell21.east) {};
\node [alignmentnode,anchor=west] (cell31) at ([xshift=2em]s3.east) {};
\node [alignmentnode,anchor=west] (cell32) at ([xshift=3.5em]cell31.east) {};
\node [alignmentnode,anchor=west] (cell33) at ([xshift=3.5em]cell32.east) {};
\node [alignmentnode,anchor=west] (cell41) at ([xshift=1.9em]s4.east) {};
\node [alignmentnode,anchor=west] (cell42) at ([xshift=3.5em]cell41.east) {};
\node [alignmentnode,anchor=west] (cell43) at ([xshift=3.5em]cell42.east) {};
\node [alignmentnode,anchor=west] (cell44) at ([xshift=3.5em]cell43.east) {};
\node [alignmentnode,anchor=west] (cell51) at ([xshift=1.9em]s5.east) {};
\node [alignmentnode,anchor=west] (cell52) at ([xshift=3.5em]cell51.east) {};
\node [alignmentnode,anchor=west] (cell53) at ([xshift=3.5em]cell52.east) {};
\node [alignmentnode,anchor=west] (cell54) at ([xshift=3.5em]cell53.east) {};
\node [alignmentnode,anchor=west] (cell55) at ([xshift=3.5em]cell54.east) {};
\node [anchor=north] (l1) at ([yshift=-1em]cell51.south) {\tiny{$l$=1}};
\node [anchor=north] (l2) at ([yshift=-1em]cell52.south) {\tiny{$l$=2}};
\node [anchor=north] (l3) at ([yshift=-1em]cell53.south) {\tiny{$l$=3}};
\node [anchor=north] (l4) at ([yshift=-1em]cell54.south) {\tiny{$l$=4}};
\node [anchor=north] (l5) at ([yshift=-1em]cell55.south) {\tiny{$l$=5}};
\node [anchor=north] (caption4) at ([xshift=0.0em,yshift=0.0em]l5.south) {(d)};
\node [anchor=center] (y1) at ([xshift=-2.1em,yshift=2em]cell11.center) {\tiny{\blue 0}};
\node [anchor=center] (y2) at ([xshift=-2.1em,yshift=2em]cell21.center) {\tiny{\blue 1}};
\node [anchor=center] (y3) at ([xshift=-2.1em,yshift=2em]cell31.center) {\tiny{\blue 2}};
\node [anchor=center] (y4) at ([xshift=-2.1em,yshift=2em]cell41.center) {\tiny{\blue 3}};
\node [anchor=center] (y5) at ([xshift=-2.1em,yshift=2em]cell51.center) {\tiny{\blue 4}};
\node [anchor=center] (y6) at ([xshift=-2.1em,yshift=-2em]cell51.center) {\tiny{\blue 5}};
\node [anchor=west] (num) at ([xshift=22.3em,yshift=2em]s1.east) {\scriptsize{序号}};
\node [anchor=west] (kua) at ([xshift=0em]num.east) {\scriptsize{跨度}};
\node [anchor=west] (tui) at ([xshift=0.4em]kua.east) {\scriptsize{推导}};
\draw[-] ([yshift=-0.1em]num.south west)--([xshift=17em,yshift=-0.1em]num.south west);
{
\node [anchor=west] (n1) at ([xshift=1.4em,yshift=-1.5em]num.south west) {\scriptsize{1}};
\node [anchor=west] (k1) at ([yshift=-1.8em]kua.south west) {\scriptsize{[{\blue 0},{\blue 1}]}};
\node [anchor=west] (t1) at ([xshift=-0.8em,yshift=-1.5em]tui.south west) {\scriptsize{C $\to$ a}};
\node [anchor=center,selectnode,fill=red!20] (alig11) at (cell11.center) {\tiny{C}};
}
{
\node [anchor=center] (n2) at ([yshift=-2.2em]n1.center) {\scriptsize{2}};
\node [anchor=center] (k2) at ([yshift=-2.2em]k1.center) {\scriptsize{[{\blue 1},{\blue 2}]}};
\node [anchor=west] (t2) at ([yshift=-2.2em]t1.west) {\scriptsize{C $\to$ a}};
\node [anchor=center,selectnode,fill=red!20] (alig21) at (cell21.center) {\tiny{C}};
}
{
\node [anchor=center] (n3) at ([yshift=-2.2em]n2.center) {\scriptsize{3}};
\node [anchor=center] (k3) at ([yshift=-2.2em]k2.center) {\scriptsize{[{\blue 2},{\blue 3}]}};
\node [anchor=west] (t3) at ([yshift=-2.2em]t2.west) {\scriptsize{D $\to$ b}};
\node [anchor=center,selectnode,fill=red!20] (alig31) at (cell31.center) {\tiny{D}};
}
{
\node [anchor=center] (n4) at ([yshift=-2.2em]n3.center) {\scriptsize{4}};
\node [anchor=center] (k4) at ([yshift=-2.2em]k3.center) {\scriptsize{[{\blue 3},{\blue 4}]}};
\node [anchor=west] (t4) at ([yshift=-2.2em]t3.west) {\scriptsize{D $\to$ b}};
\node [anchor=center,selectnode,fill=red!20] (alig41) at (cell41.center) {\tiny{D}};
}
{
\node [anchor=center] (n5) at ([yshift=-2.2em]n4.center) {\scriptsize{5}};
\node [anchor=center] (k5) at ([yshift=-2.2em]k4.center) {\scriptsize{[{\blue 4},{\blue 5}]}};
\node [anchor=west] (t5) at ([yshift=-2.2em]t4.west) {\scriptsize{B $\to$ c , }};
\node [anchor=east] (t52) at ([xshift=-1em,yshift=-2em]t5.east) {\scriptsize{E $\to$ c}};
\node [anchor=center,selectnode,fill=red!20] (alig51) at (cell51.center) {\tiny{B,E}};
}
{
\node [anchor=center] (n6) at ([yshift=-4em]n5.center) {\scriptsize{6}};
\node [anchor=center] (k6) at ([yshift=-4em]k5.center) {\scriptsize{[{\blue 0},{\blue 2}]}};
\node [anchor=west] (t6) at ([xshift=0.2em,yshift=-4.2em]t5.west) {\scriptsize{none}};
\node [anchor=center,selectnode,fill=red!20] (alig22) at (cell22.center) {\tiny{}};
}
{
\node [anchor=center] (n7) at ([yshift=-2.2em]n6.center) {\scriptsize{7}};
\node [anchor=center] (k7) at ([yshift=-2.2em]k6.center) {\scriptsize{[{\blue 1},{\blue 3}]}};
\node [anchor=west] (t7) at ([yshift=-2.2em]t6.west) {\scriptsize{A $\to$ CD}};
\node [anchor=center,selectnode,fill=red!20] (alig32) at (cell32.center) {\tiny{A}};
}
{
\node [anchor=center] (sep1) at ([yshift=-1.7em]n7.center) {\scriptsize{...}};
\node [anchor=center] (n8) at ([yshift=-3.4em]n7.center) {\scriptsize{15}};
\node [anchor=center] (k8) at ([yshift=-3.4em]k7.center) {\scriptsize{[{\blue 0},{\blue 5}]}};
\node [anchor=west] (t8) at ([yshift=-3.4em]t7.west) {\tiny{S $\to$ AB}};
\node [anchor=center,selectnode,fill=red!20] (alig33) at (cell33.center) {\tiny{}};
\node [anchor=center,selectnode,fill=red!20] (alig42) at (cell42.center) {\tiny{}};
\node [anchor=center,selectnode,fill=red!20] (alig43) at (cell43.center) {\tiny{F}};
\node [anchor=center,selectnode,fill=red!20] (alig44) at (cell44.center) {\tiny{A}};
\node [anchor=center,selectnode,fill=red!20] (alig52) at (cell52.center) {\tiny{}};
\node [anchor=center,selectnode,fill=red!20] (alig53) at (cell53.center) {\tiny{}};
\node [anchor=center,selectnode,fill=red!20] (alig54) at (cell54.center) {\tiny{}};
\node [anchor=center,selectnode,fill=red!20] (alig55) at (cell55.center) {\tiny{S}};
}
\end{scope}
\end{tikzpicture}
Book/Chapter4/Figures/example-of-phrase-table.tex
......@@ -10,7 +10,7 @@
\node [anchor=west] (s7) at ([yshift=-1.1em]s5.west) {\small{...}};
\node [anchor=west] (s6) at ([yshift=1.0em]s1.west) {\small{...}};
\begin{pgfonlayer}{background}
\node [rectangle,inner sep=0.3em,fill=red!20] [fit = (s1) (s3) (s4) (s6) (s7)] (box1) {};
\node [rectangle,inner sep=0.3em,fill=red!10] [fit = (s1) (s3) (s4) (s6) (s7)] (box1) {};
\end{pgfonlayer}
\end{tikzpicture}
\end{center}
\ No newline at end of file
Book/Chapter4/Figures/example-of-zh2en-translation-base-phrase.tex
......@@ -19,8 +19,8 @@
\path[<->, thick] (s3.south) edge (t3.north);
}
\node[anchor=south] (s0) at ([xshift=-3em,yshift=0em]s1.south) {源语:};
\node[anchor=east] (t0) at ([xshift=0em,yshift=-3.5em]s0.east) {目标语:};
\node[anchor=south] (s0) at ([xshift=-3em,yshift=0em]s1.south) {源语言:};
\node[anchor=east] (t0) at ([xshift=0em,yshift=-3.5em]s0.east) {目标语言:};
\end{scope}
\end{tikzpicture}
......
Book/Chapter4/Figures/grid-search-1.tex
......@@ -29,7 +29,7 @@
\node [anchor=center,draw,circle,inner sep=1.5pt,blue!30,fill=blue!30] (f11) at ([xshift=0em,yshift=23em]y2.north) {};
\node[anchor=south] (f12) at ([xshift=5em,yshift=-0.5em]f11.south) {\scriptsize{fixed}};
\node [anchor=center,draw,circle,inner sep=1.5pt,purple!30,fill=ugreen!50] (f21) at ([xshift=0em,yshift=-4em]f11.north) {};
\node [anchor=center,draw,circle,inner sep=1.5pt,ugreen!50,fill=ugreen!50] (f21) at ([xshift=0em,yshift=-4em]f11.north) {};
\node[anchor=south] (f22) at ([xshift=8.5em,yshift=-0.5em]f21.south) {\scriptsize{valid choices}};
\node [anchor=center,draw,circle,inner sep=1.5pt,red!30,fill=red!30] (f31) at ([xshift=0em,yshift=-4em]f21.north) {};
\node[anchor=south] (f32) at ([xshift=9.5em,yshift=-0.5em]f31.south) {\scriptsize{invalid choices}};
......
Book/Chapter4/Figures/grid-search-2.tex
......@@ -26,7 +26,7 @@
\node [anchor=center,draw,circle,inner sep=1.5pt,red!30,fill=red!30] (r33) at (2,2) {};
\node [anchor=center,draw,circle,inner sep=1.5pt,red!30,fill=red!30] (r35) at (2,1) {};
\node [anchor=center,draw,circle,inner sep=1.5pt,purple!30,fill=purple!30] (r34) at (2,3) {};
\node [anchor=center,draw,circle,inner sep=1.5pt,ugreen!50,fill=ugreen!50] (r34) at (2,3) {};
\draw [-,very thick,red!50, dashed] (1,2) -- (2,4) -- (3,2) -- (2,3) -- (1,2) -- (3,2) -- (2,1) -- (1,2) -- (2,0) -- (3,2);
\draw [-,very thick,blue!50] (0,1) -- (1,2);
......
Book/Chapter4/Figures/long-distance-dependence-in-zh2en-translation.tex
......@@ -7,9 +7,9 @@
\node[anchor=west, fill=red!30, inner sep=0.05cm] (sp3) at (14em, 0) {有了\ 大幅度\ 下降};
\draw[->] (sp1) edge [out=15, in=170] (sp3);
\node[anchor=west, fill=blue!30, inner sep=0.05cm] (tp1) at (0, -0.8) {The imports};
\node[anchor=west, fill=red!30, inner sep=0.05cm] (tp2) at (5.3em, -0.8) {drastically fell};
\node[anchor=west] (tp3) at (11.3em, -0.8) {in the past five to ten years};
\node[anchor=west, fill=blue!30, inner sep=0.05cm] (tp1) at (0, -1.2) {The imports};
\node[anchor=west, fill=red!30, inner sep=0.05cm] (tp2) at (5.3em, -1.2) {drastically fell};
\node[anchor=west] (tp3) at (11.3em, -1.2) {in the past five to ten years};
\path[->] (tp1) edge [out=30, in=150] (tp2);
\end{tikzpicture}
\ No newline at end of file
Book/Chapter4/Figures/one-best-node-alignment-and-alignment-matrix.tex
......@@ -105,7 +105,7 @@
\end{flushright}
\begin{center}
\vspace{-1em}
(a)节点对齐矩阵（1-best vs. Matrix）
\footnotesize{(a)节点对齐矩阵（1-best vs. Matrix）}
\end{center}
\begin{center}
......@@ -147,7 +147,7 @@
\begin{center}
\vspace{-2em}
(b) 抽取得到的树到树翻译规则
\footnotesize{(b) 抽取得到的树到树翻译规则}
\end{center}
\end{center}
Book/Chapter4/Figures/reorder-base-distance.tex
......@@ -34,8 +34,8 @@
\node[anchor=north] (d1) at ([xshift=-0.1em,yshift=-0.1em]distance.south) {+4};
\node[anchor=north] (d2) at ([yshift=-1.8em]d1.south) {-5};
\node[anchor=north west,fill=blue!20] (m1) at ([xshift=-1em,yshift=-0.0em]t1.south west) {\small{$\textrm{start}_1\ -\ \textrm{end}_{0}\ -\ 1$\ =\ 5\ -\ 0\ -\ 1}};
\node[anchor=north west,fill=blue!20] (m2) at ([xshift=-1em,yshift=-0.0em]t2.south west) {\small{$\textrm{start}_2\ -\ \textrm{end}_{1}\ -\ 1$\ =\ 1\ -\ 5\ -\ 1}};
\node[anchor=north west,fill=blue!20] (m1) at ([xshift=-1em,yshift=-0.0em]t1.south west) {\small{$start_1\ -\ end_{0}\ -\ 1$\ =\ 5\ -\ 0\ -\ 1}};
\node[anchor=north west,fill=blue!20] (m2) at ([xshift=-1em,yshift=-0.0em]t2.south west) {\small{$start_2\ -\ end_{1}\ -\ 1$\ =\ 1\ -\ 5\ -\ 1}};
\draw[-] ([xshift=0.02in]target.south west)--([xshift=2in]target.south west);
......
Book/Chapter4/chapter4.tex
This source diff could not be displayed because it is too large. You can view the blob instead.
Book/Chapter5/Figures/fig-4-gram.tex
......@@ -14,19 +14,19 @@
\node [anchor=south,draw,minimum width=9em,inner sep=3pt] (h1) at ([yshift=1.5em]h0.north) {\tiny{$\textbf{y}=\textrm{Softmax}(\textbf{h}_0 \textbf{U})$}};
\node [anchor=south] (ylabel) at ([yshift=1em]h1.north) {\footnotesize{$\textrm{P}(w_i|w_{i-3}w_{i-2}w_{i-1})$}};
\draw [->] ([yshift=0.1em]w0.north) -- ([yshift=-0.1em]e0.south);
\draw [->] ([yshift=0.1em]w1.north) -- ([yshift=-0.1em]e1.south);
\draw [->] ([yshift=0.1em]w2.north) -- ([yshift=-0.1em]e2.south);
\draw [->] ([yshift=0.1em]e0.north) -- ([xshift=-2em,yshift=-0.1em]h0.south);
\draw [->] ([yshift=0.1em]e1.north) -- ([yshift=-0.1em]h0.south);
\draw [->] ([yshift=0.1em]e2.north) -- ([xshift=2em,yshift=-0.1em]h0.south);
\draw [->] ([yshift=0.1em]h0.north) -- ([yshift=-0.1em]h1.south);
\draw [->] ([yshift=0.1em]h1.north) -- ([yshift=-0.1em]ylabel.south);
\draw [->,line width=1pt] ([yshift=0.1em]w0.north) -- ([yshift=-0.1em]e0.south);
\draw [->,line width=1pt] ([yshift=0.1em]w1.north) -- ([yshift=-0.1em]e1.south);
\draw [->,line width=1pt] ([yshift=0.1em]w2.north) -- ([yshift=-0.1em]e2.south);
\draw [->,line width=1pt] ([yshift=0.1em]e0.north) -- ([xshift=-2em,yshift=-0.1em]h0.south);
\draw [->,line width=1pt] ([yshift=0.1em]e1.north) -- ([yshift=-0.1em]h0.south);
\draw [->,line width=1pt] ([yshift=0.1em]e2.north) -- ([xshift=2em,yshift=-0.1em]h0.south);
\draw [->,line width=1pt] ([yshift=0.1em]h0.north) -- ([yshift=-0.1em]h1.south);
\draw [->,line width=1pt] ([yshift=0.1em]h1.north) -- ([yshift=-0.1em]ylabel.south);
{
\draw [->,dashed,red,thick] ([xshift=1em,yshift=0.1em]e1.north) -- ([xshift=1em,yshift=-0.1em]h1.south);
\draw [->,dashed,red,thick] ([xshift=-1em,yshift=0.1em]e0.north) .. controls +(north:2) and +(south:1) .. ([xshift=-3em,yshift=-0.1em]h1.south);
\draw [->,dashed,red,thick] ([xshift=1em,yshift=0.1em]e2.north) .. controls +(north:2) and +(south:1) .. ([xshift=3em,yshift=-0.1em]h1.south);
\draw [->,dashed,red,line width=1pt] ([xshift=1em,yshift=0.1em]e1.north) -- ([xshift=1em,yshift=-0.1em]h1.south);
\draw [->,dashed,red,line width=1pt] ([xshift=-1em,yshift=0.1em]e0.north) .. controls +(north:2) and +(south:1) .. ([xshift=-3em,yshift=-0.1em]h1.south);
\draw [->,dashed,red,line width=1pt] ([xshift=1em,yshift=0.1em]e2.north) .. controls +(north:2) and +(south:1) .. ([xshift=3em,yshift=-0.1em]h1.south);
}
\begin{pgfonlayer}{background}
......
Book/Chapter5/Figures/fig-activation.tex deleted 100644 → 0
%%%------------------------------------------------------------------------------------------------------------
\begin{figure}
\subfigure[softplus]{
\centering
\begin{minipage}{.2\textwidth}
\begin{tikzpicture}
\draw[->](-1.2,0)--(1.2,0)node[left,below,font=\tiny]{$x$};
\draw[->](0,-1.2)--(0,1.2)node[right,font=\tiny]{$y$};
\foreach \x in {-1.0,-0.5,0.0,0.5,1.0}{\draw(\x,0)--(\x,0.05)node[below,outer sep=2pt,font=\tiny]at(\x,0){\x};}
\foreach \y in {1.0,0.5}{\draw(0,\y)--(0.05,\y)node[left,outer sep=2pt,font=\tiny]at(0,\y){\y};}
\draw[color=red ,domain=-1.2:1]plot(\x,{ln(1+(exp(\x))});
\node[black,anchor=south] at (0,1.2) {\small $y = ln(1+e^x)$};
\end{tikzpicture}
\end{minipage}%
}
\hfill
\subfigure[sigmoid]{
\centering
\begin{minipage}{.2\textwidth}
\begin{tikzpicture}
\draw[->](-1.2,0)--(1.2,0)node[left,below,font=\tiny]{$x$};
\draw[->](0,-1.2)--(0,1.2)node[right,font=\tiny]{$y$};
\draw[dashed](-1.2,1)--(1.2,1);
\foreach \x in {-1,-0.5,0,0.5,1}{\draw(\x,0)--(\x,0.05)node[below,outer sep=2pt,font=\tiny]at(\x,0){
\pgfmathparse{(\x)*5}
\pgfmathresult};}
\foreach \y in {0.5,1.0}{\draw(0,\y)--(0.05,\y)node[left,outer sep=2pt,font=\tiny]at(0,\y){\y};}
\draw[color=red,domain=-1.2:1.2]plot(\x,{1/(1+(exp(-5*\x)))});
\node[black,anchor=south] at (0,1.2) {\small $y = \frac{1}{1+e^{-x}}$};
\end{tikzpicture}
\end{minipage}%
}
\hfill
\subfigure[tanh]{
\centering
\begin{minipage}{.2\textwidth}
\begin{tikzpicture}
\draw[->](-1.2,0)--(1.2,0)node[left,below,font=\tiny]{$x$};
\draw[->](0,-1.2)--(0,1.2)node[right,font=\tiny]{$y$};
\draw[dashed](-1.2,1)--(1.2,1);
\draw[dashed](-1.2,-1)--(1.2,-1);
\foreach \x in {-1.0,-0.5,0.0,0.5,1.0}{\draw(\x,0)--(\x,0.05)node[below,outer sep=2pt,font=\tiny]at(\x,0){\x};}
\foreach \y in {0.5,1.0}{\draw(0,\y)--(0.05,\y)node[left,outer sep=2pt,font=\tiny]at(0,\y){\y};}
\draw[color=red ,domain=-1.2:1.2]plot(\x,{tanh(\x)});
\node[black,anchor=south] at (0,1.2) {\small $y = \frac{e^{x}-e^{-x}}{e^{x}+e^{-x}}$};
\end{tikzpicture}
\end{minipage}
}\\ \vspace{-1em}
\subfigure[relu]{
\centering
\begin{minipage}{.2\textwidth}
\begin{tikzpicture}
\draw[->](-1.2,0)--(1.2,0)node[left,below,font=\tiny]{$x$};
\draw[->](0,-1.2)--(0,1.2)node[right,font=\tiny]{$y$};
\draw[dashed](-1.2,1)--(1.2,1);
\draw[dashed](-1.2,-1)--(1.2,-1);
\foreach \x in {-1.0,-0.5,0.0,0.5,1.0}{\draw(\x,0)--(\x,0.05)node[below,outer sep=2pt,font=\tiny]at(\x,0){\x};}
\foreach \y in {0.5,1.0}{\draw(0,\y)--(0.05,\y)node[left,outer sep=2pt,font=\tiny]at(0,\y){\y};}
\draw[color=red ,domain=-1.2:1.2]plot(\x,{max(\x,0)});
\node[black,anchor=south] at (0,1.2) {\small $y =\max (0, x)$};
\end{tikzpicture}
\end{minipage}%
}
\hfill
\subfigure[gaussian]{
\centering
\begin{minipage}{.2\textwidth}
\begin{tikzpicture}
\draw[->](-1.2,0)--(1.2,0)node[left,below,font=\tiny]{$x$};
\draw[->](0,-1.2)--(0,1.2)node[right,font=\tiny]{$y$};
\draw[dashed](-1.2,1)--(1.2,1);
\foreach \x in {-1.0,-0.5,0.0,0.5,1.0}{\draw(\x,0)--(\x,0.05)node[below,outer sep=2pt,font=\tiny]at(\x,0){\x};}
\foreach \y in {0.5,1.0}{\draw(0,\y)--(0.05,\y)node[left,outer sep=2pt,font=\tiny]at(0,\y){\y};}
\draw[color=red ,domain=-1.2:1.2]plot(\x,{exp(-1*((\x)^2))});
\node[black,anchor=south] at (0,1.2) {\small $y =e^{-x^2}$};
\end{tikzpicture}
\end{minipage}%
}
\hfill
\subfigure[identity]{
\centering
\begin{minipage}{.2\textwidth}
\begin{tikzpicture}
\draw[->](-1.2,0)--(1.2,0)node[left,below,font=\tiny]{$x$};
\draw[->](0,-1.2)--(0,1.2)node[right,font=\tiny]{$y$};
\foreach \x in {-1.0,-0.5,0.0,0.5,1.0}{\draw(\x,0)--(\x,0.05)node[below,outer sep=2pt,font=\tiny]at(\x,0){\x};}
\foreach \y in {0.5,1.0}{\draw(0,\y)--(0.05,\y)node[left,outer sep=2pt,font=\tiny]at(0,\y){\y};}
\draw[color=red ,domain=-1:1]plot(\x,\x);
\node[black,anchor=south] at (0,1.2) {\small $y =x$};
\end{tikzpicture}
\end{minipage}
}
\end{figure}
%%%------------------------------------------------------------------------------------------------------------
Book/Chapter5/Figures/fig-code-back-propagation-2.tex
......@@ -35,14 +35,14 @@
\node [anchor=south,draw,rounded corners,inner sep=2pt,minimum width=8em,minimum height=1.0em,fill=green!30!white,blur shadow={shadow xshift=1pt,shadow yshift=-1pt}] (h3) at ([yshift=1.0em]h2.north) {\tiny{h2 = Relu(h1 * w2 + b2)}};
\node [anchor=south,draw,rounded corners,inner sep=2pt,minimum width=8em,minimum height=1.0em,fill=green!30!white,blur shadow={shadow xshift=1pt,shadow yshift=-1pt}] (h4) at ([yshift=1.0em]h3.north) {\tiny{h3 = HardTanh(h2)}};
\draw [->,thick] (h1.north) -- (h2.south);
\draw [->,thick] (h2.north) -- (h3.south);
\draw [->,thick] (h3.north) -- (h4.south);
\draw [->,very thick] (h1.north) -- (h2.south);
\draw [->,very thick] (h2.north) -- (h3.south);
\draw [->,very thick] (h3.north) -- (h4.south);
\node [anchor=south,draw,rounded corners,inner sep=2pt,minimum width=8.0em,minimum height=1.0em,fill=red!30!white,blur shadow={shadow xshift=1pt,shadow yshift=-1pt}] (slayer) at ([yshift=1.0em]h4.north) {\tiny{h4 = Softmax(h3 * w4) (output)}};
\node [anchor=south] (losslabel) at (slayer.north) {\scriptsize{\textbf{Cross Entropy Loss}}};
\draw [->,thick] (h4.north) -- (slayer.south);
\draw [->,very thick] (h4.north) -- (slayer.south);
\end{tikzpicture}
\end{center}
......
Book/Chapter5/Figures/fig-four-layers-of-neural-network.tex
......@@ -25,10 +25,10 @@
\node [rectangle,inner sep=0.2em,fill=red!20] [fit = (neuron01) (neuron05)] (layer01) {};
\end{pgfonlayer}
\node [anchor=west] (layer00label) at ([xshift=1.25em]x5.east) {\footnotesize{\red{{输入层}}}};
\node [anchor=west] (layer00label) at ([xshift=1.3em]x5.east) {\footnotesize{第0层}};
\node [anchor=west] (layer00label2) at (layer00label.east) {\footnotesize{\red{(输入层)}}};
{
\node [anchor=west] (layer01label) at ([xshift=1em]layer01.east) {\footnotesize{第二层}};
\node [anchor=west] (layer01label) at ([xshift=1em]layer01.east) {\footnotesize{第1层}};
}
{
\node [anchor=west] (layer01label2) at (layer01label.east) {\footnotesize{\red{({隐层})}}};
......@@ -55,7 +55,7 @@
}
\end{pgfonlayer}
\node [anchor=west] (layer02label) at ([xshift=4.5em]layer02.east) {\footnotesize{第三层}};
\node [anchor=west] (layer02label) at ([xshift=4.4em]layer02.east) {\footnotesize{第2层}};
{
\node [anchor=west] (layer02label2) at (layer02label.east) {\footnotesize{\red{({隐层})}}};
}
......@@ -85,7 +85,7 @@
}
\end{pgfonlayer}
\node [anchor=west] (layer03label) at ([xshift=1em]layer03.east) {\footnotesize{第四层}};
\node [anchor=west] (layer03label) at ([xshift=1em]layer03.east) {\footnotesize{第3层}};
{
\node [anchor=west] (layer03label2) at (layer03label.east) {\footnotesize{\red{({输出层})}}};
}
......
Book/Chapter5/Figures/fig-gaussian.tex
%%%------------------------------------------------------------------------------------------------------------
\begin{tikzpicture}
\draw[->](-1.2,0)--(1.2,0)node[left,below,font=\tiny]{$x$};
\draw[->](0,-1.2)--(0,1.2)node[right,font=\tiny]{$y$};
\draw[->, line width=1pt](-1.2,0)--(1.2,0)node[left,below,font=\tiny]{$x$};
\draw[->, line width=1pt](0,-1.2)--(0,1.2)node[right,font=\tiny]{$y$};
\draw[dashed](-1.2,1)--(1.2,1);
\foreach \x in {-1.0,-0.5,0.0,0.5,1.0}{\draw(\x,0)--(\x,0.05)node[below,outer sep=2pt,font=\tiny]at(\x,0){\x};}
\foreach \y in {0.5,1.0}{\draw(0,\y)--(0.05,\y)node[left,outer sep=2pt,font=\tiny]at(0,\y){\y};}
\draw[color=red ,domain=-1.2:1.2]plot(\x,{exp(-1*((\x)^2))});
\draw[color=red ,domain=-1.2:1.2, line width=1pt]plot(\x,{exp(-1*((\x)^2))});
\node[black,anchor=south] at (0,1.2) {\small $y =e^{-x^2}$};
\end{tikzpicture}
%%%------------------------------------------------------------------------------------------------------------
......
Book/Chapter5/Figures/fig-identity.tex
%%%------------------------------------------------------------------------------------------------------------
\begin{tikzpicture}
\draw[->](-1.2,0)--(1.2,0)node[left,below,font=\tiny]{$x$};
\draw[->](0,-1.2)--(0,1.2)node[right,font=\tiny]{$y$};
\draw[->, line width=1pt](-1.2,0)--(1.2,0)node[left,below,font=\tiny]{$x$};
\draw[->, line width=1pt](0,-1.2)--(0,1.2)node[right,font=\tiny]{$y$};
\foreach \x in {-1.0,-0.5,0.0,0.5,1.0}{\draw(\x,0)--(\x,0.05)node[below,outer sep=2pt,font=\tiny]at(\x,0){\x};}
\foreach \y in {0.5,1.0}{\draw(0,\y)--(0.05,\y)node[left,outer sep=2pt,font=\tiny]at(0,\y){\y};}
\draw[color=red ,domain=-1:1]plot(\x,\x);
\draw[color=red ,domain=-1:1, line width=1pt]plot(\x,\x);
\node[black,anchor=south] at (0,1.2) {\small $y =x$};
\end{tikzpicture}
%%%------------------------------------------------------------------------------------------------------------
......
Book/Chapter5/Figures/fig-parallel.tex
......@@ -10,9 +10,9 @@
\tikzstyle{parametershard} = [draw,thick,minimum width=4em,align=left,rounded corners=2pt]
{
\node[parametershard,anchor=west,fill=yellow!10] (param1) at (0,0) {$W_o$, $b_o$};
\node[parametershard,anchor=west,fill=yellow!10] (param1) at (0,0) {$W_o$};
\node (param2) at ([xshift=1em]param1.east) {};
\node[parametershard,anchor=west,fill=red!10] (param3) at ([xshift=1em]param2.east) {$W_h$, $b_h$};
\node[parametershard,anchor=west,fill=red!10] (param3) at ([xshift=1em]param2.east) {$W_h$};
\node[anchor=south,inner sep=1pt] (serverlabel) at ([yshift=0.2em]param2.north) {\footnotesize{\textbf{parameter server}: $\mathbf w_{new} = \mathbf w - \alpha\cdot \frac{\partial L}{\partial \mathbf w}$}};
}
......@@ -88,9 +88,9 @@
\tikzstyle{parametershard} = [draw,thick,minimum width=4em,align=left,rounded corners=2pt]
{
\node[parametershard,anchor=west,fill=yellow!10] (param1) at (0,0) {$W_o$, $b_o$};
\node[parametershard,anchor=west,fill=yellow!10] (param1) at (0,0) {$W_o$};
\node (param2) at ([xshift=1em]param1.east) {};
\node[parametershard,anchor=west,fill=red!10] (param3) at ([xshift=1em]param2.east) {$W_h$, $b_h$};
\node[parametershard,anchor=west,fill=red!10] (param3) at ([xshift=1em]param2.east) {$W_h$};
\node[anchor=south,inner sep=1pt] (serverlabel) at ([yshift=0.2em]param2.north) {\footnotesize{\textbf{parameter server}: $\mathbf w_{new} = \mathbf w - \alpha\cdot \frac{\partial L}{\partial \mathbf w}$}};
}
......
Book/Chapter5/Figures/fig-piecewise.tex
......@@ -2,7 +2,7 @@
\begin{tikzpicture}
%% a two-layer neural network
\begin{scope}
\begin{scope}[xshift=2in]
\tikzstyle{neuronnode} = [minimum size=1.7em,circle,draw,ublue,very thick,inner sep=1pt, fill=white,align=center,drop shadow={shadow xshift=0.1em,shadow yshift=-0.1em}]
%% output illustration
\begin{scope}[xshift=2.8in,yshift=0.1in]
......@@ -10,7 +10,7 @@
\draw [->,thick] (-2.2,0) -- (2.2,0);
\draw [->,thick] (0,0) -- (0,2);
\draw [-] (-0.05,1) -- (0.05,1);
\node [anchor=north,inner sep=1pt] (labela) at (0,-0.2) {\footnotesize{(a)}};
\node [anchor=north,inner sep=1pt] (labelb) at (0,-0.2) {\footnotesize{(b)}};
}
{
\draw [->,thick] (-2.2,0) -- (2.2,0);
......@@ -25,7 +25,7 @@
\end{scope}
%% a two-layer neural network
\begin{scope}[xshift=2in]
\begin{scope}[xshift=0in]
\tikzstyle{neuronnode} = [minimum size=1.7em,circle,draw,ublue,very thick,inner sep=1pt, fill=white,align=center,drop shadow={shadow xshift=0.1em,shadow yshift=-0.1em}]
%% output illustration
\begin{scope}[xshift=2.8in,yshift=0.1in]
......@@ -35,7 +35,7 @@
\draw [-] (-0.05,1) -- (0.05,1);
\node [anchor=east,inner sep=1pt] (label1) at (0,1) {\tiny{1}};
\node [anchor=south east,inner sep=1pt] (label2) at (0,0) {\tiny{0}};
\node [anchor=north,inner sep=1pt] (labelb) at (0,-0.2) {\footnotesize{(b)}};
\node [anchor=north,inner sep=1pt] (labela) at (0,-0.2) {\footnotesize{(a)}};
}
{
\draw [->,thick] (-2.2,0) -- (2.2,0);
......
Book/Chapter5/Figures/fig-relu.tex
%%%------------------------------------------------------------------------------------------------------------
\begin{tikzpicture}
\draw[->](-1.2,0)--(1.2,0)node[left,below,font=\tiny]{$x$};
\draw[->](0,-1.2)--(0,1.2)node[right,font=\tiny]{$y$};
\draw[->, line width=1pt](-1.2,0)--(1.2,0)node[left,below,font=\tiny]{$x$};
\draw[->, line width=1pt](0,-1.2)--(0,1.2)node[right,font=\tiny]{$y$};
\draw[dashed](-1.2,1)--(1.2,1);
\draw[dashed](-1.2,-1)--(1.2,-1);
\foreach \x in {-1.0,-0.5,0.0,0.5,1.0}{\draw(\x,0)--(\x,0.05)node[below,outer sep=2pt,font=\tiny]at(\x,0){\x};}
\foreach \y in {0.5,1.0}{\draw(0,\y)--(0.05,\y)node[left,outer sep=2pt,font=\tiny]at(0,\y){\y};}
\draw[color=red ,domain=-1.2:1.2]plot(\x,{max(\x,0)});
\draw[color=red ,domain=-1.2:1.2, line width=1pt]plot(\x,{max(\x,0)});
\node[black,anchor=south] at (0,1.2) {\small $y =\max (0, x)$};
\end{tikzpicture}
%%%------------------------------------------------------------------------------------------------------------
......
Book/Chapter5/Figures/fig-sigmoid.tex
%%%------------------------------------------------------------------------------------------------------------
\begin{tikzpicture}
\draw[->](-1.2,0)--(1.2,0)node[left,below,font=\tiny]{$x$};
\draw[->](0,-1.2)--(0,1.2)node[right,font=\tiny]{$y$};
\draw[->, line width=1pt](-1.2,0)--(1.2,0)node[left,below,font=\tiny]{$x$};
\draw[->, line width=1pt](0,-1.2)--(0,1.2)node[right,font=\tiny]{$y$};
\draw[dashed](-1.2,1)--(1.2,1);
\foreach \x in {-1,-0.5,0,0.5,1}{\draw(\x,0)--(\x,0.05)node[below,outer sep=2pt,font=\tiny]at(\x,0){
\pgfmathparse{(\x)*5}
\pgfmathresult};}
\foreach \y in {0.5,1.0}{\draw(0,\y)--(0.05,\y)node[left,outer sep=2pt,font=\tiny]at(0,\y){\y};}
\draw[color=red,domain=-1.2:1.2]plot(\x,{1/(1+(exp(-5*\x)))});
\draw[color=red,domain=-1.2:1.2, line width=1pt]plot(\x,{1/(1+(exp(-5*\x)))});
\node[black,anchor=south] at (0,1.2) {\small $y = \frac{1}{1+e^{-x}}$};
\end{tikzpicture}
%%%------------------------------------------------------------------------------------------------------------
......
Book/Chapter5/Figures/fig-softplus.tex
%%%------------------------------------------------------------------------------------------------------------
\begin{tikzpicture}
\draw[->](-1.2,0)--(1.2,0)node[left,below,font=\tiny]{$x$};
\draw[->](0,-1.2)--(0,1.2)node[right,font=\tiny]{$y$};
\draw[->, line width=1pt](-1.2,0)--(1.2,0)node[left,below,font=\tiny]{$x$};
\draw[->, line width=1pt](0,-1.2)--(0,1.2)node[right,font=\tiny]{$y$};
\foreach \x in {-1.0,-0.5,0.0,0.5,1.0}{\draw(\x,0)--(\x,0.05)node[below,outer sep=2pt,font=\tiny]at(\x,0){\x};}
\foreach \y in {1.0,0.5}{\draw(0,\y)--(0.05,\y)node[left,outer sep=2pt,font=\tiny]at(0,\y){\y};}
\draw[color=red ,domain=-1.2:1]plot(\x,{ln(1+(exp(\x))});
\draw[color=red ,domain=-1.2:1, line width=1pt]plot(\x,{ln(1+(exp(\x))});
\node[black,anchor=south] at (0,1.2) {\small $y = ln(1+e^x)$};
\end{tikzpicture}
%%%------------------------------------------------------------------------------------------------------------
......
Book/Chapter5/Figures/fig-tanh.tex
%%%------------------------------------------------------------------------------------------------------------
\begin{tikzpicture}
\draw[->](-1.2,0)--(1.2,0)node[left,below,font=\tiny]{$x$};
\draw[->](0,-1.2)--(0,1.2)node[right,font=\tiny]{$y$};
\draw[->, line width=1pt](-1.2,0)--(1.2,0)node[left,below,font=\tiny]{$x$};
\draw[->, line width=1pt](0,-1.2)--(0,1.2)node[right,font=\tiny]{$y$};
\draw[dashed](-1.2,1)--(1.2,1);
\draw[dashed](-1.2,-1)--(1.2,-1);
\foreach \x in {-1.0,-0.5,0.0,0.5,1.0}{\draw(\x,0)--(\x,0.05)node[below,outer sep=2pt,font=\tiny]at(\x,0){\x};}
\foreach \y in {0.5,1.0}{\draw(0,\y)--(0.05,\y)node[left,outer sep=2pt,font=\tiny]at(0,\y){\y};}
\draw[color=red ,domain=-1.2:1.2]plot(\x,{tanh(\x)});
\draw[color=red ,domain=-1.2:1.2, line width=1pt]plot(\x,{tanh(\x)});
\node[black,anchor=south] at (0,1.2) {\small $y = \frac{e^{x}-e^{-x}}{e^{x}+e^{-x}}$};
\end{tikzpicture}
%%%------------------------------------------------------------------------------------------------------------
......
Book/Chapter5/Figures/fig-transformer-LM.tex
%%%------------------------------------------------------------------------------------------------------------
\begin{tikzpicture}
\begin{tikzpicture}
\begin{scope}
\node [anchor=west] (w0) at (0,0) {\footnotesize{$w_{0}$}};
\node [anchor=west] (w1) at ([xshift=5em]w0.east) {\footnotesize{$w_{1}$}};
\node [anchor=west] (w2) at ([xshift=5em]w1.east) {\footnotesize{$w_{2}$}};
\node [anchor=west] (w3) at ([xshift=5em]w2.east) {\footnotesize{$w_{3}$}};
\node [anchor=north] (index0) at ([yshift=0.5em]w0.south) {\tiny(index)};
\node [anchor=north] (index1) at ([yshift=0.5em]w1.south) {\tiny(index)};
\node [anchor=north] (index2) at ([yshift=0.5em]w2.south) {\tiny(index)};
\node [anchor=north] (index3) at ([yshift=0.5em]w3.south) {\tiny(index)};
\node [anchor=south,draw,inner sep=3pt] (e0) at ([yshift=1em]w0.north) {\tiny{$\textbf{e}_0=w_{0} \textbf{C} + \textrm{PE}(0)$}};
\node [anchor=south,draw,inner sep=3pt] (e1) at ([yshift=1em]w1.north) {\tiny{$\textbf{e}_1=w_{1} \textbf{C} + \textrm{PE}(1)$}};
\node [anchor=south,draw,inner sep=3pt] (e2) at ([yshift=1em]w2.north) {\tiny{$\textbf{e}_2=w_{2} \textbf{C} + \textrm{PE}(2)$}};
\node [anchor=south,draw,inner sep=3pt] (e3) at ([yshift=1em]w3.north) {\tiny{$\textbf{e}_3=w_{3} \textbf{C} + \textrm{PE}(3)$}};
\node [anchor=south,draw,inner sep=3pt] (h0) at ([xshift=-0.5em, yshift=1.5em]e0.north) {\tiny{$\textbf{h}_{0}=\textrm{SelfAtt}(\textbf{e}_0,\textbf{e}_3)$}};
\node [anchor=south,draw,inner sep=3pt] (h1) at ([xshift=0.5em, yshift=1.5em]e1.north) {\tiny{$\textbf{h}_{1}=\textrm{SelfAtt}(\textbf{e}_1,\textbf{e}_3)$}};
\node [anchor=south,draw,inner sep=3pt] (h2) at ([xshift=1.5em, yshift=1.5em]e2.north) {\tiny{$\textbf{h}_{2}=\textrm{SelfAtt}(\textbf{e}_2,\textbf{e}_3)$}};
\node [anchor=south,draw,minimum width=9em,inner sep=3pt] (f1) at ([xshift=0.5em, yshift=1.5em]h2.north) {\tiny{$\textbf{f}_3=\textrm{FNN}([\textbf{h}_0,\textbf{h}_1,\textbf{h}_2,\textbf{e}_3])$}};
\node [anchor=south,draw,minimum width=9em,inner sep=3pt] (o1) at ([yshift=1em]f1.north) {\tiny{$\textbf{y}=\textrm{Softmax}(f_3 \textbf{U})$}};
\node [anchor=south] (ylabel) at ([yshift=1em]o1.north) {\footnotesize{$\textrm{P}(w_4|w_{0}w_{1}w_{2}w_{3})$}};
\draw [->] ([yshift=0.1em]w0.north) -- ([yshift=-0.1em]e0.south);
\draw [->] ([yshift=0.1em]w1.north) -- ([yshift=-0.1em]e1.south);
\draw [->] ([yshift=0.1em]w2.north) -- ([yshift=-0.1em]e2.south);
\draw [->] ([yshift=0.1em]w3.north) -- ([yshift=-0.1em]e3.south);
\draw [->] ([yshift=0.1em]e0.north) -- ([xshift=0em,yshift=-0.1em]h0.south);
\draw [->] ([yshift=0.1em]e1.north) -- ([xshift=-0.5em,yshift=-0.1em]h1.south);
\draw [->] ([yshift=0.1em]e2.north) -- ([xshift=-1em,yshift=-0.1em]h2.south);
\draw [->] ([yshift=0.1em]e3.north) -- ([xshift=1em,yshift=-0.1em]h0.south);
\draw [->] ([yshift=0.1em]e3.north) -- ([xshift=1em,yshift=-0.1em]h1.south);
\draw [->] ([yshift=0.1em]e3.north) -- ([xshift=1em,yshift=-0.1em]h2.south);
\draw [->] ([yshift=0.1em]h0.north) -- ([xshift=-2em,yshift=-0.1em]f1.south);
\draw [->] ([yshift=0.1em]e3.north) -- ([xshift=2em,yshift=-0.1em]f1.south);
\draw [->] ([yshift=0.1em]h1.north) -- ([xshift=-1em,yshift=-0.1em]f1.south);
\draw [->] ([yshift=0.1em]h2.north) -- ([xshift=0em,yshift=-0.1em]f1.south);
\draw [->] ([yshift=0.1em]f1.north) -- ([yshift=-0.1em]o1.south);
\draw [->] ([yshift=0.1em]o1.north) -- ([yshift=-0.1em]ylabel.south);
\node [anchor=west] (w1) at ([xshift=6.5em]w0.east) {\footnotesize{$w_{1}$}};
\node [anchor=west] (w2) at ([xshift=6.5em]w1.east) {\footnotesize{$w_{2}$}};
\node [anchor=west] (w3) at ([xshift=6.5em]w2.east) {\footnotesize{$w_{3}$}};
\node [anchor=north] (index0) at ([yshift=0.5em]w0.south) {\footnotesize(index)};
\node [anchor=north] (index1) at ([yshift=0.5em]w1.south) {\footnotesize(index)};
\node [anchor=north] (index2) at ([yshift=0.5em]w2.south) {\footnotesize(index)};
\node [anchor=north] (index3) at ([yshift=0.5em]w3.south) {\footnotesize(index)};
{
\node [anchor=south,draw,inner sep=3pt,fill=blue!20!white] (e0) at ([yshift=1em]w0.north) {\tiny{$\textbf{e}_0=w_{0} \textbf{C} + \textrm{PE}(0)$}};
\node [anchor=south,draw,inner sep=3pt,fill=blue!20!white] (e1) at ([yshift=1em]w1.north) {\tiny{$\textbf{e}_1=w_{1} \textbf{C} + \textrm{PE}(1)$}};
\node [anchor=south,draw,inner sep=3pt,fill=blue!20!white] (e2) at ([yshift=1em]w2.north) {\tiny{$\textbf{e}_2=w_{2} \textbf{C} + \textrm{PE}(2)$}};
\node [anchor=south,draw,inner sep=3pt,fill=blue!20!white] (e3) at ([yshift=1em]w3.north) {\tiny{$\textbf{e}_3=w_{3} \textbf{C} + \textrm{PE}(3)$}};
\node [anchor=south,line width=0.8pt,draw,inner sep=3pt,fill=blue!20!white] (e0) at ([yshift=1em]w0.north) {\footnotesize{$\textbf{e}_0=w_{0} \textbf{C} + \textrm{PE}(0)$}};
\node [anchor=south,line width=0.8pt,draw,inner sep=3pt,fill=blue!20!white] (e1) at ([yshift=1em]w1.north) {\footnotesize{$\textbf{e}_1=w_{1} \textbf{C} + \textrm{PE}(1)$}};
\node [anchor=south,line width=0.8pt,draw,inner sep=3pt,fill=blue!20!white] (e2) at ([yshift=1em]w2.north) {\footnotesize{$\textbf{e}_2=w_{2} \textbf{C} + \textrm{PE}(2)$}};
\node [anchor=south,line width=0.8pt,draw,inner sep=3pt,fill=blue!20!white] (e3) at ([yshift=1em]w3.north) {\footnotesize{$\textbf{e}_3=w_{3} \textbf{C} + \textrm{PE}(3)$}};
}
{
\node [anchor=south,draw,inner sep=3pt,fill=ugreen!20!white] (h0) at ([xshift=-0.5em, yshift=1.5em]e0.north) {\tiny{$\textbf{h}_{0}=\textrm{SelfAtt}(\textbf{e}_0,\textbf{e}_3)$}};
\node [anchor=south,draw,inner sep=3pt,fill=ugreen!20!white] (h1) at ([xshift=0.5em, yshift=1.5em]e1.north) {\tiny{$\textbf{h}_{1}=\textrm{SelfAtt}(\textbf{e}_1,\textbf{e}_3)$}};
\node [anchor=south,draw,inner sep=3pt,fill=ugreen!20!white] (h2) at ([xshift=1.5em, yshift=1.5em]e2.north) {\tiny{$\textbf{h}_{2}=\textrm{SelfAtt}(\textbf{e}_2,\textbf{e}_3)$}};
\node [anchor=south,line width=0.8pt,draw,inner sep=3pt,fill=ugreen!20!white] (h0) at ([xshift=-0.5em, yshift=2.0em]e0.north) {\footnotesize{$\textbf{h}_{0}=\textrm{SelfAtt}(\textbf{e}_0,\textbf{e}_3)$}};
\node [anchor=south,line width=0.8pt,draw,inner sep=3pt,fill=ugreen!20!white] (h1) at ([xshift=0.5em, yshift=2.0em]e1.north) {\footnotesize{$\textbf{h}_{1}=\textrm{SelfAtt}(\textbf{e}_1,\textbf{e}_3)$}};
\node [anchor=south,line width=0.8pt,draw,inner sep=3pt,fill=ugreen!20!white] (h2) at ([xshift=1.5em, yshift=2.0em]e2.north) {\footnotesize{$\textbf{h}_{2}=\textrm{SelfAtt}(\textbf{e}_2,\textbf{e}_3)$}};
}
{
\node [anchor=south,draw,minimum width=9em,inner sep=3pt,fill=orange!20!white] (f1) at ([xshift=0.5em, yshift=1.5em]h2.north) {\tiny{$\textbf{f}_3=\textrm{FNN}([\textbf{h}_0,\textbf{h}_1,\textbf{h}_2,\textbf{e}_3])$}};
\node [anchor=south,draw,minimum width=9em,inner sep=3pt,fill=orange!20!white] (o1) at ([yshift=1em]f1.north) {\tiny{$\textbf{y}=\textrm{Softmax}(f_3 \textbf{U})$}};
\node [anchor=south,line width=0.8pt,draw,minimum width=9em,inner sep=3pt,fill=orange!20!white] (f1) at ([xshift=0.5em, yshift=2.0em]h2.north) {\footnotesize{$\textbf{f}_3=\textrm{FNN}([\textbf{h}_0,\textbf{h}_1,\textbf{h}_2,\textbf{e}_3])$}};
\node [anchor=south,line width=0.8pt,draw,minimum width=9em,inner sep=3pt,fill=orange!20!white] (o1) at ([yshift=1.2em]f1.north) {\footnotesize{$\textbf{y}=\textrm{Softmax}(f_3 \textbf{U})$}};
}
\node [anchor=south] (ylabel) at ([yshift=1.2em]o1.north) {\footnotesize{$\textrm{P}(w_4|w_{0}w_{1}w_{2}w_{3})$}};
\draw [->,line width=0.6pt] ([yshift=0.1em]w0.north) -- ([yshift=-0.1em]e0.south);
\draw [->,line width=0.6pt] ([yshift=0.1em]w1.north) -- ([yshift=-0.1em]e1.south);
\draw [->,line width=0.6pt] ([yshift=0.1em]w2.north) -- ([yshift=-0.1em]e2.south);
\draw [->,line width=0.6pt] ([yshift=0.1em]w3.north) -- ([yshift=-0.1em]e3.south);
\draw [->,line width=0.6pt] ([yshift=0.1em]e0.north) -- ([xshift=0em,yshift=-0.1em]h0.south);
\draw [->,line width=0.6pt] ([yshift=0.1em]e1.north) -- ([xshift=-0.5em,yshift=-0.1em]h1.south);
\draw [->,line width=0.6pt] ([yshift=0.1em]e2.north) -- ([xshift=-1em,yshift=-0.1em]h2.south);
\draw [->,line width=0.6pt] ([yshift=0.1em]e3.north) -- ([xshift=1em,yshift=-0.1em]h0.south);
\draw [->,line width=0.6pt] ([yshift=0.1em]e3.north) -- ([xshift=1em,yshift=-0.1em]h1.south);
\draw [->,line width=0.6pt] ([yshift=0.1em]e3.north) -- ([xshift=1em,yshift=-0.1em]h2.south);
\draw [->,line width=0.6pt] ([yshift=0.1em]h0.north) -- ([xshift=-2em,yshift=-0.1em]f1.south);
\draw [->,line width=0.6pt] ([yshift=0.1em]e3.north) -- ([xshift=2em,yshift=-0.1em]f1.south);
\draw [->,line width=0.6pt] ([yshift=0.1em]h1.north) -- ([xshift=-1em,yshift=-0.1em]f1.south);
\draw [->,line width=0.6pt] ([yshift=0.1em]h2.north) -- ([xshift=0em,yshift=-0.1em]f1.south);
\draw [->,line width=0.6pt] ([yshift=0.1em]f1.north) -- ([yshift=-0.1em]o1.south);
\draw [->,line width=0.6pt] ([yshift=0.1em]o1.north) -- ([yshift=-0.1em]ylabel.south);
\end{scope}
\end{tikzpicture}
\ No newline at end of file
%%%------------------------------------------------------------------------------------------------------------
Book/Chapter5/Figures/fig-two-layer-neural-network.tex
......@@ -34,12 +34,12 @@
\draw [->,thick] ([yshift=0.1em]n20.north) -- (y.south);
%% weight and bias
{\node [anchor=center,rotate=90,fill=white,inner sep=1pt] (b0) at ([yshift=2em,xshift=-0.5em]b.north) {\tiny{$b_1$}};}
{\node [anchor=center,rotate=-59,fill=white,inner sep=1pt] (w2) at ([yshift=1em,xshift=-1.0em]x1.north) {\tiny{$w_1$}};}
{\node [anchor=center,rotate=62,fill=white,inner sep=1pt] (w21) at ([yshift=1.2em,xshift=0em]n10.north) {\tiny{$w'_1$}};}
{\node [anchor=center,rotate=-62,fill=white,inner sep=1pt] (w22) at ([yshift=1.2em,xshift=-0em]n11.north) {\tiny{$w'_2$}};}
{\node [anchor=center,rotate=59,fill=white,inner sep=1pt] (b1) at ([yshift=3.4em,xshift=1.5em]b.north) {\tiny{$b_2$}};}
{\node [anchor=center,rotate=90,fill=white,inner sep=1pt] (w1) at ([yshift=2em,xshift=0.5em]x1.north) {\tiny{$w_2$}};}
{\node [anchor=center,rotate=90,fill=white,inner sep=1pt] (b0) at ([yshift=2em,xshift=-0.5em]b.north) {\scriptsize{$b_1$}};}
{\node [anchor=center,rotate=-59,fill=white,inner sep=1pt] (w2) at ([yshift=1em,xshift=-1.0em]x1.north) {\scriptsize{$w_1$}};}
{\node [anchor=center,rotate=62,fill=white,inner sep=1pt] (w21) at ([yshift=1.2em,xshift=0em]n10.north) {\scriptsize{$w'_1$}};}
{\node [anchor=center,rotate=-62,fill=white,inner sep=1pt] (w22) at ([yshift=1.2em,xshift=-0em]n11.north) {\scriptsize{$w'_2$}};}
{\node [anchor=center,rotate=59,fill=white,inner sep=1pt] (b1) at ([yshift=3.4em,xshift=1.5em]b.north) {\scriptsize{$b_2$}};}
{\node [anchor=center,rotate=90,fill=white,inner sep=1pt] (w1) at ([yshift=2em,xshift=0.5em]x1.north) {\scriptsize{$w_2$}};}
%% sigmoid box
\begin{scope}
......
Book/Chapter5/Figures/fig-weather-2layer.tex deleted 100644 → 0
%%%------------------------------------------------------------------------------------------------------------
\begin{tikzpicture}
%左
\node [anchor=west,draw=ublue,minimum width=1.5em,minimum height=1.5em] (part1-1) at (0,0) {\small{$x_1$}};
\node [anchor=north,draw=ublue,minimum width=1.5em,minimum height=1.5em] (part1-2) at ([yshift=-2.5em]part1-1.south) {\small {$x_2$}};
\node [anchor=north,draw=ublue,minimum width=1.5em,minimum height=1.5em] (part1-3) at ([yshift=-2.5em]part1-2.south) {\small {$x_3$}};
\node [anchor=north,minimum width=3.0em] (part1-4) at ([yshift=-1.0em]part1-3.south) {\small {输入层}};
\node[anchor=south,minimum height=13em,minimum width=4.0em,draw=ublue,dotted,thick] (part1out) at ([xshift=0.0em,yshift=-8.5em]part1-2.north) {};
%中
\node [circle,anchor=west,draw=ublue,minimum width=2.0em] (part2-1) at ([xshift=4.6em,yshift=2.0em]part1-2.east) {\small {$a_1$}};
\node [circle,anchor=west,draw=ublue,minimum width=2.0em] (part2-2) at ([xshift=4.6em,yshift=-2.0em]part1-2.east) {\small {$a_2$}};
\node [anchor=north,minimum width=3.0em] (part2-3) at ([xshift=0.0em,yshift=-2.79em]part2-2.south) {\small {隐藏层}};
\node[anchor=south,minimum height=13em,minimum width=4.0em,draw=ublue,dotted,thick] (part2out) at ([xshift=6.4em,yshift=-8.5em]part1-2.north) {};
%右
\node [circle,anchor=west,draw=ublue,minimum width=2.0em] (part3-1) at ([xshift=10.5em,yshift=0.0em]part1-2.east) {\small {$y$}};
\node [anchor=west,draw=ublue,minimum width=1.5em,minimum height=1.5em] (part3-2) at ([xshift=1.2em]part3-1.east) {\small {$y$}};
\node [anchor=north,minimum width=3.0em] (part3-3) at ([xshift=1.4em,yshift=-4.79em]part3-1.south) {\small {输出层}};
\node[anchor=south,minimum height=13em,minimum width=6.0em,draw=ublue,dotted,thick] (part3out) at ([xshift=13.8em,yshift=-8.5em]part1-2.north) {};
%连线
\draw [->,thick,ublue](part1-1.east)--(part2-1.west);
\draw [->,thick,ublue](part1-1.east)--(part2-2.west);
\draw [->,thick,ublue](part1-2.east)--(part2-1.west);
\draw [->,thick,ublue](part1-2.east)--(part2-2.west);
\draw [->,thick,ublue](part1-3.east)--(part2-1.west);
\draw [->,thick,ublue](part1-3.east)--(part2-2.west);
\draw [->,thick,ublue](part2-1.east)--(part3-1.west);
\draw [->,thick,ublue](part2-2.east)--(part3-1.west);
\draw [->,thick,ublue](part3-1.east)--(part3-2.west);
\end{tikzpicture}
%%------------------------------------------------------------------------------------------------------------
Book/Chapter5/Figures/fig-weather-forward.tex
......@@ -18,8 +18,8 @@
\node [anchor=north,minimum width=4.0em,minimum height=1.5em] (part5) at ([yshift=-1.4em]part4.south) {\footnotesize {$\mathbf a$}};
\draw [-,thick](part4.south)--([yshift=-0.1em]part5.north);
%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\node [anchor=west,minimum width=2.0em,minimum height=1.5em,draw,fill=red!20] (part5-3) at ([xshift=0.0em,yshift=0.1em]part5.east) {\footnotesize {$\mathbf w^2$}};
\node [anchor=west,minimum width=2.0em,minimum height=1.5em,draw,fill=orange!40] (part5-4) at ([xshift=2.0em,yshift=0.0em]part5-3.east) {\footnotesize {$\mathbf b^2$}};
\node [anchor=west,minimum width=2.0em,minimum height=1.5em,draw,fill=red!20] (part5-3) at ([xshift=0.0em,yshift=0.1em]part5.east) {\footnotesize {$\mathbf w^{[2]}$}};
\node [anchor=west,minimum width=2.0em,minimum height=1.5em,draw,fill=orange!40] (part5-4) at ([xshift=2.0em,yshift=0.0em]part5-3.east) {\footnotesize {$ b^{[2]}$}};
\draw[-,thick](part4.south)--(part5-3.north);
\draw[-,thick](part3.south)--(part5-4.north);
\node [anchor=south,minimum width=1.5em,minimum height=1.5em] (part5-3-1) at ([xshift=1.1em,yshift=-0.45em]part5-3.north) {\scriptsize {$1\times 2$}};
......@@ -41,8 +41,8 @@
\node [anchor=north,minimum width=4.0em,minimum height=1.5em] (part9) at ([yshift=-1.4em]part8.south) {\footnotesize {$\mathbf x$}};
\draw [-,thick](part8.south)--([yshift=-0.1em]part9.north);
%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\node [anchor=west,minimum width=2.0em,minimum height=1.5em,draw,fill=red!20] (part9-3) at ([xshift=0.0em,yshift=0.1em]part9.east) {\footnotesize {$\mathbf w^1$}};
\node [anchor=west,minimum width=2.0em,minimum height=1.5em,draw,fill=orange!40] (part9-4) at ([xshift=2.0em,yshift=0.0em]part9-3.east) {\footnotesize {$\mathbf b^1$}};
\node [anchor=west,minimum width=2.0em,minimum height=1.5em,draw,fill=red!20] (part9-3) at ([xshift=0.0em,yshift=0.1em]part9.east) {\footnotesize {$\mathbf w^{[1]}$}};
\node [anchor=west,minimum width=2.0em,minimum height=1.5em,draw,fill=orange!40] (part9-4) at ([xshift=2.0em,yshift=0.0em]part9-3.east) {\footnotesize {$\mathbf b^{[1]}$}};
\draw[-,thick](part8.south)--(part9-3.north);
\draw[-,thick](part7.south)--(part9-4.north);
\node [anchor=south,minimum width=1.5em,minimum height=1.5em] (part9-3-1) at ([xshift=1.1em,yshift=-0.45em]part9-3.north) {\scriptsize {$3\times 2$}};
......
Book/Chapter5/Figures/fig-weather-model.tex deleted 100644 → 0
%%%------------------------------------------------------------------------------------------------------------
\begin{tikzpicture}
%左
\node [anchor=west,draw=ublue,minimum width=3.0em] (part1-1) at (0,0) {\small{天空状况}};
\node [anchor=north,draw=ublue,minimum width=3.0em] (part1-2) at ([yshift=-2.5em]part1-1.south) {\small {低空气温}};
\node [anchor=north,draw=ublue,minimum width=3.0em] (part1-3) at ([yshift=-2.5em]part1-2.south) {\small {水平气压}};
\node [anchor=north,minimum width=3.0em] (part1-4) at ([yshift=-1.0em]part1-3.south) {\small {输入层}};
\node[anchor=south,minimum height=13em,minimum width=6.0em,draw=ublue,dotted,thick] (part1out) at ([xshift=0.0em,yshift=-8.5em]part1-2.north) {};
%中
\node [circle,anchor=west,draw=ublue,minimum width=2.2em] (part2-1) at ([xshift=4.0em,yshift=2.0em]part1-2.east) {\footnotesize {温度}};
\node [circle,anchor=west,draw=ublue,minimum width=2.2em] (part2-2) at ([xshift=4.0em,yshift=-2.0em]part1-2.east) {\footnotesize {风速}};
\node [anchor=north,minimum width=3.0em] (part2-3) at ([xshift=0.0em,yshift=-2.42em]part2-2.south) {\small {隐藏层}};
\node[anchor=south,minimum height=13em,minimum width=4.0em,draw=ublue,dotted,thick] (part2out) at ([xshift=7.4em,yshift=-8.5em]part1-2.north) {};
%右
\node [anchor=west,draw=ublue,minimum width=3.0em] (part3-1) at ([xshift=10.0em,yshift=0.0em]part1-2.east) {\small {穿衣指数}};
\node [anchor=north,minimum width=3.0em] (part3-2) at ([yshift=-5.05em]part3-1.south) {\small {输出层}};
\node[anchor=south,minimum height=13em,minimum width=6.0em,draw=ublue,dotted,thick] (part3out) at ([xshift=14.4em,yshift=-8.5em]part1-2.north) {};
%连线
\draw [->,thick,ublue](part1-1.east)--(part2-1.west);
\draw [->,thick,ublue](part1-1.east)--(part2-2.west);
\draw [->,thick,ublue](part1-2.east)--(part2-1.west);
\draw [->,thick,ublue](part1-2.east)--(part2-2.west);
\draw [->,thick,ublue](part1-3.east)--(part2-1.west);
\draw [->,thick,ublue](part1-3.east)--(part2-2.west);
\draw [->,thick,ublue](part2-1.east)--(part3-1.west);
\draw [->,thick,ublue](part2-2.east)--(part3-1.west);
\end{tikzpicture}
%%%------------------------------------------------------------------------------------------------------------
Book/Chapter5/Figures/fig-weather.tex
......@@ -7,7 +7,7 @@
\node [anchor=north,draw=ublue,minimum width=3.55em,fill=yellow!20] (part1-2) at ([yshift=-2.0em]part1-1.south) {\scriptsize {低空气温}};
\node [anchor=north,draw=ublue,minimum width=3.55em,fill=yellow!20] (part1-3) at ([yshift=-2.0em]part1-2.south) {\scriptsize {水平气压}};
\node [rectangle,rounded corners,draw=black!50,densely dashed,inner sep=0.4em] [fit = (part1-1) (part1-2) (part1-3) (inputlabel)] (inputshadow) {};
\node [anchor=north,draw=ublue,minimum width=3.55em,fill=yellow!20] (part1-4) at ([yshift=-2.0em]part1-3.south) {\scriptsize {偏移1}};
\node [anchor=north,draw=ublue,minimum width=3.55em,fill=yellow!20] (part1-4) at ([yshift=-2.0em]part1-3.south) {\scriptsize {偏置1}};
\node [anchor=north,minimum width=2.5em] (part1-5) at ([yshift=-0.5em]part1-4.south) {\scriptsize {输入层}};
......@@ -16,7 +16,7 @@
\node [anchor=north] (hidlabel) at ([yshift=3.1em]part2-1.north) {\scriptsize{特征}};
\node [circle,anchor=west,draw=ublue,minimum width=2.5em,fill=blue!20] (part2-2) at ([xshift=2.0em,yshift=-1.7em]part1-2.east) {\scriptsize {风速}};
\node [rectangle,rounded corners,draw=black!50,densely dashed,inner sep=0.4em] [fit = (part2-1) (part2-2) (hidlabel) ] (inputshadow) {};
\node [circle,anchor=west,draw=ublue,minimum width=2.5em,fill=blue!20,inner sep=2pt] (part2-3) at ([xshift=2.0em,yshift=-1.7em]part1-3.east) {\scriptsize {偏移2}};
\node [circle,anchor=west,draw=ublue,minimum width=2.5em,fill=blue!20,inner sep=2pt] (part2-3) at ([xshift=2.0em,yshift=-1.7em]part1-3.east) {\scriptsize {偏置2}};
\node [anchor=north,minimum width=3.0em] (part2-4) at ([xshift=0.0em,yshift=-1.6em]part2-3.south) {\scriptsize{隐藏层}};
\node [anchor=north] (labela) at ([xshift=0.0em,yshift=-4em]part2-3.south) {\footnotesize {(a)}};
......@@ -54,7 +54,7 @@
y
\node [rectangle,rounded corners,draw=black!50,densely dashed,inner sep=0.4em] [fit = (part1-1) (part1-2) (part1-3) (inputlabel)] (inputshadow) {};
\node [anchor=north,draw=ublue,minimum width=3.55em,fill=yellow!20] (part1-4) at ([yshift=-2.0em]part1-3.south) {\footnotesize {$\mathbf b^1 $}};
\node [anchor=north,draw=ublue,minimum width=3.55em,fill=yellow!20] (part1-4) at ([yshift=-2.0em]part1-3.south) {\footnotesize {$\mathbf b^{[1]} $}};
\node [anchor=north,minimum width=2.5em] (part1-5) at ([yshift=-0.5em]part1-4.south) {\scriptsize {输入层}};
......@@ -65,7 +65,7 @@ y
\node [circle,anchor=west,draw=ublue,minimum width=2.5em,fill=blue!20] (part2-2) at ([xshift=2.0em,yshift=-1.7em]part1-2.east) {\large{$a_2$}};
\node [rectangle,rounded corners,draw=black!50,densely dashed,inner sep=0.4em] [fit = (part2-1) (part2-2) (hidlabel) ] (inputshadow) {};
\node [circle,anchor=west,draw=ublue,minimum width=2.5em,fill=blue!20,inner sep=2pt] (part2-3) at ([xshift=2.0em,yshift=-1.7em]part1-3.east) {\large {$b^2 $}};
\node [circle,anchor=west,draw=ublue,minimum width=2.5em,fill=blue!20,inner sep=2pt] (part2-3) at ([xshift=2.0em,yshift=-1.7em]part1-3.east) {\large {$b^{[2]} $}};
\node [anchor=north,minimum width=3.0em] (part2-4) at ([xshift=0.0em,yshift=-1.6em]part2-3.south) {\scriptsize{隐藏层}};
\node [anchor=north] (labelb) at ([xshift=0.0em,yshift=-4em]part2-3.south) {\footnotesize {(b)}};
......
Book/Chapter5/chapter5.tex
This source diff could not be displayed because it is too large. You can view the blob instead.
Book/Chapter6/Chapter6.tex
This source diff could not be displayed because it is too large. You can view the blob instead.
Book/Chapter6/Figures/figure-A-working-example-of-neural-machine-translation.tex → Book/Chapter6/Figures/figure-a-working-example-of-neural-machine-translation.tex
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\begin{tikzpicture}
\setlength{\base}{0.9cm}
\setlength{\base}{1cm}
\tikzstyle{rnnnode} = [rounded corners=1pt,minimum size=0.5\base,draw,inner sep=0pt,outer sep=0pt]
\tikzstyle{wordnode} = [font=\tiny]
\tikzstyle{wordnode} = [font=\footnotesize]
% RNN translation model
\begin{scope}[local bounding box=RNNMT]
......@@ -19,58 +13,62 @@
\node[rnnnode,minimum height=0.5\base,fill=green!30!white,anchor=west] (eemb\x) at ([xshift=0.4\base]eemb\y.east) {};
\foreach \x in {1,2,...,4}
\node[rnnnode,fill=blue!30!white,anchor=south] (enc\x) at ([yshift=0.5\base]eemb\x.north) {};
\node[wordnode,left=0.4\base of enc1,font=\scriptsize] (init) {0};
\node[wordnode,left=0.4\base of enc1,font=\footnotesize] (init) {0};
\node[wordnode,anchor=east] (init2) at ([xshift=-3.0em]init.west){};
{
\node[rnnnode,fill=purple] (repr) at (enc4) {};
\node[wordnode] (label) at ([xshift=3.5em]enc4.east) {源语言句子表示};
\draw[->,dashed,thick] (label.west) -- (enc4.east);
\node[wordnode] (label) at ([yshift=2.5em]enc4.north) {
\begin{tabular}{c}
源语言句\\子表示
\end{tabular}
};
\draw[->,dashed,thick] (label.south) -- (enc4.north);
}
\node[wordnode,below=0pt of eemb1,font=\scriptsize] (encwordin1) {我};
\node[wordnode,below=0pt of eemb2,font=\scriptsize] (encwordin2) {很};
\node[wordnode,below=0pt of eemb3,font=\scriptsize] (encwordin3) {好};
\node[wordnode,below=0pt of eemb4,font=\scriptsize] (encwordin4) {$\langle$eos$\rangle$};
\node[wordnode,below=0pt of eemb1,font=\footnotesize] (encwordin1) {我};
\node[wordnode,below=0pt of eemb2,font=\footnotesize] (encwordin2) {很};
\node[wordnode,below=0pt of eemb3,font=\footnotesize] (encwordin3) {好};
\node[wordnode,below=0pt of eemb4,font=\footnotesize] (encwordin4) {$\langle$eos$\rangle$};
%大括号
\draw[decorate,thick,decoration={mirror,brace}]([xshift=0.0em,yshift=-1.5em]eemb1.south west) --([xshift=0.0em,yshift=-1.5em]eemb4.south east) node [font=\scriptsize,xshift=-3.8em,yshift=-1.0em,align=center](label2) {编码器};
\draw[decorate,thick,decoration={mirror,brace}]([xshift=0.0em,yshift=-1.5em]eemb1.south west) --([xshift=0.0em,yshift=-1.5em]eemb4.south east) node [font=\footnotesize,xshift=-3.8em,yshift=-1.0em,align=center](label2) {编码器};
% RNN Decoder
\foreach \x in {1,2,...,4}
\node[rnnnode,minimum height=0.5\base,fill=green!30!white,anchor=south] (demb\x) at ([xshift=0.0em,yshift=3.0em]enc\x.north) {};
\node[rnnnode,minimum height=0.5\base,fill=green!30!white,anchor=south] (demb\x) at ([xshift=9.5em,yshift=-3.9em]enc\x.north) {};
\foreach \x in {1,2,...,4}
\node[rnnnode,fill=blue!30!white,anchor=south] (dec\x) at ([yshift=0.5\base]demb\x.north) {};
\foreach \x in {1,2,...,4}
\node[rnnnode,minimum height=0.5\base,fill=red!30!white,anchor=south] (softmax\x) at ([yshift=0.5\base]dec\x.north) {};
% Decoder input words
\node[wordnode,below=0pt of demb1,font=\scriptsize] (decwordin) {$\langle$sos$\rangle$};
\node[wordnode,below=0pt of demb1,font=\footnotesize] (decwordin) {$\langle$sos$\rangle$};
\ExtractX{$(demb2.south)$}
\ExtractY{$(decwordin.base)$}
\node[wordnode,anchor=base,font=\scriptsize] () at (\XCoord,\YCoord) {I};
\node[wordnode,anchor=base,font=\footnotesize] () at (\XCoord,\YCoord) {I};
\ExtractX{$(demb3.south)$}
\ExtractY{$(decwordin.base)$}
\node[wordnode,anchor=base,font=\scriptsize] () at (\XCoord,\YCoord) {am};
\node[wordnode,anchor=base,font=\footnotesize] () at (\XCoord,\YCoord) {am};
\ExtractX{$(demb4.south)$}
\ExtractY{$(decwordin.base)$}
\node[wordnode,anchor=base,font=\scriptsize] () at (\XCoord,\YCoord) {fine};
\node[wordnode,anchor=base,font=\footnotesize] () at (\XCoord,\YCoord) {fine};
% Decoder output words
\node[wordnode,above=0pt of softmax1,font=\scriptsize] (decwordout1) {I};
\node[wordnode,above=0pt of softmax1,font=\footnotesize] (decwordout1) {I};
\ExtractX{$(softmax2.north)$}
\ExtractY{$(decwordout1.base)$}
\node[wordnode,anchor=base,font=\scriptsize] (decwordout2) at (\XCoord,\YCoord) {am};
\node[wordnode,anchor=base,font=\footnotesize] (decwordout2) at (\XCoord,\YCoord) {am};
\ExtractX{$(softmax3.north)$}
\ExtractY{$(decwordout1.base)$}
\node[wordnode,anchor=base,font=\scriptsize] (decwordout3) at (\XCoord,\YCoord) {fine};
\node[wordnode,anchor=base,font=\footnotesize] (decwordout3) at (\XCoord,\YCoord) {fine};
\ExtractX{$(softmax4.north)$}
\ExtractY{$(decwordout1.base)$}
\node[wordnode,anchor=base,font=\scriptsize] (decwordout4) at (\XCoord,\YCoord) {$\langle$eos$\rangle$};
\node[wordnode,anchor=base,font=\footnotesize] (decwordout4) at (\XCoord,\YCoord) {$\langle$eos$\rangle$};
%大括号
\draw[decorate,thick,decoration={brace}]([xshift=0.0em,yshift=1.3em]softmax1.north west) --([xshift=0.0em,yshift=1.3em]softmax4.north east) node [font=\scriptsize,xshift=-3.8em,yshift=1.0em,align=center](label1) {解码器};
\draw[decorate,thick,decoration={brace}]([xshift=0.0em,yshift=1.3em]softmax1.north west) --([xshift=0.0em,yshift=1.3em]softmax4.north east) node [font=\footnotesize,xshift=-3.8em,yshift=1.0em,align=center](label1) {解码器};
% Connections
\draw[-latex'] (init.east) to (enc1.west);
......@@ -86,41 +84,9 @@
\draw[-latex'] (dec\x.east) to (dec\y.west);
}
\coordinate (bridge) at ([yshift=-1.15\base]demb2);
\draw[-latex'] (enc4.north) .. controls +(north:0.4\base) and +(east:0.5\base) .. (bridge) .. controls +(west:2.4\base) and +(west:0.5\base) .. (dec1.west);
\draw[-latex'] (enc4.east) -- (dec1.west);
\node[] at ([xshift=1.5cm]dec4.east){};
\end{scope}
\end{tikzpicture}
\ No newline at end of file
Book/Chapter6/Figures/figure-Automatically-generate-instances-of-couplets.tex → Book/Chapter6/Figures/figure-automatically-generate-instances-of-couplets.tex
\begin{tikzpicture}
\begin{scope}
\tikzstyle{lnode} = [minimum height=2.5em,minimum width=12em,inner sep=3pt,rounded corners=2pt,draw=red!75!black,fill=red!5];
......
Book/Chapter6/Figures/figure-Calculation-of-context-vector-C.tex → Book/Chapter6/Figures/figure-calculation-of-context-vector-C.tex
\begin{tikzpicture}
\begin{scope}
......@@ -50,7 +49,7 @@
\vspace{-1.0em}
\footnotesize{
\begin{eqnarray}
\textbf{C}(\textrm{''机票''}) & = & 0.2 \times \textbf{h}(\textrm{``沈阳''}) + 0.3 \times \textbf{h}(\textrm{``到''}) + \nonumber \\
\tilde{\mathbf{h}} (\textrm{''机票''}) & = & 0.2 \times \textbf{h}(\textrm{``沈阳''}) + 0.3 \times \textbf{h}(\textrm{``到''}) + \nonumber \\
& & 0.1 \times \textbf{h}(\textrm{``广州''}) + ... + 0.3 \times \textbf{h}(\textrm{``机票''}) \nonumber
\end{eqnarray}
}
\ No newline at end of file
Book/Chapter6/Figures/figure-convergence&lr.tex
......@@ -2,14 +2,14 @@
\begin{tikzpicture}
\begin{axis}[
name=s1,
width=7cm, height=4cm,
width=7cm, height=4.5cm,
xtick={-4,-3,-2,-1,0,1,2,3,4},
ytick={0,1,...,4},
xticklabel style={opacity=0},
yticklabel style={opacity=0},
xlabel={$w$},
ylabel={$L(w)$},
axis line style={->},
axis line style={->,very thick},
xlabel style={xshift=2.2cm,yshift=1.2cm},
ylabel style={rotate=-90,xshift=1.5cm,yshift=1.6cm},
tick align=inside,
......@@ -19,7 +19,7 @@
xmin=-4,
xmax=4,
ymin=0,
ymax=4]
ymax=4.5]
\addplot [dashed,ublue,thick] {x^2/4};
\addplot [quiver={u=1,v=x/2,scale arrows = 0.25},domain=-4:-0.3,->,samples=10,red!60,ultra thick] {x^2/4};
\addplot [draw=ublue,fill=red,mark=*] coordinates{(0,0)};
......@@ -29,14 +29,14 @@
anchor=south,
xshift=6cm,
yshift=0cm,
width=7cm, height=4cm,
width=7cm, height=4.5cm,
xtick={-4,-3,-2,-1,0,1,2,3,4},
ytick={0,1,...,4},
xticklabel style={opacity=0},
yticklabel style={opacity=0},
xlabel={$w$},
ylabel={$L(w)$},
axis line style={->},
axis line style={->,very thick},
xlabel style={xshift=2.2cm,yshift=1.2cm},
ylabel style={rotate=-90,xshift=1.5cm,yshift=1.6cm},
tick align=inside,
......@@ -46,7 +46,7 @@
xmin=-4,
xmax=4,
ymin=0,
ymax=4]
ymax=4.5]
\addplot [dashed,ublue,thick] {x^2/4};
\addplot [quiver={u=-x-(x/abs(x))*(1+x^2-4)^(1/2),v=-0.7},domain=-4:3.6,->,samples=2,red!60,ultra thick] {x^2/4};
\addplot [quiver={u=-x-(x/abs(x))*(1+x^2-4)^(1/2),v=-0.7},domain=-3.13:2.6,->,samples=2,red!60,ultra thick] {x^2/4};
......
Book/Chapter6/Figures/figure-Decode-the-word-probability-distribution-at-the-first-position.tex → Book/Chapter6/Figures/figure-decode-the-word-probability-distribution-at-the-first-position.tex
......@@ -3,7 +3,6 @@
\begin{tikzpicture}
\begin{scope}
\tikzstyle{rnnnode} = [minimum height=1.1em,minimum width=3.5em,inner sep=2pt,rounded corners=1pt,draw,fill=red!20];
......
Book/Chapter6/Figures/figure-Decoding-process-based-on-greedy-method.tex → Book/Chapter6/Figures/figure-decoding-process-based-on-greedy-method.tex
\begin{tikzpicture}
\begin{scope}
\tikzstyle{rnnnode} = [minimum height=1.1em,minimum width=2.1em,inner sep=2pt,rounded corners=1pt,draw,fill=red!20];
......
Book/Chapter6/Figures/figure-Different-regularization-methods.tex → Book/Chapter6/Figures/figure-different-regularization-methods.tex
\begin{tikzpicture}
\begin{scope}
\tikzstyle{lnode} = [minimum height=1.5em,minimum width=3em,inner sep=3pt,rounded corners=1.5pt,draw,fill=orange!20];
......
Book/Chapter6/Figures/figure-Double-layer-RNN.tex → Book/Chapter6/Figures/figure-double-layer-RNN.tex
%--------------------------------------------------------------------------------
\begin{tikzpicture}
\setlength{\base}{0.9cm}
......
Book/Chapter6/Figures/figure-Generate-summary.tex → Book/Chapter6/Figures/figure-generate-summary.tex
%%% outline
%-------------------------------------------------------------------------
\begin{tikzpicture}
......@@ -16,7 +15,7 @@ Jenson Button was denied his 100th race for McLaren after an ERS prevented him f
};
%译文1--------------mt1
\node[font=\small] (mt1) at ([xshift=0em,yshift=-16.8em]original0.south) {系统生成\quad};
\node[font=\small] (mt-2) at ([xshift=0em,yshift=-0.5em]mt1.south) {\quad 的摘要：};
\node[font=\small] (mt-2) at ([xshift=0em,yshift=-0.5em]mt1.south) {的摘要：\quad };
\node[font=\small] (ts1) at ([xshift=0em,yshift=-3em]original1.south) {
\begin{tabular}[t]{l}
\parbox{32em}{
......
Book/Chapter6/Figures/figure-Multi-Head-Attention-Model.tex → Book/Chapter6/Figures/figure-multi-head-attention-model.tex
\begin{tikzpicture}
\begin{scope}
\node [anchor=west,draw=black!30,inner sep=4pt,fill=ugreen!20!white] (Linear0) at (0,0) {\tiny{Linear}};
\node [anchor=south west,draw=black!50,fill=ugreen!20!white,draw,inner sep=4pt] (Linear01) at ([shift={(-0.2em,-0.2em)}]Linear0.south west) {\tiny{Linear}};
\node [anchor=south west,fill=ugreen!20!white,draw,inner sep=4pt] (Linear02) at ([shift={(-0.2em,-0.2em)}]Linear01.south west) {\tiny{Linear}};
\node [anchor=west,draw=black!30,inner sep=4pt,fill=ugreen!20!white] (Linear0) at (0,0) {\footnotesize{Linear}};
\node [anchor=south west,draw=black!50,fill=ugreen!20!white,draw,inner sep=4pt] (Linear01) at ([shift={(-0.2em,-0.2em)}]Linear0.south west) {\footnotesize{Linear}};
\node [anchor=south west,fill=ugreen!20!white,draw,inner sep=4pt] (Linear02) at ([shift={(-0.2em,-0.2em)}]Linear01.south west) {\footnotesize{Linear}};
\node [anchor=north] (Q) at ([xshift=0em,yshift=-1em]Linear02.south) {\footnotesize{$\mathbf{Q}$}};
\node [anchor=west,draw=black!30,inner sep=4pt,fill=ugreen!20!white] (Linear1) at ([xshift=1.5em]Linear0.east) {\tiny{Linear}};
\node [anchor=south west,draw=black!50,fill=ugreen!20!white,draw,inner sep=4pt] (Linear11) at ([shift={(-0.2em,-0.2em)}]Linear1.south west) {\tiny{Linear}};
\node [anchor=south west,fill=ugreen!20!white,draw,inner sep=4pt] (Linear12) at ([shift={(-0.2em,-0.2em)}]Linear11.south west) {\tiny{Linear}};
\node [anchor=west,draw=black!30,inner sep=4pt,fill=ugreen!20!white] (Linear1) at ([xshift=1.5em]Linear0.east) {\footnotesize{Linear}};
\node [anchor=south west,draw=black!50,fill=ugreen!20!white,draw,inner sep=4pt] (Linear11) at ([shift={(-0.2em,-0.2em)}]Linear1.south west) {\footnotesize{Linear}};
\node [anchor=south west,fill=ugreen!20!white,draw,inner sep=4pt] (Linear12) at ([shift={(-0.2em,-0.2em)}]Linear11.south west) {\footnotesize{Linear}};
\node [anchor=north] (K) at ([xshift=0em,yshift=-1em]Linear12.south) {\footnotesize{$\mathbf{K}$}};
\node [anchor=west,draw=black!30,inner sep=4pt,fill=ugreen!20!white] (Linear2) at ([xshift=1.5em]Linear1.east) {\tiny{Linear}};
\node [anchor=south west,draw=black!50,fill=ugreen!20!white,draw,inner sep=4pt] (Linear21) at ([shift={(-0.2em,-0.2em)}]Linear2.south west) {\tiny{Linear}};
\node [anchor=south west,fill=ugreen!20!white,draw,inner sep=4pt] (Linear22) at ([shift={(-0.2em,-0.2em)}]Linear21.south west) {\tiny{Linear}};
\node [anchor=west,draw=black!30,inner sep=4pt,fill=ugreen!20!white] (Linear2) at ([xshift=1.5em]Linear1.east) {\footnotesize{Linear}};
\node [anchor=south west,draw=black!50,fill=ugreen!20!white,draw,inner sep=4pt] (Linear21) at ([shift={(-0.2em,-0.2em)}]Linear2.south west) {\footnotesize{Linear}};
\node [anchor=south west,fill=ugreen!20!white,draw,inner sep=4pt] (Linear22) at ([shift={(-0.2em,-0.2em)}]Linear21.south west) {\footnotesize{Linear}};
\node [anchor=north] (V) at ([xshift=0em,yshift=-1em]Linear22.south) {\footnotesize{$\mathbf{V}$}};
\node [anchor=south,draw=black!30,minimum width=9em,inner sep=4pt,fill=blue!20!white] (Scale) at ([yshift=1em]Linear1.north) {\tiny{Scaled Dot-Product Attention}};
\node [anchor=south west,draw=black!50,minimum width=9em,fill=blue!20!white,draw,inner sep=4pt] (Scale1) at ([shift={(-0.2em,-0.2em)}]Scale.south west) {\tiny{Scaled Dot-Product Attention}};
\node [anchor=south west,fill=blue!20!white,draw,minimum width=9em,inner sep=4pt] (Scale2) at ([shift={(-0.2em,-0.2em)}]Scale1.south west) {\tiny{Scaled Dot-Product Attention}};
\node [anchor=south,draw=black!30,minimum width=12em,minimum height=2em,inner sep=4pt,fill=blue!20!white] (Scale) at ([yshift=1em]Linear1.north) {\footnotesize{Scaled Dot-Product Attention}};
\node [anchor=south west,draw=black!50,minimum width=12em,minimum height=2em,fill=blue!20!white,draw,inner sep=4pt] (Scale1) at ([shift={(-0.2em,-0.2em)}]Scale.south west) {\footnotesize{Scaled Dot-Product Attention}};
\node [anchor=south west,fill=blue!20!white,draw,minimum width=12em,minimum height=2em,inner sep=4pt] (Scale2) at ([shift={(-0.2em,-0.2em)}]Scale1.south west) {\footnotesize{Scaled Dot-Product Attention}};
\node [anchor=south,draw,minimum width=4em,inner sep=4pt,fill=yellow!30] (Concat) at ([yshift=1em]Scale2.north) {\tiny{Concat}};
\node [anchor=south,draw,minimum width=4em,inner sep=4pt,fill=yellow!30] (Concat) at ([yshift=1em]Scale2.north) {\footnotesize{Concat}};
\node [anchor=south,draw,minimum width=4em,inner sep=4pt,fill=ugreen!20!white] (Linear) at ([yshift=1em]Concat.north) {\tiny{Linear}};
\node [anchor=south,draw,minimum width=4em,inner sep=4pt,fill=ugreen!20!white] (Linear) at ([yshift=1em]Concat.north) {\footnotesize{Linear}};
\draw [->] ([yshift=0.1em]Q.north) -- ([yshift=-0.1em]Linear02.south);
......
Book/Chapter6/Figures/figure-Point-product-attention-model.tex → Book/Chapter6/Figures/figure-point-product-attention-model.tex
......@@ -3,7 +3,6 @@
\begin{tikzpicture}
\begin{scope}
......@@ -23,11 +22,11 @@
\draw [->] ([yshift=0.1em]Scale3.north) -- ([yshift=-0.1em]Mask.south);
\draw [->] ([yshift=0.1em]Mask.north) -- ([yshift=-0.1em]SoftMax.south);
\draw [->] ([yshift=0.1em]SoftMax.north) -- ([yshift=0.9em]SoftMax.north);
\draw [->] ([yshift=0.1em]V1.north) -- ([yshift=9.1em]V1.north);
\draw [->] ([yshift=0.1em]V1.north) -- ([yshift=9.3em]V1.north);
\draw [->] ([yshift=0.1em]MatMul1.north) -- ([yshift=0.8em]MatMul1.north);
{
\node [anchor=east] (line1) at ([xshift=-3em,yshift=1em]MatMul.west) {\scriptsize{自注意力机制的Query}};
\node [anchor=east] (line1) at ([xshift=-4em,yshift=1em]MatMul.west) {\scriptsize{自注意力机制的Query}};
\node [anchor=north west] (line2) at ([yshift=0.3em]line1.south west) {\scriptsize{Key和Value均来自同一句子}};
\node [anchor=north west] (line3) at ([yshift=0.3em]line2.south west) {\scriptsize{编码-解码注意力机制}};
\node [anchor=north west] (line4) at ([yshift=0.3em]line3.south west) {\scriptsize{与前面讲的一样}};
......@@ -60,7 +59,7 @@
{
\node [rectangle,inner sep=0.2em,rounded corners=1pt,fill=green!10,drop shadow,draw=ugreen] [fit = (line1) (line2) (line3) (line4)] (box1) {};
\node [rectangle,inner sep=0.1em,rounded corners=1pt,very thick,dotted,draw=ugreen] [fit = (Q1) (K1) (V1)] (box0) {};
\draw [->,dotted,very thick,ugreen] ([yshift=-1.5em,xshift=0.8em]box1.east) -- ([yshift=-1.5em,xshift=0.1em]box1.east);
\draw [->,dotted,very thick,ugreen] ([yshift=-1.5em,xshift=1.2em]box1.east) -- ([yshift=-1.5em,xshift=0.1em]box1.east);
}
{
\node [rectangle,inner sep=0.2em,rounded corners=1pt,fill=blue!20!white,drop shadow,draw=blue] [fit = (line11) (line12) (line13)] (box2) {};
......@@ -74,7 +73,7 @@
{
\node [rectangle,inner sep=0.2em,rounded corners=1pt,fill=red!10,drop shadow,draw=red] [fit = (line31) (line32) (line33) (line34)] (box4) {};
\draw [->,dotted,very thick,red] ([yshift=-1.5em,xshift=1.5em]box4.east) -- ([yshift=-1.5em,xshift=0.1em]box4.east);
\draw [->,dotted,very thick,red] ([yshift=-1.2em,xshift=2.2em]box4.east) -- ([yshift=-1.2em,xshift=0.1em]box4.east);
}
{
......
Book/Chapter6/Figures/figure-Position-of-difference-and-layer-regularization-in-the-model.tex → Book/Chapter6/Figures/figure-position-of-difference-and-layer-regularization-in-the-model.tex
......@@ -2,7 +2,6 @@
\begin{tikzpicture}
\begin{scope}
\tikzstyle{Sanode} = [minimum height=1.4em,minimum width=7em,inner sep=3pt,rounded corners=1.5pt,draw,fill=orange!20];
......
Book/Chapter6/Figures/figure-Position-of-feedforward-neural-network-in-the-model.tex → Book/Chapter6/Figures/figure-position-of-feedforward-neural-network-in-the-model.tex
\begin{tikzpicture}
\begin{scope}
\tikzstyle{Sanode} = [minimum height=1.4em,minimum width=7em,inner sep=3pt,rounded corners=1.5pt,draw,fill=orange!20];
......
Book/Chapter6/Figures/figure-Position-of-self-attention-mechanism-in-the-model.tex → Book/Chapter6/Figures/figure-position-of-self-attention-mechanism-in-the-model.tex
\begin{tikzpicture}
\begin{scope}
\tikzstyle{Sanode} = [minimum height=1.4em,minimum width=7em,inner sep=3pt,rounded corners=1.5pt,draw,fill=orange!20];
......
Book/Chapter6/Figures/figure-Query-model-corresponding-to-traditional-query-model-vs-attention-mechanism.tex → Book/Chapter6/Figures/figure-query-model-corresponding-to-traditional-query-model-vs-attention-mechanism.tex
%-----------------------------------------------------
\begin{tikzpicture}
......@@ -28,46 +27,7 @@
\draw [->] ([yshift=1pt]query.north) .. controls +(90:2em) and +(90:2em) .. ([yshift=1pt]key3.north) node [pos=0.5,below,yshift=0.2em] {\scriptsize{匹配}};
\node [anchor=north] (result) at (value3.south) {\scriptsize{ {\red 返回结果} }};
\node [anchor=north] (result2) at ([xshift=-2em,yshift=-2em]value2.south) {\footnotesize{ { (a)索引的查询过程} }};
\end{scope}
\end{tikzpicture}
\begin{tikzpicture}
\begin{scope}
\tikzstyle{rnode} = [draw,minimum width=3em,minimum height=1.2em]
\node [rnode,anchor=south west,fill=red!20!white] (value1) at (0,0) {\scriptsize{value$_1$}};
\node [rnode,anchor=south west,fill=red!20!white] (value2) at ([xshift=1em]value1.south east) {\scriptsize{value$_2$}};
\node [rnode,anchor=south west,fill=red!20!white] (value3) at ([xshift=1em]value2.south east) {\scriptsize{value$_3$}};
\node [rnode,anchor=south west,fill=red!20!white] (value4) at ([xshift=1em]value3.south east) {\scriptsize{value$_4$}};
\node [rnode,anchor=south west,pattern=north east lines] (key1) at ([yshift=0.2em]value1.north west) {};
\node [rnode,anchor=south west,pattern=dots] (key2) at ([yshift=0.2em]value2.north west) {};
\node [rnode,anchor=south west,pattern=horizontal lines] (key3) at ([yshift=0.2em]value3.north west) {};
\node [rnode,anchor=south west,pattern=crosshatch dots] (key4) at ([yshift=0.2em]value4.north west) {};
\node [fill=white,inner sep=1pt] (key1label) at (key1) {\scriptsize{key$_1$}};
\node [fill=white,inner sep=1pt] (key1label) at (key2) {\scriptsize{key$_2$}};
\node [fill=white,inner sep=1pt] (key1label) at (key3) {\scriptsize{key$_3$}};
\node [fill=white,inner sep=1pt] (key1label) at (key4) {\scriptsize{key$_4$}};
\node [rnode,anchor=east,pattern=vertical lines] (query) at ([xshift=-3em]key1.west) {};
\node [anchor=east] (querylabel) at ([xshift=-0.2em]query.west) {\scriptsize{query}};
\draw [->] ([yshift=1pt,xshift=6pt]query.north) .. controls +(90:1em) and +(90:1em) .. ([yshift=1pt]key1.north);
\draw [->] ([yshift=1pt,xshift=3pt]query.north) .. controls +(90:1.5em) and +(90:1.5em) .. ([yshift=1pt]key2.north);
\draw [->] ([yshift=1pt]query.north) .. controls +(90:2em) and +(90:2em) .. ([yshift=1pt]key3.north);
\draw [->] ([yshift=1pt,xshift=-3pt]query.north) .. controls +(90:2.5em) and +(90:2.5em) .. ([yshift=1pt]key4.north);
\node [anchor=south east] (alpha1) at (key1.north east) {\scriptsize{$\alpha_1$}};
\node [anchor=south east] (alpha2) at (key2.north east) {\scriptsize{$\alpha_2$}};
\node [anchor=south east] (alpha3) at (key3.north east) {\scriptsize{$\alpha_3$}};
\node [anchor=south east] (alpha4) at (key4.north east) {\scriptsize{$\alpha_4$}};
\node [anchor=north] (result) at ([xshift=-1.5em]value2.south east) {\scriptsize{{\red 返回结果}=$\alpha_1 \cdot \textrm{value}_1 + \alpha_2 \cdot \textrm{value}_2 + \alpha_3 \cdot \textrm{value}_3 + \alpha_4 \cdot \textrm{value}_4$}};
\node [anchor=north] (result2) at ([xshift=-1em,yshift=-2.5em]value2.south) {\footnotesize{ { (b)注意力机制查询过程} }};
\end{scope}
\end{tikzpicture}
\ No newline at end of file
Book/Chapter6/Figures/figure-Residual-network-structure.tex → Book/Chapter6/Figures/figure-residual-network-structure.tex
\begin{tikzpicture}
\begin{scope}
\tikzstyle{lnode} = [minimum height=1.5em,minimum width=3em,inner sep=3pt,rounded corners=1.5pt,draw,fill=orange!20];
......
Book/Chapter6/Figures/figure-Structure-of-the-network-during-Transformer-training.tex → Book/Chapter6/Figures/figure-structure-of-the-network-during-transformer-training.tex
\begin{tikzpicture}
\begin{scope}
\tikzstyle{rnnnode} = [minimum height=1.1em,minimum width=2.1em,inner sep=2pt,rounded corners=1pt,draw,fill=red!20];
......
Book/Chapter6/Figures/figure-Transformer-input-and-position-encoding.tex → Book/Chapter6/Figures/figure-transformer-input-and-position-encoding.tex
\begin{tikzpicture}
\begin{scope}
\tikzstyle{Sanode} = [minimum height=1.4em,minimum width=7em,inner sep=3pt,rounded corners=1.5pt,draw];
......
Book/Chapter6/Figures/figure-transformer.tex
......@@ -62,5 +62,8 @@
\node [rectangle,inner sep=0.7em,rounded corners=1pt,very thick,dotted,draw=ugreen!70] [fit = (sa1) (res1) (ffn1) (res2)] (box0) {};
\node [rectangle,inner sep=0.7em,rounded corners=1pt,very thick,dotted,draw=red!60] [fit = (sa2) (res3) (res5)] (box1) {};
\node [ugreen,font=\scriptsize] (count) at ([xshift=-1.5em,yshift=-1em]encoder.south) {$6\times$};
\node [red,font=\scriptsize] (count) at ([xshift=10.8em,yshift=0em]decoder.south) {$\times 6$};
\end{scope}
\end{tikzpicture}
\ No newline at end of file
Book/Chapter6/Figures/figure-Word-embedding-structure.tex → Book/Chapter6/Figures/figure-word-embedding-structure.tex
%------------------------------------------------------
\begin{tikzpicture}
\setlength{\base}{0.9cm}
......
Book/Chapter7/Chapter7.tex
This source diff could not be displayed because it is too large. You can view the blob instead.
Book/Chapter7/Figures/figure-MASS.tex
......@@ -3,48 +3,47 @@
\tikzstyle{word} = [font=\scriptsize]
\tikzstyle{model} = [rectangle,draw,minimum height=3em,minimum width=6em,rounded corners=4pt,fill=red!15!white]
\node [model,fill=blue!15!white] (ate) at (0,0) {Attention};
\node [anchor=center] (ate) at (0,0) {};
\node [model,minimum width=10.5em] (decoder) at ([xshift=8em]ate.east) {Decoder};
\node [word] (w1) at ([yshift=-2em,xshift=1em]decoder.south) {$x_3$};
\node [model,minimum width=10.5em,line width=0.7pt] (decoder) at ([xshift=6em]ate.east) {Decoder};
\node [word] (w1) at ([yshift=-2em,xshift=1em]decoder.south) {\small{$x_3$}};
\node [word] (w2) at ([xshift=-1em]w1.west) {\#};
\node [word] (w3) at ([xshift=-1em]w2.west) {\#};
\node [word] (w4) at ([xshift=-1em]w3.west) {\#};
\node [word] (w5) at ([xshift=1em]w1.east) {$x_4$};
\node [word] (w5) at ([xshift=1em]w1.east) {\small{$x_4$}};
\node [word] (w6) at ([xshift=1em]w5.east) {\#};
\node [word] (w7) at ([yshift=2em,xshift=1em]decoder.north) {$x_4$};
\node [word] (w8) at ([yshift=0em,xshift=-1em]w7.west) {$x_3$};
\node [word] (w9) at ([yshift=0em,xshift=1em]w7.east) {$x_5$};
\node [word] (w7) at ([yshift=2em,xshift=1em]decoder.north) {\small{$x_4$}};
\node [word] (w8) at ([yshift=0em,xshift=-1em]w7.west) {\small{$x_3$}};
\node [word] (w9) at ([yshift=0em,xshift=1em]w7.east) {\small{$x_5$}};
\draw [->] (w1.north) -- ([yshift=1.4em]w1.north);
\draw [->] (w2.north) -- ([yshift=1.3em]w2.north);
\draw [->] (w3.north) -- ([yshift=1.3em]w3.north);
\draw [->] (w4.north) -- ([yshift=1.3em]w4.north);
\draw [->] (w5.north) -- ([yshift=1.4em]w5.north);
\draw [->] (w6.north) -- ([yshift=1.4em]w6.north);
\draw [->,thick] (w1.north) -- ([yshift=1.35em]w1.north);
\draw [->,thick] (w2.north) -- ([yshift=1.35em]w2.north);
\draw [->,thick] (w3.north) -- ([yshift=1.35em]w3.north);
\draw [->,thick] (w4.north) -- ([yshift=1.35em]w4.north);
\draw [->,thick] (w5.north) -- ([yshift=1.35em]w5.north);
\draw [->,thick] (w6.north) -- ([yshift=1.35em]w6.north);
\draw [->] ([yshift=-1.4em]w7.south) -- (w7.south);
\draw [->] ([yshift=-1.4em]w8.south) -- (w8.south);
\draw [->] ([yshift=-1.4em]w9.south) -- (w9.south);
\draw [->,thick] ([yshift=-1.4em]w7.south) -- (w7.south);
\draw [->,thick] ([yshift=-1.4em]w8.south) -- (w8.south);
\draw [->,thick] ([yshift=-1.4em]w9.south) -- (w9.south);
%encoder
\node [model,minimum width=10.5em] (encoder) at ([xshift=-8em]ate.west) {Encoder};
\node [model,minimum width=10.5em,line width=0.7pt] (encoder) at ([xshift=-6em]ate.west) {Encoder};
\node [word] (we1) at ([yshift=-2em,xshift=1em]encoder.south) {\#};
\node [word] (we2) at ([xshift=-1em]we1.west) {\#};
\node [word] (we3) at ([xshift=-1em]we2.west) {$x_2$};
\node [word] (we4) at ([xshift=-1em]we3.west) {$x_3$};
\node [word] (we3) at ([xshift=-1em]we2.west) {\small{$x_2$}};
\node [word] (we4) at ([xshift=-1em]we3.west) {\small{$x_3$}};
\node [word] (we5) at ([xshift=1em]we1.east) {\#};
\node [word] (we6) at ([xshift=1em]we5.east) {$x_6$};
\node [word] (we6) at ([xshift=1em]we5.east) {\small{$x_6$}};
\draw [->] (we1.north) -- ([yshift=1.3em]we1.north);
\draw [->] (we2.north) -- ([yshift=1.3em]we2.north);
\draw [->] (we3.north) -- ([yshift=1.4em]we3.north);
\draw [->] (we4.north) -- ([yshift=1.4em]we4.north);
\draw [->] (we5.north) -- ([yshift=1.3em]we5.north);
\draw [->] (we6.north) -- ([yshift=1.4em]we6.north);
\draw [->,thick] (we1.north) -- ([yshift=1.35em]we1.north);
\draw [->,thick] (we2.north) -- ([yshift=1.35em]we2.north);
\draw [->,thick] (we3.north) -- ([yshift=1.35em]we3.north);
\draw [->,thick] (we4.north) -- ([yshift=1.35em]we4.north);
\draw [->,thick] (we5.north) -- ([yshift=1.35em]we5.north);
\draw [->,thick] (we6.north) -- ([yshift=1.35em]we6.north);
\draw [->,very thick] ([xshift=0.5em]encoder) -- ([xshift=-0.5em]ate);
\draw [->,very thick] ([xshift=0.5em]ate) -- ([xshift=-0.5em]decoder);
\draw [->,line width=2pt] ([xshift=0.5em]encoder)--([xshift=-0.5em]decoder);
\end{scope}
\end{tikzpicture}
\ No newline at end of file
Book/Chapter7/Figures/figure-application-process-of-back-translation.tex
\begin{tikzpicture}
\begin{scope}
\tikzstyle{node1} = [rectangle,draw,minimum height=2em,minimum width=8em,rounded corners=2pt,fill=orange!10]
\tikzstyle{node2} = [rectangle,draw,minimum height=1.3em,minimum width=10em,rounded corners=2pt,fill=blue!15!white]
\tikzstyle{node3} = [rectangle,draw,minimum height=2em,minimum width=4em,rounded corners=2pt,fill=orange!10]
\node [anchor=north,inner sep=0mm,node1] (n1) at (0,0) {Parallel\ Data};
\node [anchor=north,node2] (n2) at ([xshift=0em,yshift=-2em]n1.south) {Reverse\ NMT\ System};
\node [anchor=north,node3] (n3) at ([xshift=-3em,yshift=-2em]n2.south) {M$_{\textrm{pseudo}}$};
\node [anchor=west,node3] (n31) at ([xshift=2em,yshift=0em]n3.east) {M$_{\textrm{target}}$};
\node [anchor=north west,node1,minimum height=4em,minimum width=8em] (n4) at ([xshift=5em,yshift=0em]n1.north east) {};
\node [anchor=south west,fill=orange!10,minimum height=1.6em,minimum width=3.6em] (n41) at ([xshift=0.2em,yshift=0.2em]n4.south west) {M$_{\textrm{pseudo}}$};
\node [anchor=south east,fill=orange!10,minimum height=1.6em,minimum width=3.6em] (n42) at ([xshift=-0.2em,yshift=0.2em]n4.south east) {M$_{\textrm{target}}$};
\node [anchor=north,fill=orange!10,minimum height=1.6em,minimum width=7.6em] (n43) at ([xshift=0em,yshift=-0.2em]n4.north) {Parallel\ Data};
\node [anchor=north,node2] (n5) at ([xshift=0em,yshift=-3em]n4.south) {Final\ NMT\ System};
\draw [->,thick,black!60,line width=1mm] (n1.east) -- ([xshift=0em,yshift=1em]n4.west);
\draw [->,thick,black!20,line width=1mm] (n1.south) -- (n2.north);
\draw [->,thick,black!20,line width=1mm] (n2.south) -- ([xshift=0em,yshift=-2em]n2.south);
\draw [->,thick,black!40,line width=1mm] (n3.east) -- (n31.west);
\draw [->,thick,black!60,line width=1mm] (n31.north east) -- (n4.south west);
\draw [->,thick,black!20,line width=1mm] (n4.south) -- (n5.north);
\node [anchor=center] (node1) at (4.9,1) {\small{训练：}};
\node [anchor=center] (node11) at (5.5,1) {};
\node [anchor=center] (node12) at (6.7,1) {};
\node [anchor=center] (node2) at (4.9,0.5) {\small{推理：}};
\node [anchor=center] (node21) at (5.5,0.5) {};
\node [anchor=center] (node22) at (6.7,0.5) {};
\node [anchor=west,line width=0.6pt,draw=black,minimum width=5.6em,minimum height=2.2em,fill=blue!20,rounded corners=2pt] (node1-1) at (0,0) {\footnotesize{双语数据}};
\node [anchor=south,line width=0.6pt,draw=black,minimum width=4.5em,minimum height=2.2em,fill=blue!20,rounded corners=2pt] (node1-2) at ([yshift=-5em]node1-1.south) {\footnotesize{目标语伪数据}};
\node [anchor=west,line width=0.6pt,draw=black,minimum width=4.5em,minimum height=2.2em,fill=red!20,rounded corners=2pt] (node2-1) at ([xshift=-8.8em,yshift=-2.5em]node1-1.west) {\footnotesize{反向NMT系统}};
\node [anchor=west,line width=0.6pt,draw=black,minimum width=4.5em,minimum height=2.2em,fill=red!20,rounded corners=2pt] (node3-1) at ([xshift=3em,yshift=-2.5em]node1-1.east) {\footnotesize{前向NMT系统}};
\draw [-,thick] (n4.west) -- (n4.east);
\draw [-,thick] (n4.south) -- ([xshift=0em,yshift=2em]n4.south);
\draw [->,line width=1pt](node1-1.west)--([xshift=3em]node2-1.north);
\draw [->,line width=1pt](node1-1.east)--([xshift=-3em]node3-1.north);
\draw [->,line width=1pt](node1-2.east)--([xshift=-3em]node3-1.south);
\draw [->,line width=1pt](node11.east)--(node12.west);
\draw [->,line width=1pt,dashed](node21.east)--(node22.west);
\draw [->,line width=1pt,dashed]([xshift=3em]node2-1.south)--(node1-2.west);
\end{scope}
\end{tikzpicture}
\ No newline at end of file
Book/Chapter7/Figures/figure-batch-generation-method.tex
\begin{tikzpicture}
\tikzstyle{node} = [minimum height=1.0em,draw=teal,fill=teal!10]
\tikzstyle{legend} = [minimum height=1.0em,minimum width=1.0em,draw]
\tikzstyle{node2} = [minimum width=1.0em,minimum height=4.1em,draw=blue,fill=blue!10]
\node[node,minimum width=2.8em] (node1) at (0,0) {};
\node[node,minimum width=4.0em,anchor=north west] (node2) at (node1.south west) {};
\node[node,minimum width=3.2em,anchor=north west] (node3) at (node2.south west) {};
\node[node,minimum width=3.0em,anchor=north west] (node4) at (node3.south west) {};
\tikzstyle{node} = [minimum height=1.0*1.2em,draw=teal,fill=teal!10]
\tikzstyle{legend} = [minimum height=1.0*1.2em,minimum width=1.0*1.2em,draw]
\tikzstyle{node2} = [minimum width=1.0*1.2em,minimum height=4.1*1.2em,draw=blue,fill=blue!10]
\node[node,minimum width=2.8*1.2em] (node1) at (0,0) {};
\node[node,minimum width=4.0*1.2em,anchor=north west] (node2) at (node1.south west) {};
\node[node,minimum width=3.2*1.2em,anchor=north west] (node3) at (node2.south west) {};
\node[node,minimum width=3.0*1.2em,anchor=north west] (node4) at (node3.south west) {};
\node[node2,anchor = north west] (grad1) at ([xshift=1.2em]node1.north east) {};
\node[node,minimum width=3.7em,anchor=north west] (node5) at (grad1.north east) {};
\node[node,minimum width=2.8em,anchor=north west] (node6) at (node5.south west) {};
\node[node,minimum width=3.2em,anchor=north west] (node7) at (node6.south west) {};
\node[node,minimum width=4.0em,anchor=north west] (node8) at (node7.south west) {};
\node[font=\scriptsize,anchor=east] (line1) at (node1.west) {gpu1};
\node[font=\scriptsize,anchor=east] (line2) at (node2.west) {gpu2};
\node[font=\scriptsize,anchor=east] (line3) at (node3.west) {gpu3};
\node[font=\scriptsize,anchor=east] (line4) at (node4.west) {gpu4};
\node[node,minimum width=3.7*1.2em,anchor=north west] (node5) at (grad1.north east) {};
\node[node,minimum width=2.8*1.2em,anchor=north west] (node6) at (node5.south west) {};
\node[node,minimum width=3.2*1.2em,anchor=north west] (node7) at (node6.south west) {};
\node[node,minimum width=4.0*1.2em,anchor=north west] (node8) at (node7.south west) {};
\node[font=\footnotesize,anchor=east] (line1) at (node1.west) {gpu1};
\node[font=\footnotesize,anchor=east] (line2) at (node2.west) {gpu2};
\node[font=\footnotesize,anchor=east] (line3) at (node3.west) {gpu3};
\node[font=\footnotesize,anchor=east] (line4) at (node4.west) {gpu4};
\node[node2,anchor = north west] (grad2) at ([xshift=0.3em]node5.north east) {};
\draw[->] (-1.4em,-3.62em) -- (9.5em,-3.62em);
\draw[->] (-1.4em*1.2,-3.62*1.2em) -- (9em*1.2,-3.62*1.2em);
\node[node,minimum width=2.8em] (node9) at (15em,0) {};
\node[node,minimum width=4.0em,anchor=north west] (node10) at (node9.south west) {};
\node[node,minimum width=3.2em,anchor=north west] (node11) at (node10.south west) {};
\node[node,minimum width=3.0em,anchor=north west] (node12) at (node11.south west) {};
\node[node,minimum width=2.8*1.2em] (node9) at (16em,0) {};
\node[node,minimum width=4.0*1.2em,anchor=north west] (node10) at (node9.south west) {};
\node[node,minimum width=3.2*1.2em,anchor=north west] (node11) at (node10.south west) {};
\node[node,minimum width=3.0*1.2em,anchor=north west] (node12) at (node11.south west) {};
\node[node,minimum width=3.7em,anchor=north west] (node13) at (node9.north east) {};
\node[node,minimum width=2.8em,anchor=north west] (node14) at (node10.north east) {};
\node[node,minimum width=3.2em,anchor=north west] (node15) at (node11.north east) {};
\node[node,minimum width=4.0em,anchor=north west] (node16) at (node12.north east) {};
\node[node,minimum width=3.7*1.2em,anchor=north west] (node13) at (node9.north east) {};
\node[node,minimum width=2.8*1.2em,anchor=north west] (node14) at (node10.north east) {};
\node[node,minimum width=3.2*1.2em,anchor=north west] (node15) at (node11.north east) {};
\node[node,minimum width=4.0*1.2em,anchor=north west] (node16) at (node12.north east) {};
\node[node2,anchor = north west] (grad3) at ([xshift=0.5em]node13.north east) {};
\node[font=\scriptsize,anchor=east] (line1) at (node9.west) {gpu1};
\node[font=\scriptsize,anchor=east] (line2) at (node10.west) {gpu2};
\node[font=\scriptsize,anchor=east] (line3) at (node11.west) {gpu3};
\node[font=\scriptsize,anchor=east] (line4) at (node12.west) {gpu4};
\draw[->] (13.6em,-3.62em) -- (22.2em,-3.62em);
\node[font=\footnotesize,anchor=east] (line1) at (node9.west) {gpu1};
\node[font=\footnotesize,anchor=east] (line2) at (node10.west) {gpu2};
\node[font=\footnotesize,anchor=east] (line3) at (node11.west) {gpu3};
\node[font=\footnotesize,anchor=east] (line4) at (node12.west) {gpu4};
\draw[->] (13.6*1.2em,-3.62*1.2em) -- (20.5*1.2em,-3.62*1.2em);
\begin{pgfonlayer}{background}
\node [rectangle,inner sep=-0.0em,draw] [fit = (node1) (node2) (node3) (node4)] (box1) {};
\node [rectangle,inner sep=-0.0em,draw] [fit = (node5) (node6) (node7) (node8)] (box2) {};
\node [rectangle,inner sep=-0.0em,draw] [fit = (node9) (node13) (node12) (node16)] (box2) {};
\end{pgfonlayer}
\node[font=\scriptsize,anchor=north] (legend1) at ([xshift=3em]node4.south) {一步一更新};
\node[font=\scriptsize,anchor=north] (legend2) at ([xshift=2.5em]node12.south) {累积两步更新};
\node[font=\scriptsize,anchor=north] (time1) at (grad2.south) {time};
\node[font=\scriptsize,anchor=north] (time1) at (grad3.south) {time};
\node[font=\footnotesize,anchor=north] (legend1) at ([xshift=3em]node4.south) {一步一更新};
\node[font=\footnotesize,anchor=north] (legend2) at ([xshift=2.5em]node12.south) {累积两步更新};
\node[font=\footnotesize,anchor=north] (time1) at (grad2.south) {time};
\node[font=\footnotesize,anchor=north] (time1) at (grad3.south) {time};
\node[legend] (legend3) at (2em,2em) {};
\node[font=\scriptsize,anchor=west] (idle) at (legend3.east) {:空闲};
\node[font=\footnotesize,anchor=west] (idle) at (legend3.east) {:空闲};
\node[legend,anchor=west,draw=teal,fill=teal!10] (legend4) at ([xshift = 2em]idle.east) {};
\node[font=\scriptsize,anchor=west] (FB) at (legend4.east) {:前向/反向};
\node[font=\footnotesize,anchor=west] (FB) at (legend4.east) {:前向/反向};
\node[legend,anchor=west,draw=blue,fill=blue!10] (legend5) at ([xshift = 2em]FB.east) {};
\node[font=\scriptsize,anchor=west] (grad_sync) at (legend5.east) {:梯度更新};
\node[font=\footnotesize,anchor=west] (grad_sync) at (legend5.east) {:梯度更新};
\end{tikzpicture}
\ No newline at end of file
Book/Chapter7/Figures/figure-construction-steps-of-MT-system.tex
......@@ -21,8 +21,8 @@
{\footnotesize
\node [anchor=west] (n11) at ([xshift=-13em,yshift=2em]n1.west) {对训练和测试数据进行};
\node [anchor=west] (n12) at ([xshift=0em,yshift=-1.5em]n11.west) {处理，包括：数据清洗、};
\node [anchor=west] (n13) at ([xshift=0em,yshift=-1.5em]n12.west) {翻译单元切分、译文后};
\node [anchor=west] (n14) at ([xshift=0em,yshift=-1.5em]n13.west) {处理等};
\node [anchor=west] (n13) at ([xshift=0em,yshift=-1.5em]n12.west) {子词切分、译文后处理};
\node [anchor=west] (n14) at ([xshift=0em,yshift=-1.5em]n13.west) {等};
\node [anchor=west] (n31) at ([xshift=2em,yshift=0em]n3.north east) {神经网络模型设计，包括};
......
Book/Chapter7/Figures/figure-different-softmax.tex
\begin{tikzpicture}
\tikzstyle{layer} = [rectangle,draw,rounded corners=3pt,minimum width=1cm,minimum height=0.5cm];
\tikzstyle{layer} = [rectangle,draw,rounded corners=3pt,minimum width=1cm,minimum height=0.5cm,line width=1pt];
\tikzstyle{prob} = [minimum width=0.3cm,rectangle,fill=ugreen!20!white,inner sep=0pt];
\begin{scope}[local bounding box=STANDARD]
......@@ -22,8 +22,8 @@
\path [fill=blue!20!white,draw=white] (out1.north west) -- (prob1.south west) -- (prob9.south east) -- (out1.north east) -- (out1.north west);
\draw [->] (input1) to (net1);
\draw [->] (net1) to (out1);
\draw [->,line width=1pt] (input1) to (net1);
\draw [->,line width=1pt] (net1) to (out1);
\node [font=\small] (label1) at ([yshift=0.6cm]out1.north) {Softmax};
\end{scope}
......@@ -51,8 +51,8 @@
\path [fill=blue!20!white,draw=white] (out2.north west) -- (prob1.south west) -- (prob9.south east) -- (out2.north east) -- (out2.north west);
\draw [->] (input2) to (net2);
\draw [->] (net2) to (out2);
\draw [->,line width=1pt] (input2) to (net2);
\draw [->,line width=1pt] (net2) to (out2);
\node [font=\small] (label2) at ([yshift=0.6cm]out2.north) {Softmax};
......@@ -60,9 +60,9 @@
\node [anchor=north,font=\scriptsize] (input3) at ([yshift=-0.5cm]net3.south) {源语};
\node [anchor=south,layer,align=center,font=\scriptsize,fill=yellow!10!white] (out3) at ([yshift=0.9cm]net3.north) {Candidate\\List};
\draw [->] (input3) to (net3);
\draw [->] (net3) to (out3);
\draw [->] (out3) |- (plabel9.east);
\draw [->,line width=1pt] (input3) to (net3);
\draw [->,line width=1pt] (net3) to (out3);
\draw [->,line width=1pt] (out3) |- (plabel9.east);
\end{scope}
\node [anchor=north,font=\scriptsize] () at ([yshift=-0.2em]STANDARD.south) {(a) 标准方法};
......
Book/Chapter7/Figures/figure-encoder-fin.tex
\begin{tikzpicture}
\begin{scope}
\tikzstyle{word} = [font=\scriptsize]
\tikzstyle{model} = [rectangle,draw,minimum height=3em,minimum width=5em,rounded corners=4pt,fill=blue!15!white]
\tikzstyle{model} = [rectangle,draw,minimum height=2.5em,minimum width=5em,rounded corners=4pt,fill=blue!15!white,line width=0.7pt]
\node [model,minimum width=10.5em] (encoder0) at (0,0) {Encoder};
\node [word] (w1) at ([yshift=-2em,xshift=1em]encoder0.south) {\#};
\node [word] (w2) at ([xshift=-1em]w1.west) {\#};
\node [word] (w3) at ([xshift=-1em]w2.west) {$x_2$};
\node [word] (w4) at ([xshift=-1em]w3.west) {$x_1$};
\node [word] (w3) at ([xshift=-1em]w2.west) {\small{$x_2$}};
\node [word] (w4) at ([xshift=-1em]w3.west) {\small{$x_1$}};
\node [word] (w5) at ([xshift=1em]w1.east) {\#};
\node [word] (w6) at ([xshift=1em]w5.east) {$x_6$};
\node [word] (w6) at ([xshift=1em]w5.east) {\small{$x_6$}};
\node [word] (w7) at ([yshift=2em,xshift=1em]encoder0.north) {$x_4$};
\node [word] (w8) at ([yshift=0em,xshift=-1em]w7.west) {$x_3$};
\node [word] (w9) at ([yshift=0em,xshift=1em]w7.east) {$x_5$};
\node [word] (w7) at ([yshift=2em,xshift=1.0em]encoder0.north) {\small{$x_4$}};
\node [word] (w8) at ([yshift=5.95em]w2.north) {\small{$x_3$}};
\node [word] (w9) at ([yshift=5.95em]w5.north) {\small{$x_5$}};
\draw [->] (w1.north) -- ([yshift=1.3em]w1.north);
\draw [->] (w2.north) -- ([yshift=1.3em]w2.north);
\draw [->] (w3.north) -- ([yshift=1.4em]w3.north);
\draw [->] (w4.north) -- ([yshift=1.4em]w4.north);
\draw [->] (w5.north) -- ([yshift=1.3em]w5.north);
\draw [->] (w6.north) -- ([yshift=1.4em]w6.north);
\draw [->,line width=1pt] (w1.north) -- ([yshift=1.35em]w1.north);
\draw [->,line width=1pt] (w2.north) -- ([yshift=1.35em]w2.north);
\draw [->,line width=1pt] (w3.north) -- ([yshift=1.35em]w3.north);
\draw [->,line width=1pt] (w4.north) -- ([yshift=1.35em]w4.north);
\draw [->,line width=1pt] (w5.north) -- ([yshift=1.35em]w5.north);
\draw [->,line width=1pt] (w6.north) -- ([yshift=1.35em]w6.north);
\draw [<-] (w7.south) -- ([yshift=-1.4em]w7.south);
\draw [<-] (w8.south) -- ([yshift=-1.4em]w8.south);
\draw [<-] (w9.south) -- ([yshift=-1.4em]w9.south);
\draw [->,line width=1pt] ([yshift=-1.4em]w7.south) -- (w7.south);
\draw [->,line width=1pt] ([yshift=-1.4em]w8.south) -- (w8.south);
\draw [->,line width=1pt] ([yshift=-1.4em]w9.south) -- (w9.south);
\node [model] (encoder1) at ([xshift=8em]encoder0.east) {Encoder};
\node [model,fill=red!15!white] (decoder) at ([xshift=6em]encoder1.east) {Decoder};
\node [] (sinput) at ([yshift=-3em]encoder1.south) {source input};
\node [] (tinput) at ([yshift=-3em]decoder.south) {target input};
\node [] (output) at ([yshift=3em]decoder.north) {target output};
\node [model,fill=red!15!white] (decoder) at ([xshift=5em]encoder1.east) {Decoder};
\node [] (sinput) at ([yshift=-3em]encoder1.south) {\footnotesize{源语输入}};
\node [] (tinput) at ([yshift=-3em]decoder.south) {\footnotesize{目标语输入}};
\node [] (output) at ([yshift=3em]decoder.north) {\footnotesize{目标语输出}};
\draw [->] (sinput) -- (encoder1);
\draw [->] (tinput) -- (decoder);
\draw [->] (decoder) -- (output);
\draw [->,line width=1pt] (sinput) -- (encoder1);
\draw [->,line width=1pt] (tinput) -- (decoder);
\draw [->,line width=1pt] (decoder) -- (output);
\coordinate (do0) at ([yshift=1em]encoder1.north);
\coordinate (do1) at ([xshift=4em]do0.east);
\coordinate (do2) at ([yshift=-2.5em]do1.south);
\coordinate (do1) at ([xshift=3.5em]do0.east);
\coordinate (do2) at ([yshift=-2.3em]do1.south);
\draw [-] (encoder1.north) -- (do0);
\draw [-] (do0) -- (do1);
\draw [-] (do1) -- (do2);
\draw [->] (do2) -- (decoder.west);
\draw [-,line width=1pt] (encoder1.north) -- (do0);
\draw [-,line width=1pt] (do0) -- (do1);
\draw [-,line width=1pt] (do1) -- (do2);
\draw [->,line width=1pt] (do2) -- (decoder.west);
\begin{pgfonlayer}{background}
\node [rectangle,inner sep=2em,fill=black!5,rounded corners=4pt] [fit =(w4) (w6) (w9) (encoder0) ] (box) {};
\node [rectangle,inner sep=1em,fill=black!5,rounded corners=4pt] [fit =(w4) (w6) (w9) (encoder0) ] (box) {};
\end{pgfonlayer}
\node [] (left) at ([yshift=-1.5em]box.south) {Pre-training with monolingual data};
\node [] (right) at ([xshift=9.8em]left.east) {Fine-tune on translation task};
\node [font=\footnotesize] (left) at ([yshift=-1.5em]box.south) {编码器使用单语数据预训练};
\node [font=\footnotesize] (right) at ([xshift=11em]left.east) {在翻译任务上进行微调};
\node[anchor=north] (arrow1) at (3.85,0.1){};
\draw[fill=yellow!20,draw=yellow]([yshift=-0.3em]arrow1.north)--([xshift=-1em,yshift=0.5em]arrow1.north west)--([xshift=-1em,yshift=0.1em]arrow1.north west)--([xshift=-2.6em,yshift=0.1em]arrow1.north west)--([xshift=-2.6em,yshift=-0.1em]arrow1.south west)--([xshift=-1em,yshift=-0.1em]arrow1.south west)--([xshift=-1em,yshift=-0.5em]arrow1.south west)--([yshift=-0.3em]arrow1.north);
\draw [->,black!50!white,line width=3pt,draw] ([xshift=1em]encoder0.east) -- ([xshift=-1em]encoder1.west);
\end{scope}
\end{tikzpicture}
\ No newline at end of file
Book/Chapter7/Figures/figure-example-of-iterative-back-translation.tex
\begin{tikzpicture}
\begin{scope}
\tikzstyle{node1} = [rectangle,draw,minimum height=2em,minimum width=8em,rounded corners=2pt,fill=orange!10]
\tikzstyle{node2} = [rectangle,draw,minimum height=1.3em,minimum width=10em,rounded corners=2pt,fill=blue!15!white]
\tikzstyle{node3} = [rectangle,draw,minimum height=2em,minimum width=4em,rounded corners=2pt,fill=orange!10]
\node [anchor=center] (node1) at (9.6,1) {\small{训练：}};
\node [anchor=center] (node11) at (10.2,1) {};
\node [anchor=center] (node12) at (11.4,1) {};
\node [anchor=center] (node2) at (9.6,0.5) {\small{推理：}};
\node [anchor=center] (node21) at (10.2,0.5) {};
\node [anchor=center] (node22) at (11.4,0.5) {};
\node [anchor=west,draw=black,line width=0.6pt,minimum width=5.6em,minimum height=2.2em,fill=blue!20,rounded corners=2pt] (node1-1) at (0,0) {\footnotesize{双语数据}};
\node [anchor=south,draw=black,line width=0.6pt,minimum width=4.5em,minimum height=2.2em,fill=blue!20,rounded corners=2pt] (node1-2) at ([yshift=-5em]node1-1.south) {\footnotesize{目标语伪数据}};
\node [anchor=west,draw=black,line width=0.6pt,minimum width=4.5em,minimum height=2.2em,fill=red!20,rounded corners=2pt] (node2-1) at ([xshift=-7.7em,yshift=-2.5em]node1-1.west) {\footnotesize{前向NMT系统}};
\node [anchor=west,draw=black,line width=0.6pt,minimum width=4.5em,minimum height=2.2em,fill=red!20,rounded corners=2pt] (node3-1) at ([xshift=1.5em,yshift=-2.5em]node1-1.east) {\footnotesize{反向NMT系统}};
\node [anchor=north,inner sep=0mm,node1] (n1) at (0,0) {Parallel\ Data};
\node [anchor=north,node2] (n2) at ([xshift=0em,yshift=-2em]n1.south) {NMT\ System};
\node [anchor=north,node3] (n3) at ([xshift=-3em,yshift=-2em]n2.south) {Mono$_{src}$};
\node [anchor=west,node3] (n31) at ([xshift=2em,yshift=0em]n3.east) {Pseudo$_{tgt}$};
\node [anchor=east,draw=black,line width=0.6pt,minimum width=5.6em,minimum height=2.2em,fill=blue!20,rounded corners=2pt] (node4-1) at ([xshift=18em]node1-1) {\footnotesize{双语数据}};
\node [anchor=south,draw=black,line width=0.6pt,minimum width=4.5em,minimum height=2.2em,fill=blue!20,rounded corners=2pt] (node4-2) at ([yshift=-5em]node4-1.south) {\footnotesize{目标语伪数据}};
\node [anchor=north west,node1,minimum height=4em,minimum width=8em] (n4) at ([xshift=4.7em,yshift=0em]n1.north east) {};
\node [anchor=south west,fill=orange!10,minimum height=1.6em,minimum width=3.6em] (n41) at ([xshift=0.2em,yshift=0.2em]n4.south west) {M$_{src}$};
\node [anchor=south east,fill=orange!10,minimum height=1.6em,minimum width=3.6em] (n42) at ([xshift=-0.2em,yshift=0.2em]n4.south east) {P$_{tgt}$};
\node [anchor=north,fill=orange!10,minimum height=1.6em,minimum width=7.6em] (n43) at ([xshift=0em,yshift=-0.2em]n4.north) {Parallel\ Data};
\node [anchor=north,node2] (n5) at ([xshift=0em,yshift=-2em]n4.south) {Reverse\ NMT\ System};
\node [anchor=north,node3] (n6) at ([xshift=-3.3em,yshift=-2em]n5.south) {Pseudo$_{src}$};
\node [anchor=west,node3] (n61) at ([xshift=2em,yshift=0em]n6.east) {Mono$_{tgt}$};
\node [anchor=east,draw=black,line width=0.6pt,minimum width=4.5em,minimum height=2.2em,fill=red!20,rounded corners=2pt] (node5-1) at ([xshift=15.2em]node3-1.east) {\footnotesize{前向NMT系统}};
\node [anchor=north west,node1,minimum height=4em,minimum width=8em] (n7) at ([xshift=4.7em,yshift=0em]n4.north east) {};
\node [anchor=south west,fill=orange!10,minimum height=1.6em,minimum width=3.6em] (n71) at ([xshift=0.2em,yshift=0.2em]n7.south west) {P$_{src}$};
\node [anchor=south east,fill=orange!10,minimum height=1.6em,minimum width=3.6em] (n72) at ([xshift=-0.2em,yshift=0.2em]n7.south east) {M$_{tgt}$};
\node [anchor=north,fill=orange!10,minimum height=1.6em,minimum width=7.6em] (n73) at ([xshift=0em,yshift=-0.2em]n7.north) {Parallel\ Data};
\node [anchor=north,node2] (n8) at ([xshift=0em,yshift=-3em]n7.south) {Final\ NMT\ System};
\draw [->,thick,black!60,line width=1mm] (n1.east) -- ([xshift=0em,yshift=1em]n4.west);
\draw [->,thick,black!20,line width=1mm] (n1.south) -- (n2.north);
\draw [->,thick,black!20,line width=1mm] (n2.south) -- ([xshift=0em,yshift=-2em]n2.south);
\draw [->,thick,black!40,line width=1mm] (n3.east) -- (n31.west);
\draw [->,thick,black!60,line width=1mm] (n31.north east) -- (n4.south west);
\draw [->,thick,black!60,line width=1mm] ([xshift=0em,yshift=1em]n4.east) -- ([xshift=0em,yshift=1em]n7.west);
\draw [->,thick,black!20,line width=1mm] (n4.south) -- (n5.north);
\draw [->,thick,black!20,line width=1mm] (n5.south) -- ([xshift=0em,yshift=-2em]n5.south);
\draw [->,thick,black!40,line width=1mm] (n61.west) -- (n6.east);
\draw [->,black!60,line width=1mm] (n61.north east) -- (n7.south west);
\draw [->,thick,black!20,line width=1mm] (n7.south) -- (n8.north);
\draw [-,thick] (n4.west) -- (n4.east);
\draw [-,thick] (n4.south) -- ([xshift=0em,yshift=2em]n4.south);
\draw [-,thick] (n7.west) -- (n7.east);
\draw [-,thick] (n7.south) -- ([xshift=0em,yshift=2em]n7.south);
\draw [->,line width=1pt](node1-1.west)--([xshift=3em]node2-1.north);
\draw [->,line width=1pt](node1-1.east)--([xshift=-3em]node3-1.north);
\draw [->,line width=1pt](node1-2.east)--([xshift=-3em]node3-1.south);
\draw [->,line width=1pt](node11.east)--(node12.west);
\draw [->,line width=1pt,dashed](node21.east)--(node22.west);
\draw [->,line width=1pt,dashed]([xshift=3em]node2-1.south)--([xshift=-0.1em]node1-2.west);
\draw [->,line width=1pt,dashed]([xshift=3em]node3-1.south)--([xshift=-0.1em]node4-2.west);
\draw [->,line width=1pt](node4-1.east)--([xshift=-3em]node5-1.north);
\draw [->,line width=1pt](node4-2.east)--([xshift=-3em]node5-1.south);
\end{scope}
\end{tikzpicture}
\ No newline at end of file
Book/Chapter7/Figures/figure-increase-the-encoder.tex
......@@ -15,7 +15,7 @@
\node[node] (enc10) at ([xshift = \base]enc9.east) {};
\node[font=\scriptsize,rotate=270] (src) at ([xshift = -\base]enc1.west) {src};
\draw [->] ([xshift=-0.75em]enc1.west) -- (enc1.west);
\draw [decorate,decoration={brace}] ([yshift=0.3em]enc1.north west) to node [auto,anchor=south,font=\scriptsize] {Nx} ([yshift=0.3em]enc10.north east);
\draw [decorate,decoration={brace}] ([yshift=0.3em]enc1.north west) to node [auto,anchor=south,font=\scriptsize] {$N$x} ([yshift=0.3em]enc10.north east);
\draw [->] (enc1.east) -- (enc2.west);
\draw [->] (enc2.east) -- (enc3.west);
\draw [->] (enc3.east) -- (enc4.west);
......
Book/Chapter7/Figures/figure-label-smoothing.tex
......@@ -4,8 +4,8 @@
\node[font=\scriptsize] (model) at (0,0) {Model Output:};
\node[anchor=north west,font=\scriptsize] (label_smooth) at ([yshift=-1.8em]model.south west) {Label Smoothing:};
\node[anchor=south west,font=\scriptsize] (one-hot) at ([yshift=2em]model.north west) {One-hot:};
\node [anchor=west,minimum width=1em,minimum height=0.2em,fill=ublue!80,inner sep=0pt] (model_label1) at ([xshift=2em,yshift=-0.5em]model.east) {};
%model out
\node [anchor=west,minimum width=1.2em,minimum height=0.2em,fill=ublue!80,inner sep=0pt] (model_label1) at ([xshift=1.5em,yshift=-0.5em]model.east) {};
\node [anchor=south,font=\scriptsize] (model_w1) at (model_label1.north) {$p_{1}$};
\node [anchor=south west,minimum width=1.2em,minimum height=0.1em,fill=ublue!80,inner sep=0pt] (model_label2) at (model_label1.south east) {};
\node [anchor=south,font=\scriptsize] (model_w2) at (model_label2.north) {$p_{2}$};
......@@ -21,18 +21,24 @@
\node [anchor=south west,minimum width=1.2em,minimum height=0.2em,fill=ublue!80,inner sep=0pt] (model_label7) at (model_label6.south east) {};
\node [anchor=south,font=\scriptsize] (model_w8) at (model_label7.north) {$p_{7}$};
%no label smooth
\node [anchor=west,minimum width=1.2em,minimum height=0em,inner sep=0pt,font=\scriptsize] (one_hot_label1) at ([xshift=2em,yshift=3em]model.east) {$0$};
\node [anchor=south west,minimum width=1.2em,minimum height=0em,inner sep=0pt,font=\scriptsize] (one_hot_label2) at (one_hot_label1.south east) {$0$};
\node [anchor=west,minimum width=1.2em,minimum height=0.05em,fill=orange!50,inner sep=0pt,font=\tiny] (one_hot_label1) at ([xshift=1.5em,yshift=3em]model.east) {};
\node [anchor=south,font=\scriptsize] (one_hot_w1) at (one_hot_label1.north) {$0$};
\node [anchor=south west,minimum width=1.2em,minimum height=0.05em,fill=orange!50,inner sep=0pt,font=\tiny] (one_hot_label2) at (one_hot_label1.south east) {};
\node [anchor=south,font=\scriptsize] (one_hot_w2) at (one_hot_label2.north) {$0$};
\node [anchor=south west,minimum width=1.2em,minimum height=1.5em,fill=orange!50,inner sep=0pt] (one_hot_label3) at (one_hot_label2.south east) {};
\node [anchor=south,font=\scriptsize] (one_hot_w3) at (one_hot_label3.north) {{\color{red} $1$}};
\node [anchor=south west,minimum width=1.2em,minimum height=0em,inner sep=0pt,font=\scriptsize] (one_hot_label4) at (one_hot_label3.south east) {$0$};
\node [anchor=south west,minimum width=1.2em,minimum height=0.05em,fill=orange!50,inner sep=0pt,font=\tiny] (one_hot_label4) at (one_hot_label3.south east) {};
\node [anchor=south,font=\scriptsize] (one_hot_w4) at (one_hot_label4.north) {$0$};
\node [anchor=south west,minimum width=1.2em,minimum height=0.1em,inner sep=0pt,font=\scriptsize] (one_hot_label5) at (one_hot_label4.south east) {$0$};
\node [anchor=south west,minimum width=1.2em,minimum height=0.3em,inner sep=0pt,font=\scriptsize] (one_hot_label6) at (one_hot_label5.south east) {$0$};
\node [anchor=south west,minimum width=1em,minimum height=0.4em,inner sep=0pt,font=\scriptsize] (one_hot_label7) at (one_hot_label6.south east) {$0$};
\node [anchor=south west,minimum width=1.2em,minimum height=0.05em,fill=orange!50,inner sep=0pt,font=\tiny] (one_hot_label5) at (one_hot_label4.south east) {};
\node [anchor=south,font=\scriptsize] (one_hot_w5) at (one_hot_label5.north) {$0$};
\node [anchor=south west,minimum width=1.2em,minimum height=0.05em,fill=orange!50,inner sep=0pt,font=\tiny] (one_hot_label6) at (one_hot_label5.south east) {};
\node [anchor=south,font=\scriptsize] (one_hot_w6) at (one_hot_label6.north) {$0$};
\node [anchor=south west,minimum width=1.12em,minimum height=0.05em,fill=orange!50,inner sep=0pt,font=\tiny] (one_hot_label7) at (one_hot_label6.south east) {};
\node [anchor=south,font=\scriptsize] (one_hot_w7) at (one_hot_label7.north) {$0$};
%label smoothing
\node [anchor=west,minimum width=1em,minimum height=0.2em,fill=red!50,inner sep=0pt] (label1) at ([xshift=2em,yshift=-3.2em]model.east) {};
\node [anchor=west,minimum width=1.2em,minimum height=0.2em,fill=red!50,inner sep=0pt] (label1) at ([xshift=1.5em,yshift=-3.2em]model.east) {};
\node [anchor=south,font=\scriptsize] (w1) at (label1.north) {$0.1$};
\node [anchor=south west,minimum width=1.2em,minimum height=0.2em,fill=red!50,inner sep=0pt] (label2) at (label1.south east) {};
\node [anchor=south,font=\scriptsize] (w2) at (label2.north) {$0.1$};
......
Book/Chapter7/Figures/learning-rate.tex → Book/Chapter7/Figures/figure-learning-rate.tex
\begin{center}
\begin{tikzpicture}[scale=1.3]
\begin{tikzpicture}[scale=1.0]
\footnotesize{
\begin{axis}[
width=.40\textwidth,
height=.30\textwidth,
width=.50\textwidth,
height=.40\textwidth,
legend style={at={(0.60,0.08)}, anchor=south west},
xlabel={\scriptsize{更新次数（10k）}},
ylabel={\scriptsize{学习率 （$10^{-3}$）}},
......
Book/Chapter7/Figures/figure-mt-data-processing-flow.tex
\begin{tikzpicture}
\tikzstyle{op} =[rounded corners=1pt,thick,minimum width=4.0em,minimum height=3.0em,draw,fill=red!5!white,font=\scriptsize]
\tikzstyle{data} = [cylinder,draw=black,thick,minimum height=3em,minimum width=3em,shape border rotate=0,cylinder uses custom fill, cylinder body fill=blue!10,cylinder end fill=blue!5,anchor = east,font=\scriptsize]
\tikzstyle{data} = [cylinder,draw=black,thick,minimum height=2.5em,minimum width=3em,shape border rotate=0,cylinder uses custom fill, cylinder body fill=blue!10,cylinder end fill=blue!5,anchor = east,font=\scriptsize]
\node[op] (node1) at (0,0) {分词};
\node[op,anchor = west] (node2) at ([xshift = 2.0em]node1.east) {符号标准化};
\node[op,anchor = west] (node3) at ([xshift = 2.0em]node2.east) {数据过滤};
\node[op,anchor = west] (node4) at ([xshift = 2.0em]node3.east) {子词切分};
\node [data,anchor = east] (data1) at ([xshift = -2.0em]node1.west){原始数据};
\node [data,anchor = west] (data2) at ([xshift = 2.0em]node3.east){训练数据};
\node [data,anchor = west] (data2) at ([xshift = 2.0em]node4.east){训练数据};
\draw[-stealth,line width=.05cm] ([xshift=0.25em]data1.east) -- ([xshift=-0.25em]node1.west);
\draw[-stealth,line width=.05cm] ([xshift=0.25em]node1.east) -- ([xshift=-0.25em]node2.west);
\draw[-stealth,line width=.05cm] ([xshift=0.25em]node2.east) -- ([xshift=-0.25em]node3.west);
\draw[-stealth,line width=.05cm] ([xshift=0.25em]node3.east) -- ([xshift=-0.25em]data2.west);
\draw[-stealth,line width=.05cm] ([xshift=0.25em]node3.east) -- ([xshift=-0.25em]node4.west);
\draw[-stealth,line width=.05cm] ([xshift=0.25em]node4.east) -- ([xshift=-0.25em]data2.west);
\end{tikzpicture}
\ No newline at end of file
Book/Chapter7/Figures/figure-randomly-generation-vs-generate-by-sentence-length.tex
\begin{tikzpicture}
\tikzstyle{node} = [minimum height=1.0em,draw=teal,fill=teal!10]
\node[node,minimum width=2.0em] (sent1) at (0,0) {};
\node[node,minimum width=5.0em,anchor=north west] (sent2) at (sent1.south west) {};
\node[node,minimum width=1.0em,anchor=north west] (sent3) at (sent2.south west) {};
\node[node,minimum width=3.0em,anchor=north west] (sent4) at (sent3.south west) {};
\tikzstyle{node} = [minimum height=1.0*1.2em,draw=teal,fill=teal!10]
\node[node,minimum width=2.0*1.2em] (sent1) at (0,0) {};
\node[node,minimum width=5.0*1.2em,anchor=north west] (sent2) at (sent1.south west) {};
\node[node,minimum width=1.0*1.2em,anchor=north west] (sent3) at (sent2.south west) {};
\node[node,minimum width=3.0*1.2em,anchor=north west] (sent4) at (sent3.south west) {};
\node[node,minimum width=4.0em] (sent5) at (12em,0) {};
\node[node,minimum width=4.5em,anchor=north west] (sent6) at (sent5.south west) {};
\node[node,minimum width=4.5em,anchor=north west] (sent7) at (sent6.south west) {};
\node[node,minimum width=5em,anchor=north west] (sent8) at (sent7.south west) {};
\node[node,minimum width=4.0*1.2em] (sent5) at (14em,0) {};
\node[node,minimum width=4.5*1.2em,anchor=north west] (sent6) at (sent5.south west) {};
\node[node,minimum width=4.5*1.2em,anchor=north west] (sent7) at (sent6.south west) {};
\node[node,minimum width=5*1.2em,anchor=north west] (sent8) at (sent7.south west) {};
\node[font=\scriptsize,anchor=east] (line1) at (sent1.west) {sent1};
\node[font=\scriptsize,anchor=east] (line2) at (sent2.west) {sent2};
\node[font=\scriptsize,anchor=east] (line3) at (sent3.west) {sent3};
\node[font=\scriptsize,anchor=east] (line4) at (sent4.west) {sent4};
\node[font=\footnotesize,anchor=east] (line1) at (sent1.west) {sent1};
\node[font=\footnotesize,anchor=east] (line2) at (sent2.west) {sent2};
\node[font=\footnotesize,anchor=east] (line3) at (sent3.west) {sent3};
\node[font=\footnotesize,anchor=east] (line4) at (sent4.west) {sent4};
\node[font=\scriptsize,anchor=east] (line5) at (sent5.west) {sent1};
\node[font=\scriptsize,anchor=east] (line6) at (sent6.west) {sent2};
\node[font=\scriptsize,anchor=east] (line7) at (sent7.west) {sent3};
\node[font=\scriptsize,anchor=east] (line8) at (sent8.west) {sent4};
\node[font=\footnotesize,anchor=east] (line5) at (sent5.west) {sent1};
\node[font=\footnotesize,anchor=east] (line6) at (sent6.west) {sent2};
\node[font=\footnotesize,anchor=east] (line7) at (sent7.west) {sent3};
\node[font=\footnotesize,anchor=east] (line8) at (sent8.west) {sent4};
\begin{pgfonlayer}{background}
\node [rectangle,inner sep=-0.0em,draw] [fit = (sent1) (sent2) (sent3) (sent4)] (box1) {};
\node [rectangle,inner sep=-0.0em,draw] [fit = (sent5) (sent6) (sent7) (sent8)] (box2) {};
\end{pgfonlayer}
\node[font=\scriptsize] (node1) at ([yshift=-3em]sent2.south) {随机生成};
\node[font=\scriptsize] (node2) at ([yshift=-1em]sent8.south) {排序生成};
\node[font=\footnotesize] (node1) at ([yshift=-3.4em]sent2.south) {随机生成};
\node[font=\footnotesize] (node2) at ([yshift=-1em]sent8.south) {排序生成};
\end{tikzpicture}
\ No newline at end of file
Book/Chapter7/Figures/figure-target-side-multi-task-learning.tex
\begin{tikzpicture}
\begin{scope}
\node [anchor=center] (node1) at (-2.3,0) {\small{$x,y$：双语数据}};
\node [anchor=center] (node2) at (-2.1,-0.5) {\small{$z$}：单语数据};
\node [anchor=center] (node1-1) at (0,0) {\small{$y'$}};
\node [anchor=center] (node1) at (-2.3,1.5) {\small{$x,y$：双语数据}};
\node [anchor=center] (node2) at (-2.1,1) {\small{$z$}：单语数据};
\node[anchor=south,draw,rounded corners,minimum height=1.5em,minimum width=4em,fill=blue!20](node1-2) at ([yshift=-3em]node1-1.south) {\small{softamx}};
\node[anchor=south,draw,rounded corners,minimum height=2.5em,minimum width=4em,fill=red!20](node1-3) at ([yshift=-4.5em]node1-2.south) {\small{Decoder}};
\node[anchor=south](node1-4) at ([yshift=-3em]node1-3.south) {\small{$y$}};
\node[anchor=west](node2-2) at ([xshift=-5.5em]node1-4.west) {\small{$x$}};
\node [anchor=center] (labela) at ([xshift=3.5em,yshift=-1.5em]node2-2.south) {\small{(a) Baseline}};
\node[anchor=north,draw,rounded corners,minimum height=2.5em,minimum width=4em,fill=red!20](node2-1) at ([yshift=3.5em]node2-2.north) {\small{Encoder}};
\node [anchor=center] (node3-1) at ([xshift=5.5em,yshift=-0.1em]node1-1.east) {\small{$z'$}};
\node[anchor=south,line width=0.6pt,draw,rounded corners,minimum height=1.5em,minimum width=4em,fill=blue!20](node1-2) at ([yshift=-3em]node1-1.south) {\small{softamx}};
\node[anchor=south,line width=0.6pt,draw,rounded corners,minimum height=1.5em,minimum width=4em,fill=blue!20](node3-2) at ([yshift=-3em]node3-1.south) {\small{softamx}};
\node[anchor=south,line width=0.6pt,draw,rounded corners,minimum height=2.2em,minimum width=4em,fill=red!20](node1-3) at ([yshift=-4.0em]node1-2.south) {\small{Decoder}};
\node[anchor=south,line width=0.6pt,draw,rounded corners,minimum height=2.2em,minimum width=4em,fill=yellow!20](node3-3) at ([yshift=-4.0em]node3-2.south) {\small{LM}};
\node[anchor=south](node1-4) at ([xshift=-0.6em,yshift=-3em]node1-3.south) {\gray{\small{$y$}}};
\node[anchor=south](node3-41) at ([xshift=-0.6em,yshift=-3em]node3-3.south) {\small{$y$}};
\node[anchor=south](node3-42) at ([xshift=0.6em,yshift=-2.9em]node3-3.south) {\small{$z$}};
\node[anchor=west](node2-2) at ([xshift=-4.9em]node1-4.west) {\small{$x$}};
\node[anchor=north,line width=0.6pt,draw,rounded corners,minimum height=2.2em,minimum width=4em,fill=red!20](node2-1) at ([yshift=4em]node2-2.north) {\small{Encoder}};
\draw [->](node1-4.north)--(node1-3);
\draw [->](node1-3.north)--(node1-2);
\draw [->](node1-2.north)--(node1-1);
\draw [->](node2-2.north)--(node2-1);
\node [rectangle,rounded corners,draw=red,line width=0.2mm,densely dashed,inner sep=0.4em] [fit = (node3-2) (node3-3)] (inputshadow) {};
\draw [->,thick,draw=gray](node1-4.north)--([xshift=-0.6em]node1-3.south);
\draw [->,thick](node1-3.north)--(node1-2);
\draw [->,thick](node1-2.north)--(node1-1);
\draw [->,thick](node2-2.north)--(node2-1);
\draw[->,thick](node2-1.east)--(node1-3.west);
\draw[->](node2-1.north)--([yshift=1em]node2-1.north)--([xshift=2.5em,yshift=1em]node2-1.north)--([xshift=2.5em,yshift=-0.4em]node2-1.north)--(node1-3.west);
\end{scope}
\begin{scope}[xshift=2.3in,yshift=0.6in]
\draw [->,thick](node3-41.north)--([xshift=-0.6em]node3-3.south);
\draw [->,thick](node3-42.north)--([xshift=0.6em]node3-3.south);
\draw [->,thick]([xshift=0.6em]node3-3.north)--([xshift=0.6em]node3-2.south);
\draw [->,thick](node3-2.north)--(node3-1);
\draw[->,thick]([xshift=-0.6em]node3-3.north)--([xshift=-0.6em,yshift=0.6em]node3-3.north)--([xshift=-3em,yshift=0.6em]node3-3.north)--([xshift=-3em,yshift=-3em]node3-3.north)--([xshift=-5.6em,yshift=-3em]node3-3.north)--([xshift=0.6em]node1-3.south);
\node [anchor=center] (node1-1) at (0,0) {\small{$y'$}};
\node[anchor=south,draw,rounded corners,minimum height=1.5em,minimum width=4em,fill=blue!20](node1-2) at ([yshift=-3em]node1-1.south) {\small{softamx}};
\node[anchor=south,draw,rounded corners,minimum height=2.5em,minimum width=4em,fill=red!20](node1-3) at ([yshift=-4.5em]node1-2.south) {\small{Decoder}};
\node[anchor=south,draw,rounded corners,minimum height=1.5em,minimum width=4em,fill=blue!20](node3-1) at ([xshift=6em,yshift=0em]node1-3.south) {\small{softmax}};
\node[anchor=south](node3-2) at ([yshift=3em]node3-1.south) {\small{$z'$}};
\node[anchor=south,draw,rounded corners,minimum height=2em,minimum width=4em,,fill=yellow!20](node1-4) at ([yshift=-4em]node1-3.south) {\small{LM}};
\node[anchor=south](node1-5) at ([yshift=-3em]node1-4.south) {\small{$y$}};
\node [anchor=center] (labelb) at ([yshift=-1.5em]node1-5.south) {\small{(b) Multi task learning}};
\node[anchor=west](node2-2) at ([xshift=-5em]node1-5.west) {\small{$x$}};
\node[anchor=north,draw,rounded corners,minimum height=2.5em,minimum width=4em,fill=red!20](node2-1) at ([yshift=5.3em]node2-2.north) {\small{Encoder}};
\draw [->](node1-5.north)--(node1-4);
\draw [->](node1-4.north)--(node1-3);
\draw [->](node1-3.north)--(node1-2);
\draw [->](node1-2.north)--(node1-1);
\draw [->](node2-2.north)--(node2-1);
\draw [->](node3-1.north)--(node3-2);
\draw[->](node2-1.north)--([yshift=1.8em]node2-1.north)--(node1-3.west);
\draw [->]([yshift=0.8em]node1-4.north)--([xshift=6em,yshift=0.8em]node1-4.north)--(node3-1.south);
%\draw[->](node2-1.north)--([yshift=1em]node2-1.north)--([xshift=2.5em,yshift=1em]node2-1.north)--([xshift=2.5em,yshift=-0.4em]node2-1.north)--(node1-3.west);
\end{scope}
\end{tikzpicture}
\ No newline at end of file
Book/Chapter7/Figures/figure-Underfitting-vs-Overfitting.tex → Book/Chapter7/Figures/figure-underfitting-vs-overfitting.tex
\begin{tabular}{l l l}
\begin{tikzpicture}
\draw[->, thick] (0,0) to (3,0);
......@@ -22,7 +21,7 @@
\fill [black] (h) circle(1pt);
% y=0.73x + 2.54
\draw [thick,red] (-1*0.3,1.81*0.2) to (10*0.3,9.84*0.2);
\draw [thick,red] (0.5*0.3,1.81*0.2) to (10*0.3,9.84*0.2);
\node [font=\footnotesize] at (1.5,-0.5) {欠拟合};
\end{tikzpicture}
......
Book/Chapter7/Figures/figure-word-change.tex
\begin{center}
\centerline{以英语为例：}
\vspace{0.5em}
\begin{tikzpicture}
\node[rounded corners=3pt,minimum width=10.0em,minimum height=2.0em,draw,thick,fill=green!5,font=\scriptsize,drop shadow,inner sep=0.5em] (left) at (0,0) {
\begin{tabular}{c}
......
Book/ChapterAppend/ChapterAppend.tex
% !Mode:: "TeX:UTF-8"
% !TEX encoding = UTF-8 Unicode
\part{附录}
%----------------------------------------------------------------------------------------
% CHAPTER APPENDIX
% CONFIGURATIONS
%----------------------------------------------------------------------------------------
\part{附录}
\renewcommand\figurename{图}%将figure改为图
\renewcommand\tablename{表}%将figure改为图
\chapterimage{fig-NEU-1.jpg} % Chapter heading image
%------------------------------------------------
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%第一章附录
%----------------------------------------------------------------------------------------
% CHAPTER APPENDIX A
%----------------------------------------------------------------------------------------
\begin{appendices}
\chapter{附录A}
\label{appendix-A}
\parinterval 在构建机器翻译系统的过程中，数据是必不可少的，尤其是现在主流的神经机器翻译系统，系统的性能往往受限于语料库规模和质量。所幸的是，随着语料库语言学的发展，一些主流语种的相关语料资源已经十分丰富。
\parinterval 为了方便读者进行相关研究，我们汇总了几个常用的基准数据集，这些数据集已经在机器翻译领域中被广泛使用，有很多之前的相关工作可以进行复现和对比。同时，我们收集了一下常用的平行语料，方便读者进行一些探索。
%%%%%%%%%%%%%%%%%%%%%
%----------------------------------------------------------------------------------------
% NEW SECTION
%----------------------------------------------------------------------------------------
\section{基准数据集}
%----------------------------------------------
% 表1.1-1
\begin{table}[htp]{
\footnotesize
\begin{center}
......@@ -41,9 +48,9 @@
\end{tabular}
\end{center}
}\end{table}
%-------------------------------------------
%----------------------------------------------
% 表1.1-2
%----------------------------------------------
\begin{table}[htp]{
\footnotesize
\begin{center}
......@@ -66,8 +73,12 @@
\end{tabular}
\end{center}
}\end{table}
%-------------------------------------------
%%%%%%%%%%%%%%%%%%%%%%%%%%%
%----------------------------------------------
%----------------------------------------------------------------------------------------
% NEW SECTION
%----------------------------------------------------------------------------------------
\section{平行语料}
\parinterval 神经机器翻译系统的训练需要大量的双语数据，这里我们汇总了一些公开的平行语料，方便读者获取。
\vspace{0.5em}
......@@ -96,14 +107,23 @@
\vspace{0.5em}
\item Tilde MODEL Corpus：欧洲语言的多语言开放数据，包含多个数据集，数据来自于经济、新闻、政府、旅游等门户网站。URL：\url{https://tilde-model.s3-eu-west-1.amazonaws.com/Tilde_MODEL_Corpus.html}
\vspace{0.5em}
\item Setimes Corpus：包括克罗地亚语、阿尔巴尼亚等9种巴尔干语言，72种个语言对的双语数据，来源于东南欧时报的新闻报道。URL：\url{http://www.statmt.org/setimes/}
\item Setimes Corpus：包括克罗地亚语、阿尔巴尼亚等9种巴尔干语言，72个语言对的双语数据，来源于东南欧时报的新闻报道。URL：\url{http://www.statmt.org/setimes/}
\vspace{0.5em}
\item TVsub：收集自电视剧集字幕的中英文对话语料库，包含超过200万的句对，可用于对话领域和长距离上下文信息的研究。URL：\url{https://github.com/longyuewangdcu/tvsub}
\vspace{0.5em}
\item Recipe Corpus：由Cookpad公司创建的日英食谱语料库，包含10万多的句对。URL：\url{http://lotus.kuee.kyoto-u.ac.jp/WAT/recipe-corpus/}
\end{itemize}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%----------------------------------------------------------------------------------------
% NEW SECTION
%----------------------------------------------------------------------------------------
\section{相关工具}
%----------------------------------------------------------------------------------------
% NEW SUB-SECTION
%----------------------------------------------------------------------------------------
\subsection{数据预处理工具}
\parinterval 数据处理是搭建神经机器翻译系统的重要步骤，这里我们提供了一些开源工具供读者进行使用。
\vspace{0.5em}
......@@ -115,6 +135,10 @@
\item Subword-nmt：基于BPE算法的子词切分工具。URL：\url{https://github.com/rsennrich/subword-nmt}
\end{itemize}
%----------------------------------------------------------------------------------------
% NEW SUB-SECTION
%----------------------------------------------------------------------------------------
\subsection{评价工具}
\parinterval 机器翻译领域已经有多种自动评价指标，包括BLEU、TER和METEOR等，这里我们提供了一些自动评价指标的工具，方便读者使用。
\vspace{0.5em}
......@@ -126,16 +150,20 @@
\item Meteor：自动评价指标METEOR的实现。URL：\url{https://www.cs.cmu.edu/~alavie/METEOR/}
\end{itemize}
\end{appendices}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%第二章附录
%----------------------------------------------------------------------------------------
% CHAPTER APPENDIX B
%----------------------------------------------------------------------------------------
\begin{appendices}
\chapter{附录B}
\label{appendix-B}
%----------------------------------------------------------------------------------------
% NEW SECTION
%----------------------------------------------------------------------------------------
\section{IBM模型3训练方法}
\parinterval 模型3的参数估计与模型1和模型2采用相同的方法。这里直接给出辅助函数。
\begin{eqnarray}
......@@ -144,7 +172,7 @@ h(t,d,n,p, \lambda,\mu, \nu, \zeta) & = & \textrm{P}_{\theta}(\mathbf{s}|\mathb
& & -\sum_{t}\nu_{t}\big(\sum_{\varphi}n(\varphi|t)-1\big)-\zeta(p^0+p^1-1)
\label{eq:1.1}
\end{eqnarray}
%----------------------------------------------
\parinterval 由于篇幅所限这里略去了推导步骤直接给出一些用于参数估计的等式。
\begin{eqnarray}
c(s|t,\mathbf{s},\mathbf{t}) & = & \sum_{\mathbf{a}}\big[\textrm{P}_{\theta}(\mathbf{s},\mathbf{a}|\mathbf{t}) \times \sum_{j=1}^{m} (\delta(s_j,s) \cdot \delta(t_{a_{j}},t))\big] \label{eq:1.2} \\
......@@ -157,7 +185,6 @@ c(\varphi|t;\mathbf{s},\mathbf{t}) & = & \sum_{\mathbf{a}}\big[\textrm{P}_{\thet
c(0|\mathbf{s},\mathbf{t}) & = & \sum_{\mathbf{a}}\big[\textrm{P}_{\theta}(\mathbf{s},\mathbf{a}|\mathbf{t}) \times (m-2\varphi_0) \big] \label{eq:1.5} \\
c(1|\mathbf{s},\mathbf{t}) & = & \sum_{\mathbf{a}}\big[\textrm{P}_{\theta}(\mathbf{s},\mathbf{a}|\mathbf{t}) \times \varphi_0 \big] \label{eq:1.6}
\end{eqnarray}
%----------------------------------------------
\parinterval 进一步，对于由$K$个样本组成的训练集，有：
\begin{eqnarray}
......@@ -166,32 +193,27 @@ d(j|i,m,l) & = & \mu_{iml}^{-1} \times \sum_{k=1}^{K}c(j|i,m,l;\mathbf{s}^{[k]},
n(\varphi|t) & = & \nu_{t}^{-1} \times \sum_{s=1}^{K}c(\varphi |t;\mathbf{s}^{[k]},\mathbf{t}^{[k]}) \label{eq:1.9} \\
p_x & = & \zeta^{-1} \sum_{k=1}^{K}c(x;\mathbf{s}^{[k]},\mathbf{t}^{[k]}) \label{eq:1.10}
\end{eqnarray}
%----------------------------------------------
\parinterval 在模型3中，因为产出率的引入，并不能像模型1和模型2那样，在保证正确性的情况下加速参数估计的过程。这就使得每次迭代过程中，都不得不面对大小为$(l+1)^m$的词对齐空间。遍历所有$(l+1)^m$个词对齐所带来的高时间复杂度显然是不能被接受的。因此就要考虑能否仅利用词对齐空间中的部分词对齐对这些参数进行估计。比较简单且直接的方法就是仅利用Viterbi对齐来进行参数估计\footnote{Viterbi词对齐可以被简单的看作搜索到的最好词对齐。}。 遗憾的是，在模型3中并没有方法直接获得Viterbi对齐。这样只能采用一种折中的策略，即仅考虑那些使得$\textrm{P}_{\theta}(\mathbf{s},\mathbf{a}|\mathbf{t})$达到较高值的词对齐。这里把这部分词对齐组成的集合记为$S$。式\ref{eq:1.2}可以被修改为：
\begin{eqnarray}
c(s|t,\mathbf{s},\mathbf{t}) \approx \sum_{\mathbf{a} \in \mathbf{S}}\big[\textrm{P}_{\theta}(\mathbf{s},\mathbf{a}|\mathbf{t}) \times \sum_{j=1}^{m}(\delta(s_j,\mathbf{s}) \cdot \delta(t_{a_{j}},\mathbf{t})) \big]
\label{eq:1.11}
\end{eqnarray}
%----------------------------------------------
\parinterval 同理可以获得式\ref{eq:1.3}-\ref{eq:1.6}的修改结果。进一步，在IBM模型3中，可以定义$S$如下：
\begin{eqnarray}
S = N(b^{\infty}(V(\mathbf{s}|\mathbf{t};2))) \cup (\mathop{\cup}\limits_{ij} N(b_{i \leftrightarrow j}^{\infty}(V_{i \leftrightarrow j}(\mathbf{s}|\mathbf{t},2))))
\label{eq:1.12}
\end{eqnarray}
%----------------------------------------------
\parinterval 为了理解这个公式，先介绍几个概念。
\begin{itemize}
\item $V(\mathbf{s}|\mathbf{t})$表示Viterbi词对齐，$V(\mathbf{s}|\mathbf{t},1)$、$V(\mathbf{s}|\mathbf{t},2)$和$V(\mathbf{s}|\mathbf{t},3)$就分别对应了模型1、2 和3 的Viterbi 词对齐；
\item 把那些满足第$j$个源语言单词对应第$i$个目标语言单词（$a_j=i$）的词对齐构成的集合记为$\mathbf{A}_{i \leftrightarrow j}(\mathbf{s},\mathbf{t})$。通常称这些对齐中$j$和$i$被``钉''在了一起。在$\mathbf{A}_{i \leftrightarrow j}(\mathbf{s},\mathbf{t})$中使$\textrm{P}(\mathbf{a}|\mathbf{s},\mathbf{t})$达到最大的那个词对齐被记为$V_{i \leftrightarrow j}(\mathbf{s},\mathbf{t})$；
\item 如果两个词对齐，通过交换两个词对齐连接就能互相转化，则称它们为邻居。一个词对齐$\mathbf{a}$的所有邻居记为$N(\mathbf{a})$。
\end{itemize}
\vspace{0.3em}
\vspace{0.5em}
\parinterval 公式\ref{eq:1.12}中，$b^{\infty}(V(\mathbf{s}|\mathbf{t};2))$ 和 $b_{i \leftrightarrow j}^{\infty}(V_{i \leftrightarrow j}(\mathbf{s}|\mathbf{t},2))$ 分别是对 $V(\mathbf{s}|\mathbf{t};3)$ 和 $V_{i \leftrightarrow j}(\mathbf{s}|\mathbf{t},3)$ 的估计。在计算$S$的过程中，需要知道一个对齐$\bf{a}$的邻居$\bf{a}^{'}$的概率，即通过$\textrm{P}_{\theta}(\mathbf{a},\mathbf{s}|\mathbf{t})$计算$\textrm{p}_{\theta}(\mathbf{a}',\mathbf{s}|\mathbf{t})$。在模型3中，如果$\bf{a}$和$\bf{a}'$仅区别于某个源语单词对齐到的目标位置上（$a_j \neq a_{j}'$），那么
\begin{eqnarray}
......@@ -200,17 +222,19 @@ S = N(b^{\infty}(V(\mathbf{s}|\mathbf{t};2))) \cup (\mathop{\cup}\limits_{ij} N(
& & \frac{t(s_j|t_{i'})}{t(s_{j}|t_{i})} \cdot \frac{d(j|i',m,l)}{d(j|i,m,l)}
\label{eq:1.13}
\end{eqnarray}
%----------------------------------------------
\parinterval 如果$\bf{a}$和$\bf{a}'$区别于两个位置$j_1$和$j_2$的对齐上，$a_{j_{1}}=a_{j_{2}^{'}}$且$a_{j_{2}}=a_{j_{1}^{'}}$，那么
\begin{eqnarray}
\textrm{P}_{\theta}(\mathbf{a'},\mathbf{s}|\mathbf{t}) = \textrm{P}_{\theta}(\mathbf{a},\mathbf{s}|\mathbf{t}) \cdot \frac{t(s_{j_{2}}|t_{a_{j_{2}}})}{t(s_{j_{1}}|t_{a_{j_{1}}})} \cdot \frac{d(j_{2}|a_{j_{2}},m,l)}{d(j_{1}|a_{j_{1}},m,l)}
\label{eq:1.14}
\end{eqnarray}
%----------------------------------------------
\parinterval 相比整个词对齐空间，$S$只是一个非常小的子集，因此运算复杂度可以被大大降低。可以看到，模型3的参数估计过程是建立在模型1和模型2的参数估计结果上的。这不仅是因为模型3要利用模型2的Viterbi对齐，而且还因为模型3参数的初值也要直接利用模型2的参数。从这个角度说，模型1，2，3是有序的且向前依赖的。单独的对模型3的参数进行估计是极其困难的。实际上IBM的模型4和模型5也具有这样的性质，即它们都可以利用前一个模型参数估计的结果作为自身参数的初始值。
%----------------------------------------------------------------------------------------
% NEW SECTION
%----------------------------------------------------------------------------------------
\section{IBM模型4训练方法}
\parinterval 模型4的参数估计基本与模型3一致。需要修改的是扭曲度的估计公式，对于目标语第$i$个cept.生成的第一单词，可以得到（假设有$K$个训练样本）：
......@@ -252,19 +276,20 @@ s_{>1}(\Delta_j|cb;\mathbf{a},\mathbf{s},\mathbf{t}) & = & \sum_{i=1}^l \big[\va
& & \delta(B(\tau_{[i]k}),cb) \big] \label{eq:1.20}
\end{eqnarray}
%----------------------------------------------
\noindent 这里，$ca$和$cb$分别表示目标语言和源语言的某个词类。模型4需要像模型3一样，通过定义一个词对齐集合$S$，使得每次迭代都在$S$上进行，进而降低运算量。模型4中$S$的定义为：
\begin{eqnarray}
\textrm{S} = N(\tilde{b}^{\infty}(V(\mathbf{s}|\mathbf{t};2))) \cup (\mathop{\cup}\limits_{ij} N(\tilde{b}_{i \leftrightarrow j}^{\infty}(V_{i \leftrightarrow j}(\mathbf{s}|\mathbf{t},2))))
\label{eq:1.22}
\end{eqnarray}
%----------------------------------------------
\parinterval 对于一个对齐$\mathbf{a}$，可用模型3对它的邻居进行排名，即按$\textrm{P}_{\theta}(b(\mathbf{a})|\mathbf{s},\mathbf{t};3)$排序，其中$b(\mathbf{a})$表示$\mathbf{a}$的邻居。$\tilde{b}(\mathbf{a})$ 表示这个排名表中满足$\textrm{P}_{\theta}(\mathbf{a}'|\mathbf{s},\mathbf{t};4) > \textrm{P}_{\theta}⁡(\mathbf{a}|\mathbf{s},\mathbf{t};4)$的最高排名的$\mathbf{a}'$。同理可知$\tilde{b}_{i \leftrightarrow j}^{\infty}(\mathbf{a})$ 的意义。这里之所以不用模型3中采用的方法直接利用$b^{\infty}(\mathbf{a})$得到模型4中高概率的对齐，是因为模型4中，要想获得某个对齐$\mathbf{a}$的邻居$\mathbf{a}'$，必须做很大调整，比如：调整$\tau_{[i]1}$和$\odot_{i}$等等。这个过程要比模型3的相应过程复杂得多。因此在模型4中只能借助于模型3的中间步骤来进行参数估计。
\setlength{\belowdisplayskip}{3pt}%调整空白大小
%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%----------------------------------------------------------------------------------------
% NEW SECTION
%----------------------------------------------------------------------------------------
\section{IBM模型5训练方法}
\parinterval 模型5的参数估计过程也与模型3的过程基本一致，二者的区别在于扭曲度的估计公式。在模型5中，对于目标语第$i$个cept.生成的第一单词，可以得到（假设有$K$个训练样本）：
......@@ -298,7 +323,6 @@ s_{>1}(\Delta_j|cb,v;\mathbf{a},\mathbf{s},\mathbf{t}) & = & \sum_{i=1}^l\Big[\v
& & \cdot \delta(v_{\pi_{i1}},v_{\pi_{i1}-1}) \big] \Big] \label{eq:1.28}
\end{eqnarray}
%----------------------------------------------
\vspace{0.5em}
\parinterval 从式(\ref{eq:1.24})中可以看出因子$\delta(v_{\pi_{i1}},v_{\pi_{i1}-1})$保证了，即使对齐$\mathbf{a}$不合理（一个源语位置对应多个目标语位置）也可以避免在这个不合理的对齐上计算结果。需要注意的是因子$\delta(v_{\pi_{p1}},v_{\pi_{p1-1}})$，确保了$\mathbf{a}$中不合理的部分不产生坏的影响，而$\mathbf{a}$中其他正确的部分仍会参与迭代。
......@@ -314,7 +338,6 @@ s_{>1}(\Delta_j|cb,v;\mathbf{a},\mathbf{s},\mathbf{t}) & = & \sum_{i=1}^l\Big[\v
\end{eqnarray}
\vspace{0.5em}
%----------------------------------------------
\parinterval 这里$\tilde{\tilde{b}}(\mathbf{a})$借用了模型4中$\tilde{b}(\mathbf{a})$的概念。不过$\tilde{\tilde{b}}(\mathbf{a})$表示在利用模型3进行排名的列表中满足$\textrm{P}_{\theta}(\mathbf{a}'|\mathbf{s},\mathbf{t};5)$的最高排名的词对齐。
\end{appendices}
......
Book/ChapterPreface/ChapterPreface.tex
% !Mode:: "TeX:UTF-8"
% !TEX encoding = UTF-8 Unicode
\newpage
\renewcommand\figurename{图}
%----------------------------------------------------------------------------------------
% PREFACE
%----------------------------------------------------------------------------------------
{\color{white} 空}
\vspace{0.5em}
\begin{center}
{\Huge \bfnew{导\ \ \ \ 读}}
\end{center}
\vspace{2em}
\begin{spacing}{1.18}
让计算机进行自然语言的翻译是人类长期的梦想，也是人工智能的终极目标之一。自上世纪九十年代起，机器翻译迈入了基于统计建模的时代，发展到今天，深度学习等机器学习方法已经在机器翻译中得到了大量的应用，取得了令人瞩目的进步。
在这个时代背景下，对机器翻译的模型、方法和实现技术进行深入了解是自然语言处理领域研究者和实践者所渴望的。本书全面回顾了近三十年内机器翻译的技术发展历程，并围绕统计建模和深度学习两个主题对机器翻译的技术方法进行了全面介绍。在写作中，笔者力求用朴实的语言和简洁的实例阐述机器翻译的基本模型和方法，同时对相关的技术前沿进行讨论。本书可以供计算机相关专业高年级本科生及研究生学习之用，也可以作为自然语言处理，特别是机器翻译领域相关研究人员的参考资料。
本书共分为七个章节，章节的顺序参考了机器翻译技术发展的时间脉络，同时兼顾了机器翻译知识体系的内在逻辑。各章节的主要内容包括：
\begin{itemize}
\vspace{0.5em}
\item 第一章：机器翻译简介
\vspace{0.5em}
\item 第二章：词法、语法及统计建模基础
\vspace{0.5em}
\item 第三章：基于词的机器翻译模型
\vspace{0.5em}
\item 第四章：基于短语和句法的机器翻译模型
\vspace{0.5em}
\item 第五章：人工神经网络和神经语言建模
\vspace{0.5em}
\item 第六章：神经机器翻译模型
\vspace{0.5em}
\item 第七章：神经机器翻译实战 \ \dash \ 参加一次比赛
\vspace{0.5em}
\end{itemize}
其中，第一章是对机器翻译的整体介绍。第二章和第五章是对统计建模和深度学习方法的介绍，分别建立了两个机器翻译范式的基础知识体系 \ \dash \ 统计机器翻译和神经机器翻译。统计机器翻译部分（第三、四章）涉及早期的基于单词的翻译模型，以及本世纪初流行的基于短语和句法的翻译模型。神经机器翻译（第六、七章）代表了当今机器翻译的前沿，内容主要涉及了基于端到端表示学习的机器翻译建模方法。特别的，第七章对一些最新的神经机器翻译方法进行了讨论，为相关科学问题的研究和实用系统的开发提供了可落地的思路。图\ref{fig:preface}展示了本书各个章节及核心概念之间的关系。
{\red 用最简单的方式阐述机器翻译的基本思想}是笔者所期望达到的目标。但是，书中不可避免会使用一些形式化定义和算法的抽象描述，因此，笔者尽所能通过图例进行解释（本书共320张插图）。不过，本书所包含的内容较为广泛，难免会有疏漏，望读者海涵，并指出不当之处。
%-------------------------------------------
\begin{figure}[htp]
\centering
\centering
\input{./ChapterPreface/Figures/figure-preface}
\caption{本书各章节及核心概念关系图}
\label{fig:preface}
\end{figure}
%-------------------------------------------
\end{spacing}
......
Book/ChapterPreface/Figures/figure-preface.tex 0 → 100644
% !Mode:: "TeX:UTF-8"
% !TEX encoding = UTF-8 Unicode
\begin{tikzpicture}
\tikzstyle{secnode} =[font=\scriptsize,minimum height=4.0em,minimum width=22em,draw,thick,fill=white,drop shadow]
\tikzstyle{conceptnode} =[font=\scriptsize,minimum height=1.5em,minimum width=5em]
\tikzstyle{conceptnodesmall} =[font=\scriptsize,minimum height=1.0em,minimum width=4.4em]
% section 1
\node [secnode,anchor=south west,minimum width=22.5em,red,fill=white] (sec1) at (0,0) {};
\node [anchor=north] (sec1label) at ([yshift=-0.2em]sec1.north) {\small{机器翻译简介}};
\node [anchor=north west,draw=red,thick,fill=white,rounded corners] (sec1title) at ([xshift=-0.3em,yshift=0.3em]sec1.north west) {{\footnotesize\bfnew{\color{red} 第一章}}};
\node [conceptnode,anchor=south west,fill=red!15,thin] (sec1box1) at ([xshift=0.5em,yshift=0.5em]sec1.south west) {\footnotesize{发展历史}};
\node [conceptnode,anchor=west,fill=red!15,thin] (sec1box2) at ([xshift=0.5em]sec1box1.east) {\footnotesize{评价方法}};
\node [conceptnode,anchor=west,fill=red!15,thin] (sec1box3) at ([xshift=0.5em]sec1box2.east) {\footnotesize{应用情况}};
\node [conceptnode,anchor=west,fill=red!15,thin] (sec1box4) at ([xshift=0.5em]sec1box3.east) {\footnotesize{系统\&数据}};
% section 2
\node [secnode,anchor=south,blue,fill=white] (sec2) at ([xshift=-6.5em,yshift=3em]sec1.north) {};
\node [anchor=north] (sec2label) at (sec2.north) {\small{词法、语法及统计建模基础}};
\node [anchor=north west,draw=blue,thick,fill=white,rounded corners] (sec2title) at ([xshift=-0.3em,yshift=0.3em]sec2.north west) {{\footnotesize\bfnew{\color{blue} 第二章}}};
\node [conceptnode,anchor=south west,fill=ublue!15,thin,minimum width=4em,align=left] (sec2box1) at ([xshift=0.5em,yshift=0.4em]sec2.south west) {\tiny{概率论与}\\\tiny{统计建模基础}};
\node [anchor=west,draw,dotted,thick,minimum height=2em,minimum width=16.6em,align=left] (sec2box2) at ([xshift=0.3em]sec2box1.east) {};
\node [conceptnodesmall,minimum width=5em,anchor=south west,fill=blue!15,thin] (sec2box3) at ([xshift=0.4em,yshift=0.3em]sec2box2.south west) {\scriptsize{中文分词}};
\node [conceptnodesmall,minimum width=5em,anchor=west,fill=blue!15,thin] (sec2box4) at ([xshift=0.4em]sec2box3.east) {\scriptsize{$n$元语法模型}};
\node [conceptnodesmall,minimum width=5em,anchor=west,fill=blue!15,thin] (sec2box5) at ([xshift=0.4em]sec2box4.east) {\scriptsize{句法分析}};
\draw [->,very thick] ([xshift=-1em,yshift=0.2em]sec1.north) .. controls +(north:2.5em) and +(south:2.5em) .. ([xshift=-3em,yshift=-0.2em]sec2.south);
% section 5
\node [secnode,anchor=south,orange,fill=white] (sec5) at ([xshift=7em,yshift=10em]sec1.north) {};
\node [anchor=north] (sec5label) at (sec5.north) {\small{人工神经网络和神经语言建模}};
\node [anchor=north west,draw=orange,thick,fill=white,rounded corners] (sec5title) at ([xshift=-0.3em,yshift=0.3em]sec5.north west) {{\footnotesize\bfnew{\color{orange} 第五章}}};
\node [conceptnode,minimum width=4em,anchor=south west,fill=orange!15,thin,align=left] (sec5box1) at ([xshift=0.5em,yshift=0.4em]sec5.south west) {\tiny{线性代数基础}\\\tiny{与感知机}};
\node [conceptnode,minimum width=4em,anchor=west,fill=orange!15,thin,align=left] (sec5box2) at ([xshift=0.2em]sec5box1.east) {\tiny{多层神经网络}\\\tiny{与实现方法}};
\node [conceptnode,minimum width=4em,anchor=west,fill=orange!15,thin,align=left] (sec5box3) at ([xshift=0.2em]sec5box2.east) {\tiny{模型训练}\\\tiny{(反向传播)}};
\node [conceptnode,minimum width=4em,anchor=west,fill=purple!15,thin,align=left] (sec5box4) at ([xshift=0.2em]sec5box3.east) {\tiny{神经语言模型}\\\tiny{(FNN等)}};
\node [conceptnode,minimum width=4em,anchor=west,fill=purple!15,thin,align=left] (sec5box5) at ([xshift=0.2em]sec5box4.east) {\tiny{表示学习与}\\\tiny{预训练模型}};
\node [draw,dotted,thick,inner sep=1pt] [fit = (sec5box4) (sec5box5)] (pretrainbox) {};
\draw [->,very thick] ([yshift=-9.8em]sec5.south) -- ([yshift=-0.2em]sec5.south);
\draw [->,thick,dotted] ([xshift=0.2em,yshift=1em]sec2.east) .. controls +(east:3em) and +(south:4em) .. ([xshift=3em,yshift=-0.0em]pretrainbox.south);
% section 3
\node [secnode,anchor=south,ugreen,fill=white] (sec3) at ([yshift=10em]sec2.north) {};
\node [anchor=north] (sec3label) at (sec3.north) {\small{基于词的机器翻译模型}};
\node [anchor=north west,draw=ugreen,thick,fill=white,rounded corners] (sec3title) at ([xshift=-0.3em,yshift=0.3em]sec3.north west) {{\footnotesize\bfnew{\color{ugreen} 第三章}}};
\node [conceptnode,minimum width=4em,anchor=south west,fill=ublue!15,thin,align=left] (sec3box1) at ([xshift=0.5em,yshift=0.4em]sec3.south west) {\tiny{机器翻译的统计}\\\tiny{描述（实例）}};
\node [conceptnode,minimum width=4em,anchor=west,fill=green!20,thin,align=left] (sec3box2) at ([xshift=0.2em]sec3box1.east) {\tiny{噪声信道模型}\\\tiny{与生成式模型}};
\node [conceptnode,minimum width=4em,anchor=west,fill=green!20,thin,align=left] (sec3box3) at ([xshift=0.2em]sec3box2.east) {\tiny{IBM模型、隐}\\\tiny{马尔可夫模型}};
\node [conceptnode,minimum width=4em,anchor=west,fill=green!20,thin,align=left] (sec3box4) at ([xshift=0.2em]sec3box3.east) {\tiny{\hspace{0.9em}参数学习}\\\tiny{=优化}};
\node [conceptnode,minimum width=3.8em,anchor=west,fill=green!20,thin,align=left,minimum height=2em,inner sep=2pt] (sec3box5) at ([xshift=0.2em]sec3box4.east) {\scriptsize{EM算法}};
\draw [->,very thick] ([yshift=0.2em,xshift=-3em]sec2.north) -- ([yshift=-0.2em,xshift=-3em]sec3.south);
% section 4
\node [secnode,anchor=south,ugreen,fill=white] (sec4) at ([yshift=3em]sec3.north) {};
\node [anchor=north] (sec4label) at (sec4.north) {\small{基于短语和句法的机器翻译模型}};
\node [anchor=north west,draw=ugreen,thick,fill=white,rounded corners] (sec4title) at ([xshift=-0.3em,yshift=0.3em]sec4.north west) {{\footnotesize\bfnew{\color{ugreen} 第四章}}};
\node [conceptnode,minimum width=4em,anchor=south west,fill=ublue!15,thin,align=left] (sec4box1) at ([xshift=0.5em,yshift=0.4em]sec4.south west) {\tiny{判别式模型与}\\\tiny{最小错误率训练}};
\node [conceptnode,minimum width=4em,anchor=west,fill=green!20,thin,align=left] (sec4box2) at ([xshift=0.2em]sec4box1.east) {\tiny{基于翻译推导}\\\tiny{的建模}};
\node [conceptnode,minimum width=4em,anchor=west,fill=green!20,thin,align=left] (sec4box3) at ([xshift=0.2em]sec4box2.east) {\tiny{短语及句法}\\\tiny{翻译规则抽取}};
\node [conceptnode,minimum width=4em,anchor=west,fill=green!20,thin,align=left,minimum height=2em,inner sep=2pt] (sec4box4) at ([xshift=0.2em]sec4box3.east) {\scriptsize{调序模型}};
\node [conceptnode,minimum width=3.8em,anchor=west,fill=green!20,thin,align=left,minimum height=2em,inner sep=2pt] (sec4box5) at ([xshift=0.2em]sec4box4.east) {\scriptsize{解码}};
\draw [->,very thick] ([yshift=0.2em,xshift=-3em]sec3.north) -- ([yshift=-0.2em,xshift=-3em]sec4.south);
% section 6
\node [secnode,anchor=south,purple,fill=white] (sec6) at ([yshift=19em]sec5.north) {};
\node [anchor=north] (sec6label) at (sec6.north) {\small{神经机器翻译模型}};
\node [anchor=north west,draw=purple,thick,fill=white,rounded corners] (sec6title) at ([xshift=-0.3em,yshift=0.3em]sec6.north west) {{\footnotesize\bfnew{\color{purple} 第六章}}};
\node [conceptnode,minimum width=4em,anchor=south west,fill=ublue!15,thin,align=left] (sec6box1) at ([xshift=0.5em,yshift=0.4em]sec6.south west) {\tiny{编码器-解码器}\\\tiny{框架}};
\node [conceptnode,minimum width=4em,anchor=west,fill=ublue!15,thin,align=left,minimum height=2em] (sec6box2) at ([xshift=0.2em]sec6box1.east) {\scriptsize{注意力机制}};
\node [conceptnode,minimum width=7.5em,anchor=west,fill=purple!15,thin,align=left] (sec6box3) at ([xshift=0.2em]sec6box2.east) {\tiny{基于RNN和Transformer}\\\tiny{的神经机器翻译建模}};
\node [conceptnode,minimum width=4em,anchor=west,fill=purple!15,thin,align=left,minimum height=2em] (sec6box4) at ([xshift=0.2em]sec6box3.east) {\scriptsize{训练与推断}};
\draw [->,very thick] ([yshift=0.2em]sec5.north) -- ([yshift=-0.2em]sec6.south);
\draw [->,very thick,dotted] ([yshift=0.2em,xshift=-2em]sec4.north) .. controls +(north:5.0em) and +(west:4em) .. ([xshift=-0.2em]sec6.west);
\draw [->,thick,dotted] ([xshift=3em,yshift=0.2em]pretrainbox.north) .. controls +(north:15em) and +(south:15em) .. ([xshift=0em,yshift=-0.0em]sec6box3.south);
% section 7
\node [secnode,anchor=south,purple,fill=white,minimum height=6.3em] (sec7) at ([yshift=3em]sec6.north) {};
\node [anchor=north] (sec7label) at (sec7.north) {\small{神经机器翻译实战\ \dash \ 参加一次比赛}};
\node [anchor=north west,draw=purple,thick,fill=white,rounded corners] (sec7title) at ([xshift=-0.3em,yshift=0.3em]sec7.north west) {{\footnotesize\bfnew{\color{purple} 第七章}}};
\node [conceptnode,minimum width=4em,anchor=south west,fill=ublue!15,thin,align=left,minimum height=4.2em] (sec7box1) at ([xshift=0.5em,yshift=0.4em]sec7.south west) {\tiny{数据处理、}\\\tiny{子词切分}};
\node [anchor=north west,minimum width=5em,anchor=north west,fill=purple!15] (sec7box2) at ([xshift=0.5em,yshift=-0.2em]sec7box1.north east) {\tiny{正则化}};
\node [anchor=north west,minimum width=5em,anchor=north west,fill=purple!15] (sec7box3) at ([yshift=-0.1em]sec7box2.south west) {\tiny{增大模型容量}};
\node [anchor=north west,minimum width=5em,anchor=north west,fill=purple!15] (sec7box4) at ([yshift=-0.1em]sec7box3.south west) {\tiny{大批量训练}};
\node [anchor=north west,minimum width=5em,anchor=north west,fill=purple!15] (sec7box5) at ([xshift=0.6em]sec7box2.north east) {\tiny{推断优化}};
\node [anchor=north west,minimum width=5em,anchor=north west,fill=purple!15] (sec7box6) at ([yshift=-0.1em]sec7box5.south west) {\tiny{译文长度控制}};
\node [anchor=north west,minimum width=5em,anchor=north west,fill=purple!15] (sec7box7) at ([yshift=-0.1em]sec7box6.south west) {\tiny{多模型集成}};
\node [anchor=north west,minimum width=5em,anchor=north west,fill=purple!15] (sec7box8) at ([xshift=0.6em]sec7box5.north east) {\tiny{深层模型}};
\node [anchor=north west,minimum width=5em,anchor=north west,fill=purple!15] (sec7box9) at ([yshift=-0.1em]sec7box8.south west) {\tiny{知识精炼}};
\node [anchor=north west,minimum width=5em,anchor=north west,fill=purple!15] (sec7box10) at ([yshift=-0.1em]sec7box9.south west) {\tiny{单语数据使用}};
\node [draw,dotted,thick,inner sep=1pt] [fit = (sec7box2) (sec7box3) (sec7box4)] (trainbox) {};
\node [draw,dotted,thick,inner sep=1pt] [fit = (sec7box5) (sec7box6) (sec7box7)] (inferencebox) {};
\node [draw,dotted,thick,inner sep=1pt] [fit = (sec7box8) (sec7box9) (sec7box10)] (advancedbox) {};
\draw [->,very thick] ([yshift=0.2em]sec6.north) -- ([yshift=-0.2em]sec7.south);
\draw [->,very thick,dotted] ([yshift=0.2em,xshift=-3em]sec4.north) .. controls +(north:7.0em) and +(west:6em) .. ([xshift=-0.2em]sec7.west);
\end{tikzpicture}
Book/bibliography.bib
......@@ -685,6 +685,12 @@
year ={2016},
publisher ={清华大学出版社}
}
@book{李航2012统计学习方法,
title ={统计学习方法},
author ={李航},
year ={2012},
publisher ={清华大学出版社}
}
@book{宗成庆2013统计自然语言处理,
title ={统计自然语言处理},
author ={宗成庆},
......@@ -1289,18 +1295,23 @@
//bibsource = {dblp computer science bibliography, https://dblp.org}
}
@article{crammer2003ultraconservative,
author = {Koby Crammer and
Yoram Singer},
title = {Ultraconservative Online Algorithms for Multiclass Problems},
journal = {Journal of Machine Learning Research},
volume = {3},
pages = {951--991},
year = {2003},
//url = {http://jmlr.org/papers/v3/crammer03a.html},
//timestamp = {Wed, 10 Jul 2019 15:28:37 +0200},
//biburl = {https://dblp.org/rec/journals/jmlr/CrammerS03.bib},
//bibsource = {dblp computer science bibliography, https://dblp.org}
@inproceedings{DBLP:conf/emnlp/ChiangMR08,
author = {David Chiang and
Yuval Marton and
Philip Resnik},
title = {Online Large-Margin Training of Syntactic and Structural Translation
Features},
booktitle = {2008 Conference on Empirical Methods in Natural Language Processing,
{EMNLP} 2008, Proceedings of the Conference, 25-27 October 2008, Honolulu,
Hawaii, USA, {A} meeting of SIGDAT, a Special Interest Group of the
{ACL}},
pages = {224--233},
publisher = {{ACL}},
year = {2008},
url = {https://www.aclweb.org/anthology/D08-1024/},
timestamp = {Fri, 13 Sep 2019 13:08:45 +0200},
biburl = {https://dblp.org/rec/conf/emnlp/ChiangMR08.bib},
bibsource = {dblp computer science bibliography, https://dblp.org}
}
@inproceedings{dreyer2015apro,
......@@ -1798,21 +1809,23 @@ year ={2008},
//biburl = {https://dblp.org/rec/conf/naacl/OchGKSYFKSSEJJR04.bib},
//bibsource = {dblp computer science bibliography, https://dblp.org}
}
@article{OchA,
@inproceedings{DBLP:conf/coling/OchN00,
author = {Franz Josef Och and
Hermann Ney},
title = {A Systematic Comparison of Various Statistical Alignment Models},
journal = {Computational Linguistics},
volume = {29},
number = {1},
pages = {19--51},
year = {2003},
//url = {https://doi.org/10.1162/089120103321337421},
//doi = {10.1162/089120103321337421},
//timestamp = {Tue, 21 May 2019 18:03:32 +0200},
//biburl = {https://dblp.org/rec/journals/coling/OchN03.bib},
//bibsource = {dblp computer science bibliography, https://dblp.org}
title = {A Comparison of Alignment Models for Statistical Machine Translation},
booktitle = {{COLING} 2000, 18th International Conference on Computational Linguistics,
Proceedings of the Conference, 2 Volumes, July 31 - August 4, 2000,
Universit{\"{a}}t des Saarlandes, Saarbr{\"{u}}cken, Germany},
pages = {1086--1090},
publisher = {Morgan Kaufmann},
year = {2000},
url = {https://www.aclweb.org/anthology/C00-2163/},
timestamp = {Mon, 16 Sep 2019 17:08:53 +0200},
biburl = {https://dblp.org/rec/conf/coling/OchN00.bib},
bibsource = {dblp computer science bibliography, https://dblp.org}
}
@article{powell1964an,
author = {M. J. D. Powell},
title = {An efficient method for finding the minimum of a function of several
......@@ -1942,14 +1955,40 @@ year ={2008},
//biburl = {https://dblp.org/rec/conf/naacl/TaskarLK05.bib},
//bibsource = {dblp computer science bibliography, https://dblp.org}
}
@article{tinsley2007robust,
author ={Tinsley, John and Zhechev, Ventsislav and Hearne, Mary and Way, Andy},
title ={Robust language pair-independent sub-tree alignment},
journal ={Machine Translation Summit XI},
pages ={467–474},
year ={2007},
//url ={http://doras.dcu.ie/15230/},
@inproceedings{DBLP:conf/coling/GrovesHW04,
author = {Declan Groves and
Mary Hearne and
Andy Way},
title = {Robust Sub-Sentential Alignment of Phrase-Structure Trees},
booktitle = {{COLING} 2004, 20th International Conference on Computational Linguistics,
Proceedings of the Conference, 23-27 August 2004, Geneva, Switzerland},
year = {2004},
url = {https://www.aclweb.org/anthology/C04-1154/},
timestamp = {Mon, 16 Sep 2019 17:08:53 +0200},
biburl = {https://dblp.org/rec/conf/coling/GrovesHW04.bib},
bibsource = {dblp computer science bibliography, https://dblp.org}
}
@inproceedings{DBLP:conf/coling/SunZT10,
author = {Jun Sun and
Min Zhang and
Chew Lim Tan},
editor = {Chu{-}Ren Huang and
Dan Jurafsky},
title = {Discriminative Induction of Sub-Tree Alignment using Limited Labeled
Data},
booktitle = {{COLING} 2010, 23rd International Conference on Computational Linguistics,
Proceedings of the Conference, 23-27 August 2010, Beijing, China},
pages = {1047--1055},
publisher = {Tsinghua University Press},
year = {2010},
url = {https://www.aclweb.org/anthology/C10-1118/},
timestamp = {Mon, 16 Sep 2019 17:08:53 +0200},
biburl = {https://dblp.org/rec/conf/coling/SunZT10.bib},
bibsource = {dblp computer science bibliography, https://dblp.org}
}
@inproceedings{Tong2009Better,
author = {Tong Xiao and
Mu Li and
......@@ -1968,6 +2007,33 @@ year ={2008},
//biburl = {https://dblp.org/rec/conf/emnlp/XiaoLZZZ09.bib},
//bibsource = {dblp computer science bibliography, https://dblp.org}
}
@inproceedings{DBLP:conf/acl/KleinM03,
author = {Dan Klein and
Christopher D. Manning},
editor = {Erhard W. Hinrichs and
Dan Roth},
title = {Accurate Unlexicalized Parsing},
booktitle = {Proceedings of the 41st Annual Meeting of the Association for Computational
Linguistics, 7-12 July 2003, Sapporo Convention Center, Sapporo, Japan},
pages = {423--430},
publisher = {{ACL}},
year = {2003},
url = {https://www.aclweb.org/anthology/P03-1054/},
timestamp = {Mon, 19 Aug 2019 18:09:53 +0200},
biburl = {https://dblp.org/rec/conf/acl/KleinM03.bib},
bibsource = {dblp computer science bibliography, https://dblp.org}
}
@inproceedings{charniak2006multilevel,
title="Multilevel Coarse-to-Fine PCFG Parsing",
author="Eugene {Charniak} and Mark {Johnson} and Micha {Elsner} and Joseph {Austerweil} and David {Ellis} and Isaac {Haxton} and Catherine {Hill} and R. {Shrivaths} and Jeremy {Moore} and Michael {Pozar} and Theresa {Vu}",
booktitle="Proceedings of the Human Language Technology Conference of the NAACL, Main Conference",
pages="168--175",
notes="Sourced from Microsoft Academic - https://academic.microsoft.com/paper/2101714644",
year="2006"
}
@inproceedings{Tong2016Syntactic,
author = {Tong Xiao and
Jingbo Zhu and
......@@ -2281,7 +2347,7 @@ year ={2008},
Liang Huang and
Daniel Gildea and
Kevin Knight},
editor = {Robert C. Moore and
//editor = {Robert C. Moore and
Jeff A. Bilmes and
Jennifer Chu{-}Carroll and
Mark Sanderson},
......@@ -2396,23 +2462,27 @@ year ={2008},
//biburl = {https://dblp.org/rec/journals/jmlr/CollobertWBKKK11.bib},
//bibsource = {dblp computer science bibliography, https://dblp.org}
}
@article{devlin2018bert,
@inproceedings{DBLP:conf/naacl/DevlinCLT19,
author = {Jacob Devlin and
Ming{-}Wei Chang and
Kenton Lee and
Kristina Toutanova},
title = {{BERT:} Pre-training of Deep Bidirectional Transformers for Language
Understanding},
journal = {CoRR},
volume = {abs/1810.04805},
year = {2018},
//url = {http://arxiv.org/abs/1810.04805},
//archivePrefix = {arXiv},
//eprint = {1810.04805},
//timestamp = {Tue, 30 Oct 2018 20:39:56 +0100},
//biburl = {https://dblp.org/rec/journals/corr/abs-1810-04805.bib},
booktitle = {Proceedings of the 2019 Conference of the North American Chapter of
the Association for Computational Linguistics: Human Language Technologies,
{NAACL-HLT} 2019, Minneapolis, MN, USA, June 2-7, 2019, Volume 1 (Long
and Short Papers)},
pages = {4171--4186},
publisher = {Association for Computational Linguistics},
year = {2019},
//url = {https://doi.org/10.18653/v1/n19-1423},
//doi = {10.18653/v1/n19-1423},
//timestamp = {Tue, 28 Jan 2020 10:30:29 +0100},
//biburl = {https://dblp.org/rec/conf/naacl/DevlinCLT19.bib},
//bibsource = {dblp computer science bibliography, https://dblp.org}
}
@article{duchi2011adaptive,
author = {John C. Duchi and
Elad Hazan and
......@@ -2731,8 +2801,6 @@ year ={2008},
Tao Qin and
Jianfeng Lu and
Tie{-}Yan Liu},
editor = {Kamalika Chaudhuri and
Ruslan Salakhutdinov},
title = {{MASS:} Masked Sequence to Sequence Pre-training for Language Generation},
booktitle = {Proceedings of the 36th International Conference on Machine Learning,
{ICML} 2019, 9-15 June 2019, Long Beach, California, {USA}},
......@@ -2850,22 +2918,6 @@ year ={2008},
year={1992},
publisher={MIT Press}
}
@article{bengio2003a,
title={A neural probabilistic language model},
author={Bengio and
Yoshua and
Ducharme and
Rejean and
Vincent and
Pascal and
Janvin and
Christian},
journal={Journal of Machine Learning Research},
volume={3},
number={6},
pages={1137--1155},
year={2003}
}
@inproceedings{mikolov2012context,
title={Context dependent recurrent neural network language model},
author={Mikolov and
......@@ -4171,29 +4223,21 @@ pages ={157-166},
pages={72-78},
year={2011},
}
%%%%%%%%%%%%%%%
@misc{provilkov2019bpedropout,
title={BPE-Dropout: Simple and Effective Subword Regularization},
author={Ivan Provilkov and Dmitrii Emelianenko and Elena Voita},
year={2019},
//eprint={1910.13267},
//archivePrefix={arXiv},
//primaryClass={cs.CL}
}
%%%%%%%%%%%%%%%%%%%
@article{DBLP:journals/corr/SennrichHB15,
@inproceedings{DBLP:conf/acl/SennrichHB16a,
author = {Rico Sennrich and
Barry Haddow and
Alexandra Birch},
title = {Neural Machine Translation of Rare Words with Subword Units},
journal = {CoRR},
volume = {abs/1508.07909},
year = {2015},
//url = {http://arxiv.org/abs/1508.07909},
//archivePrefix = {arXiv},
//eprint = {1508.07909},
//timestamp = {Mon, 13 Aug 2018 16:47:17 +0200},
//biburl = {https://dblp.org/rec/journals/corr/SennrichHB15.bib},
booktitle = {Proceedings of the 54th Annual Meeting of the Association for Computational
Linguistics, {ACL} 2016, August 7-12, 2016, Berlin, Germany, Volume
1: Long Papers},
publisher = {The Association for Computer Linguistics},
year = {2016},
//url = {https://doi.org/10.18653/v1/p16-1162},
//doi = {10.18653/v1/p16-1162},
//timestamp = {Tue, 28 Jan 2020 10:28:06 +0100},
//biburl = {https://dblp.org/rec/conf/acl/SennrichHB16a.bib},
//bibsource = {dblp computer science bibliography, https://dblp.org}
}
......@@ -4225,26 +4269,36 @@ pages ={157-166},
year={1989},
}
@article{DBLP:journals/corr/abs-1809-10853,
@inproceedings{DBLP:conf/iclr/BaevskiA19,
author = {Alexei Baevski and
Michael Auli},
title = {Adaptive Input Representations for Neural Language Modeling},
journal = {CoRR},
volume = {abs/1809.10853},
year = {2018},
//url = {http://arxiv.org/abs/1809.10853},
/archivePrefix = {arXiv},
//eprint = {1809.10853},
//timestamp = {Fri, 05 Oct 2018 11:34:52 +0200},
//biburl = {https://dblp.org/rec/journals/corr/abs-1809-10853.bib},
booktitle = {7th International Conference on Learning Representations, {ICLR} 2019,
New Orleans, LA, USA, May 6-9, 2019},
publisher = {OpenReview.net},
year = {2019},
//url = {https://openreview.net/forum?id=ByxZX20qFQ},
//timestamp = {Thu, 25 Jul 2019 14:26:00 +0200},
//biburl = {https://dblp.org/rec/conf/iclr/BaevskiA19.bib},
//bibsource = {dblp computer science bibliography, https://dblp.org}
}
@inproceedings{Stahlberg2019OnNS,
title={On NMT Search Errors and Model Errors: Cat Got Your Tongue?},
author={Felix Stahlberg and Bill Byrne},
booktitle={EMNLP/IJCNLP},
year={2019}
@inproceedings{DBLP:conf/emnlp/StahlbergB19,
author = {Felix Stahlberg and
Bill Byrne},
title = {On {NMT} Search Errors and Model Errors: Cat Got Your Tongue?},
booktitle = {Proceedings of the 2019 Conference on Empirical Methods in Natural
Language Processing and the 9th International Joint Conference on
Natural Language Processing, {EMNLP-IJCNLP} 2019, Hong Kong, China,
November 3-7, 2019},
pages = {3354--3360},
publisher = {Association for Computational Linguistics},
year = {2019},
//url = {https://doi.org/10.18653/v1/D19-1331},
//doi = {10.18653/v1/D19-1331},
//timestamp = {Thu, 12 Dec 2019 13:23:43 +0100},
//biburl = {https://dblp.org/rec/conf/emnlp/StahlbergB19.bib},
//bibsource = {dblp computer science bibliography, https://dblp.org}
}
@article{DBLP:journals/corr/abs-1810-08398,
......@@ -4270,37 +4324,40 @@ pages ={157-166},
//bibsource = {dblp computer science bibliography, https://dblp.org}
}
@article{DBLP:journals/corr/StahlbergHSB17,
@inproceedings{DBLP:conf/emnlp/StahlbergHSB17,
author = {Felix Stahlberg and
Eva Hasler and
Danielle Saunders and
Bill Byrne},
title = {{SGNMT} - {A} Flexible {NMT} Decoding Platform for Quick Prototyping
of New Models and Search Strategies},
journal = {CoRR},
volume = {abs/1707.06885},
booktitle = {Proceedings of the 2017 Conference on Empirical Methods in Natural
Language Processing, {EMNLP} 2017, Copenhagen, Denmark, September
9-11, 2017 - System Demonstrations},
pages = {25--30},
publisher = {Association for Computational Linguistics},
year = {2017},
//url = {http://arxiv.org/abs/1707.06885},
//archivePrefix = {arXiv},
//eprint = {1707.06885},
//timestamp = {Mon, 13 Aug 2018 16:48:37 +0200},
//biburl = {https://dblp.org/rec/journals/corr/StahlbergHSB17.bib},
//url = {https://doi.org/10.18653/v1/d17-2005},
//doi = {10.18653/v1/d17-2005},
//timestamp = {Tue, 28 Jan 2020 10:28:17 +0100},
//biburl = {https://dblp.org/rec/conf/emnlp/StahlbergHSB17.bib},
//bibsource = {dblp computer science bibliography, https://dblp.org}
}
@article{DBLP:journals/corr/SennrichHB16,
@inproceedings{DBLP:conf/wmt/SennrichHB16,
author = {Rico Sennrich and
Barry Haddow and
Alexandra Birch},
title = {Edinburgh Neural Machine Translation Systems for {WMT} 16},
journal = {CoRR},
volume = {abs/1606.02891},
booktitle = {Proceedings of the First Conference on Machine Translation, {WMT}
2016, colocated with {ACL} 2016, August 11-12, Berlin, Germany},
pages = {371--376},
publisher = {The Association for Computer Linguistics},
year = {2016},
//url = {http://arxiv.org/abs/1606.02891},
//archivePrefix = {arXiv},
//eprint = {1606.02891},
//timestamp = {Mon, 13 Aug 2018 16:46:23 +0200},
//biburl = {https://dblp.org/rec/journals/corr/SennrichHB16.bib},
//url = {https://doi.org/10.18653/v1/w16-2323},
//doi = {10.18653/v1/w16-2323},
//timestamp = {Tue, 28 Jan 2020 10:31:04 +0100},
//biburl = {https://dblp.org/rec/conf/wmt/SennrichHB16.bib},
//bibsource = {dblp computer science bibliography, https://dblp.org}
}
......@@ -4322,23 +4379,6 @@ pages ={157-166},
Qiang Wang and
Tong Xiao and
Jingbo Zhu},
editor = {Ondrej Bojar and
Rajen Chatterjee and
Christian Federmann and
Mark Fishel and
Yvette Graham and
Barry Haddow and
Matthias Huck and
Antonio Jimeno{-}Yepes and
Philipp Koehn and
Andr{\'{e}} Martins and
Christof Monz and
Matteo Negri and
Aur{\'{e}}lie N{\'{e}}v{\'{e}}ol and
Mariana L. Neves and
Matt Post and
Marco Turchi and
Karin Verspoor},
title = {The NiuTrans Machine Translation Systems for {WMT19}},
booktitle = {Proceedings of the Fourth Conference on Machine Translation, {WMT}
2019, Florence, Italy, August 1-2, 2019 - Volume 2: Shared Task Papers,
......@@ -4373,7 +4413,7 @@ pages ={157-166},
//bibsource = {dblp computer science bibliography, https://dblp.org}
}
@article{DBLP:journals/corr/abs-1712-05877,
@inproceedings{DBLP:conf/cvpr/JacobKCZTHAK18,
author = {Benoit Jacob and
Skirmantas Kligys and
Bo Chen and
......@@ -4384,14 +4424,15 @@ pages ={157-166},
Dmitry Kalenichenko},
title = {Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only
Inference},
journal = {CoRR},
volume = {abs/1712.05877},
year = {2017},
//url = {http://arxiv.org/abs/1712.05877},
//archivePrefix = {arXiv},
//eprint = {1712.05877},
//timestamp = {Mon, 13 Aug 2018 16:48:27 +0200},
//biburl = {https://dblp.org/rec/journals/corr/abs-1712-05877.bib},
booktitle = {2018 {IEEE} Conference on Computer Vision and Pattern Recognition,
{CVPR} 2018, Salt Lake City, UT, USA, June 18-22, 2018},
pages = {2704--2713},
publisher = {{IEEE} Computer Society},
year = {2018},
//url = {http://openaccess.thecvf.com/content\_cvpr\_2018/html/Jacob\_Quantization\_and\_Training\_CVPR\_2018\_paper.html},
//doi = {10.1109/CVPR.2018.00286},
//timestamp = {Wed, 16 Oct 2019 14:14:50 +0200},
//biburl = {https://dblp.org/rec/conf/cvpr/JacobKCZTHAK18.bib},
//bibsource = {dblp computer science bibliography, https://dblp.org}
}
......@@ -4411,7 +4452,7 @@ pages ={157-166},
//bibsource = {dblp computer science bibliography, https://dblp.org}
}
@article{DBLP:journals/corr/abs-1801-05122,
@inproceedings{DBLP:conf/aaai/ZhangSQLJW18,
author = {Xiangwen Zhang and
Jinsong Su and
Yue Qin and
......@@ -4419,31 +4460,36 @@ pages ={157-166},
Rongrong Ji and
Hongji Wang},
title = {Asynchronous Bidirectional Decoding for Neural Machine Translation},
journal = {CoRR},
volume = {abs/1801.05122},
booktitle = {Proceedings of the Thirty-Second {AAAI} Conference on Artificial Intelligence,
(AAAI-18), the 30th innovative Applications of Artificial Intelligence
(IAAI-18), and the 8th {AAAI} Symposium on Educational Advances in
Artificial Intelligence (EAAI-18), New Orleans, Louisiana, USA, February
2-7, 2018},
pages = {5698--5705},
publisher = {{AAAI} Press},
year = {2018},
//url = {http://arxiv.org/abs/1801.05122},
//archivePrefix = {arXiv},
//eprint = {1801.05122},
//timestamp = {Mon, 15 Jul 2019 14:17:41 +0200},
//biburl = {https://dblp.org/rec/journals/corr/abs-1801-05122.bib},
//url = {https://www.aaai.org/ocs/index.php/AAAI/AAAI18/paper/view/16784},
//timestamp = {Sun, 31 Mar 2019 12:09:17 +0200},
//biburl = {https://dblp.org/rec/conf/aaai/ZhangSQLJW18.bib},
//bibsource = {dblp computer science bibliography, https://dblp.org}
}
@article{DBLP:journals/corr/abs-1809-00069,
@inproceedings{DBLP:conf/emnlp/HuangZM17,
author = {Liang Huang and
Kai Zhao and
Mingbo Ma},
title = {When to Finish? Optimal Beam Search for Neural Text Generation (modulo
beam size)},
journal = {CoRR},
volume = {abs/1809.00069},
year = {2018},
//url = {http://arxiv.org/abs/1809.00069},
//archivePrefix = {arXiv},
//eprint = {1809.00069},
//timestamp = {Fri, 05 Oct 2018 11:34:52 +0200},
//biburl = {https://dblp.org/rec/journals/corr/abs-1809-00069.bib},
booktitle = {Proceedings of the 2017 Conference on Empirical Methods in Natural
Language Processing, {EMNLP} 2017, Copenhagen, Denmark, September
9-11, 2017},
pages = {2134--2139},
publisher = {Association for Computational Linguistics},
year = {2017},
//url = {https://doi.org/10.18653/v1/d17-1227},
//doi = {10.18653/v1/d17-1227},
//timestamp = {Tue, 28 Jan 2020 10:28:22 +0100},
//biburl = {https://dblp.org/rec/conf/emnlp/HuangZM17.bib},
//bibsource = {dblp computer science bibliography, https://dblp.org}
}
......@@ -4452,7 +4498,7 @@ pages ={157-166},
Robert E. Schapire},
title = {A Decision-Theoretic Generalization of On-Line Learning and an Application
to Boosting},
journal = {J. Comput. Syst. Sci.},
journal = {Journal of Computer and System Sciences},
volume = {55},
number = {1},
pages = {119--139},
......@@ -4469,9 +4515,6 @@ pages ={157-166},
Jingbo Zhu and
Muhua Zhu and
Huizhen Wang},
editor = {Jan Hajic and
Sandra Carberry and
Stephen Clark},
title = {Boosting-Based System Combination for Machine Translation},
booktitle = {{ACL} 2010, Proceedings of the 48th Annual Meeting of the Association
for Computational Linguistics, July 11-16, 2010, Uppsala, Sweden},
......@@ -4509,7 +4552,7 @@ pages ={157-166},
author = {Antti{-}Veikko I. Rosti and
Spyridon Matsoukas and
Richard M. Schwartz},
editor = {John A. Carroll and
//editor = {John A. Carroll and
Antal van den Bosch and
Annie Zaenen},
title = {Improved Word-Level System Combination for Machine Translation},
......@@ -4528,7 +4571,7 @@ pages ={157-166},
Bing Zhang and
Spyros Matsoukas and
Richard M. Schwartz},
editor = {Chris Callison{-}Burch and
//editor = {Chris Callison{-}Burch and
Philipp Koehn and
Christof Monz and
Josh Schroeder and
......@@ -4588,8 +4631,6 @@ pages ={157-166},
Rongrong Ji and
Xiaodong Shi and
Yang Liu},
editor = {Satinder P. Singh and
Shaul Markovitch},
title = {Lattice-Based Recurrent Neural Network Encoders for Neural Machine
Translation},
booktitle = {Proceedings of the Thirty-First {AAAI} Conference on Artificial Intelligence,
......@@ -4603,17 +4644,20 @@ pages ={157-166},
//bibsource = {dblp computer science bibliography, https://dblp.org}
}
@inproceedings{DBLP:conf/acl/BirdL04,
author = {Steven Bird and
Edward Loper},
@inproceedings{DBLP:conf/acl/Bird06,
author = {Steven Bird},
//editor = {Nicoletta Calzolari and
Claire Cardie and
Pierre Isabelle},
title = {{NLTK:} The Natural Language Toolkit},
booktitle = {Proceedings of the 42nd Annual Meeting of the Association for Computational
Linguistics, Barcelona, Spain, July 21-26, 2004 - Poster and Demonstration},
publisher = {{ACL}},
year = {2004},
//url = {https://www.aclweb.org/anthology/P04-3031/},
//timestamp = {Wed, 18 Sep 2019 12:15:54 +0200},
//biburl = {https://dblp.org/rec/conf/acl/BirdL04.bib},
booktitle = {{ACL} 2006, 21st International Conference on Computational Linguistics
and 44th Annual Meeting of the Association for Computational Linguistics,
Proceedings of the Conference, Sydney, Australia, 17-21 July 2006},
publisher = {The Association for Computer Linguistics},
year = {2006},
//url = {https://www.aclweb.org/anthology/P06-4018/},
//timestamp = {Fri, 13 Sep 2019 13:00:43 +0200},
//biburl = {https://dblp.org/rec/conf/acl/Bird06.bib},
//bibsource = {dblp computer science bibliography, https://dblp.org}
}
......@@ -4648,67 +4692,57 @@ pages ={157-166},
//HAL_VERSION = {v1},
}
@article{DBLP:journals/corr/JeanCMB14,
@inproceedings{DBLP:conf/acl/JeanCMB15,
author = {S{\'{e}}bastien Jean and
Kyunghyun Cho and
KyungHyun Cho and
Roland Memisevic and
Yoshua Bengio},
title = {On Using Very Large Target Vocabulary for Neural Machine Translation},
journal = {CoRR},
volume = {abs/1412.2007},
year = {2014},
//url = {http://arxiv.org/abs/1412.2007},
//archivePrefix = {arXiv},
//eprint = {1412.2007},
//timestamp = {Mon, 13 Aug 2018 16:46:10 +0200},
//biburl = {https://dblp.org/rec/journals/corr/JeanCMB14.bib},
booktitle = {Proceedings of the 53rd Annual Meeting of the Association for Computational
Linguistics and the 7th International Joint Conference on Natural
Language Processing of the Asian Federation of Natural Language Processing,
{ACL} 2015, July 26-31, 2015, Beijing, China, Volume 1: Long Papers},
pages = {1--10},
publisher = {The Association for Computer Linguistics},
year = {2015},
//url = {https://doi.org/10.3115/v1/p15-1001},
//doi = {10.3115/v1/p15-1001},
//timestamp = {Tue, 28 Jan 2020 10:27:50 +0100},
//biburl = {https://dblp.org/rec/conf/acl/JeanCMB15.bib},
//bibsource = {dblp computer science bibliography, https://dblp.org}
}
@article{DBLP:journals/corr/abs-1804-10959,
@inproceedings{DBLP:conf/acl/Kudo18,
author = {Taku Kudo},
title = {Subword Regularization: Improving Neural Network Translation Models
with Multiple Subword Candidates},
journal = {CoRR},
volume = {abs/1804.10959},
booktitle = {Proceedings of the 56th Annual Meeting of the Association for Computational
Linguistics, {ACL} 2018, Melbourne, Australia, July 15-20, 2018, Volume
1: Long Papers},
pages = {66--75},
publisher = {Association for Computational Linguistics},
year = {2018},
//url = {http://arxiv.org/abs/1804.10959},
//archivePrefix = {arXiv},
//eprint = {1804.10959},
//timestamp = {Mon, 13 Aug 2018 16:48:57 +0200},
//biburl = {https://dblp.org/rec/journals/corr/abs-1804-10959.bib},
//url = {https://www.aclweb.org/anthology/P18-1007/},
//doi = {10.18653/v1/P18-1007},
//timestamp = {Mon, 16 Sep 2019 13:46:41 +0200},
//biburl = {https://dblp.org/rec/conf/acl/Kudo18.bib},
//bibsource = {dblp computer science bibliography, https://dblp.org}
}
@article{DBLP:journals/corr/ZagoruykoK16,
@inproceedings{DBLP:conf/bmvc/ZagoruykoK16,
author = {Sergey Zagoruyko and
Nikos Komodakis},
//editor = {Richard C. Wilson and
Edwin R. Hancock and
William A. P. Smith},
title = {Wide Residual Networks},
journal = {CoRR},
volume = {abs/1605.07146},
booktitle = {Proceedings of the British Machine Vision Conference 2016, {BMVC}
2016, York, UK, September 19-22, 2016},
publisher = {{BMVA} Press},
year = {2016},
//url = {http://arxiv.org/abs/1605.07146},
//archivePrefix = {arXiv},
//eprint = {1605.07146},
//timestamp = {Mon, 13 Aug 2018 16:46:42 +0200},
//biburl = {https://dblp.org/rec/journals/corr/ZagoruykoK16.bib},
//bibsource = {dblp computer science bibliography, https://dblp.org}
}
@article{DBLP:journals/iet-bmt/Sepas-Moghaddam20,
author = {Alireza Sepas{-}Moghaddam and
Fernando Pereira and
Paulo Lobato Correia},
title = {Face recognition: a novel multi-level taxonomy based survey},
journal = {{IET} Biom.},
volume = {9},
number = {2},
pages = {58--67},
year = {2020},
//url = {https://doi.org/10.1049/iet-bmt.2019.0001},
//doi = {10.1049/iet-bmt.2019.0001},
//timestamp = {Wed, 01 Apr 2020 08:42:20 +0200},
//biburl = {https://dblp.org/rec/journals/iet-bmt/Sepas-Moghaddam20.bib},
//url = {http://www.bmva.org/bmvc/2016/papers/paper087/index.html},
//timestamp = {Thu, 07 Jun 2018 10:06:28 +0200},
//biburl = {https://dblp.org/rec/conf/bmvc/ZagoruykoK16.bib},
//bibsource = {dblp computer science bibliography, https://dblp.org}
}
......@@ -4731,7 +4765,7 @@ pages ={157-166},
author = {Ganesh Jawahar and
Beno{\^{\i}}t Sagot and
Djam{\'{e}} Seddah},
editor = {Anna Korhonen and
//editor = {Anna Korhonen and
David R. Traum and
Llu{\'{\i}}s M{\`{a}}rquez},
title = {What Does {BERT} Learn about the Structure of Language?},
......@@ -4748,20 +4782,21 @@ pages ={157-166},
//bibsource = {dblp computer science bibliography, https://dblp.org}
}
@article{DBLP:journals/corr/abs-1806-00187,
@inproceedings{DBLP:conf/wmt/OttEGA18,
author = {Myle Ott and
Sergey Edunov and
David Grangier and
Michael Auli},
title = {Scaling Neural Machine Translation},
journal = {CoRR},
volume = {abs/1806.00187},
booktitle = {Proceedings of the Third Conference on Machine Translation: Research
Papers, {WMT} 2018, Belgium, Brussels, October 31 - November 1, 2018},
pages = {1--9},
publisher = {Association for Computational Linguistics},
year = {2018},
//url = {http://arxiv.org/abs/1806.00187},
//archivePrefix = {arXiv},
//eprint = {1806.00187},
//timestamp = {Mon, 13 Aug 2018 16:47:40 +0200},
//biburl = {https://dblp.org/rec/journals/corr/abs-1806-00187.bib},
//url = {https://doi.org/10.18653/v1/w18-6301},
//doi = {10.18653/v1/w18-6301},
//timestamp = {Tue, 28 Jan 2020 10:31:02 +0100},
//biburl = {https://dblp.org/rec/conf/wmt/OttEGA18.bib},
//bibsource = {dblp computer science bibliography, https://dblp.org}
}
......@@ -4831,10 +4866,6 @@ pages ={157-166},
Orhan Firat and
Yuan Cao and
Yonghui Wu},
editor = {Ellen Riloff and
David Chiang and
Julia Hockenmaier and
Jun'ichi Tsujii},
title = {Training Deeper Neural Machine Translation Models with Transparent
Attention},
booktitle = {Proceedings of the 2018 Conference on Empirical Methods in Natural
......@@ -4853,10 +4884,6 @@ pages ={157-166},
author = {Biao Zhang and
Ivan Titov and
Rico Sennrich},
editor = {Kentaro Inui and
Jing Jiang and
Vincent Ng and
Xiaojun Wan},
title = {Improving Deep Transformer with Depth-Scaled Initialization and Merged
Attention},
booktitle = {Proceedings of the 2019 Conference on Empirical Methods in Natural
......@@ -4878,10 +4905,6 @@ pages ={157-166},
Xiangyu Zhang and
Shaoqing Ren and
Jian Sun},
editor = {Bastian Leibe and
Jiri Matas and
Nicu Sebe and
Max Welling},
title = {Identity Mappings in Deep Residual Networks},
booktitle = {Computer Vision - {ECCV} 2016 - 14th European Conference, Amsterdam,
The Netherlands, October 11-14, 2016, Proceedings, Part {IV}},
......@@ -4906,9 +4929,6 @@ pages ={157-166},
Tao Qin and
Jianhuang Lai and
Tie{-}Yan Liu},
editor = {Anna Korhonen and
David R. Traum and
Llu{\'{\i}}s M{\`{a}}rquez},
title = {Depth Growing for Neural Machine Translation},
booktitle = {Proceedings of the 57th Conference of the Association for Computational
Linguistics, {ACL} 2019, Florence, Italy, July 28- August 2, 2019,
......@@ -4923,37 +4943,40 @@ pages ={157-166},
//bibsource = {dblp computer science bibliography, https://dblp.org}
}
@article{DBLP:journals/corr/HuangLW16a,
@inproceedings{DBLP:conf/cvpr/HuangLMW17,
author = {Gao Huang and
Zhuang Liu and
Laurens van der Maaten and
Kilian Q. Weinberger},
title = {Densely Connected Convolutional Networks},
journal = {CoRR},
volume = {abs/1608.06993},
year = {2016},
//url = {http://arxiv.org/abs/1608.06993},
//archivePrefix = {arXiv},
//eprint = {1608.06993},
//timestamp = {Mon, 10 Sep 2018 15:49:32 +0200},
//biburl = {https://dblp.org/rec/journals/corr/HuangLW16a.bib},
booktitle = {2017 {IEEE} Conference on Computer Vision and Pattern Recognition,
{CVPR} 2017, Honolulu, HI, USA, July 21-26, 2017},
pages = {2261--2269},
publisher = {{IEEE} Computer Society},
year = {2017},
//url = {https://doi.org/10.1109/CVPR.2017.243},
//doi = {10.1109/CVPR.2017.243},
//timestamp = {Wed, 16 Oct 2019 14:14:50 +0200},
//biburl = {https://dblp.org/rec/conf/cvpr/HuangLMW17.bib},
//bibsource = {dblp computer science bibliography, https://dblp.org}
}
@article{DBLP:journals/corr/abs-1810-10181,
@inproceedings{DBLP:conf/emnlp/DouTWSZ18,
author = {Zi{-}Yi Dou and
Zhaopeng Tu and
Xing Wang and
Shuming Shi and
Tong Zhang},
title = {Exploiting Deep Representations for Neural Machine Translation},
journal = {CoRR},
volume = {abs/1810.10181},
booktitle = {Proceedings of the 2018 Conference on Empirical Methods in Natural
Language Processing, Brussels, Belgium, October 31 - November 4, 2018},
pages = {4253--4262},
publisher = {Association for Computational Linguistics},
year = {2018},
//url = {http://arxiv.org/abs/1810.10181},
//archivePrefix = {arXiv},
//eprint = {1810.10181},
//timestamp = {Tue, 15 Jan 2019 11:48:13 +0100},
//biburl = {https://dblp.org/rec/journals/corr/abs-1810-10181.bib},
//url = {https://doi.org/10.18653/v1/d18-1457},
//doi = {10.18653/v1/d18-1457},
//timestamp = {Tue, 28 Jan 2020 10:28:31 +0100},
//biburl = {https://dblp.org/rec/conf/emnlp/DouTWSZ18.bib},
//bibsource = {dblp computer science bibliography, https://dblp.org}
}
......@@ -4973,22 +4996,26 @@ pages ={157-166},
//bibsource = {dblp computer science bibliography, https://dblp.org}
}
@article{DBLP:journals/corr/XiaQCBYL17,
@inproceedings{DBLP:conf/icml/XiaQCBYL17,
author = {Yingce Xia and
Tao Qin and
Wei Chen and
Jiang Bian and
Nenghai Yu and
Tie{-}Yan Liu},
//editor = {Doina Precup and
Yee Whye Teh},
title = {Dual Supervised Learning},
journal = {CoRR},
volume = {abs/1707.00415},
booktitle = {Proceedings of the 34th International Conference on Machine Learning,
{ICML} 2017, Sydney, NSW, Australia, 6-11 August 2017},
series = {Proceedings of Machine Learning Research},
volume = {70},
pages = {3789--3798},
publisher = {{PMLR}},
year = {2017},
//url = {http://arxiv.org/abs/1707.00415},
//archivePrefix = {arXiv},
//eprint = {1707.00415},
//timestamp = {Tue, 03 Sep 2019 16:31:11 +0200},
//biburl = {https://dblp.org/rec/journals/corr/XiaQCBYL17.bib},
//url = {http://proceedings.mlr.press/v70/xia17a.html},
//timestamp = {Tue, 03 Sep 2019 16:31:10 +0200},
//biburl = {https://dblp.org/rec/conf/icml/XiaQCBYL17.bib},
//bibsource = {dblp computer science bibliography, https://dblp.org}
}
......@@ -5000,11 +5027,6 @@ pages ={157-166},
Nenghai Yu and
Tie{-}Yan Liu and
Wei{-}Ying Ma},
editor = {Daniel D. Lee and
Masashi Sugiyama and
Ulrike von Luxburg and
Isabelle Guyon and
Roman Garnett},
title = {Dual Learning for Machine Translation},
booktitle = {Advances in Neural Information Processing Systems 29: Annual Conference
on Neural Information Processing Systems 2016, December 5-10, 2016,
......@@ -5022,9 +5044,6 @@ pages ={157-166},
David A. McAllester and
Satinder P. Singh and
Yishay Mansour},
editor = {Sara A. Solla and
Todd K. Leen and
Klaus{-}Robert M{\"{u}}ller},
title = {Policy Gradient Methods for Reinforcement Learning with Function Approximation},
booktitle = {Advances in Neural Information Processing Systems 12, {[NIPS} Conference,
Denver, Colorado, USA, November 29 - December 4, 1999]},
......@@ -5063,16 +5082,6 @@ pages ={157-166},
author = {Anna Currey and
Antonio Valerio Miceli Barone and
Kenneth Heafield},
editor = {Ondrej Bojar and
Christian Buck and
Rajen Chatterjee and
Christian Federmann and
Yvette Graham and
Barry Haddow and
Matthias Huck and
Antonio Jimeno{-}Yepes and
Philipp Koehn and
Julia Kreutzer},
title = {Copied Monolingual Data Improves Low-Resource Neural Machine Translation},
booktitle = {Proceedings of the Second Conference on Machine Translation, {WMT}
2017, Copenhagen, Denmark, September 7-8, 2017},
......@@ -5108,10 +5117,6 @@ pages ={157-166},
Myle Ott and
Michael Auli and
David Grangier},
editor = {Ellen Riloff and
David Chiang and
Julia Hockenmaier and
Jun'ichi Tsujii},
title = {Understanding Back-Translation at Scale},
booktitle = {Proceedings of the 2018 Conference on Empirical Methods in Natural
Language Processing, Brussels, Belgium, October 31 - November 4, 2018},
......@@ -5128,9 +5133,6 @@ pages ={157-166},
@inproceedings{DBLP:conf/emnlp/DomhanH17,
author = {Tobias Domhan and
Felix Hieber},
editor = {Martha Palmer and
Rebecca Hwa and
Sebastian Riedel},
title = {Using Target-side Monolingual Data for Neural Machine Translation
through Multi-task Learning},
booktitle = {Proceedings of the 2017 Conference on Empirical Methods in Natural
......@@ -5167,24 +5169,10 @@ pages ={157-166},
//bibsource = {dblp computer science bibliography, https://dblp.org}
}
@inproceedings{DBLP:conf/iclr/FrankleC19,
author = {Jonathan Frankle and
Michael Carbin},
title = {The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks},
booktitle = {7th International Conference on Learning Representations, {ICLR} 2019,
New Orleans, LA, USA, May 6-9, 2019},
publisher = {OpenReview.net},
year = {2019},
//url = {https://openreview.net/forum?id=rJl-b3RcF7},
//timestamp = {Thu, 25 Jul 2019 13:03:15 +0200},
//biburl = {https://dblp.org/rec/conf/iclr/FrankleC19.bib},
//bibsource = {dblp computer science bibliography, https://dblp.org}
}
@inproceedings{DBLP:conf/emnlp/KimR16,
author = {Yoon Kim and
Alexander M. Rush},
editor = {Jian Su and
//editor = {Jian Su and
Xavier Carreras and
Kevin Duh},
title = {Sequence-Level Knowledge Distillation},
......@@ -5301,38 +5289,62 @@ pages ={157-166},
year= {1994}
}
@article{bengioCurriculumlearning,
author = {Yoshu Bengio and
Jerome Louradour and
Ronman Collobert and
@inproceedings{DBLP:conf/icml/BengioLCW09,
author = {Yoshua Bengio and
J{\'{e}}r{\^{o}}me Louradour and
Ronan Collobert and
Jason Weston},
//editor = {Andrea Pohoreckyj Danyluk and
L{\'{e}}on Bottou and
Michael L. Littman},
title = {Curriculum learning},
booktitle={Proceedings of the 26th annual international conference on machine learning},
pages={41--48},
year={2009}
booktitle = {Proceedings of the 26th Annual International Conference on Machine
Learning, {ICML} 2009, Montreal, Quebec, Canada, June 14-18, 2009},
series = {{ACM} International Conference Proceeding Series},
volume = {382},
pages = {41--48},
publisher = {{ACM}},
year = {2009},
//url = {https://doi.org/10.1145/1553374.1553380},
//doi = {10.1145/1553374.1553380},
//timestamp = {Wed, 14 Nov 2018 10:58:56 +0100},
//biburl = {https://dblp.org/rec/conf/icml/BengioLCW09.bib},
//bibsource = {dblp computer science bibliography, https://dblp.org}
}
@inproceedings{Hubara2016BinarizedNN,
title={Binarized Neural Networks},
author={Itay Hubara and
@inproceedings{DBLP:conf/nips/HubaraCSEB16,
author = {Itay Hubara and
Matthieu Courbariaux and
Daniel Soudry and
Ran El-Yaniv and
Ran El{-}Yaniv and
Yoshua Bengio},
booktitle={Advances in neural information processing systems},
pages={4107--4115},
year={2016}}
title = {Binarized Neural Networks},
booktitle = {Advances in Neural Information Processing Systems 29: Annual Conference
on Neural Information Processing Systems 2016, December 5-10, 2016,
Barcelona, Spain},
pages = {4107--4115},
year = {2016},
//url = {http://papers.nips.cc/paper/6573-binarized-neural-networks},
//timestamp = {Fri, 06 Mar 2020 17:00:15 +0100},
//biburl = {https://dblp.org/rec/conf/nips/HubaraCSEB16.bib},
//bibsource = {dblp computer science bibliography, https://dblp.org}
}
@article{deeplearning,
title={deep learning},
author={Yann LeCun and
@article{DBLP:journals/nature/LeCunBH15,
author = {Yann LeCun and
Yoshua Bengio and
Geoffrey Hinton},
journal={nature},
volume={521},
number={7553},
pages={436--444},
year={2015},
publisher={Nature Publishing Group}
Geoffrey E. Hinton},
title = {Deep learning},
journal = {Nature},
volume = {521},
number = {7553},
pages = {436--444},
year = {2015},
//url = {https://doi.org/10.1038/nature14539},
//doi = {10.1038/nature14539},
//timestamp = {Wed, 14 Nov 2018 10:30:42 +0100},
//biburl = {https://dblp.org/rec/journals/nature/LeCunBH15.bib},
//bibsource = {dblp computer science bibliography, https://dblp.org}
}
%%%%%%%%%%%%%%%%%%7.6%%%%%%%%%%%%%%%%
@inproceedings{DBLP:conf/acl/ArtetxeLA19,
......@@ -5386,11 +5398,23 @@ pages ={157-166},
//biburl = {https://dblp.org/rec/conf/acl/HitschlerSR16.bib},
//bibsource = {dblp computer science bibliography, https://dblp.org}
}
@article{elliott2015multilingual,
title={Multilingual Image Description with Neural Sequence Models},
author={Elliott, Desmond and Frank, Stella and Hasler, Eva},
journal={arXiv: Computation and Language},
year={2015}}
@article{DBLP:journals/corr/ElliottFH15,
author = {Desmond Elliott and
Stella Frank and
Eva Hasler},
title = {Multi-Language Image Description with Neural Sequence Models},
journal = {CoRR},
volume = {abs/1510.04709},
year = {2015},
//url = {http://arxiv.org/abs/1510.04709},
archivePrefix = {arXiv},
eprint = {1510.04709},
//timestamp = {Mon, 13 Aug 2018 16:46:09 +0200},
//biburl = {https://dblp.org/rec/journals/corr/ElliottFH15.bib},
//bibsource = {dblp computer science bibliography, https://dblp.org}
}
@inproceedings{DBLP:conf/wmt/HuangLSOD16,
author = {Po{-}Yao Huang and
Frederick Liu and
......@@ -5449,37 +5473,7 @@ year={2015}}
//biburl = {https://dblp.org/rec/conf/acl/SaundersSGB18.bib},
//bibsource = {dblp computer science bibliography, https://dblp.org}
}
@inproceedings{DBLP:conf/wmt/NadejdeRSDJKB17,
author = {Maria Nadejde and
Siva Reddy and
Rico Sennrich and
Tomasz Dwojak and
Marcin Junczys{-}Dowmunt and
Philipp Koehn and
Alexandra Birch},
//editor = {Ondrej Bojar and
Christian Buck and
Rajen Chatterjee and
Christian Federmann and
Yvette Graham and
Barry Haddow and
Matthias Huck and
Antonio Jimeno{-}Yepes and
Philipp Koehn and
Julia Kreutzer},
title = {Predicting Target Language {CCG} Supertags Improves Neural Machine
Translation},
booktitle = {Proceedings of the Second Conference on Machine Translation, {WMT}
2017, Copenhagen, Denmark, September 7-8, 2017},
pages = {68--79},
publisher = {Association for Computational Linguistics},
year = {2017},
//url = {https:////doi.org/10.18653/v1/w17-4707},
//doi = {10.18653/v1/w17-4707},
//timestamp = {Tue, 28 Jan 2020 10:31:04 +0100},
//biburl = {https://dblp.org/rec/conf/wmt/NadejdeRSDJKB17.bib},
//bibsource = {dblp computer science bibliography, https://dblp.org}
}
@inproceedings{DBLP:conf/acl/SumitaUZTM18,
author = {Chunpeng Ma and
Akihiro Tamura and
......@@ -5713,7 +5707,7 @@ year={2015}}
Shuming Shi and
Tong Zhang},
title = {Learning to Remember Translation History with a Continuous Cache},
journal = {Trans. Assoc. Comput. Linguistics},
journal = {Transactions of the Association for Computational Linguistics},
volume = {6},
pages = {407--420},
year = {2018},
......@@ -5930,22 +5924,24 @@ year={2015}}
//biburl = {https://dblp.org/rec/conf/icassp/BerardBKP18.bib},
//bibsource = {dblp computer science bibliography, https://dblp.org}
}
@article{DBLP:journals/corr/abs-1802-06003,
@inproceedings{DBLP:conf/interspeech/KanoS017,
author = {Takatomo Kano and
Sakriani Sakti and
Satoshi Nakamura},
title = {Structured-based Curriculum Learning for End-to-end English-Japanese
title = {Structured-Based Curriculum Learning for End-to-End English-Japanese
Speech Translation},
journal = {CoRR},
volume = {abs/1802.06003},
year = {2018},
//url = {http://arxiv.org/abs/1802.06003},
//archivePrefix = {arXiv},
//eprint = {1802.06003},
//timestamp = {Mon, 13 Aug 2018 16:47:19 +0200},
//biburl = {https://dblp.org/rec/journals/corr/abs-1802-06003.bib},
booktitle = {Interspeech 2017, 18th Annual Conference of the International Speech
Communication Association, Stockholm, Sweden, August 20-24, 2017},
pages = {2630--2634},
publisher = {{ISCA}},
year = {2017},
//url = {http://www.isca-speech.org/archive/Interspeech\_2017/abstracts/0944.html},
//timestamp = {Mon, 15 Jul 2019 08:29:02 +0200},
//biburl = {https://dblp.org/rec/conf/interspeech/KanoS017.bib},
//bibsource = {dblp computer science bibliography, https://dblp.org}
}
@article{DBLP:journals/tacl/SperberNNW19,
author = {Matthias Sperber and
Graham Neubig and
......@@ -5953,7 +5949,7 @@ year={2015}}
Alex Waibel},
title = {Attention-Passing Models for Robust and Data-Efficient End-to-End
Speech Translation},
journal = {Trans. Assoc. Comput. Linguistics},
journal = {Transactions of the Association for Computational Linguistics},
volume = {7},
pages = {313--325},
year = {2019},
......@@ -6022,7 +6018,7 @@ year={2015}}
Jeffrey Dean},
title = {Google's Multilingual Neural Machine Translation System: Enabling
Zero-Shot Translation},
journal = {Trans. Assoc. Comput. Linguistics},
journal = {Transactions of the Association for Computational Linguistics},
volume = {5},
pages = {339--351},
year = {2017},
......@@ -6047,21 +6043,7 @@ year={2015}}
//biburl = {https://dblp.org/rec/journals/corr/HaNW16.bib},
//bibsource = {dblp computer science bibliography, https://dblp.org}
}
@article{DBLP:journals/corr/abs-1711-07893,
author = {Thanh{-}Le Ha and
Jan Niehues and
Alexander H. Waibel},
title = {Effective Strategies in Zero-Shot Neural Machine Translation},
journal = {CoRR},
volume = {abs/1711.07893},
year = {2017},
//url = {http://arxiv.org/abs/1711.07893},
//archivePrefix = {arXiv},
//eprint = {1711.07893},
//timestamp = {Mon, 13 Aug 2018 16:46:07 +0200},
//biburl = {https://dblp.org/rec/journals/corr/abs-1711-07893.bib},
//bibsource = {dblp computer science bibliography, https://dblp.org}
}
@inproceedings{DBLP:conf/naacl/FiratCB16,
author = {Orhan Firat and
Kyunghyun Cho and
......@@ -6083,31 +6065,32 @@ year={2015}}
//biburl = {https://dblp.org/rec/conf/naacl/FiratCB16.bib},
//bibsource = {dblp computer science bibliography, https://dblp.org}
}
@article{luong2015multi-task,
title={Multi-task Sequence to Sequence Learning},
author={Luong, Minhthang and Le, Quoc V and Sutskever, Ilya and Vinyals, Oriol and Kaiser, Lukasz},
journal={arXiv: Learning},
year={2015}}
@article{Och2001Statistical,
title={Statistical multi-source translation},
author={Och, Franz Josef and Ney, Hermann},
journal={Mt Summit},
year={2001},
}
@article{DBLP:journals/jmlr/ElskenMH19,
author = {Thomas Elsken and
Jan Hendrik Metzen and
Frank Hutter},
title = {Neural Architecture Search: {A} Survey},
journal = {J. Mach. Learn. Res.},
volume = {20},
pages = {55:1--55:21},
year = {2019},
//url = {http://jmlr.org/papers/v20/18-598.html},
//timestamp = {Wed, 10 Jul 2019 15:28:24 +0200},
//biburl = {https://dblp.org/rec/journals/jmlr/ElskenMH19.bib},
@inproceedings{DBLP:journals/corr/LuongLSVK15,
author = {Minh{-}Thang Luong and
Quoc V. Le and
Ilya Sutskever and
Oriol Vinyals and
Lukasz Kaiser},
title = {Multi-task Sequence to Sequence Learning},
booktitle = {4th International Conference on Learning Representations, {ICLR} 2016,
San Juan, Puerto Rico, May 2-4, 2016, Conference Track Proceedings},
year = {2016},
//url = {http://arxiv.org/abs/1511.06114},
//timestamp = {Thu, 25 Jul 2019 14:25:37 +0200},
//biburl = {https://dblp.org/rec/journals/corr/LuongLSVK15.bib},
//bibsource = {dblp computer science bibliography, https://dblp.org}
}
@article{elsken2019neural,
title={Neural Architecture Search: A Survey},
author={Elsken, Thomas and Metzen, Jan Hendrik and Hutter, Frank},
journal={Journal of Machine Learning Research},
volume={20},
number={55},
pages={1--21},
year={2019}}
@inproceedings{DBLP:conf/iclr/ZophL17,
author = {Barret Zoph and
Quoc V. Le},
......@@ -6161,11 +6144,25 @@ for Language Modeling},
publisher = {Association for Computational Linguistics},
year = {2020},
}
@article{Luo2018Neural,
title={Neural Architecture Optimization},
author={Luo, Renqian and Tian, Fei and Qin, Tao and Liu, Tie-Yan},
year={2018},
@inproceedings{DBLP:conf/nips/LuoTQCL18,
author = {Renqian Luo and
Fei Tian and
Tao Qin and
Enhong Chen and
Tie{-}Yan Liu},
title = {Neural Architecture Optimization},
booktitle = {Advances in Neural Information Processing Systems 31: Annual Conference
on Neural Information Processing Systems 2018, NeurIPS 2018, 3-8 December
2018, Montr{\'{e}}al, Canada},
pages = {7827--7838},
year = {2018},
//url = {http://papers.nips.cc/paper/8007-neural-architecture-optimization},
//timestamp = {Fri, 06 Mar 2020 17:00:31 +0100},
//biburl = {https://dblp.org/rec/conf/nips/LuoTQCL18.bib},
//bibsource = {dblp computer science bibliography, https://dblp.org}
}
@inproceedings{DBLP:conf/aclnmt/KoehnK17,
author = {Philipp Koehn and
Rebecca Knowles},
......@@ -6224,23 +6221,7 @@ year = {2020},
//biburl = {https://dblp.org/rec/conf/ijcnlp/KhayrallahKDPK17.bib},
//bibsource = {dblp computer science bibliography, https://dblp.org}
}
@inproceedings{DBLP:conf/acl/StahlbergHWB16,
author = {Felix Stahlberg and
Eva Hasler and
Aurelien Waite and
Bill Byrne},
title = {Syntactically Guided Neural Machine Translation},
booktitle = {Proceedings of the 54th Annual Meeting of the Association for Computational
Linguistics, {ACL} 2016, August 7-12, 2016, Berlin, Germany, Volume
2: Short Papers},
publisher = {The Association for Computer Linguistics},
year = {2016},
//url = {https:////doi.org/10.18653/v1/p16-2049},
//doi = {10.18653/v1/p16-2049},
//timestamp = {Tue, 28 Jan 2020 10:27:31 +0100},
//biburl = {https://dblp.org/rec/conf/acl/StahlbergHWB16.bib},
//bibsource = {dblp computer science bibliography, https://dblp.org}
}
@inproceedings{DBLP:conf/aclwat/NeubigMN15,
author = {Graham Neubig and
Makoto Morishita and
......@@ -6263,27 +6244,7 @@ year = {2020},
//biburl = {https://dblp.org/rec/conf/aclwat/NeubigMN15.bib},
//bibsource = {dblp computer science bibliography, https://dblp.org}
}
@inproceedings{DBLP:conf/naacl/GrundkiewiczJ18,
author = {Roman Grundkiewicz and
Marcin Junczys{-}Dowmunt},
//editor = {Marilyn A. Walker and
Heng Ji and
Amanda Stent},
title = {Near Human-Level Performance in Grammatical Error Correction with
Hybrid Machine Translation},
booktitle = {Proceedings of the 2018 Conference of the North American Chapter of
the Association for Computational Linguistics: Human Language Technologies,
NAACL-HLT, New Orleans, Louisiana, USA, June 1-6, 2018, Volume 2 (Short
Papers)},
pages = {284--290},
publisher = {Association for Computational Linguistics},
year = {2018},
//url = {https:////doi.org/10.18653/v1/n18-2046},
//doi = {10.18653/v1/n18-2046},
//timestamp = {Tue, 28 Jan 2020 10:30:23 +0100},
//biburl = {https://dblp.org/rec/conf/naacl/GrundkiewiczJ18.bib},
//bibsource = {dblp computer science bibliography, https://dblp.org}
}
@inproceedings{niehues-etal-2016-pre,
title = "Pre-Translation for Neural Machine Translation",
author = "Niehues, Jan and
......@@ -6334,15 +6295,179 @@ year = {2020},
@article{Alvaro2017Interactive,
title={Interactive neural machine translation},
author={Alvaro Peris and Domingo, Miguel and Casacuberta, Francisco},
journal={Computer Speech & Language},
journal={Computer Speech \& Language},
volume={45},
number={Sep.},
pages={201-220},
year={2017},
}
@inproceedings{Nepveu2004Adaptive,
title={Adaptive Language and Translation Models for Interactive Machine Translation},
author={Nepveu, Laurent and Lapalme, Guy and Langlais, Philippe and Foster, George F.},
booktitle={Conference on Empirical Methods in Natural Language Processing},
year={2004},
@inproceedings{DBLP:conf/emnlp/NepveuLLF04,
author = {Laurent Nepveu and
Guy Lapalme and
Philippe Langlais and
George F. Foster},
title = {Adaptive Language and Translation Models for Interactive Machine Translation},
booktitle = {Proceedings of the 2004 Conference on Empirical Methods in Natural
Language Processing , {EMNLP} 2004, {A} meeting of SIGDAT, a Special
Interest Group of the ACL, held in conjunction with {ACL} 2004, 25-26
July 2004, Barcelona, Spain},
pages = {190--197},
publisher = {{ACL}},
year = {2004},
//url = {https://www.aclweb.org/anthology/W04-3225/},
//timestamp = {Fri, 13 Sep 2019 13:08:45 +0200},
//biburl = {https://dblp.org/rec/conf/emnlp/NepveuLLF04.bib},
//bibsource = {dblp computer science bibliography, https://dblp.org}
}
@inproceedings{wang-etal-2018-tencent,
title = "Tencent Neural Machine Translation Systems for {WMT}18",
author = "Wang, Mingxuan and
Gong, Li and
Zhu, Wenhuan and
Xie, Jun and
Bian, Chao",
booktitle = "Proceedings of the Third Conference on Machine Translation: Shared Task Papers",
month = oct,
year = "2018",
address = "Belgium, Brussels",
publisher = "Association for Computational Linguistics",
//url = "https://www.aclweb.org/anthology/W18-6429",
//doi = "10.18653/v1/W18-6429",
pages = "522--527",
//abstract = "We participated in the WMT 2018 shared news translation task on English鈫擟hinese language pair. Our systems are based on attentional sequence-to-sequence models with some form of recursion and self-attention. Some data augmentation methods are also introduced to improve the translation performance. The best translation result is obtained with ensemble and reranking techniques. Our Chinese鈫扙nglish system achieved the highest cased BLEU score among all 16 submitted systems, and our English鈫扖hinese system ranked the third out of 18 submitted systems.",
}
@article{DBLP:journals/tacl/LeeCH17,
author = {Jason Lee and
Kyunghyun Cho and
Thomas Hofmann},
title = {Fully Character-Level Neural Machine Translation without Explicit
Segmentation},
journal = {Trans. Assoc. Comput. Linguistics},
volume = {5},
pages = {365--378},
year = {2017},
//url = {https://transacl.org/ojs/index.php/tacl/article/view/1051},
//timestamp = {Thu, 02 Apr 2020 08:34:57 +0200},
//biburl = {https://dblp.org/rec/journals/tacl/LeeCH17.bib},
//bibsource = {dblp computer science bibliography, https://dblp.org}
}
@INPROCEEDINGS{6289079,
author={M. {Schuster} and K. {Nakajima}},
booktitle={2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)}, title={Japanese and Korean voice search},
year={2012},
volume={},
number={},
pages={5149-5152}
}
@ARTICLE{61115,
author={J. {Lin}},
journal={IEEE Transactions on Information Theory},
title={Divergence measures based on the Shannon entropy},
year={1991},
volume={37},
number={1},
pages={145-151},}
@inproceedings{DBLP:conf/aaai/XiaHLS19,
author = {Mengzhou Xia and
Guoping Huang and
Lemao Liu and
Shuming Shi},
title = {Graph Based Translation Memory for Neural Machine Translation},
booktitle = {The Thirty-Third {AAAI} Conference on Artificial Intelligence, {AAAI}
2019, The Thirty-First Innovative Applications of Artificial Intelligence
Conference, {IAAI} 2019, The Ninth {AAAI} Symposium on Educational
Advances in Artificial Intelligence, {EAAI} 2019, Honolulu, Hawaii,
USA, January 27 - February 1, 2019},
pages = {7297--7304},
publisher = {{AAAI} Press},
year = {2019},
//url = {https://doi.org/10.1609/aaai.v33i01.33017297},
//doi = {10.1609/aaai.v33i01.33017297},
//timestamp = {Wed, 25 Sep 2019 11:05:09 +0200},
//biburl = {https://dblp.org/rec/conf/aaai/XiaHLS19.bib},
//bibsource = {dblp computer science bibliography, https://dblp.org}
}
@inproceedings{DBLP:conf/nlpcc/HeHLL19,
author = {Qiuxiang He and
Guoping Huang and
Lemao Liu and
Li Li},
title = {Word Position Aware Translation Memory for Neural Machine Translation},
booktitle = {Natural Language Processing and Chinese Computing - 8th {CCF} International
Conference, {NLPCC} 2019, Dunhuang, China, October 9-14, 2019, Proceedings,
Part {I}},
series = {Lecture Notes in Computer Science},
volume = {11838},
pages = {367--379},
publisher = {Springer},
year = {2019},
//url = {https://doi.org/10.1007/978-3-030-32233-5\_29},
//doi = {10.1007/978-3-030-32233-5\_29},
//timestamp = {Fri, 04 Oct 2019 08:40:42 +0200},
//biburl = {https://dblp.org/rec/conf/nlpcc/HeHLL19.bib},
//bibsource = {dblp computer science bibliography, https://dblp.org}
}
@INPROCEEDINGS{Och01statisticalmulti-source,
author = {Franz Josef Och and Hermann Ney},
title = {Statistical multi-source translation},
booktitle = {In MT Summit 2001},
year = {2001},
pages = {253--258}
}
@inproceedings{Weiss2017SequencetoSequenceMC,
title={Sequence-to-Sequence Models Can Directly Translate Foreign Speech},
author={Ron J. Weiss and Jan Chorowski and Navdeep Jaitly and Yonghui Wu and Zhifeng Chen},
booktitle={INTERSPEECH},
pages = {2625-2629},
year={2017}
}
@inproceedings{dong-etal-2015-multi,
title = "Multi-Task Learning for Multiple Language Translation",
author = "Dong, Daxiang and
Wu, Hua and
He, Wei and
Yu, Dianhai and
Wang, Haifeng",
booktitle = "Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)",
month = jul,
year = "2015",
//address = "Beijing, China",
publisher = "Association for Computational Linguistics",
//url = "https://www.aclweb.org/anthology/P15-1166",
//doi = "10.3115/v1/P15-1166",
pages = "1723--1732",
}
@inproceedings{firat-etal-2016-zero,
title = "Zero-Resource Translation with Multi-Lingual Neural Machine Translation",
author = "Firat, Orhan and
Sankaran, Baskaran and
Al-onaizan, Yaser and
Yarman Vural, Fatos T. and
Cho, Kyunghyun",
booktitle = "Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing",
month = nov,
year = "2016",
//address = "Austin, Texas",
publisher = "Association for Computational Linguistics",
//url = "https://www.aclweb.org/anthology/D16-1026",
//doi = "10.18653/v1/D16-1026",
pages = "268--277",
}
@inproceedings{Zoph2016MultiSourceNT,
title={Multi-Source Neural Translation},
author={Barret Zoph and Kevin Knight},
booktitle={HLT-NAACL},
year={2016}
}
Book/mt-book-xelatex.idx
\indexentry{未登录词|hyperpage}{11}
\indexentry{Out of Vocabulary Word，OOV Word|hyperpage}{11}
\indexentry{子词切分|hyperpage}{11}
\indexentry{Sub-word Segmentation|hyperpage}{11}
\indexentry{标准化|hyperpage}{11}
\indexentry{Normalization|hyperpage}{11}
\indexentry{数据清洗|hyperpage}{11}
\indexentry{Dada Cleaning|hyperpage}{11}
\indexentry{数据选择|hyperpage}{13}
\indexentry{Data Selection|hyperpage}{13}
\indexentry{数据过滤|hyperpage}{13}
\indexentry{Data Filtering|hyperpage}{13}
\indexentry{开放词表|hyperpage}{16}
\indexentry{Open-Vocabulary|hyperpage}{16}
\indexentry{子词|hyperpage}{17}
\indexentry{Sub-word|hyperpage}{17}
\indexentry{字节对编码|hyperpage}{17}
\indexentry{双字节编码|hyperpage}{17}
\indexentry{Byte Pair Encoding，BPE|hyperpage}{17}
\indexentry{正则化|hyperpage}{20}
\indexentry{Regularization|hyperpage}{20}
\indexentry{过拟合问题|hyperpage}{20}
\indexentry{Overfitting Problem|hyperpage}{20}
\indexentry{反问题|hyperpage}{20}
\indexentry{Inverse Problem|hyperpage}{20}
\indexentry{适定的|hyperpage}{20}
\indexentry{Well-posed|hyperpage}{20}
\indexentry{不适定问题|hyperpage}{20}
\indexentry{Ill-posed Problem|hyperpage}{20}
\indexentry{降噪|hyperpage}{21}
\indexentry{Denoising|hyperpage}{21}
\indexentry{泛化|hyperpage}{21}
\indexentry{Generalization|hyperpage}{21}
\indexentry{标签平滑|hyperpage}{23}
\indexentry{Label Smoothing|hyperpage}{23}
\indexentry{相互适应|hyperpage}{24}
\indexentry{Co-Adaptation|hyperpage}{24}
\indexentry{集成学习|hyperpage}{25}
\indexentry{Ensemble Learning|hyperpage}{25}
\indexentry{容量|hyperpage}{26}
\indexentry{Capacity|hyperpage}{26}
\indexentry{宽残差网络|hyperpage}{27}
\indexentry{Wide Residual Network|hyperpage}{27}
\indexentry{探测任务|hyperpage}{28}
\indexentry{Probing Task|hyperpage}{28}
\indexentry{表面信息|hyperpage}{28}
\indexentry{Surface Information|hyperpage}{28}
\indexentry{语法信息|hyperpage}{28}
\indexentry{Syntactic Information|hyperpage}{28}
\indexentry{语义信息|hyperpage}{28}
\indexentry{Semantic Information|hyperpage}{28}
\indexentry{词嵌入|hyperpage}{29}
\indexentry{Embedding|hyperpage}{29}
\indexentry{数据并行|hyperpage}{29}
\indexentry{Data Parallelism|hyperpage}{29}
\indexentry{模型并行|hyperpage}{29}
\indexentry{Model Parallelism|hyperpage}{29}
\indexentry{小批量训练|hyperpage}{29}
\indexentry{Mini-batch Training|hyperpage}{29}
\indexentry{课程学习|hyperpage}{31}
\indexentry{Curriculum Learning|hyperpage}{31}
\indexentry{推断|hyperpage}{32}
\indexentry{Inference|hyperpage}{32}
\indexentry{解码|hyperpage}{32}
\indexentry{Decoding|hyperpage}{32}
\indexentry{搜索错误|hyperpage}{32}
\indexentry{Search Error|hyperpage}{32}
\indexentry{模型错误|hyperpage}{32}
\indexentry{Modeling Error|hyperpage}{32}
\indexentry{重排序|hyperpage}{34}
\indexentry{Re-ranking|hyperpage}{34}
\indexentry{双向推断|hyperpage}{34}
\indexentry{Bidirectional Inference|hyperpage}{34}
\indexentry{批量推断|hyperpage}{38}
\indexentry{Batch Inference|hyperpage}{38}
\indexentry{批量处理|hyperpage}{38}
\indexentry{Batching|hyperpage}{38}
\indexentry{二值网络|hyperpage}{39}
\indexentry{Binarized Neural Networks|hyperpage}{39}
\indexentry{自回归翻译|hyperpage}{40}
\indexentry{Autoregressive Translation|hyperpage}{40}
\indexentry{非自回归翻译|hyperpage}{40}
\indexentry{Regressive Translation|hyperpage}{40}
\indexentry{繁衍率|hyperpage}{40}
\indexentry{Fertility|hyperpage}{40}
\indexentry{偏置|hyperpage}{41}
\indexentry{Bias|hyperpage}{41}
\indexentry{退化|hyperpage}{42}
\indexentry{Degenerate|hyperpage}{42}
\indexentry{过翻译|hyperpage}{43}
\indexentry{Over Translation|hyperpage}{43}
\indexentry{欠翻译|hyperpage}{43}
\indexentry{Under Translation|hyperpage}{43}
\indexentry{充分性|hyperpage}{44}
\indexentry{Adequacy|hyperpage}{44}
\indexentry{系统融合|hyperpage}{44}
\indexentry{System Combination|hyperpage}{44}
\indexentry{假设选择|hyperpage}{45}
\indexentry{Hypothesis Selection|hyperpage}{45}
\indexentry{多样性|hyperpage}{45}
\indexentry{Diversity|hyperpage}{45}
\indexentry{重排序|hyperpage}{46}
\indexentry{Re-ranking|hyperpage}{46}
\indexentry{混淆网络|hyperpage}{47}
\indexentry{Confusion Network|hyperpage}{47}
\indexentry{动态线性层聚合方法|hyperpage}{51}
\indexentry{Dynamic Linear Combination of Layers，DLCL|hyperpage}{51}
\indexentry{相互适应|hyperpage}{55}
\indexentry{Co-adaptation|hyperpage}{55}
\indexentry{数据增强|hyperpage}{57}
\indexentry{Data Augmentation|hyperpage}{57}
\indexentry{回译|hyperpage}{57}
\indexentry{Back Translation|hyperpage}{57}
\indexentry{迭代式回译|hyperpage}{58}
\indexentry{Iterative Back Translation|hyperpage}{58}
\indexentry{前向翻译|hyperpage}{59}
\indexentry{Forward Translation|hyperpage}{59}
\indexentry{分布式表示|hyperpage}{59}
\indexentry{Distributed Representation|hyperpage}{59}
\indexentry{预训练|hyperpage}{59}
\indexentry{Pre-training|hyperpage}{59}
\indexentry{微调|hyperpage}{59}
\indexentry{Fine-tuning|hyperpage}{59}
\indexentry{多任务学习|hyperpage}{61}
\indexentry{Multitask Learning|hyperpage}{61}
\indexentry{知识精炼|hyperpage}{62}
\indexentry{Knowledge Distillation|hyperpage}{62}
\indexentry{模型压缩|hyperpage}{62}
\indexentry{Model Compression|hyperpage}{62}
\indexentry{学习难度|hyperpage}{62}
\indexentry{Learning Difficulty|hyperpage}{62}
\indexentry{教师模型|hyperpage}{63}
\indexentry{Teacher Model|hyperpage}{63}
\indexentry{学生模型|hyperpage}{63}
\indexentry{Student Model|hyperpage}{63}
\indexentry{基于单词的知识精炼|hyperpage}{63}
\indexentry{Word-level Knowledge Distillation|hyperpage}{63}
\indexentry{基于序列的知识精炼|hyperpage}{63}
\indexentry{Sequence-level Knowledge Distillation|hyperpage}{63}
\indexentry{中间层输出|hyperpage}{64}
\indexentry{Hint-based Knowledge Transfer|hyperpage}{64}
\indexentry{注意力分布|hyperpage}{64}
\indexentry{Attention To Attention Transfer|hyperpage}{65}
\indexentry{循环一致性|hyperpage}{67}
\indexentry{Circle Consistency|hyperpage}{67}
\indexentry{翻译中回译|hyperpage}{68}
\indexentry{On-the-fly Back-translation|hyperpage}{68}
\indexentry{神经架构搜索|hyperpage}{71}
\indexentry{Neural Architecture Search|hyperpage}{71}
\indexentry{组合性翻译|hyperpage}{10}
\indexentry{Compositional Translation|hyperpage}{10}
\indexentry{短语|hyperpage}{10}
\indexentry{短语切分|hyperpage}{15}
\indexentry{Phrasal Segmentation|hyperpage}{15}
\indexentry{短语对|hyperpage}{15}
\indexentry{推导|hyperpage}{15}
\indexentry{Derivation|hyperpage}{15}
\indexentry{生成式模型|hyperpage}{18}
\indexentry{Generative Model|hyperpage}{18}
\indexentry{判别式模型|hyperpage}{18}
\indexentry{Discriminative Model|hyperpage}{18}
\indexentry{对数线性模型|hyperpage}{19}
\indexentry{Log-linear Model|hyperpage}{19}
\indexentry{短语抽取|hyperpage}{20}
\indexentry{Phrase Extraction|hyperpage}{20}
\indexentry{词汇化翻译概率|hyperpage}{23}
\indexentry{Lexical Translation Probability|hyperpage}{23}
\indexentry{短语表|hyperpage}{23}
\indexentry{Phrase Table|hyperpage}{23}
\indexentry{调序|hyperpage}{24}
\indexentry{Reordering|hyperpage}{24}
\indexentry{模型训练|hyperpage}{28}
\indexentry{Model Training|hyperpage}{28}
\indexentry{权重调优|hyperpage}{28}
\indexentry{Weight Tuning|hyperpage}{28}
\indexentry{最小错误率训练|hyperpage}{28}
\indexentry{Minimum Error Rate Training|hyperpage}{28}
\indexentry{调优集合|hyperpage}{28}
\indexentry{Tuning Set|hyperpage}{28}
\indexentry{线搜索|hyperpage}{29}
\indexentry{Line Search|hyperpage}{29}
\indexentry{格搜索|hyperpage}{30}
\indexentry{Grid Search|hyperpage}{30}
\indexentry{覆盖度模型|hyperpage}{32}
\indexentry{Coverage Model|hyperpage}{32}
\indexentry{翻译候选|hyperpage}{32}
\indexentry{Translation Candidate|hyperpage}{32}
\indexentry{翻译假设|hyperpage}{33}
\indexentry{Translation Hypothesis|hyperpage}{33}
\indexentry{剪枝|hyperpage}{34}
\indexentry{Pruning|hyperpage}{34}
\indexentry{束剪枝|hyperpage}{34}
\indexentry{Beam Pruning|hyperpage}{34}
\indexentry{直方图剪枝|hyperpage}{34}
\indexentry{Histogram Pruning|hyperpage}{34}
\indexentry{阈值剪枝|hyperpage}{34}
\indexentry{Threshold Pruning|hyperpage}{34}
\indexentry{假设重组|hyperpage}{34}
\indexentry{Hypothesis Recombination|hyperpage}{34}
\indexentry{基于层次短语的模型|hyperpage}{38}
\indexentry{Hierarchical Phrase-based Model|hyperpage}{38}
\indexentry{同步上下文无关文法|hyperpage}{39}
\indexentry{Synchronous Context-free Grammar|hyperpage}{39}
\indexentry{基于层次短语的文法|hyperpage}{40}
\indexentry{Hierarchical Phrase-based Grammar|hyperpage}{40}
\indexentry{推导|hyperpage}{41}
\indexentry{Derivation|hyperpage}{41}
\indexentry{胶水规则|hyperpage}{41}
\indexentry{Glue Rule|hyperpage}{41}
\indexentry{乔姆斯基范式|hyperpage}{45}
\indexentry{Chomsky Normal Form|hyperpage}{45}
\indexentry{跨度|hyperpage}{45}
\indexentry{Span|hyperpage}{45}
\indexentry{自下而上的分析|hyperpage}{46}
\indexentry{Top-down Parsing|hyperpage}{46}
\indexentry{束剪枝|hyperpage}{48}
\indexentry{Beam Pruning|hyperpage}{48}
\indexentry{立方剪枝|hyperpage}{50}
\indexentry{Cube Pruning|hyperpage}{50}
\indexentry{序列化|hyperpage}{53}
\indexentry{线性化|hyperpage}{53}
\indexentry{Linearization|hyperpage}{53}
\indexentry{树到串翻译规则|hyperpage}{55}
\indexentry{Tree-to-String Translation Rule|hyperpage}{55}
\indexentry{树到树翻译规则|hyperpage}{55}
\indexentry{Tree-to-Tree Translation Rule|hyperpage}{55}
\indexentry{树片段|hyperpage}{56}
\indexentry{Tree Fragment|hyperpage}{56}
\indexentry{同步树替换文法规则|hyperpage}{57}
\indexentry{Synchronous Tree Substitution Grammar Rule|hyperpage}{57}
\indexentry{边缘集合|hyperpage}{63}
\indexentry{Frontier Set|hyperpage}{63}
\indexentry{最小规则|hyperpage}{64}
\indexentry{Minimal Rules|hyperpage}{64}
\indexentry{二叉化|hyperpage}{67}
\indexentry{Binarization|hyperpage}{67}
\indexentry{基于短语的特征|hyperpage}{72}
\indexentry{基于句法的特征|hyperpage}{72}
\indexentry{有向超图|hyperpage}{73}
\indexentry{Directed Hyper-graph|hyperpage}{73}
\indexentry{超边|hyperpage}{73}
\indexentry{Hyper-edge|hyperpage}{73}
\indexentry{半环分析|hyperpage}{73}
\indexentry{Semi-ring Parsing|hyperpage}{73}
\indexentry{组合|hyperpage}{75}
\indexentry{Composition|hyperpage}{75}
\indexentry{基于串的解码|hyperpage}{76}
\indexentry{String-based Decoding|hyperpage}{76}
\indexentry{基于树的解码|hyperpage}{76}
\indexentry{Tree-based Decoding|hyperpage}{76}
\indexentry{Lexicalized Norm Form|hyperpage}{79}
Book/mt-book-xelatex.ptc
\boolfalse {citerequest}\boolfalse {citetracker}\boolfalse {pagetracker}\boolfalse {backtracker}\relax
\babel@toc {english}{}
\defcounter {refsection}{0}\relax
\select@language {english}
\defcounter {refsection}{0}\relax
\contentsline {part}{\@mypartnumtocformat {I}{机器翻译基础}}{13}{part.1}
\contentsline {part}{\@mypartnumtocformat {I}{统计机器翻译}}{9}{part.1}%
\ttl@starttoc {default@1}
\defcounter {refsection}{0}\relax
\contentsline {chapter}{\numberline {1}机器翻译简介}{15}{chapter.1}
\defcounter {refsection}{0}\relax
\contentsline {section}{\numberline {1.1}机器翻译的概念}{15}{section.1.1}
\defcounter {refsection}{0}\relax
\contentsline {section}{\numberline {1.2}机器翻译简史}{18}{section.1.2}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {1.2.1}人工翻译}{18}{subsection.1.2.1}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {1.2.2}机器翻译的萌芽}{19}{subsection.1.2.2}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {1.2.3}机器翻译的受挫}{20}{subsection.1.2.3}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {1.2.4}机器翻译的快速成长}{21}{subsection.1.2.4}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {1.2.5}机器翻译的爆发}{22}{subsection.1.2.5}
\defcounter {refsection}{0}\relax
\contentsline {section}{\numberline {1.3}机器翻译现状}{23}{section.1.3}
\defcounter {refsection}{0}\relax
\contentsline {section}{\numberline {1.4}机器翻译方法}{24}{section.1.4}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {1.4.1}基于规则的机器翻译}{26}{subsection.1.4.1}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {1.4.2}基于实例的机器翻译}{26}{subsection.1.4.2}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {1.4.3}统计机器翻译}{27}{subsection.1.4.3}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {1.4.4}神经机器翻译}{28}{subsection.1.4.4}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {1.4.5}对比分析}{29}{subsection.1.4.5}
\defcounter {refsection}{0}\relax
\contentsline {section}{\numberline {1.5}翻译质量评价}{30}{section.1.5}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {1.5.1}人工评价}{30}{subsection.1.5.1}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {1.5.2}自动评价}{31}{subsection.1.5.2}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{BLEU}{31}{section*.15}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{TER}{33}{section*.16}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{基于检测点的评价}{33}{section*.17}
\defcounter {refsection}{0}\relax
\contentsline {section}{\numberline {1.6}机器翻译应用}{34}{section.1.6}
\defcounter {refsection}{0}\relax
\contentsline {section}{\numberline {1.7}开源项目与评测}{36}{section.1.7}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {1.7.1}开源机器翻译系统}{36}{subsection.1.7.1}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{统计机器翻译开源系统}{37}{section*.19}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{神经机器翻译开源系统}{38}{section*.20}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {1.7.2}常用数据集及公开评测任务}{40}{subsection.1.7.2}
\defcounter {refsection}{0}\relax
\contentsline {section}{\numberline {1.8}推荐学习资源}{42}{section.1.8}
\defcounter {refsection}{0}\relax
\contentsline {chapter}{\numberline {2}词法、语法及统计建模基础}{47}{chapter.2}
\defcounter {refsection}{0}\relax
\contentsline {section}{\numberline {2.1}问题概述 }{48}{section.2.1}
\defcounter {refsection}{0}\relax
\contentsline {section}{\numberline {2.2}概率论基础}{49}{section.2.2}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {2.2.1}随机变量和概率}{50}{subsection.2.2.1}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {2.2.2}联合概率、条件概率和边缘概率}{51}{subsection.2.2.2}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {2.2.3}链式法则}{52}{subsection.2.2.3}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {2.2.4}贝叶斯法则}{53}{subsection.2.2.4}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {2.2.5}KL距离和熵}{55}{subsection.2.2.5}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{信息熵}{55}{section*.27}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{KL距离}{56}{section*.29}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{交叉熵}{56}{section*.30}
\defcounter {refsection}{0}\relax
\contentsline {section}{\numberline {2.3}中文分词}{57}{section.2.3}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {2.3.1}基于词典的分词方法}{58}{subsection.2.3.1}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {2.3.2}基于统计的分词方法}{59}{subsection.2.3.2}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{统计模型的学习与推断}{59}{section*.34}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{掷骰子游戏}{60}{section*.36}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{全概率分词方法}{62}{section*.40}
\defcounter {refsection}{0}\relax
\contentsline {section}{\numberline {2.4}$n$-gram语言模型 }{64}{section.2.4}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {2.4.1}建模}{65}{subsection.2.4.1}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {2.4.2}未登录词和平滑算法}{67}{subsection.2.4.2}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{加法平滑方法}{68}{section*.46}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{古德-图灵估计法}{69}{section*.48}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{Kneser-Ney平滑方法}{70}{section*.50}
\defcounter {refsection}{0}\relax
\contentsline {section}{\numberline {2.5}句法分析（短语结构分析）}{72}{section.2.5}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {2.5.1}句子的句法树表示}{72}{subsection.2.5.1}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {2.5.2}上下文无关文法}{74}{subsection.2.5.2}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {2.5.3}规则和推导的概率}{78}{subsection.2.5.3}
\defcounter {refsection}{0}\relax
\contentsline {section}{\numberline {2.6}小结及深入阅读}{80}{section.2.6}
\defcounter {refsection}{0}\relax
\contentsline {part}{\@mypartnumtocformat {II}{统计机器翻译}}{83}{part.2}
\ttl@stoptoc {default@1}
\ttl@starttoc {default@2}
\defcounter {refsection}{0}\relax
\contentsline {chapter}{\numberline {3}基于词的机器翻译模型}{85}{chapter.3}
\defcounter {refsection}{0}\relax
\contentsline {section}{\numberline {3.1}什么是基于词的翻译模型}{85}{section.3.1}
\defcounter {refsection}{0}\relax
\contentsline {section}{\numberline {3.2}构建一个简单的机器翻译系统}{87}{section.3.2}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {3.2.1}如何进行翻译？}{87}{subsection.3.2.1}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{机器翻译流程}{88}{section*.63}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{人工翻译 vs. 机器翻译}{89}{section*.65}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {3.2.2}基本框架}{89}{subsection.3.2.2}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {3.2.3}单词翻译概率}{90}{subsection.3.2.3}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{什么是单词翻译概率？}{90}{section*.67}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{如何从一个双语平行数据中学习？}{90}{section*.69}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{如何从大量的双语平行数据中学习？}{92}{section*.70}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {3.2.4}句子级翻译模型}{93}{subsection.3.2.4}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{基础模型}{93}{section*.72}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{生成流畅的译文}{95}{section*.74}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {3.2.5}解码}{97}{subsection.3.2.5}
\defcounter {refsection}{0}\relax
\contentsline {section}{\numberline {3.3}基于词的翻译建模}{100}{section.3.3}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {3.3.1}噪声信道模型}{100}{subsection.3.3.1}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {3.3.2}统计机器翻译的三个基本问题}{102}{subsection.3.3.2}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{词对齐}{103}{section*.83}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{基于词对齐的翻译模型}{103}{section*.86}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{基于词对齐的翻译实例}{105}{section*.88}
\defcounter {refsection}{0}\relax
\contentsline {section}{\numberline {3.4}IBM模型1-2}{106}{section.3.4}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {3.4.1}IBM模型1}{106}{subsection.3.4.1}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {3.4.2}IBM模型2}{108}{subsection.3.4.2}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {3.4.3}解码及计算优化}{109}{subsection.3.4.3}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {3.4.4}训练}{110}{subsection.3.4.4}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{目标函数}{110}{section*.93}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{优化}{111}{section*.95}
\defcounter {refsection}{0}\relax
\contentsline {section}{\numberline {3.5}IBM模型3-5及隐马尔可夫模型}{117}{section.3.5}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {3.5.1}基于产出率的翻译模型}{117}{subsection.3.5.1}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {3.5.2}IBM 模型3}{120}{subsection.3.5.2}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {3.5.3}IBM 模型4}{121}{subsection.3.5.3}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {3.5.4} IBM 模型5}{123}{subsection.3.5.4}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {3.5.5}隐马尔可夫模型}{124}{subsection.3.5.5}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{隐马尔可夫模型}{125}{section*.107}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{词对齐模型}{126}{section*.109}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {3.5.6}解码和训练}{127}{subsection.3.5.6}
\defcounter {refsection}{0}\relax
\contentsline {section}{\numberline {3.6}问题分析}{127}{section.3.6}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {3.6.1}词对齐及对称化}{127}{subsection.3.6.1}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {3.6.2}Deficiency}{128}{subsection.3.6.2}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {3.6.3}句子长度}{129}{subsection.3.6.3}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {3.6.4}其他问题}{130}{subsection.3.6.4}
\defcounter {refsection}{0}\relax
\contentsline {section}{\numberline {3.7}小结及深入阅读}{130}{section.3.7}
\defcounter {refsection}{0}\relax
\contentsline {chapter}{\numberline {4}基于短语和句法的机器翻译模型}{133}{chapter.4}
\defcounter {refsection}{0}\relax
\contentsline {section}{\numberline {4.1}翻译中的结构信息}{133}{section.4.1}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {4.1.1}更大粒度的翻译单元}{134}{subsection.4.1.1}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {4.1.2}句子的结构信息}{136}{subsection.4.1.2}
\defcounter {refsection}{0}\relax
\contentsline {section}{\numberline {4.2}基于短语的翻译模型}{138}{section.4.2}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {4.2.1}机器翻译中的短语}{138}{subsection.4.2.1}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {4.2.2}数学建模及判别式模型}{141}{subsection.4.2.2}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{基于翻译推导的建模}{141}{section*.121}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{对数线性模型}{142}{section*.122}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{搭建模型的基本流程}{143}{section*.123}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {4.2.3}短语抽取}{144}{subsection.4.2.3}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{与词对齐一致的短语}{145}{section*.126}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{获取词对齐}{146}{section*.130}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{度量双语短语质量}{147}{section*.132}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {4.2.4}调序}{148}{subsection.4.2.4}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{基于距离的调序}{148}{section*.136}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{基于方向的调序}{149}{section*.138}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{基于分类的调序}{151}{section*.141}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {4.2.5}特征}{151}{subsection.4.2.5}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {4.2.6}最小错误率训练}{152}{subsection.4.2.6}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {4.2.7}栈解码}{155}{subsection.4.2.7}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{翻译候选匹配}{156}{section*.146}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{翻译假设扩展}{156}{section*.148}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{剪枝}{157}{section*.150}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{解码中的栈结构}{159}{section*.152}
\defcounter {refsection}{0}\relax
\contentsline {section}{\numberline {4.3}基于层次短语的模型}{160}{section.4.3}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {4.3.1}同步上下文无关文法}{163}{subsection.4.3.1}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{文法定义}{163}{section*.157}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{推导}{164}{section*.158}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{胶水规则}{165}{section*.159}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{处理流程}{166}{section*.160}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {4.3.2}层次短语规则抽取}{166}{subsection.4.3.2}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {4.3.3}翻译模型及特征}{168}{subsection.4.3.3}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {4.3.4}CYK解码}{169}{subsection.4.3.4}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {4.3.5}立方剪枝}{172}{subsection.4.3.5}
\defcounter {refsection}{0}\relax
\contentsline {section}{\numberline {4.4}基于语言学句法的模型}{175}{section.4.4}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {4.4.1}基于句法的翻译模型分类}{177}{subsection.4.4.1}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {4.4.2}基于树结构的文法}{177}{subsection.4.4.2}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{树到树翻译规则}{179}{section*.176}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{基于树结构的翻译推导}{181}{section*.178}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{树到串翻译规则}{183}{section*.181}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {4.4.3}树到串翻译规则抽取}{184}{subsection.4.4.3}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{树的切割与最小规则}{185}{section*.183}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{空对齐处理}{188}{section*.189}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{组合规则}{189}{section*.191}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{SPMT规则}{190}{section*.193}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{句法树二叉化}{191}{section*.195}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {4.4.4}树到树翻译规则抽取}{192}{subsection.4.4.4}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{基于节点对齐的规则抽取}{193}{section*.199}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{基于对齐矩阵的规则抽取}{194}{section*.202}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {4.4.5}句法翻译模型的特征}{196}{subsection.4.4.5}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {4.4.6}基于超图的推导空间表示}{197}{subsection.4.4.6}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {4.4.7}基于树的解码 vs 基于串的解码}{199}{subsection.4.4.7}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{基于树的解码}{201}{section*.209}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{基于串的解码}{202}{section*.212}
\defcounter {refsection}{0}\relax
\contentsline {section}{\numberline {4.5}小结及深入阅读}{204}{section.4.5}
\defcounter {refsection}{0}\relax
\contentsline {part}{\@mypartnumtocformat {III}{神经机器翻译}}{207}{part.3}
\ttl@stoptoc {default@2}
\ttl@starttoc {default@3}
\defcounter {refsection}{0}\relax
\contentsline {chapter}{\numberline {5}人工神经网络和神经语言建模}{209}{chapter.5}
\defcounter {refsection}{0}\relax
\contentsline {section}{\numberline {5.1}深度学习与人工神经网络}{210}{section.5.1}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {5.1.1}发展简史}{210}{subsection.5.1.1}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{早期的人工神经网络和第一次寒冬}{210}{section*.214}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{神经网络的第二次高潮和第二次寒冬}{211}{section*.215}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{深度学习和神经网络方法的崛起}{212}{section*.216}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {5.1.2}为什么需要深度学习}{213}{subsection.5.1.2}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{端到端学习和表示学习}{213}{section*.218}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{深度学习的效果}{214}{section*.220}
\defcounter {refsection}{0}\relax
\contentsline {section}{\numberline {5.2}神经网络基础}{214}{section.5.2}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {5.2.1}线性代数基础}{214}{subsection.5.2.1}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{标量、向量和矩阵}{215}{section*.222}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{矩阵的转置}{216}{section*.223}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{矩阵加法和数乘}{216}{section*.224}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{矩阵乘法和矩阵点乘}{217}{section*.225}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{线性映射}{218}{section*.226}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{范数}{219}{section*.227}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {5.2.2}人工神经元和感知机}{220}{subsection.5.2.2}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{感知机\ \raisebox {0.5mm}{------}\ 最简单的人工神经元模型}{221}{section*.230}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{神经元内部权重}{222}{section*.233}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{神经元的输入\ \raisebox {0.5mm}{------}\ 离散 vs 连续}{223}{section*.235}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{神经元内部的参数学习}{223}{section*.237}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {5.2.3}多层神经网络}{224}{subsection.5.2.3}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{线性变换和激活函数}{224}{section*.239}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{单层神经网络$\rightarrow $多层神经网络}{226}{section*.246}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {5.2.4}函数拟合能力}{227}{subsection.5.2.4}
\defcounter {refsection}{0}\relax
\contentsline {section}{\numberline {5.3}神经网络的张量实现}{231}{section.5.3}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {5.3.1} 张量及其计算}{232}{subsection.5.3.1}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{张量}{232}{section*.256}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{张量的矩阵乘法}{234}{section*.259}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{张量的单元操作}{235}{section*.261}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {5.3.2}张量的物理存储形式}{236}{subsection.5.3.2}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {5.3.3}使用开源框架实现张量计算}{236}{subsection.5.3.3}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {5.3.4}前向传播与计算图}{238}{subsection.5.3.4}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {5.3.5}神经网络实例}{241}{subsection.5.3.5}
\defcounter {refsection}{0}\relax
\contentsline {section}{\numberline {5.4}神经网络的参数训练}{242}{section.5.4}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {5.4.1}损失函数}{243}{subsection.5.4.1}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {5.4.2}基于梯度的参数优化}{243}{subsection.5.4.2}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{梯度下降}{244}{section*.279}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{梯度获取}{246}{section*.281}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{基于梯度的方法的变种和改进}{249}{section*.285}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {5.4.3}参数更新的并行化策略}{252}{subsection.5.4.3}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {5.4.4}梯度消失、梯度爆炸和稳定性训练}{254}{subsection.5.4.4}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{易于优化的激活函数}{254}{section*.288}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{梯度裁剪}{255}{section*.292}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{稳定性训练}{256}{section*.293}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {5.4.5}过拟合}{257}{subsection.5.4.5}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {5.4.6}反向传播}{258}{subsection.5.4.6}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{输出层的反向传播}{259}{section*.296}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{隐藏层的反向传播}{261}{section*.300}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{程序实现}{262}{section*.303}
\defcounter {refsection}{0}\relax
\contentsline {section}{\numberline {5.5}神经语言模型}{264}{section.5.5}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {5.5.1}基于神经网络的语言建模}{264}{subsection.5.5.1}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{基于前馈神经网络的语言模型}{265}{section*.306}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{基于循环神经网络的语言模型}{267}{section*.309}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{基于自注意力机制的语言模型}{268}{section*.311}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{语言模型的评价}{269}{section*.313}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {5.5.2}单词表示模型}{270}{subsection.5.5.2}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{One-hot编码}{270}{section*.314}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{分布式表示}{270}{section*.316}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {5.5.3}句子表示模型及预训练}{272}{subsection.5.5.3}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{简单的上下文表示模型}{272}{section*.320}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{ELMO模型}{274}{section*.323}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{GPT模型}{274}{section*.325}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{BERT模型}{275}{section*.327}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{为什么要预训练？}{276}{section*.329}
\defcounter {refsection}{0}\relax
\contentsline {section}{\numberline {5.6}小结及深入阅读}{277}{section.5.6}
\defcounter {refsection}{0}\relax
\contentsline {chapter}{\numberline {6}神经机器翻译模型}{279}{chapter.6}
\defcounter {refsection}{0}\relax
\contentsline {section}{\numberline {6.1}神经机器翻译的发展简史}{279}{section.6.1}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {6.1.1}神经机器翻译的起源}{281}{subsection.6.1.1}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {6.1.2}神经机器翻译的品质 }{283}{subsection.6.1.2}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {6.1.3}神经机器翻译的优势 }{286}{subsection.6.1.3}
\defcounter {refsection}{0}\relax
\contentsline {section}{\numberline {6.2}编码器-解码器框架}{288}{section.6.2}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {6.2.1}框架结构}{288}{subsection.6.2.1}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {6.2.2}表示学习}{289}{subsection.6.2.2}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {6.2.3}简单的运行实例}{290}{subsection.6.2.3}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {6.2.4}机器翻译范式的对比}{291}{subsection.6.2.4}
\defcounter {refsection}{0}\relax
\contentsline {section}{\numberline {6.3}基于循环神经网络的翻译模型及注意力机制}{292}{section.6.3}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {6.3.1}建模}{292}{subsection.6.3.1}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {6.3.2}输入（词嵌入）及输出（Softmax）}{296}{subsection.6.3.2}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {6.3.3}循环神经网络结构}{300}{subsection.6.3.3}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{循环神经单元（RNN）}{300}{section*.351}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{长短时记忆网络（LSTM）}{300}{section*.352}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{门控循环单元（GRU）}{302}{section*.355}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{双向模型}{304}{section*.357}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{多层循环神经网络}{304}{section*.359}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {6.3.4}注意力机制}{305}{subsection.6.3.4}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{翻译中的注意力机制}{306}{section*.362}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{上下文向量的计算}{307}{section*.365}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{注意力机制的解读}{310}{section*.370}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {6.3.5}训练}{312}{subsection.6.3.5}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{损失函数}{312}{section*.373}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{长参数初始化}{313}{section*.374}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{优化策略}{314}{section*.375}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{梯度裁剪}{314}{section*.377}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{学习率策略}{314}{section*.378}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{并行训练}{316}{section*.381}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {6.3.6}推断}{317}{subsection.6.3.6}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{贪婪搜索}{319}{section*.385}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{束搜索}{320}{section*.388}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{长度惩罚}{321}{section*.390}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {6.3.7}实例-GNMT}{322}{subsection.6.3.7}
\defcounter {refsection}{0}\relax
\contentsline {section}{\numberline {6.4}Transformer}{323}{section.6.4}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {6.4.1}自注意力模型}{325}{subsection.6.4.1}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {6.4.2}Transformer架构}{326}{subsection.6.4.2}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {6.4.3}位置编码}{328}{subsection.6.4.3}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {6.4.4}基于点乘的注意力机制}{331}{subsection.6.4.4}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {6.4.5}掩码操作}{333}{subsection.6.4.5}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {6.4.6}多头注意力}{334}{subsection.6.4.6}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {6.4.7}残差网络和层正则化}{335}{subsection.6.4.7}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {6.4.8}前馈全连接网络子层}{336}{subsection.6.4.8}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {6.4.9}训练}{337}{subsection.6.4.9}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {6.4.10}推断}{340}{subsection.6.4.10}
\defcounter {refsection}{0}\relax
\contentsline {section}{\numberline {6.5}序列到序列问题及应用}{340}{section.6.5}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {6.5.1}自动问答}{341}{subsection.6.5.1}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {6.5.2}自动文摘}{341}{subsection.6.5.2}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {6.5.3}文言文翻译}{342}{subsection.6.5.3}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {6.5.4}对联生成}{342}{subsection.6.5.4}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {6.5.5}古诗生成}{343}{subsection.6.5.5}
\defcounter {refsection}{0}\relax
\contentsline {section}{\numberline {6.6}小结及深入阅读}{344}{section.6.6}
\defcounter {refsection}{0}\relax
\contentsline {chapter}{\numberline {7}神经机器翻译实战 \ \raisebox {0.5mm}{------}\ 参加一次比赛}{347}{chapter.7}
\defcounter {refsection}{0}\relax
\contentsline {section}{\numberline {7.1}神经机器翻译并不简单}{347}{section.7.1}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {7.1.1}影响神经机器翻译性能的因素}{348}{subsection.7.1.1}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {7.1.2}搭建神经机器翻译系统的步骤 }{349}{subsection.7.1.2}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {7.1.3}架构选择 }{350}{subsection.7.1.3}
\defcounter {refsection}{0}\relax
\contentsline {section}{\numberline {7.2}数据处理}{350}{section.7.2}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {7.2.1}分词}{351}{subsection.7.2.1}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {7.2.2}标准化}{352}{subsection.7.2.2}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {7.2.3}数据清洗}{353}{subsection.7.2.3}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {7.2.4}子词切分}{355}{subsection.7.2.4}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{大词表和OOV问题}{356}{section*.428}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{子词}{356}{section*.430}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{双字节编码（BPE）}{357}{section*.432}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{其他方法}{360}{section*.435}
\defcounter {refsection}{0}\relax
\contentsline {section}{\numberline {7.3}建模与训练}{360}{section.7.3}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {7.3.1}正则化}{360}{subsection.7.3.1}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{L1/L2正则化}{362}{section*.437}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{标签平滑}{363}{section*.438}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{Dropout}{364}{section*.440}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{Layer Dropout}{365}{section*.443}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {7.3.2}增大模型容量}{366}{subsection.7.3.2}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{宽网络}{366}{section*.445}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{深网络}{367}{section*.447}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{增大输入层和输出层表示能力}{369}{section*.449}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{大模型的分布式计算}{369}{section*.450}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {7.3.3}大批量训练}{369}{subsection.7.3.3}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{为什么需要大批量训练}{370}{section*.451}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{如何构建批次}{371}{section*.454}
\defcounter {refsection}{0}\relax
\contentsline {section}{\numberline {7.4}推断}{372}{section.7.4}
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {7.4.1}推断优化}{372}{subsection.7.4.1}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{推断系统的架构}{372}{section*.456}
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{自左向右推断 vs 自右向左推断}{373}{section*.458}
\contentsline {chapter}{\numberline {1}基于词的机器翻译模型}{11}{chapter.1}%
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{推断加速}{374}{section*.459}
\contentsline {section}{\numberline {1.1}什么是基于词的翻译模型}{11}{section.1.1}%
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {7.4.2}译文长度控制}{381}{subsection.7.4.2}
\contentsline {section}{\numberline {1.2}构建一个简单的机器翻译系统}{13}{section.1.2}%
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{长度惩罚因子}{382}{section*.465}
\contentsline {subsection}{\numberline {1.2.1}如何进行翻译？}{13}{subsection.1.2.1}%
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{译文长度范围约束}{383}{section*.467}
\contentsline {subsubsection}{机器翻译流程}{14}{section*.7}%
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{覆盖度模型}{383}{section*.468}
\contentsline {subsubsection}{人工翻译 vs. 机器翻译}{15}{section*.9}%
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {7.4.3}多模型集成}{384}{subsection.7.4.3}
\contentsline {subsection}{\numberline {1.2.2}基本框架}{15}{subsection.1.2.2}%
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{假设选择}{385}{section*.469}
\contentsline {subsection}{\numberline {1.2.3}单词翻译概率}{16}{subsection.1.2.3}%
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{局部预测融合}{386}{section*.471}
\contentsline {subsubsection}{什么是单词翻译概率？}{16}{section*.11}%
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{译文重组}{387}{section*.473}
\contentsline {subsubsection}{如何从一个双语平行数据中学习？}{17}{section*.13}%
\defcounter {refsection}{0}\relax
\contentsline {section}{\numberline {7.5}进阶技术}{388}{section.7.5}
\contentsline {subsubsection}{如何从大量的双语平行数据中学习？}{18}{section*.14}%
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {7.5.1}深层模型}{388}{subsection.7.5.1}
\contentsline {subsection}{\numberline {1.2.4}句子级翻译模型}{19}{subsection.1.2.4}%
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{Post-Norm vs Pre-Norm}{388}{section*.476}
\contentsline {subsubsection}{基础模型}{19}{section*.16}%
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{层聚合}{391}{section*.479}
\contentsline {subsubsection}{生成流畅的译文}{21}{section*.18}%
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{深层模型的训练加速}{392}{section*.481}
\contentsline {subsection}{\numberline {1.2.5}解码}{23}{subsection.1.2.5}%
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{渐进式训练}{392}{section*.482}
\contentsline {section}{\numberline {1.3}基于词的翻译建模}{26}{section.1.3}%
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{分组稠密连接}{392}{section*.484}
\contentsline {subsection}{\numberline {1.3.1}噪声信道模型}{26}{subsection.1.3.1}%
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{学习率重置策略}{393}{section*.486}
\contentsline {subsection}{\numberline {1.3.2}统计机器翻译的三个基本问题}{28}{subsection.1.3.2}%
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{深层模型的鲁棒性训练}{395}{section*.488}
\contentsline {subsubsection}{词对齐}{29}{section*.27}%
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {7.5.2}单语数据的使用}{396}{subsection.7.5.2}
\contentsline {subsubsection}{基于词对齐的翻译模型}{30}{section*.30}%
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{伪数据}{397}{section*.491}
\contentsline {subsubsection}{基于词对齐的翻译实例}{31}{section*.32}%
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{预训练}{399}{section*.494}
\contentsline {section}{\numberline {1.4}IBM模型1-2}{32}{section.1.4}%
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{联合训练}{401}{section*.497}
\contentsline {subsection}{\numberline {1.4.1}IBM模型1}{32}{subsection.1.4.1}%
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {7.5.3}知识精炼}{401}{subsection.7.5.3}
\contentsline {subsection}{\numberline {1.4.2}IBM模型2}{34}{subsection.1.4.2}%
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{什么是知识精炼}{402}{section*.499}
\contentsline {subsection}{\numberline {1.4.3}解码及计算优化}{35}{subsection.1.4.3}%
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{知识精炼的基本方法}{403}{section*.500}
\contentsline {subsection}{\numberline {1.4.4}训练}{36}{subsection.1.4.4}%
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{机器翻译中的知识精炼}{404}{section*.502}
\contentsline {subsubsection}{目标函数}{36}{section*.37}%
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {7.5.4}双向训练}{406}{subsection.7.5.4}
\contentsline {subsubsection}{优化}{37}{section*.39}%
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{有监督对偶学习}{406}{section*.504}
\contentsline {section}{\numberline {1.5}IBM模型3-5及隐马尔可夫模型}{42}{section.1.5}%
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{无监督对偶学习}{407}{section*.505}
\contentsline {subsection}{\numberline {1.5.1}基于产出率的翻译模型}{44}{subsection.1.5.1}%
\defcounter {refsection}{0}\relax
\contentsline {subsubsection}{翻译中回译}{408}{section*.507}
\contentsline {subsection}{\numberline {1.5.2}IBM 模型3}{46}{subsection.1.5.2}%
\defcounter {refsection}{0}\relax
\contentsline {section}{\numberline {7.6}小结及深入阅读}{408}{section.7.6}
\contentsline {subsection}{\numberline {1.5.3}IBM 模型4}{48}{subsection.1.5.3}%
\defcounter {refsection}{0}\relax
\contentsline {part}{\@mypartnumtocformat {IV}{附录}}{413}{part.4}
\ttl@stoptoc {default@3}
\ttl@starttoc {default@4}
\contentsline {subsection}{\numberline {1.5.4} IBM 模型5}{49}{subsection.1.5.4}%
\defcounter {refsection}{0}\relax
\contentsline {chapter}{\numberline {A}附录A}{415}{Appendix.1.A}
\contentsline {subsection}{\numberline {1.5.5}隐马尔可夫模型}{51}{subsection.1.5.5}%
\defcounter {refsection}{0}\relax
\contentsline {section}{\numberline {A.1}基准数据集}{415}{section.1.A.1}
\contentsline {subsubsection}{隐马尔可夫模型}{51}{section*.51}%
\defcounter {refsection}{0}\relax
\contentsline {section}{\numberline {A.2}平行语料}{416}{section.1.A.2}
\contentsline {subsubsection}{词对齐模型}{52}{section*.53}%
\defcounter {refsection}{0}\relax
\contentsline {section}{\numberline {A.3}相关工具}{417}{section.1.A.3}
\contentsline {subsection}{\numberline {1.5.6}解码和训练}{53}{subsection.1.5.6}%
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {A.3.1}数据预处理工具}{417}{subsection.1.A.3.1}
\contentsline {section}{\numberline {1.6}问题分析}{54}{section.1.6}%
\defcounter {refsection}{0}\relax
\contentsline {subsection}{\numberline {A.3.2}评价工具}{418}{subsection.1.A.3.2}
\contentsline {subsection}{\numberline {1.6.1}词对齐及对称化}{54}{subsection.1.6.1}%
\defcounter {refsection}{0}\relax
\contentsline {chapter}{\numberline {B}附录B}{419}{Appendix.2.B}
\contentsline {subsection}{\numberline {1.6.2}Deficiency}{55}{subsection.1.6.2}%
\defcounter {refsection}{0}\relax
\contentsline {section}{\numberline {B.1}IBM模型3训练方法}{419}{section.2.B.1}
\contentsline {subsection}{\numberline {1.6.3}句子长度}{56}{subsection.1.6.3}%
\defcounter {refsection}{0}\relax
\contentsline {section}{\numberline {B.2}IBM模型4训练方法}{421}{section.2.B.2}
\contentsline {subsection}{\numberline {1.6.4}其他问题}{56}{subsection.1.6.4}%
\defcounter {refsection}{0}\relax
\contentsline {section}{\numberline {B.3}IBM模型5训练方法}{423}{section.2.B.3}
\contentsline {section}{\numberline {1.7}小结及深入阅读}{57}{section.1.7}%
\contentsfinish
Book/mt-book-xelatex.tex
......@@ -3,28 +3,29 @@
%----------------------------------------------------------------------------------------
% PACKAGES AND OTHER DOCUMENT CONFIGURATIONS
% BASIC CONFIGURATIONS
%----------------------------------------------------------------------------------------
\documentclass[11pt]{book} % Default font size and left-justified equations
\input{structure.tex} % Insert the commands.tex file which contains the majority of the structure behind the template
\documentclass[11pt]{book} % font and book template
\input{structure.tex} % self-defined template
\usepackage{hyperref}
%\hypersetup{pdftitle={Title},pdfauthor={Author}} % Uncomment and fill out to include PDF metadata for the author and title of the book
%\usepackage{CJKutf8}%CJK用的宏包
\usepackage {xeCJK}
\usepackage{ctex}
\setCJKmainfont{SimSun}
\setCJKmonofont{SimSun}
\setmainfont{Times New Roman}
%----------------------------------------------------------------------------------------
% CHINESE FONTS AND MATH FONTS
%----------------------------------------------------------------------------------------
{\newcommand{\mycfont}{song}}
{\newcommand{\mycfont}{gbsn}}
%公式字体设置为计算机现代罗马
% math fount = Computer Modern Roman
\AtBeginDocument{
\SetSymbolFont{operators}{normal}{OT1}{cmr} {m}{n}
\SetSymbolFont{letters}{normal}{OML}{cmm} {m}{it}
......@@ -44,15 +45,21 @@
\SetMathAlphabet{\mathit}{bold}{OT1}{cmr}{bx}{it}
\SetMathAlphabet{\mathtt}{bold}{OT1}{cmtt}{m}{n}
}
\renewcommand{\baselinestretch}{1.2}%设置行间距
\renewcommand{\baselinestretch}{1.2} % spacing
%----------------------------------------------------------------------------------------
% MAIN BODY OF THE BOOK
%----------------------------------------------------------------------------------------
\begin{document}
%\begin{CJK}{UTF8}{\mycfont}%原来的CJK
%----------------------------------------------------------------------------------------
% TITLE PAGE
%----------------------------------------------------------------------------------------
\begingroup
\thispagestyle{empty} % Suppress headers and footers on the title page
\thispagestyle{empty}
\begin{tikzpicture}[remember picture,overlay]
\node[inner sep=0pt] (background) at (current page.center) {\includegraphics[width=\paperwidth,height=\paperheight]{fig-cover.jpg}};
......@@ -69,15 +76,16 @@
~\vfill
\thispagestyle{empty}
\noindent Copyright \copyright\ 2020 肖桐\ \ 朱靖波\\ % Copyright notice
\noindent Copyright \copyright\ 2020 肖桐\ \ 朱靖波\\
\noindent \textsc{东北大学自然语言处理实验室\ /\ 小牛翻译}\\ % Publisher
\noindent \textsc{东北大学自然语言处理实验室\ $\cdot$\ 小牛翻译}\\
\noindent \textsc{\url{https://github.com/NiuTrans/MTBook}}\\ % URL
\noindent \textsc{\url{https://opensource.niutrans.com/mtbook/index.html}}\\
\noindent \textsc{\url{https://github.com/NiuTrans/MTBook}}\\
\noindent {\red{Licensed under the Creative Commons Attribution-NonCommercial 4.0 Unported License (the ``License''). You may not use this file except in compliance with the License. You may obtain a copy of the License at \url{http://creativecommons.org/licenses/by-nc/4.0}. Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an \textsc{``as is'' basis, without warranties or conditions of any kind}, either express or implied. See the License for the specific language governing permissions and limitations under the License.}}\\ % License information, replace this with your own license (if any)
\noindent {\red{Licensed under the Creative Commons Attribution-NonCommercial 4.0 Unported License (the ``License''). You may not use this file except in compliance with the License. You may obtain a copy of the License at \url{http://creativecommons.org/licenses/by-nc/4.0}. Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an \textsc{``as is'' basis, without warranties or conditions of any kind}, either express or implied. See the License for the specific language governing permissions and limitations under the License.}}\\
\noindent \textit{First Edition, April 2020}
\noindent \textit{\today}
%----------------------------------------------------------------------------------------
% ACKNOWLEDGE PAGE
......@@ -90,20 +98,24 @@
{\large
\noindent {\color{red} 在此感谢所有为本书做出贡献的人} \\
\noindent 曹润柘、曾信、孟霞、单韦乔、姜雨帆、王子扬、刘辉、许诺、李北、刘继强、张哲旸、周书含、周涛、张裕浩、李炎洋、林野、刘晓倩、牛蕊 \\
\noindent 曹润柘、曾信、孟霞、单韦乔、姜雨帆、王子扬、刘辉、许诺、李北、刘继强、张哲旸、周书含、周涛、张裕浩、李炎洋、林野、陈贺轩、刘晓倩、牛蕊、田丰宁、杜权 \\
}
%----------------------------------------------------------------------------------------
% PREFACE PAGES
%----------------------------------------------------------------------------------------
\newpage
\include{ChapterPreface/ChapterPreface}
%----------------------------------------------------------------------------------------
% TABLE OF CONTENTS
%----------------------------------------------------------------------------------------
%\usechapterimagefalse % If you don't want to include a chapter image, use this to toggle images off - it can be enabled later with \usechapterimagetrue
\chapterimage{fig-NEU-1.jpg} %目录标题的图案
\chapterimage{fig-NEU-1.jpg} % Image of the content page
\pagestyle{empty} % Disable headers and footers for the following pages
\tableofcontents % 打印目录
\cleardoublepage %保证章节页在奇数页
\pagestyle{fancy} % Enable headers and footers again
\tableofcontents % Show contents
\cleardoublepage % Place the first page of each chapter on odd pages
\pagestyle{fancy} % Enable headers and footers
%----------------------------------------------------------------------------------------
......@@ -113,35 +125,30 @@
%\include{Chapter1/chapter1}
%\include{Chapter2/chapter2}
%\include{Chapter3/chapter3}
%\include{Chapter4/chapter4}
\include{Chapter4/chapter4}
%\include{Chapter5/chapter5}
%\include{Chapter6/chapter6}
\include{Chapter7/chapter7}
%\include{Chapter7/chapter7}
%\include{ChapterAppend/chapterappend}
%----------------------------------------------------------------------------------------
% BIBLIOGRAPHY
%----------------------------------------------------------------------------------------
\chapterimage{fig-NEU-10.jpg} %目录标题的图案
\chapterimage{fig-NEU-10.jpg} % Image of the header
\cleardoublepage % Make sure the index starts on an odd (right side) page
\printbibliography
%----------------------------------------------------------------------------------------
% INDEX
%----------------------------------------------------------------------------------------
\chapterimage{fig-NEU-10.jpg} %目录标题的图案
\chapterimage{fig-NEU-10.jpg} % Image of the header
\cleardoublepage % Make sure the index starts on an odd (right side) page
%\phantomsection
%\setlength{\columnsep}{0.75cm} % Space between the 2 columns of the index
%\addcontentsline{toc}{chapter}{\textcolor{ocre}{Index}} % Add an Index heading to the table of contents
\printindex % Output the index
\printindex % Show index
%-------------------------
......
Book/structure.tex
......@@ -50,7 +50,7 @@
paper=b5paper, % Paper size, change to letterpaper for US letter size
%papersize={185mm,260mm}, % specify paper size by (width,height)
top=2cm, % Top margin
bottom=1.5cm, % Bottom margin
bottom=1.5cm, % Bottom margin原来1.5cm
left=1.8cm, % Left margin
right=1.8cm, % Right margin
headheight=10pt, % Header height
......@@ -76,7 +76,7 @@
% BIBLIOGRAPHY AND INDEX
%----------------------------------------------------------------------------------------
\usepackage[style=numeric,citestyle=numeric,sorting=nyt,sortcites=true,maxbibnames=40,minbibnames=30,autopunct=true,babel=hyphen,hyperref=true,abbreviate=false,backref=true,backend=biber]{biblatex}
\usepackage[style=numeric,citestyle=numeric,sorting=anyt,sortcites=true,maxbibnames=40,minbibnames=30,autopunct=true,babel=hyphen,hyperref=true,abbreviate=false,backref=true,backend=biber]{biblatex}
%maxbibnames 设置参考文献最多显示作者数目
%minbibnames 如果作者数目超过maxbibnames，则只显示minbibnames个作者
\addbibresource{bibliography.bib} % BibTeX bibliography file
......@@ -85,6 +85,7 @@
\usepackage{calc} % For simpler calculation - used for spacing the index letter headings correctly
\usepackage{makeidx} % Required to make an index
\makeindex % Tells LaTeX to create the files required for indexing
\newcommand{\upcite}[1]{\textsuperscript{\textsuperscript{\cite{#1}}}}%参考文献上标
%----------------------------------------------------------------------------------------
% MAIN TABLE OF CONTENTS
......@@ -178,14 +179,16 @@
\pagestyle{fancy} % Enable the custom headers and footers
\renewcommand{\chaptermark}[1]{\markboth{\sffamily\normalsize\bfseries\chaptername\ \thechapter.\ #1}{}} % Styling for the current chapter in the header
\renewcommand{\chaptermark}[1]{\markboth{\sffamily\normalsize\bfseries\chaptername\ \thechapter.\ #1 \quad 肖桐\ 朱靖波}{}} % Styling for the current chapter in the header
\renewcommand{\sectionmark}[1]{\markright{\sffamily\normalsize\thesection\hspace{5pt}#1}{}} % Styling for the current section in the header
\fancyhf{} % Clear default headers and footers
\fancyhead[LE,RO]{\sffamily\normalsize\thepage} % Styling for the page number in the header
\fancyhead[LO]{\rightmark} % Print the nearest section name on the left side of odd pages
\fancyhead[RE]{\leftmark} % Print the current chapter name on the right side of even pages
%\fancyfoot[C]{\thepage} % Uncomment to include a footer
%\fancyfoot[RE]{\tiny{肖桐\ 朱靖波}}
%O-odd page,E-even page,R-right area,L-left area,C-center area
%\fancyfoot[C]{\thepage} % Uncomment to include a footer底部中间页码
\renewcommand{\headrulewidth}{0.5pt} % Thickness of the rule under the header
......@@ -543,7 +546,7 @@ addtohook={%
%----------------------------------------------------------------------------------------
% NEW PAGE FOR SUBSECTION
%----------------------------------------------------------------------------------------
%\newcommand{\sectionnewpage}{\newpage}
%\newcommand{\sectionnewpage}{\clearpage}
\newcommand{\sectionnewpage}{}
%----------------------------------------------------------------------------------------
......
Markdown 格式
0% 或
您添加了 0 人 到此讨论。请谨慎行事。
请先完成此评论的编辑！
请 注册 或者 后发表评论