Skip to content

M
mtbookv2
Commit 484db460 by 孟霞
Options

合并分支 'master' 到 'mengxia' 

Master

查看合并请求 !962
parents f1caac97 ace46a9f
正在显示 包含 55 行增加 和 44 行删除
Chapter13/chapter13.tex
......@@ -570,14 +570,14 @@ Loss_{\textrm{robust}}(\theta_{\textrm{mt}}) &=& \frac{1}{N}\sum_{(\mathbi{x},\
\subsubsection{2. 演员-评论家方法}
\parinterval 基于策略的强化学习是要寻找一个策略$\funp{p}(a|\hat{{y}}_{1 \ldots j},\seq{x})$，使得该策略选择的行动$a$未来可以获得的奖励期望（也被称为{\small\bfnew{动作价值函数}}\index{动作价值函数}（Action-value Function）\index{Action-value Function}）最大化。这个过程通常用函数$Q$来描述：
\parinterval 基于策略的强化学习是要寻找一个策略$\funp{p}(a|\hat{{y}}_{1 \ldots j-1},\seq{x})$，使得该策略选择的行动$a$未来可以获得的奖励期望最大化，也被称为{\small\bfnew{动作价值函数}}\index{动作价值函数}（Action-value Function）\index{Action-value Function}最大化。这个过程通常用函数$Q$来描述：
\begin{eqnarray}
\funp{Q}(a;\hat{y}_{1 \ldots j},\seq{y}) & = & \mathbb{E}_{\hat{y}_{j+1 \ldots J} \sim \funp{p}(a|\hat{y}_{1 \ldots j} a,\seq{x})}[\funp{r}_j(a;\hat{y}_{1 \ldots j-1},\seq{y}) + \nonumber \\
& & \sum_{i=j+1}^J\funp{r}_i(\hat{{y}}_i;\hat{{y}}_{1 \ldots j-1}a\hat{{y}}_{j+1 \ldots i-1},\seq{y})]
\funp{Q}(a;\hat{y}_{1 \ldots j-1},\seq{y}) & = & \mathbb{E}_{\hat{y}_{j+1 \ldots J} \sim \funp{p}(\cdot|\hat{y}_{1 \ldots j-1} a,\seq{x})}[\funp{r}_j(a;\hat{y}_{1 \ldots j-1},\seq{y}) + \nonumber \\
& & \sum_{i=j+1}^J\funp{r}_i(\hat{{y}}_i;\hat{{y}}_{1 \ldots j-1}a\hat{{y}}_{j+1 \ldots i},\seq{y})]
\label{eq:13-16}
\end{eqnarray}
\noindent 其中，$\funp{r}_j(a;\hat{{y}}_{1 \ldots j-1},\seq{y})$是$j$时刻做出行动$a$获得的奖励，$\funp{r}_i(\hat{{y}}_i;\hat{{y}}_{1 \ldots j-1}a\hat{{y}}_{j+1 \ldots i-1},\seq{y})$是在$j$时刻的行动为$a$的前提下，$i$时刻的做出行动$\hat{{y}}_i$获得的奖励，$\seq{x}$是源语言句子，$\seq{y}$是正确译文，$\hat{{y}}_{1 \ldots j}$是策略$\funp{p}$产生的译文的前$j$个词，$J$是生成译文的长度。对于源语句子$x$，最优策略$\hat{p}$可以被定义为：
\noindent 其中，$\funp{r}_j(a;\hat{{y}}_{1 \ldots j-1},\seq{y})$是$j$时刻做出行动$a$获得的奖励，$\funp{r}_i(\hat{{y}}_i;\hat{{y}}_{1 \ldots j-1}a\hat{{y}}_{j+1 \ldots i},\seq{y})$是在$j$时刻的行动为$a$的前提下，$i$时刻的做出行动$\hat{{y}}_i$获得的奖励，$\seq{x}$是源语言句子，$\seq{y}$是正确译文，$\hat{{y}}_{1 \ldots j-1}$是策略$\funp{p}$产生的译文的前$j-1$个词，$J$是生成译文的长度。对于源语句子$x$，最优策略$\hat{p}$可以被定义为：
\begin{eqnarray}
\hat{p} & = & \argmax_{\funp{p}}\mathbb{E}_{\hat{\seq{y}} \sim \funp{p}(\hat{\seq{y}} | \seq{x})}\sum_{j=1}^J\sum_{a \in A}\funp{p}(a|\hat{{y}}_{1 \ldots j},\seq{x})\funp{Q}(a;\hat{{y}}_{1 \ldots j},\seq{y})
\label{eq:13-17}
......
Chapter15/Figures/figure-introducing-rnn-mechanism-into-transformer.tex
......@@ -5,20 +5,20 @@
\tikzstyle{tnode}=[rectangle,inner sep=0mm,minimum height=1.8em,minimum width=4em,rounded corners=5pt,thick,draw,fill=teal!20]
\tikzstyle{stnode}=[rectangle,inner sep=0mm,minimum height=1.8em,minimum width=4em,rounded corners=5pt,thick,fill=red!20,draw]
\tikzstyle{rnnnode}=[rectangle,inner sep=0mm,minimum height=2em,minimum width=6.5em,rounded corners=5pt,thick,fill=green!20,draw]
\tikzstyle{rnnnode}=[rectangle,inner sep=0mm,minimum height=1.8em,minimum width=5em,rounded corners=5pt,thick,fill=green!20,draw]
\tikzstyle{wnode}=[inner sep=0mm,minimum height=2em,minimum width=5em]
\tikzstyle{wnode}=[inner sep=0mm,minimum height=2em,minimum width=4em]
\begin{scope}[]
\node [anchor=west,rnnnode] (r1) at (0, 0) {循环单元$\mathbi{h}_1$};
\node [anchor=south,rnnnode] (r2) at ([xshift=0em,yshift=1em]r1.north){循环单元$\mathbi{h}_2$};
\node [anchor=west,rnnnode] (r1) at (0, 0) {循环单元};
\node [anchor=south,rnnnode] (r2) at ([xshift=0em,yshift=1em]r1.north){循环单元};
\node [anchor=south] (r3) at ([xshift=0em,yshift=1em]r2.north){$\cdots$};
\node [anchor=south,rnnnode] (r4) at ([xshift=0em,yshift=1em]r3.north){循环单元$\mathbi{h}_n$};
\node [anchor=south,rnnnode] (r4) at ([xshift=0em,yshift=1em]r3.north){循环单元};
\node [anchor=east,wnode,font=\footnotesize] (wr1) at ([xshift=-1em,yshift=0em]r1.west) {1时刻输入};
\node [anchor=east,wnode,font=\footnotesize] (wr2) at ([xshift=-1em,yshift=0em]r2.west) {2时刻输入};
\node [anchor=east,wnode,font=\footnotesize] (wr3) at ([xshift=-1em,yshift=0em]r4.west) {$n$时刻输入};
\node [anchor=east,wnode,font=\footnotesize,align=left,minimum width=3em] (wr1) at ([xshift=-1em,yshift=0em]r1.west) {1时刻\\ 输入};
\node [anchor=east,wnode,font=\footnotesize,align=left,minimum width=3em] (wr2) at ([xshift=-1em,yshift=0em]r2.west) {2时刻\\ 输入};
\node [anchor=east,wnode,font=\footnotesize,align=left,minimum width=3em] (wr3) at ([xshift=-1em,yshift=0em]r4.west) {$n$时刻\\ 输入};
\node [anchor=north,wnode] (input) at ([xshift=0em,yshift=-1em]r1.south) {$\mathbi{h}_0$};
\node [anchor=south,wnode] (output) at ([xshift=0em,yshift=1em]r4.north) {输出};
......@@ -34,11 +34,11 @@
\draw[->,thick] ([xshift=0em,yshift=0em]wr2.east)--([xshift=0em,yshift=0em]r2.west);
\draw[->,thick] ([xshift=0em,yshift=0em]wr3.east)--([xshift=0em,yshift=0em]r4.west);
\node [anchor=north,font=\small] (label) at ([xshift=-4em,yshift=-0.5em]input.south) {(a)RNN};
\node [anchor=north,font=\small] (label) at ([xshift=-3em,yshift=-0.7em]input.south) {(a)RNN};
\end{scope}
\begin{scope}[xshift=1.9in]
\begin{scope}[xshift=1.85in]
\node [anchor=west,stnode] (r1) at (0, 0) {第1层};
\node [anchor=south,tnode] (r2) at ([xshift=0em,yshift=1em]r1.north){第2层};
......@@ -60,20 +60,20 @@
\draw[->,thick] ([xshift=0em,yshift=0em]r3.north)--([xshift=0em,yshift=0em]r4.south);
\draw[->,thick] ([xshift=0em,yshift=0em]r4.north)--([xshift=0em,yshift=0em]output.south);
\node [anchor=north,font=\small] (label) at ([xshift=-2em,yshift=-0.5em]input.south) {(b)原始Transformer模型};
\node [anchor=north,font=\small] (label) at ([xshift=-1.5em,yshift=-0.7em]input.south) {(b)原始Transformer模型};
\end{scope}
\begin{scope}[xshift=3.7in]
\begin{scope}[xshift=3.9in]
\node [anchor=west,stnode] (r1) at (0, 0) {第1层};
\node [anchor=south,stnode] (r2) at ([xshift=0em,yshift=1em]r1.north){第2层};
\node [anchor=south] (r3) at ([xshift=0em,yshift=1em]r2.north){$\cdots$};
\node [anchor=south,stnode] (r4) at ([xshift=0em,yshift=1em]r3.north){第$n$层};
\node [anchor=east,wnode,font=\footnotesize,align=left,minimum width=4em] (wr1) at ([xshift=-1em,yshift=0em]r1.west) {1时刻编\\ 码向量};
\node [anchor=east,wnode,font=\footnotesize,align=left,minimum width=4em] (wr2) at ([xshift=-1em,yshift=0em]r2.west) {2时刻编\\ 码向量};
\node [anchor=east,wnode,font=\footnotesize,align=left,minimum width=4em] (wr3) at ([xshift=-1em,yshift=0em]r4.west) {$n$时刻编\\ 码向量};
\node [anchor=east,wnode,font=\footnotesize,align=left] (wr1) at ([xshift=-1em,yshift=0em]r1.west) {1时刻\\ 编码向量};
\node [anchor=east,wnode,font=\footnotesize,align=left] (wr2) at ([xshift=-1em,yshift=0em]r2.west) {2时刻\\ 编码向量};
\node [anchor=east,wnode,font=\footnotesize,align=left] (wr3) at ([xshift=-1em,yshift=0em]r4.west) {$n$时刻\\ 编码向量};
\node [anchor=north,wnode] (input) at ([xshift=0em,yshift=-1em]r1.south) {$\mathbi{h}_0$};
\node [anchor=south,wnode] (output) at ([xshift=0em,yshift=1em]r4.north) {输出};
......@@ -89,7 +89,7 @@
\draw[->,thick] ([xshift=0em,yshift=0em]wr2.east)--([xshift=0em,yshift=0em]r2.west);
\draw[->,thick] ([xshift=0em,yshift=0em]wr3.east)--([xshift=0em,yshift=0em]r4.west);
\node [anchor=north,font=\small,align=left] (label) at ([xshift=-2em,yshift=0.2em]input.south) {(c)共享权重的\\ Transformer模型};
\node [anchor=north,font=\small,align=left] (label) at ([xshift=-3em,yshift=-0.7em]input.south) {(c)共享权重的\\ Transformer模型};
\end{scope}
......
Chapter15/Figures/figure-relative-position-coding-and-absolute-position-coding.tex
......@@ -21,25 +21,28 @@
\node [ffnnode,anchor=south] (ffn2) at ([yshift=0.5em]res3.north) {\tiny{$\textbf{Feed Forward Network}$}};
\node [Resnode,anchor=south] (res4) at ([yshift=0.3em]ffn2.north) {\tiny{$\textbf{Add \& LayerNorm}$}};
\node [anchor=north west,font=\scriptsize,align=center] (l1) at ([xshift=0.1em]ffn1.north east) {第 \\ 一 \\ 层};
\node [anchor=north west,font=\scriptsize,align=center] (l2) at ([xshift=0.1em]ffn2.north east) {第 \\ $n$ \\ 层};
\node [anchor=north west,font=\scriptsize,align=center,ublue] (l1) at ([xshift=0.1em]ffn1.north east) {第 \\ $1$ \\ 层};
\node [anchor=north west,font=\scriptsize,align=center,ublue] (l2) at ([xshift=0.1em]ffn2.north east) {第 \\ $n$ \\ 层};
\node [anchor=south east,font=\scriptsize] (l3) at ([yshift=3em]l1.north east) {};
\node [anchor=north east,font=\scriptsize,align=center,opacity=0.0] (l4) at ([xshift=0.1em]ffn1.north west) {第 \\ 一 \\ 层};
\node [anchor=north east,font=\scriptsize,align=center,opacity=0.0] (l4) at ([xshift=0.1em]ffn1.north west) {第 \\ $1$ \\ 层};
\node [anchor=north east,font=\scriptsize,align=center,opacity=0.0] (l5) at ([xshift=0.1em]ffn2.north west) {第 \\ $n$ \\ 层};
\node [anchor=south west,font=\scriptsize] (l6) at ([yshift=3em]l4.north west) {};
\node [anchor=south west] (e1) at ([xshift=-1.4em,yshift=0.3em]res4.north west) {{编码器}};
\node [anchor=north,minimum height=0.1em,minimum width=0.1em] (set1) at ([xshift=0em,yshift=0.3em]sa1.south) {};
\node [anchor=north,minimum height=0.1em,minimum width=0.1em] (set2) at ([xshift=0em,yshift=0.3em]sa2.south) {};
\begin{pgfonlayer}{background}
\node [rectangle,inner sep=0.6em,rounded corners=5pt,very thick,dotted,minimum height=1.4em,minimum width=3.5em,draw=red!40] [fit = (e1) (sa1) (l1) (l4)] (b4) {};
\node [rectangle,inner sep=0.3em,rounded corners=5pt,very thick,dotted,draw=green!40,minimum height=1.4em,minimum width=7em] [fit = (l1) (sa1) (res2) (l4)] (b1) {};
\node [rectangle,inner sep=0.3em,rounded corners=5pt,very thick,dotted,draw=green!40,minimum height=1.4em,minimum width=7em] [fit = (dot1) (l3) (l6)] (b2) {};
\node [rectangle,inner sep=0.3em,rounded corners=5pt,very thick,dotted,draw=green!40,minimum height=1.4em,minimum width=7em] [fit = (l2) (sa2) (res4) (l5)] (b3) {};
\node [rectangle,inner sep=0.6em,rounded corners=5pt,very thick,dotted,minimum height=1.4em,minimum width=3.5em,draw=red] [fit = (e1) (sa1) (l1) (l4) (set1)] (b4) {};
\node [rectangle,inner sep=0.3em,rounded corners=5pt,very thick,dotted,draw=ublue,minimum height=1.4em,minimum width=7em] [fit = (l1) (sa1) (res2) (l4) (set1)] (b1) {};
\node [rectangle,inner sep=0.3em,rounded corners=5pt,very thick,dotted,draw=ublue,minimum height=1.4em,minimum width=7em] [fit = (dot1) (l3) (l6)] (b2) {};
\node [rectangle,inner sep=0.3em,rounded corners=5pt,very thick,dotted,draw=ublue,minimum height=1.4em,minimum width=7em] [fit = (l2) (sa2) (res4) (l5) (set2)] (b3) {};
\end{pgfonlayer}
\node [inputnode,anchor=north west] (input1) at ([yshift=-1.1em,xshift=-0.5em]sa1.south west) {\tiny{$\textbf{Embedding}$}};
\node [] (add) at ([yshift=-1.7em,xshift=3.5em]sa1.south west) {$+$};
\node [posnode,anchor=north east] (pos1) at ([yshift=-1.1em,xshift=1.6em]sa1.south east) {\tiny{$\textbf{Absolute Position}$}};
\node [inputnode,anchor=north west] (input1) at ([yshift=-1.6em,xshift=-0.5em]sa1.south west) {\tiny{$\textbf{Embedding}$}};
\node [] (add) at ([yshift=-2.2em,xshift=3.5em]sa1.south west) {$+$};
\node [posnode,anchor=north east] (pos1) at ([yshift=-1.6em,xshift=1.6em]sa1.south east) {\tiny{$\textbf{Absolute Position}$}};
\node [anchor=north] (wi) at ([yshift=-0.5em]pos1.south) {\scriptsize{词序信息}};
......@@ -83,25 +86,28 @@
\node [ffnnode,anchor=south] (ffn2) at ([yshift=0.5em]res3.north) {\tiny{$\textbf{Feed Forward Network}$}};
\node [Resnode,anchor=south] (res4) at ([yshift=0.3em]ffn2.north) {\tiny{$\textbf{Add \& LayerNorm}$}};
\node [anchor=north west,font=\scriptsize,align=center] (l1) at ([xshift=0.1em]ffn1.north east) {第 \\ 一 \\ 层};
\node [anchor=north west,font=\scriptsize,align=center] (l2) at ([xshift=0.1em]ffn2.north east) {第 \\ $n$ \\ 层};
\node [anchor=north west,font=\scriptsize,align=center,ublue] (l1) at ([xshift=0.1em]ffn1.north east) {第 \\ $1$ \\ 层};
\node [anchor=north west,font=\scriptsize,align=center,ublue] (l2) at ([xshift=0.1em]ffn2.north east) {第 \\ $n$ \\ 层};
\node [anchor=south east,font=\scriptsize] (l3) at ([yshift=3em]l1.north east) {};
\node [anchor=north east,font=\scriptsize,align=center,opacity=0.0] (l4) at ([xshift=0.1em]ffn1.north west) {第 \\ 一 \\ 层};
\node [anchor=north east,font=\scriptsize,align=center,opacity=0.0] (l4) at ([xshift=0.1em]ffn1.north west) {第 \\ $1$ \\ 层};
\node [anchor=north east,font=\scriptsize,align=center,opacity=0.0] (l5) at ([xshift=0.1em]ffn2.north west) {第 \\ $n$ \\ 层};
\node [anchor=south west,font=\scriptsize] (l6) at ([yshift=3em]l4.north west) {};
\node [anchor=south west] (e1) at ([xshift=-1.4em,yshift=0.3em]res4.north west) {{编码器}};
\node [anchor=north,minimum height=0.1em,minimum width=0.1em] (set1) at ([xshift=0em,yshift=0.3em]sa1.south) {};
\node [anchor=north,minimum height=0.1em,minimum width=0.1em] (set2) at ([xshift=0em,yshift=0.3em]sa2.south) {};
\begin{pgfonlayer}{background}
\node [rectangle,inner sep=0.6em,rounded corners=5pt,very thick,dotted,minimum height=1.4em,minimum width=3.5em,draw=red!40] [fit = (e1) (sa1) (l1) (l4)] (b4) {};
\node [rectangle,inner sep=0.3em,rounded corners=5pt,very thick,dotted,draw=green!40,minimum height=1.4em,minimum width=7em] [fit = (l1) (sa1) (res2) (l4)] (b1) {};
\node [rectangle,inner sep=0.3em,rounded corners=5pt,very thick,dotted,draw=green!40,minimum height=1.4em,minimum width=7em] [fit = (dot1) (l3) (l6)] (b2) {};
\node [rectangle,inner sep=0.3em,rounded corners=5pt,very thick,dotted,draw=green!40,minimum height=1.4em,minimum width=7em] [fit = (l2) (sa2) (res4) (l5)] (b3) {};
\node [rectangle,inner sep=0.6em,rounded corners=5pt,very thick,dotted,minimum height=1.4em,minimum width=3.5em,draw=red] [fit = (e1) (sa1) (l1) (l4) (set1)] (b4) {};
\node [rectangle,inner sep=0.3em,rounded corners=5pt,very thick,dotted,draw=ublue,minimum height=1.4em,minimum width=7em] [fit = (l1) (sa1) (res2) (l4) (set1)] (b1) {};
\node [rectangle,inner sep=0.3em,rounded corners=5pt,very thick,dotted,draw=ublue,minimum height=1.4em,minimum width=7em] [fit = (dot1) (l3) (l6)] (b2) {};
\node [rectangle,inner sep=0.3em,rounded corners=5pt,very thick,dotted,draw=ublue,minimum height=1.4em,minimum width=7em] [fit = (l2) (sa2) (res4) (l5) (set2)] (b3) {};
\end{pgfonlayer}
\node [inputnode,anchor=north west] (input1) at ([yshift=-1.1em,xshift=-0.5em]sa1.south west) {\tiny{Embedding}};
\node [] (add) at ([yshift=-1.7em,xshift=3.5em]sa1.south west) {$+$};
\node [posnode,anchor=north east] (pos1) at ([yshift=-1.1em,xshift=1.5em]sa1.south east) {\tiny{Absolute Position}};
\node [inputnode,anchor=north west] (input1) at ([yshift=-1.6em,xshift=-0.5em]sa1.south west) {\tiny{Embedding}};
\node [] (add) at ([yshift=-2.2em,xshift=3.5em]sa1.south west) {$+$};
\node [posnode,anchor=north east] (pos1) at ([yshift=-1.6em,xshift=1.5em]sa1.south east) {\tiny{Absolute Position}};
\node [anchor=north] (wi) at ([yshift=-0.5em]pos1.south) {\scriptsize{词序信息}};
......@@ -130,9 +136,9 @@
\draw[->,standard] ([xshift=0em]wi.east) -- ([xshift=3.2em,yshift=0em]wi.east) -- ([xshift=-0em,yshift=0em]pos2.south);
\draw[->,standard] ([xshift=0em]wi.east) -- ([xshift=6.7em,yshift=0em]wi.east) -- ([xshift=-0em,yshift=0em]pos3.south);
\draw[->,standard] ([xshift=0em]wi.east) -- ([xshift=10.2em,yshift=0em]wi.east) -- ([xshift=-0em,yshift=0em]pos4.south);
\draw[->,standard] ([xshift=0em]pos2.north) -- ([xshift=0em,yshift=1.6em]pos2.north) -- ([xshift=-0em,yshift=0em]sa1.east);
\draw[->,standard] ([xshift=0em]pos3.north) -- ([xshift=0em,yshift=9em]pos3.north) -- ([xshift=-0em,yshift=0em]dot1.east);
\draw[->,standard] ([xshift=0em]pos4.north) -- ([xshift=0em,yshift=11.7em]pos4.north) -- ([xshift=-0em,yshift=0em]sa2.east);
\draw[->,standard] ([xshift=0em]pos2.north) -- ([xshift=0em,yshift=2.1em]pos2.north) -- ([xshift=-0em,yshift=0em]sa1.east);
\draw[->,standard] ([xshift=0em]pos3.north) -- ([xshift=0em,yshift=9.6em]pos3.north) -- ([xshift=-0em,yshift=0em]dot1.east);
\draw[->,standard] ([xshift=0em]pos4.north) -- ([xshift=0em,yshift=12.3em]pos4.north) -- ([xshift=-0em,yshift=0em]sa2.east);
\begin{pgfonlayer}{background}
......
Chapter15/Figures/figure-relative-position-weight.tex
......@@ -27,6 +27,9 @@
{0/0/8, 1/0/9, 2/0/10, 3/0/11, 4/0/12, 5/0/13, 6/0/14}
\node[anchor=north] (n\k) at ([xshift=-0em,yshift=-0.5em]a\i\j.south) {\i};
\node [anchor=east] (l1) at ([xshift=-0.3em,yshift=0em]n8.west) {$i$};
\node [anchor=north] (l2) at ([xshift=0em,yshift=-0em]n7.south) {$j$};
%\node [anchor=north] (n1) at ([xshift=0em,yshift=0em]a00.south west) {};
\end{scope}
......
ChapterAcknowledgement/acknowledgement.tex
......@@ -32,7 +32,9 @@
\begin{spacing}{1.18}
\parinterval 写点儿什么 。。。
\parinterval 我知道这里本应该再写点儿什么，感慨一下蹉跎岁月，最后致敬所有人。
\parinterval 不过我还是最想说：谢谢你，我的妻子。没有你的支持与照顾，我应该没有勇气来完成这本书。爱你 $\sim$
\hfill 肖桐
......
Markdown 格式
0% 或
您添加了 0 人 到此讨论。请谨慎行事。
请先完成此评论的编辑！
请 注册 或者 后发表评论