Skip to content
项目
群组
代码片段
帮助
当前项目
正在载入...
登录 / 注册
切换导航面板
M
mtbookv2
概览
Overview
Details
Activity
Cycle Analytics
版本库
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Charts
问题
0
Issues
0
列表
Board
标记
里程碑
合并请求
0
Merge Requests
0
CI / CD
CI / CD
流水线
作业
日程表
图表
维基
Wiki
代码片段
Snippets
成员
Collapse sidebar
Close sidebar
活动
图像
聊天
创建新问题
作业
提交
Issue Boards
Open sidebar
NiuTrans
mtbookv2
Commits
f97b11e4
Commit
f97b11e4
authored
4 years ago
by
曹润柘
Browse files
Options
Browse Files
Download
Plain Diff
合并分支 'caorunzhe' 到 'master'
Caorunzhe 查看合并请求
!955
parents
dca0f06e
30b1412a
全部展开
显示空白字符变更
内嵌
并排
正在显示
4 个修改的文件
包含
48 行增加
和
39 行删除
+48
-39
Chapter15/Figures/figure-introducing-rnn-mechanism-into-transformer.tex
+16
-16
Chapter15/Figures/figure-relative-position-coding-and-absolute-position-coding.tex
+29
-23
Chapter15/Figures/figure-relative-position-weight.tex
+3
-0
ChapterPostscript/postscript.tex
+0
-0
没有找到文件。
Chapter15/Figures/figure-introducing-rnn-mechanism-into-transformer.tex
查看文件 @
f97b11e4
...
...
@@ -5,20 +5,20 @@
\tikzstyle
{
tnode
}
=[rectangle,inner sep=0mm,minimum height=1.8em,minimum width=4em,rounded corners=5pt,thick,draw,fill=teal!20]
\tikzstyle
{
stnode
}
=[rectangle,inner sep=0mm,minimum height=1.8em,minimum width=4em,rounded corners=5pt,thick,fill=red!20,draw]
\tikzstyle
{
rnnnode
}
=[rectangle,inner sep=0mm,minimum height=
2em,minimum width=6.
5em,rounded corners=5pt,thick,fill=green!20,draw]
\tikzstyle
{
rnnnode
}
=[rectangle,inner sep=0mm,minimum height=
1.8em,minimum width=
5em,rounded corners=5pt,thick,fill=green!20,draw]
\tikzstyle
{
wnode
}
=[inner sep=0mm,minimum height=2em,minimum width=
5
em]
\tikzstyle
{
wnode
}
=[inner sep=0mm,minimum height=2em,minimum width=
4
em]
\begin{scope}
[]
\node
[anchor=west,rnnnode] (r1) at (0, 0)
{
循环单元
$
\mathbi
{
h
}_
1
$
}
;
\node
[anchor=south,rnnnode] (r2) at ([xshift=0em,yshift=1em]r1.north)
{
循环单元
$
\mathbi
{
h
}_
2
$
}
;
\node
[anchor=west,rnnnode] (r1) at (0, 0)
{
循环单元
}
;
\node
[anchor=south,rnnnode] (r2) at ([xshift=0em,yshift=1em]r1.north)
{
循环单元
}
;
\node
[anchor=south] (r3) at ([xshift=0em,yshift=1em]r2.north)
{$
\cdots
$}
;
\node
[anchor=south,rnnnode] (r4) at ([xshift=0em,yshift=1em]r3.north)
{
循环单元
$
\mathbi
{
h
}_
n
$
}
;
\node
[anchor=south,rnnnode] (r4) at ([xshift=0em,yshift=1em]r3.north)
{
循环单元
}
;
\node
[anchor=east,wnode,font=
\footnotesize
] (wr1) at ([xshift=-1em,yshift=0em]r1.west)
{
1时刻
输入
}
;
\node
[anchor=east,wnode,font=
\footnotesize
] (wr2) at ([xshift=-1em,yshift=0em]r2.west)
{
2时刻
输入
}
;
\node
[anchor=east,wnode,font=
\footnotesize
] (wr3) at ([xshift=-1em,yshift=0em]r4.west)
{$
n
$
时刻
输入
}
;
\node
[anchor=east,wnode,font=
\footnotesize
,align=left,minimum width=3em] (wr1) at ([xshift=-1em,yshift=0em]r1.west)
{
1时刻
\\
输入
}
;
\node
[anchor=east,wnode,font=
\footnotesize
,align=left,minimum width=3em] (wr2) at ([xshift=-1em,yshift=0em]r2.west)
{
2时刻
\\
输入
}
;
\node
[anchor=east,wnode,font=
\footnotesize
,align=left,minimum width=3em] (wr3) at ([xshift=-1em,yshift=0em]r4.west)
{$
n
$
时刻
\\
输入
}
;
\node
[anchor=north,wnode] (input) at ([xshift=0em,yshift=-1em]r1.south)
{$
\mathbi
{
h
}_
0
$}
;
\node
[anchor=south,wnode] (output) at ([xshift=0em,yshift=1em]r4.north)
{
输出
}
;
...
...
@@ -34,11 +34,11 @@
\draw
[->,thick]
([xshift=0em,yshift=0em]wr2.east)--([xshift=0em,yshift=0em]r2.west);
\draw
[->,thick]
([xshift=0em,yshift=0em]wr3.east)--([xshift=0em,yshift=0em]r4.west);
\node
[anchor=north,font=
\small
] (label) at ([xshift=-
4em,yshift=-0.5
em]input.south)
{
(a)RNN
}
;
\node
[anchor=north,font=
\small
] (label) at ([xshift=-
3em,yshift=-0.7
em]input.south)
{
(a)RNN
}
;
\end{scope}
\begin{scope}
[xshift=1.
9
in]
\begin{scope}
[xshift=1.
85
in]
\node
[anchor=west,stnode] (r1) at (0, 0)
{
第1层
}
;
\node
[anchor=south,tnode] (r2) at ([xshift=0em,yshift=1em]r1.north)
{
第2层
}
;
...
...
@@ -60,20 +60,20 @@
\draw
[->,thick]
([xshift=0em,yshift=0em]r3.north)--([xshift=0em,yshift=0em]r4.south);
\draw
[->,thick]
([xshift=0em,yshift=0em]r4.north)--([xshift=0em,yshift=0em]output.south);
\node
[anchor=north,font=
\small
] (label) at ([xshift=-
2em,yshift=-0.5
em]input.south)
{
(b)原始Transformer模型
}
;
\node
[anchor=north,font=
\small
] (label) at ([xshift=-
1.5em,yshift=-0.7
em]input.south)
{
(b)原始Transformer模型
}
;
\end{scope}
\begin{scope}
[xshift=3.
7
in]
\begin{scope}
[xshift=3.
9
in]
\node
[anchor=west,stnode] (r1) at (0, 0)
{
第1层
}
;
\node
[anchor=south,stnode] (r2) at ([xshift=0em,yshift=1em]r1.north)
{
第2层
}
;
\node
[anchor=south] (r3) at ([xshift=0em,yshift=1em]r2.north)
{$
\cdots
$}
;
\node
[anchor=south,stnode] (r4) at ([xshift=0em,yshift=1em]r3.north)
{
第
$
n
$
层
}
;
\node
[anchor=east,wnode,font=
\footnotesize
,align=left
,minimum width=4em] (wr1) at ([xshift=-1em,yshift=0em]r1.west)
{
1时刻编
\\
码向量
}
;
\node
[anchor=east,wnode,font=
\footnotesize
,align=left
,minimum width=4em] (wr2) at ([xshift=-1em,yshift=0em]r2.west)
{
2时刻编
\\
码向量
}
;
\node
[anchor=east,wnode,font=
\footnotesize
,align=left
,minimum width=4em] (wr3) at ([xshift=-1em,yshift=0em]r4.west)
{$
n
$
时刻编
\\
码向量
}
;
\node
[anchor=east,wnode,font=
\footnotesize
,align=left
] (wr1) at ([xshift=-1em,yshift=0em]r1.west)
{
1时刻
\\
编
码向量
}
;
\node
[anchor=east,wnode,font=
\footnotesize
,align=left
] (wr2) at ([xshift=-1em,yshift=0em]r2.west)
{
2时刻
\\
编
码向量
}
;
\node
[anchor=east,wnode,font=
\footnotesize
,align=left
] (wr3) at ([xshift=-1em,yshift=0em]r4.west)
{$
n
$
时刻
\\
编
码向量
}
;
\node
[anchor=north,wnode] (input) at ([xshift=0em,yshift=-1em]r1.south)
{$
\mathbi
{
h
}_
0
$}
;
\node
[anchor=south,wnode] (output) at ([xshift=0em,yshift=1em]r4.north)
{
输出
}
;
...
...
@@ -89,7 +89,7 @@
\draw
[->,thick]
([xshift=0em,yshift=0em]wr2.east)--([xshift=0em,yshift=0em]r2.west);
\draw
[->,thick]
([xshift=0em,yshift=0em]wr3.east)--([xshift=0em,yshift=0em]r4.west);
\node
[anchor=north,font=
\small
,align=left] (label) at ([xshift=-
2em,yshift=0.2
em]input.south)
{
(c)共享权重的
\\
Transformer模型
}
;
\node
[anchor=north,font=
\small
,align=left] (label) at ([xshift=-
3em,yshift=-0.7
em]input.south)
{
(c)共享权重的
\\
Transformer模型
}
;
\end{scope}
...
...
This diff is collapsed.
Click to expand it.
Chapter15/Figures/figure-relative-position-coding-and-absolute-position-coding.tex
查看文件 @
f97b11e4
...
...
@@ -21,25 +21,28 @@
\node
[ffnnode,anchor=south] (ffn2) at ([yshift=0.5em]res3.north)
{
\tiny
{$
\textbf
{
Feed Forward Network
}$}}
;
\node
[Resnode,anchor=south] (res4) at ([yshift=0.3em]ffn2.north)
{
\tiny
{$
\textbf
{
Add
\&
LayerNorm
}$}}
;
\node
[anchor=north west,font=
\scriptsize
,align=center
] (l1) at ([xshift=0.1em]ffn1.north east)
{
第
\\
一
\\
层
}
;
\node
[anchor=north west,font=
\scriptsize
,align=center] (l2) at ([xshift=0.1em]ffn2.north east)
{
第
\\
$
n
$
\\
层
}
;
\node
[anchor=north west,font=
\scriptsize
,align=center
,ublue] (l1) at ([xshift=0.1em]ffn1.north east)
{
第
\\
$
1
$
\\
层
}
;
\node
[anchor=north west,font=
\scriptsize
,align=center
,ublue
] (l2) at ([xshift=0.1em]ffn2.north east)
{
第
\\
$
n
$
\\
层
}
;
\node
[anchor=south east,font=
\scriptsize
] (l3) at ([yshift=3em]l1.north east)
{}
;
\node
[anchor=north east,font=
\scriptsize
,align=center,opacity=0.0] (l4) at ([xshift=0.1em]ffn1.north west)
{
第
\\
一
\\
层
}
;
\node
[anchor=north east,font=
\scriptsize
,align=center,opacity=0.0] (l4) at ([xshift=0.1em]ffn1.north west)
{
第
\\
$
1
$
\\
层
}
;
\node
[anchor=north east,font=
\scriptsize
,align=center,opacity=0.0] (l5) at ([xshift=0.1em]ffn2.north west)
{
第
\\
$
n
$
\\
层
}
;
\node
[anchor=south west,font=
\scriptsize
] (l6) at ([yshift=3em]l4.north west)
{}
;
\node
[anchor=south west] (e1) at ([xshift=-1.4em,yshift=0.3em]res4.north west)
{{
编码器
}}
;
\node
[anchor=north,minimum height=0.1em,minimum width=0.1em] (set1) at ([xshift=0em,yshift=0.3em]sa1.south)
{}
;
\node
[anchor=north,minimum height=0.1em,minimum width=0.1em] (set2) at ([xshift=0em,yshift=0.3em]sa2.south)
{}
;
\begin{pgfonlayer}
{
background
}
\node
[rectangle,inner sep=0.6em,rounded corners=5pt,very thick,dotted,minimum height=1.4em,minimum width=3.5em,draw=red
!40] [fit = (e1) (sa1) (l1) (l4
)] (b4)
{}
;
\node
[rectangle,inner sep=0.3em,rounded corners=5pt,very thick,dotted,draw=
green!40,minimum height=1.4em,minimum width=7em] [fit = (l1) (sa1) (res2) (l4
)] (b1)
{}
;
\node
[rectangle,inner sep=0.3em,rounded corners=5pt,very thick,dotted,draw=
green!40
,minimum height=1.4em,minimum width=7em] [fit = (dot1) (l3) (l6)] (b2)
{}
;
\node
[rectangle,inner sep=0.3em,rounded corners=5pt,very thick,dotted,draw=
green!40,minimum height=1.4em,minimum width=7em] [fit = (l2) (sa2) (res4) (l5
)] (b3)
{}
;
\node
[rectangle,inner sep=0.6em,rounded corners=5pt,very thick,dotted,minimum height=1.4em,minimum width=3.5em,draw=red
] [fit = (e1) (sa1) (l1) (l4) (set1
)] (b4)
{}
;
\node
[rectangle,inner sep=0.3em,rounded corners=5pt,very thick,dotted,draw=
ublue,minimum height=1.4em,minimum width=7em] [fit = (l1) (sa1) (res2) (l4) (set1
)] (b1)
{}
;
\node
[rectangle,inner sep=0.3em,rounded corners=5pt,very thick,dotted,draw=
ublue
,minimum height=1.4em,minimum width=7em] [fit = (dot1) (l3) (l6)] (b2)
{}
;
\node
[rectangle,inner sep=0.3em,rounded corners=5pt,very thick,dotted,draw=
ublue,minimum height=1.4em,minimum width=7em] [fit = (l2) (sa2) (res4) (l5) (set2
)] (b3)
{}
;
\end{pgfonlayer}
\node
[inputnode,anchor=north west] (input1) at ([yshift=-1.
1
em,xshift=-0.5em]sa1.south west)
{
\tiny
{$
\textbf
{
Embedding
}$}}
;
\node
[] (add) at ([yshift=-
1.7
em,xshift=3.5em]sa1.south west)
{$
+
$}
;
\node
[posnode,anchor=north east] (pos1) at ([yshift=-1.
1
em,xshift=1.6em]sa1.south east)
{
\tiny
{$
\textbf
{
Absolute Position
}$}}
;
\node
[inputnode,anchor=north west] (input1) at ([yshift=-1.
6
em,xshift=-0.5em]sa1.south west)
{
\tiny
{$
\textbf
{
Embedding
}$}}
;
\node
[] (add) at ([yshift=-
2.2
em,xshift=3.5em]sa1.south west)
{$
+
$}
;
\node
[posnode,anchor=north east] (pos1) at ([yshift=-1.
6
em,xshift=1.6em]sa1.south east)
{
\tiny
{$
\textbf
{
Absolute Position
}$}}
;
\node
[anchor=north] (wi) at ([yshift=-0.5em]pos1.south)
{
\scriptsize
{
词序信息
}}
;
...
...
@@ -83,25 +86,28 @@
\node
[ffnnode,anchor=south] (ffn2) at ([yshift=0.5em]res3.north)
{
\tiny
{$
\textbf
{
Feed Forward Network
}$}}
;
\node
[Resnode,anchor=south] (res4) at ([yshift=0.3em]ffn2.north)
{
\tiny
{$
\textbf
{
Add
\&
LayerNorm
}$}}
;
\node
[anchor=north west,font=
\scriptsize
,align=center
] (l1) at ([xshift=0.1em]ffn1.north east)
{
第
\\
一
\\
层
}
;
\node
[anchor=north west,font=
\scriptsize
,align=center] (l2) at ([xshift=0.1em]ffn2.north east)
{
第
\\
$
n
$
\\
层
}
;
\node
[anchor=north west,font=
\scriptsize
,align=center
,ublue] (l1) at ([xshift=0.1em]ffn1.north east)
{
第
\\
$
1
$
\\
层
}
;
\node
[anchor=north west,font=
\scriptsize
,align=center
,ublue
] (l2) at ([xshift=0.1em]ffn2.north east)
{
第
\\
$
n
$
\\
层
}
;
\node
[anchor=south east,font=
\scriptsize
] (l3) at ([yshift=3em]l1.north east)
{}
;
\node
[anchor=north east,font=
\scriptsize
,align=center,opacity=0.0] (l4) at ([xshift=0.1em]ffn1.north west)
{
第
\\
一
\\
层
}
;
\node
[anchor=north east,font=
\scriptsize
,align=center,opacity=0.0] (l4) at ([xshift=0.1em]ffn1.north west)
{
第
\\
$
1
$
\\
层
}
;
\node
[anchor=north east,font=
\scriptsize
,align=center,opacity=0.0] (l5) at ([xshift=0.1em]ffn2.north west)
{
第
\\
$
n
$
\\
层
}
;
\node
[anchor=south west,font=
\scriptsize
] (l6) at ([yshift=3em]l4.north west)
{}
;
\node
[anchor=south west] (e1) at ([xshift=-1.4em,yshift=0.3em]res4.north west)
{{
编码器
}}
;
\node
[anchor=north,minimum height=0.1em,minimum width=0.1em] (set1) at ([xshift=0em,yshift=0.3em]sa1.south)
{}
;
\node
[anchor=north,minimum height=0.1em,minimum width=0.1em] (set2) at ([xshift=0em,yshift=0.3em]sa2.south)
{}
;
\begin{pgfonlayer}
{
background
}
\node
[rectangle,inner sep=0.6em,rounded corners=5pt,very thick,dotted,minimum height=1.4em,minimum width=3.5em,draw=red
!40] [fit = (e1) (sa1) (l1) (l4
)] (b4)
{}
;
\node
[rectangle,inner sep=0.3em,rounded corners=5pt,very thick,dotted,draw=
green!40,minimum height=1.4em,minimum width=7em] [fit = (l1) (sa1) (res2) (l4
)] (b1)
{}
;
\node
[rectangle,inner sep=0.3em,rounded corners=5pt,very thick,dotted,draw=
green!40
,minimum height=1.4em,minimum width=7em] [fit = (dot1) (l3) (l6)] (b2)
{}
;
\node
[rectangle,inner sep=0.3em,rounded corners=5pt,very thick,dotted,draw=
green!40,minimum height=1.4em,minimum width=7em] [fit = (l2) (sa2) (res4) (l5
)] (b3)
{}
;
\node
[rectangle,inner sep=0.6em,rounded corners=5pt,very thick,dotted,minimum height=1.4em,minimum width=3.5em,draw=red
] [fit = (e1) (sa1) (l1) (l4) (set1
)] (b4)
{}
;
\node
[rectangle,inner sep=0.3em,rounded corners=5pt,very thick,dotted,draw=
ublue,minimum height=1.4em,minimum width=7em] [fit = (l1) (sa1) (res2) (l4) (set1
)] (b1)
{}
;
\node
[rectangle,inner sep=0.3em,rounded corners=5pt,very thick,dotted,draw=
ublue
,minimum height=1.4em,minimum width=7em] [fit = (dot1) (l3) (l6)] (b2)
{}
;
\node
[rectangle,inner sep=0.3em,rounded corners=5pt,very thick,dotted,draw=
ublue,minimum height=1.4em,minimum width=7em] [fit = (l2) (sa2) (res4) (l5) (set2
)] (b3)
{}
;
\end{pgfonlayer}
\node
[inputnode,anchor=north west] (input1) at ([yshift=-1.
1
em,xshift=-0.5em]sa1.south west)
{
\tiny
{
Embedding
}}
;
\node
[] (add) at ([yshift=-
1.7
em,xshift=3.5em]sa1.south west)
{$
+
$}
;
\node
[posnode,anchor=north east] (pos1) at ([yshift=-1.
1
em,xshift=1.5em]sa1.south east)
{
\tiny
{
Absolute Position
}}
;
\node
[inputnode,anchor=north west] (input1) at ([yshift=-1.
6
em,xshift=-0.5em]sa1.south west)
{
\tiny
{
Embedding
}}
;
\node
[] (add) at ([yshift=-
2.2
em,xshift=3.5em]sa1.south west)
{$
+
$}
;
\node
[posnode,anchor=north east] (pos1) at ([yshift=-1.
6
em,xshift=1.5em]sa1.south east)
{
\tiny
{
Absolute Position
}}
;
\node
[anchor=north] (wi) at ([yshift=-0.5em]pos1.south)
{
\scriptsize
{
词序信息
}}
;
...
...
@@ -130,9 +136,9 @@
\draw
[->,standard]
([xshift=0em]wi.east) -- ([xshift=3.2em,yshift=0em]wi.east) -- ([xshift=-0em,yshift=0em]pos2.south);
\draw
[->,standard]
([xshift=0em]wi.east) -- ([xshift=6.7em,yshift=0em]wi.east) -- ([xshift=-0em,yshift=0em]pos3.south);
\draw
[->,standard]
([xshift=0em]wi.east) -- ([xshift=10.2em,yshift=0em]wi.east) -- ([xshift=-0em,yshift=0em]pos4.south);
\draw
[->,standard]
([xshift=0em]pos2.north) -- ([xshift=0em,yshift=
1.6
em]pos2.north) -- ([xshift=-0em,yshift=0em]sa1.east);
\draw
[->,standard]
([xshift=0em]pos3.north) -- ([xshift=0em,yshift=9em]pos3.north) -- ([xshift=-0em,yshift=0em]dot1.east);
\draw
[->,standard]
([xshift=0em]pos4.north) -- ([xshift=0em,yshift=1
1.7
em]pos4.north) -- ([xshift=-0em,yshift=0em]sa2.east);
\draw
[->,standard]
([xshift=0em]pos2.north) -- ([xshift=0em,yshift=
2.1
em]pos2.north) -- ([xshift=-0em,yshift=0em]sa1.east);
\draw
[->,standard]
([xshift=0em]pos3.north) -- ([xshift=0em,yshift=9
.6
em]pos3.north) -- ([xshift=-0em,yshift=0em]dot1.east);
\draw
[->,standard]
([xshift=0em]pos4.north) -- ([xshift=0em,yshift=1
2.3
em]pos4.north) -- ([xshift=-0em,yshift=0em]sa2.east);
\begin{pgfonlayer}
{
background
}
...
...
This diff is collapsed.
Click to expand it.
Chapter15/Figures/figure-relative-position-weight.tex
查看文件 @
f97b11e4
...
...
@@ -27,6 +27,9 @@
{
0/0/8, 1/0/9, 2/0/10, 3/0/11, 4/0/12, 5/0/13, 6/0/14
}
\node
[anchor=north]
(n
\k
) at ([xshift=-0em,yshift=-0.5em]a
\i\j
.south)
{
\i
}
;
\node
[anchor=east] (l1) at ([xshift=-0.3em,yshift=0em]n8.west)
{$
i
$}
;
\node
[anchor=north] (l2) at ([xshift=0em,yshift=-0em]n7.south)
{$
j
$}
;
%\node [anchor=north] (n1) at ([xshift=0em,yshift=0em]a00.south west) {};
\end{scope}
...
...
This diff is collapsed.
Click to expand it.
ChapterPostscript/postscript.tex
查看文件 @
f97b11e4
差异被折叠。
点击展开。
编写
预览
Markdown
格式
0%
重试
或
添加新文件
添加附件
取消
您添加了
0
人
到此讨论。请谨慎行事。
请先完成此评论的编辑!
取消
请
注册
或者
登录
后发表评论