Skip to content
项目
群组
代码片段
帮助
当前项目
正在载入...
登录 / 注册
切换导航面板
T
Toy-MT-Introduction
概览
Overview
Details
Activity
Cycle Analytics
版本库
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Charts
问题
0
Issues
0
列表
Board
标记
里程碑
合并请求
0
Merge Requests
0
CI / CD
CI / CD
流水线
作业
日程表
图表
维基
Wiki
代码片段
Snippets
成员
Collapse sidebar
Close sidebar
活动
图像
聊天
创建新问题
作业
提交
Issue Boards
Open sidebar
NiuTrans
Toy-MT-Introduction
Commits
65fd1ab3
Commit
65fd1ab3
authored
Mar 09, 2020
by
姜雨帆
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
update slides
parent
19043049
隐藏空白字符变更
内嵌
并排
正在显示
1 个修改的文件
包含
105 行增加
和
99 行删除
+105
-99
Section05-Neural-Networks-and-Language-Modeling/section05.tex
+105
-99
没有找到文件。
Section05-Neural-Networks-and-Language-Modeling/section05.tex
查看文件 @
65fd1ab3
...
@@ -3923,11 +3923,11 @@ NLP问题的\alert{隐含结构}假设 & 无隐含结构假设，\alert{端到
...
@@ -3923,11 +3923,11 @@ NLP问题的\alert{隐含结构}假设 & 无隐含结构假设，\alert{端到
\node
[anchor=north] (index0) at ([yshift=0.5em]w0.south)
{
\tiny
(index)
}
;
\node
[anchor=north] (index0) at ([yshift=0.5em]w0.south)
{
\tiny
(index)
}
;
\node
[anchor=north] (index1) at ([yshift=0.5em]w1.south)
{
\tiny
(index)
}
;
\node
[anchor=north] (index1) at ([yshift=0.5em]w1.south)
{
\tiny
(index)
}
;
\node
[anchor=north] (index2) at ([yshift=0.5em]w2.south)
{
\tiny
(index)
}
;
\node
[anchor=north] (index2) at ([yshift=0.5em]w2.south)
{
\tiny
(index)
}
;
\node
[anchor=south,draw,inner sep=3pt] (e0) at ([yshift=1em]w0.north)
{
\tiny
{$
e
_
0
=
w
_{
i
-
3
}
\textbf
{
C
}$}}
;
\node
[anchor=south,draw,inner sep=3pt] (e0) at ([yshift=1em]w0.north)
{
\tiny
{$
\textbf
{
e
}
_
0
=
w
_{
i
-
3
}
\textbf
{
C
}$}}
;
\node
[anchor=south,draw,inner sep=3pt] (e1) at ([yshift=1em]w1.north)
{
\tiny
{$
e
_
1
=
w
_{
i
-
2
}
\textbf
{
C
}$}}
;
\node
[anchor=south,draw,inner sep=3pt] (e1) at ([yshift=1em]w1.north)
{
\tiny
{$
\textbf
{
e
}
_
1
=
w
_{
i
-
2
}
\textbf
{
C
}$}}
;
\node
[anchor=south,draw,inner sep=3pt] (e2) at ([yshift=1em]w2.north)
{
\tiny
{$
e
_
2
=
w
_{
i
-
1
}
\textbf
{
C
}$}}
;
\node
[anchor=south,draw,inner sep=3pt] (e2) at ([yshift=1em]w2.north)
{
\tiny
{$
\textbf
{
e
}
_
2
=
w
_{
i
-
1
}
\textbf
{
C
}$}}
;
\node
[anchor=south,draw,minimum width=9em,inner sep=3pt] (h0) at ([yshift=1.5em]e1.north)
{
\tiny
{$
h
_
0
=
\textrm
{
Tanh
}
([
e
_
0
,e
_
1
,e
_
2
]
\textbf
{
H
}
+
\textbf
{
d
}
)
$}}
;
\node
[anchor=south,draw,minimum width=9em,inner sep=3pt] (h0) at ([yshift=1.5em]e1.north)
{
\tiny
{$
\textbf
{
h
}_
0
=
\textrm
{
Tanh
}
([
\textbf
{
e
}_
0
,
\textbf
{
e
}_
1
,
\textbf
{
e
}
_
2
]
\textbf
{
H
}
+
\textbf
{
d
}
)
$}}
;
\node
[anchor=south,draw,minimum width=9em,inner sep=3pt] (h1) at ([yshift=1.5em]h0.north)
{
\tiny
{$
y
=
\textrm
{
Softmax
}
(
h
_
0
\textbf
{
U
}
)
$}}
;
\node
[anchor=south,draw,minimum width=9em,inner sep=3pt] (h1) at ([yshift=1.5em]h0.north)
{
\tiny
{$
\textbf
{
y
}
=
\textrm
{
Softmax
}
(
h
_
0
\textbf
{
U
}
)
$}}
;
\node
[anchor=south] (ylabel) at ([yshift=1em]h1.north)
{
\footnotesize
{$
\textrm
{
P
}
(
w
_
i|w
_{
i
-
3
}
w
_{
i
-
2
}
w
_{
i
-
1
}
)
$}}
;
\node
[anchor=south] (ylabel) at ([yshift=1em]h1.north)
{
\footnotesize
{$
\textrm
{
P
}
(
w
_
i|w
_{
i
-
3
}
w
_{
i
-
2
}
w
_{
i
-
1
}
)
$}}
;
\draw
[->] ([yshift=0.1em]w0.north) -- ([yshift=-0.1em]e0.south);
\draw
[->] ([yshift=0.1em]w0.north) -- ([yshift=-0.1em]e0.south);
...
@@ -4483,23 +4483,23 @@ $\textbf{V}, \textbf{U}, \textbf{W}$: 参数
...
@@ -4483,23 +4483,23 @@ $\textbf{V}, \textbf{U}, \textbf{W}$: 参数
\begin{tikzpicture}
\begin{tikzpicture}
\begin{scope}
\begin{scope}
\node
[anchor=west] (w0) at (0,0)
{
\footnotesize
{$
w
_{
0
}$}}
;
\node
[anchor=west] (w0) at (0,0)
{
\footnotesize
{$
w
_{
0
}$}}
;
\node
[anchor=west] (w1) at ([xshift=
4
em]w0.east)
{
\footnotesize
{$
w
_{
1
}$}}
;
\node
[anchor=west] (w1) at ([xshift=
5
em]w0.east)
{
\footnotesize
{$
w
_{
1
}$}}
;
\node
[anchor=west] (w2) at ([xshift=
4
em]w1.east)
{
\footnotesize
{$
w
_{
2
}$}}
;
\node
[anchor=west] (w2) at ([xshift=
5
em]w1.east)
{
\footnotesize
{$
w
_{
2
}$}}
;
\node
[anchor=west] (w3) at ([xshift=
4
em]w2.east)
{
\footnotesize
{$
w
_{
3
}$}}
;
\node
[anchor=west] (w3) at ([xshift=
5
em]w2.east)
{
\footnotesize
{$
w
_{
3
}$}}
;
\node
[anchor=north] (index0) at ([yshift=0.5em]w0.south)
{
\tiny
(index)
}
;
\node
[anchor=north] (index0) at ([yshift=0.5em]w0.south)
{
\tiny
(index)
}
;
\node
[anchor=north] (index1) at ([yshift=0.5em]w1.south)
{
\tiny
(index)
}
;
\node
[anchor=north] (index1) at ([yshift=0.5em]w1.south)
{
\tiny
(index)
}
;
\node
[anchor=north] (index2) at ([yshift=0.5em]w2.south)
{
\tiny
(index)
}
;
\node
[anchor=north] (index2) at ([yshift=0.5em]w2.south)
{
\tiny
(index)
}
;
\node
[anchor=north] (index3) at ([yshift=0.5em]w3.south)
{
\tiny
(index)
}
;
\node
[anchor=north] (index3) at ([yshift=0.5em]w3.south)
{
\tiny
(index)
}
;
\node
[anchor=south,draw,inner sep=3pt] (e0) at ([yshift=1em]w0.north)
{
\tiny
{$
e
_
0
=
w
_{
0
}
\textbf
{
C
}
$}}
;
\node
[anchor=south,draw,inner sep=3pt] (e0) at ([yshift=1em]w0.north)
{
\tiny
{$
\textbf
{
e
}_
0
=
w
_{
0
}
\textbf
{
C
}
+
\textrm
{
PE
}
(
0
)
$}}
;
\node
[anchor=south,draw,inner sep=3pt] (e1) at ([yshift=1em]w1.north)
{
\tiny
{$
e
_
1
=
w
_{
1
}
\textbf
{
C
}
$}}
;
\node
[anchor=south,draw,inner sep=3pt] (e1) at ([yshift=1em]w1.north)
{
\tiny
{$
\textbf
{
e
}_
1
=
w
_{
1
}
\textbf
{
C
}
+
\textrm
{
PE
}
(
1
)
$}}
;
\node
[anchor=south,draw,inner sep=3pt] (e2) at ([yshift=1em]w2.north)
{
\tiny
{$
e
_
2
=
w
_{
2
}
\textbf
{
C
}
$}}
;
\node
[anchor=south,draw,inner sep=3pt] (e2) at ([yshift=1em]w2.north)
{
\tiny
{$
\textbf
{
e
}_
2
=
w
_{
2
}
\textbf
{
C
}
+
\textrm
{
PE
}
(
2
)
$}}
;
\node
[anchor=south,draw,inner sep=3pt] (e3) at ([yshift=1em]w3.north)
{
\tiny
{$
e
_
3
=
w
_{
3
}
\textbf
{
C
}
$}}
;
\node
[anchor=south,draw,inner sep=3pt] (e3) at ([yshift=1em]w3.north)
{
\tiny
{$
\textbf
{
e
}_
3
=
w
_{
3
}
\textbf
{
C
}
+
\textrm
{
PE
}
(
3
)
$}}
;
\node
[anchor=south,draw,inner sep=3pt] (h0) at ([xshift=-0.5em, yshift=1.5em]e0.north)
{
\tiny
{$
h
_{
0
}
=
\textrm
{
SelfAtt
}
(
e
_
0
,e
_
3
)
$}}
;
\node
[anchor=south,draw,inner sep=3pt] (h0) at ([xshift=-0.5em, yshift=1.5em]e0.north)
{
\tiny
{$
\textbf
{
h
}_{
0
}
=
\textrm
{
SelfAtt
}
(
\textbf
{
e
}_
0
,
\textbf
{
e
}
_
3
)
$}}
;
\node
[anchor=south,draw,inner sep=3pt] (h1) at ([xshift=0.5em, yshift=1.5em]e1.north)
{
\tiny
{$
h
_{
1
}
=
\textrm
{
SelfAtt
}
(
e
_
1
,e
_
3
)
$}}
;
\node
[anchor=south,draw,inner sep=3pt] (h1) at ([xshift=0.5em, yshift=1.5em]e1.north)
{
\tiny
{$
\textbf
{
h
}_{
1
}
=
\textrm
{
SelfAtt
}
(
\textbf
{
e
}_
1
,
\textbf
{
e
}
_
3
)
$}}
;
\node
[anchor=south,draw,inner sep=3pt] (h2) at ([xshift=1.5em, yshift=1.5em]e2.north)
{
\tiny
{$
h
_{
2
}
=
\textrm
{
SelfAtt
}
(
e
_
2
,e
_
3
)
$}}
;
\node
[anchor=south,draw,inner sep=3pt] (h2) at ([xshift=1.5em, yshift=1.5em]e2.north)
{
\tiny
{$
\textbf
{
h
}_{
2
}
=
\textrm
{
SelfAtt
}
(
\textbf
{
e
}_
2
,
\textbf
{
e
}
_
3
)
$}}
;
\node
[anchor=south,draw,minimum width=9em,inner sep=3pt] (f1) at ([xshift=0.5em, yshift=1.5em]h2.north)
{
\tiny
{$
f
_
3
=
\textrm
{
FNN
}
([
h
_
0
,h
_
1
,h
_
2
,e
_
3
])
$}}
;
\node
[anchor=south,draw,minimum width=9em,inner sep=3pt] (f1) at ([xshift=0.5em, yshift=1.5em]h2.north)
{
\tiny
{$
\textbf
{
f
}_
3
=
\textrm
{
FNN
}
([
\textbf
{
h
}_
0
,
\textbf
{
h
}_
1
,
\textbf
{
h
}_
2
,
\textbf
{
e
}
_
3
])
$}}
;
\node
[anchor=south,draw,minimum width=9em,inner sep=3pt] (o1) at ([yshift=1em]f1.north)
{
\tiny
{$
y
=
\textrm
{
Softmax
}
(
f
_
3
\textbf
{
U
}
)
$}}
;
\node
[anchor=south,draw,minimum width=9em,inner sep=3pt] (o1) at ([yshift=1em]f1.north)
{
\tiny
{$
\textbf
{
y
}
=
\textrm
{
Softmax
}
(
f
_
3
\textbf
{
U
}
)
$}}
;
\node
[anchor=south] (ylabel) at ([yshift=1em]o1.north)
{
\footnotesize
{$
\textrm
{
P
}
(
w
_
4
|w
_{
0
}
w
_{
1
}
w
_{
2
}
w
_{
3
}
)
$}}
;
\node
[anchor=south] (ylabel) at ([yshift=1em]o1.north)
{
\footnotesize
{$
\textrm
{
P
}
(
w
_
4
|w
_{
0
}
w
_{
1
}
w
_{
2
}
w
_{
3
}
)
$}}
;
\draw
[->] ([yshift=0.1em]w0.north) -- ([yshift=-0.1em]e0.south);
\draw
[->] ([yshift=0.1em]w0.north) -- ([yshift=-0.1em]e0.south);
...
@@ -4521,23 +4521,23 @@ $\textbf{V}, \textbf{U}, \textbf{W}$: 参数
...
@@ -4521,23 +4521,23 @@ $\textbf{V}, \textbf{U}, \textbf{W}$: 参数
\draw
[->] ([yshift=0.1em]o1.north) -- ([yshift=-0.1em]ylabel.south);
\draw
[->] ([yshift=0.1em]o1.north) -- ([yshift=-0.1em]ylabel.south);
\visible
<2->
{
\visible
<2->
{
\node
[anchor=south,draw,inner sep=3pt,fill=blue!20!white] (e0) at ([yshift=1em]w0.north)
{
\tiny
{$
e
_
0
=
w
_{
0
}
\textbf
{
C
}
$}}
;
\node
[anchor=south,draw,inner sep=3pt,fill=blue!20!white] (e0) at ([yshift=1em]w0.north)
{
\tiny
{$
\textbf
{
e
}_
0
=
w
_{
0
}
\textbf
{
C
}
+
\textrm
{
PE
}
(
0
)
$}}
;
\node
[anchor=south,draw,inner sep=3pt,fill=blue!20!white] (e1) at ([yshift=1em]w1.north)
{
\tiny
{$
e
_
1
=
w
_{
1
}
\textbf
{
C
}
$}}
;
\node
[anchor=south,draw,inner sep=3pt,fill=blue!20!white] (e1) at ([yshift=1em]w1.north)
{
\tiny
{$
\textbf
{
e
}_
1
=
w
_{
1
}
\textbf
{
C
}
+
\textrm
{
PE
}
(
1
)
$}}
;
\node
[anchor=south,draw,inner sep=3pt,fill=blue!20!white] (e2) at ([yshift=1em]w2.north)
{
\tiny
{$
e
_
2
=
w
_{
2
}
\textbf
{
C
}
$}}
;
\node
[anchor=south,draw,inner sep=3pt,fill=blue!20!white] (e2) at ([yshift=1em]w2.north)
{
\tiny
{$
\textbf
{
e
}_
2
=
w
_{
2
}
\textbf
{
C
}
+
\textrm
{
PE
}
(
2
)
$}}
;
\node
[anchor=south,draw,inner sep=3pt,fill=blue!20!white] (e3) at ([yshift=1em]w3.north)
{
\tiny
{$
e
_
3
=
w
_{
3
}
\textbf
{
C
}
$}}
;
\node
[anchor=south,draw,inner sep=3pt,fill=blue!20!white] (e3) at ([yshift=1em]w3.north)
{
\tiny
{$
\textbf
{
e
}_
3
=
w
_{
3
}
\textbf
{
C
}
+
\textrm
{
PE
}
(
3
)
$}}
;
}
}
\visible
<2->
{
\visible
<2->
{
\node
[anchor=west] (embedinglabel0) at ([xshift=-5em,yshift=-2em]w0.south)
{
\scriptsize
{{
\blue
\textbf
{
词的分布式表示
}}}}
;
\node
[anchor=west] (embedinglabel0) at ([xshift=-5em,yshift=-2em]w0.south)
{
\scriptsize
{{
\blue
\textbf
{
词的分布式表示
}}}}
;
\node
[anchor=north west] (embedinglabel1) at ([yshift=0.3em]embedinglabel0.south west)
{
\scriptsize
{
前面已经介绍过！
}}
;
\node
[anchor=north west] (embedinglabel1) at ([yshift=0.3em]embedinglabel0.south west)
{
\scriptsize
{
前面已经介绍过！
}}
;
\node
[anchor=north west] (embedinglabel2) at ([yshift=0.3em]embedinglabel1.south west)
{
\scriptsize
{
基于One-hot表示获得
}}
;
\node
[anchor=north west] (embedinglabel2) at ([yshift=0.3em]embedinglabel1.south west)
{
\scriptsize
{
基于One-hot表示获得
}}
;
\node
[anchor=north west] (embedinglabel3) at ([yshift=0.3em]embedinglabel2.south west)
{
\scriptsize
{
新加入位置向量
}}
;
\node
[anchor=north west] (embedinglabel3) at ([yshift=0.3em]embedinglabel2.south west)
{
\scriptsize
{
新加入位置向量
PE
}}
;
}
}
\visible
<3->
{
\visible
<3->
{
\node
[anchor=south,draw,inner sep=3pt,fill=ugreen!20!white] (h0) at ([xshift=-0.5em, yshift=1.5em]e0.north)
{
\tiny
{$
h
_{
0
}
=
\textrm
{
SelfAtt
}
(
e
_
0
,e
_
3
)
$}}
;
\node
[anchor=south,draw,inner sep=3pt,fill=ugreen!20!white] (h0) at ([xshift=-0.5em, yshift=1.5em]e0.north)
{
\tiny
{$
\textbf
{
h
}_{
0
}
=
\textrm
{
SelfAtt
}
(
\textbf
{
e
}_
0
,
\textbf
{
e
}
_
3
)
$}}
;
\node
[anchor=south,draw,inner sep=3pt,fill=ugreen!20!white] (h1) at ([xshift=0.5em, yshift=1.5em]e1.north)
{
\tiny
{$
h
_{
1
}
=
\textrm
{
SelfAtt
}
(
e
_
1
,e
_
3
)
$}}
;
\node
[anchor=south,draw,inner sep=3pt,fill=ugreen!20!white] (h1) at ([xshift=0.5em, yshift=1.5em]e1.north)
{
\tiny
{$
\textbf
{
h
}_{
1
}
=
\textrm
{
SelfAtt
}
(
\textbf
{
e
}_
1
,
\textbf
{
e
}
_
3
)
$}}
;
\node
[anchor=south,draw,inner sep=3pt,fill=ugreen!20!white] (h2) at ([xshift=1.5em, yshift=1.5em]e2.north)
{
\tiny
{$
h
_{
2
}
=
\textrm
{
SelfAtt
}
(
e
_
2
,e
_
3
)
$}}
;
\node
[anchor=south,draw,inner sep=3pt,fill=ugreen!20!white] (h2) at ([xshift=1.5em, yshift=1.5em]e2.north)
{
\tiny
{$
\textbf
{
h
}_{
2
}
=
\textrm
{
SelfAtt
}
(
\textbf
{
e
}_
2
,
\textbf
{
e
}
_
3
)
$}}
;
}
}
\visible
<3->
{
\visible
<3->
{
...
@@ -4548,8 +4548,8 @@ $\textbf{V}, \textbf{U}, \textbf{W}$: 参数
...
@@ -4548,8 +4548,8 @@ $\textbf{V}, \textbf{U}, \textbf{W}$: 参数
}
}
\visible
<4->
{
\visible
<4->
{
\node
[anchor=south,draw,minimum width=9em,inner sep=3pt,fill=orange!20!white] (f1) at ([xshift=0.5em, yshift=1.5em]h2.north)
{
\tiny
{$
f
_
3
=
\textrm
{
FNN
}
([
h
_
0
,h
_
1
,h
_
2
,e
_
3
])
$}}
;
\node
[anchor=south,draw,minimum width=9em,inner sep=3pt,fill=orange!20!white] (f1) at ([xshift=0.5em, yshift=1.5em]h2.north)
{
\tiny
{$
\textbf
{
f
}_
3
=
\textrm
{
FNN
}
([
\textbf
{
h
}_
0
,
\textbf
{
h
}_
1
,
\textbf
{
h
}_
2
,
\textbf
{
e
}
_
3
])
$}}
;
\node
[anchor=south,draw,minimum width=9em,inner sep=3pt,fill=orange!20!white] (o1) at ([yshift=1em]f1.north)
{
\tiny
{$
y
=
\textrm
{
Softmax
}
(
f
_
3
\textbf
{
U
}
)
$}}
;
\node
[anchor=south,draw,minimum width=9em,inner sep=3pt,fill=orange!20!white] (o1) at ([yshift=1em]f1.north)
{
\tiny
{$
\textbf
{
y
}
=
\textrm
{
Softmax
}
(
f
_
3
\textbf
{
U
}
)
$}}
;
}
}
\visible
<4->
{
\visible
<4->
{
...
@@ -4731,7 +4731,7 @@ GPT-2 (Transformer) & Radford et al. & 2019 & 35.7
...
@@ -4731,7 +4731,7 @@ GPT-2 (Transformer) & Radford et al. & 2019 & 35.7
\node
[anchor=south] (w2) at (o2.north)
{
\footnotesize
{
椅子
}}
;
\node
[anchor=south] (w2) at (o2.north)
{
\footnotesize
{
椅子
}}
;
\node
[anchor=north] (label) at (o1.south)
{
\footnotesize
{
单词的one-hot表示
}}
;
\node
[anchor=north] (label) at (o1.south)
{
\footnotesize
{
单词的one-hot表示
}}
;
\visible
<3->
{
\visible
<3->
{
\node
[anchor=south,fill=red!20!white] (cosine) at (w1.north)
{
\footnotesize
{$
cosine
(
\textrm
{
`桌子'
}
,
\textrm
{
`椅子'
}
)=
0
$}}
;
\node
[anchor=south,fill=red!20!white] (cosine) at (w1.north)
{
\footnotesize
{$
\textrm
{
cosine
}
(
\textrm
{
`桌子'
}
,
\textrm
{
`椅子'
}
)=
0
$}}
;
}
}
\end{scope}
\end{scope}
...
@@ -4744,7 +4744,7 @@ GPT-2 (Transformer) & Radford et al. & 2019 & 35.7
...
@@ -4744,7 +4744,7 @@ GPT-2 (Transformer) & Radford et al. & 2019 & 35.7
\node
[anchor=south] (w2) at (o2.north)
{
\footnotesize
{
椅子
}}
;
\node
[anchor=south] (w2) at (o2.north)
{
\footnotesize
{
椅子
}}
;
\node
[anchor=north] (label) at ([yshift=-2em]o1.south)
{
\footnotesize
{
单词的分布式表示(词嵌入)
}}
;
\node
[anchor=north] (label) at ([yshift=-2em]o1.south)
{
\footnotesize
{
单词的分布式表示(词嵌入)
}}
;
\visible
<3->
{
\visible
<3->
{
\node
[anchor=south,fill=red!20!white] (cosine) at (w1.north)
{
\footnotesize
{$
cosine
(
\textrm
{
`桌子'
}
,
\textrm
{
`椅子'
}
)=
0
.
5
$}}
;
\node
[anchor=south,fill=red!20!white] (cosine) at (w1.north)
{
\footnotesize
{$
\textrm
{
cosine
}
(
\textrm
{
`桌子'
}
,
\textrm
{
`椅子'
}
)=
0
.
5
$}}
;
}
}
\end{scope}
\end{scope}
}
}
...
@@ -5015,59 +5015,59 @@ GPT-2 (Transformer) & Radford et al. & 2019 & 35.7
...
@@ -5015,59 +5015,59 @@ GPT-2 (Transformer) & Radford et al. & 2019 & 35.7
\begin{tikzpicture}
\begin{tikzpicture}
\begin{scope}
[scale=1.2]
\begin{scope}
[scale=1.2]
\node
[anchor=west,draw,inner sep=4pt,fill=blue!20!white] (Lstm0) at (0,0)
{
\scriptsize
{
L
stm
}}
;
\node
[anchor=west,draw,inner sep=4pt,fill=blue!20!white] (Lstm0) at (0,0)
{
\scriptsize
{
L
STM
}}
;
\node
[anchor=west,draw,inner sep=4pt,fill=blue!20!white] (Lstm1) at ([xshift=1em]Lstm0.east)
{
\scriptsize
{
L
stm
}}
;
\node
[anchor=west,draw,inner sep=4pt,fill=blue!20!white] (Lstm1) at ([xshift=1em]Lstm0.east)
{
\scriptsize
{
L
STM
}}
;
\node
[anchor=west,inner sep=4pt] (sep) at ([xshift=
1
em]Lstm1.east)
{
\scriptsize
{
...
}}
;
\node
[anchor=west,inner sep=4pt] (sep) at ([xshift=
0.5
em]Lstm1.east)
{
\scriptsize
{
...
}}
;
\node
[anchor=west,draw,inner sep=4pt,fill=blue!20!white] (Lstm2) at ([xshift=
1em]sep.east)
{
\scriptsize
{
Lstm
}}
;
\node
[anchor=west,draw,inner sep=4pt,fill=blue!20!white] (Lstm2) at ([xshift=
0.5em]sep.east)
{
\scriptsize
{
LSTM
}}
;
\node
[anchor=south,draw,inner sep=4pt,fill=blue!20!white] (Lstm3) at ([yshift=1em]Lstm0.north)
{
\scriptsize
{
L
stm
}}
;
\node
[anchor=south,draw,inner sep=4pt,fill=blue!20!white] (Lstm3) at ([yshift=1em]Lstm0.north)
{
\scriptsize
{
L
STM
}}
;
\node
[anchor=west,draw,inner sep=4pt,fill=blue!20!white] (Lstm4) at ([xshift=1em]Lstm3.east)
{
\scriptsize
{
L
stm
}}
;
\node
[anchor=west,draw,inner sep=4pt,fill=blue!20!white] (Lstm4) at ([xshift=1em]Lstm3.east)
{
\scriptsize
{
L
STM
}}
;
\node
[anchor=west,inner sep=4pt] (sep1) at ([xshift=
1
em]Lstm4.east)
{
\scriptsize
{
...
}}
;
\node
[anchor=west,inner sep=4pt] (sep1) at ([xshift=
0.5
em]Lstm4.east)
{
\scriptsize
{
...
}}
;
\node
[anchor=west,draw,inner sep=4pt,fill=blue!20!white] (Lstm5) at ([xshift=
1em]sep1.east)
{
\scriptsize
{
Lstm
}}
;
\node
[anchor=west,draw,inner sep=4pt,fill=blue!20!white] (Lstm5) at ([xshift=
0.5em]sep1.east)
{
\scriptsize
{
LSTM
}}
;
\node
[rectangle,rounded corners,draw=black!50,densely dashed,inner sep=0.4em] [fit = (Lstm0) (Lstm2) (Lstm3) (Lstm5)] (inputshadow)
{}
;
\node
[rectangle,rounded corners,draw=black!50,densely dashed,inner sep=0.4em] [fit = (Lstm0) (Lstm2) (Lstm3) (Lstm5)] (inputshadow)
{}
;
\node
[anchor=north,draw,inner sep=4pt,fill=ugreen!20!white,minimum width=2em] (e1) at ([xshift=-2em,yshift=-1em]Lstm2.south)
{
\scriptsize
{$
E
_
1
$}}
;
\node
[anchor=north,draw,inner sep=4pt,fill=ugreen!20!white,minimum width=2em] (e1) at ([xshift=-2em,yshift=-1em]Lstm2.south)
{
\scriptsize
{$
\textbf
{
x
}
_
1
$}}
;
\node
[anchor=west,draw,inner sep=4pt,fill=ugreen!20!white,minimum width=2em] (e2) at ([xshift=1em]e1.east)
{
\scriptsize
{$
E
_
2
$}}
;
\node
[anchor=west,draw,inner sep=4pt,fill=ugreen!20!white,minimum width=2em] (e2) at ([xshift=1em]e1.east)
{
\scriptsize
{$
\textbf
{
x
}
_
2
$}}
;
\node
[anchor=west,inner sep=4pt] (sep5) at ([xshift=1em]e2.east)
{
\scriptsize
{
...
}}
;
\node
[anchor=west,inner sep=4pt] (sep5) at ([xshift=1em]e2.east)
{
\scriptsize
{
...
}}
;
\node
[anchor=west,draw,inner sep=4pt,fill=ugreen!20!white,minimum width=2em] (e3) at ([xshift=1em]sep5.east)
{
\scriptsize
{$
E
_
3
$}}
;
\node
[anchor=west,draw,inner sep=4pt,fill=ugreen!20!white,minimum width=2em] (e3) at ([xshift=1em]sep5.east)
{
\scriptsize
{$
\textbf
{
x
}_
n
$}}
;
\node
[anchor=south,draw,inner sep=4pt,fill=yellow!30,minimum width=2em] (t1) at ([xshift=-2em,yshift=1em]Lstm5.north)
{
\scriptsize
{$
T
_
1
$}}
;
\node
[anchor=south,draw,inner sep=4pt,fill=yellow!30,minimum width=2em] (t1) at ([xshift=-2em,yshift=1em]Lstm5.north)
{
\scriptsize
{$
\textbf
{
h
}
_
1
$}}
;
\node
[anchor=west,draw,inner sep=4pt,fill=yellow!30,minimum width=2em] (t2) at ([xshift=1em]t1.east)
{
\scriptsize
{$
T
_
2
$}}
;
\node
[anchor=west,draw,inner sep=4pt,fill=yellow!30,minimum width=2em] (t2) at ([xshift=1em]t1.east)
{
\scriptsize
{$
\textbf
{
h
}
_
2
$}}
;
\node
[anchor=west,inner sep=4pt] (sep6) at ([xshift=1em]t2.east)
{
\scriptsize
{
...
}}
;
\node
[anchor=west,inner sep=4pt] (sep6) at ([xshift=1em]t2.east)
{
\scriptsize
{
...
}}
;
\node
[anchor=west,draw,inner sep=4pt,fill=yellow!30,minimum width=2em] (t3) at ([xshift=1em]sep6.east)
{
\scriptsize
{$
T
_
3
$}}
;
\node
[anchor=west,draw,inner sep=4pt,fill=yellow!30,minimum width=2em] (t3) at ([xshift=1em]sep6.east)
{
\scriptsize
{$
\textbf
{
h
}_
n
$}}
;
\node
[anchor=west,draw,inner sep=4pt,fill=blue!20!white] (Lstm6) at ([xshift=1.5em]Lstm2.east)
{
\scriptsize
{
L
stm
}}
;
\node
[anchor=west,draw,inner sep=4pt,fill=blue!20!white] (Lstm6) at ([xshift=1.5em]Lstm2.east)
{
\scriptsize
{
L
STM
}}
;
\node
[anchor=west,draw,inner sep=4pt,fill=blue!20!white] (Lstm7) at ([xshift=1em]Lstm6.east)
{
\scriptsize
{
L
stm
}}
;
\node
[anchor=west,draw,inner sep=4pt,fill=blue!20!white] (Lstm7) at ([xshift=1em]Lstm6.east)
{
\scriptsize
{
L
STM
}}
;
\node
[anchor=west,inner sep=4pt] (sep3) at ([xshift=
1
em]Lstm7.east)
{
\scriptsize
{
...
}}
;
\node
[anchor=west,inner sep=4pt] (sep3) at ([xshift=
0.5
em]Lstm7.east)
{
\scriptsize
{
...
}}
;
\node
[anchor=west,draw,inner sep=4pt,fill=blue!20!white] (Lstm8) at ([xshift=
1em]sep3.east)
{
\scriptsize
{
Lstm
}}
;
\node
[anchor=west,draw,inner sep=4pt,fill=blue!20!white] (Lstm8) at ([xshift=
0.5em]sep3.east)
{
\scriptsize
{
LSTM
}}
;
\node
[anchor=south,draw,inner sep=4pt,fill=blue!20!white] (Lstm9) at ([yshift=1em]Lstm6.north)
{
\scriptsize
{
L
stm
}}
;
\node
[anchor=south,draw,inner sep=4pt,fill=blue!20!white] (Lstm9) at ([yshift=1em]Lstm6.north)
{
\scriptsize
{
L
STM
}}
;
\node
[anchor=west,draw,inner sep=4pt,fill=blue!20!white] (Lstm10) at ([xshift=1em]Lstm9.east)
{
\scriptsize
{
L
stm
}}
;
\node
[anchor=west,draw,inner sep=4pt,fill=blue!20!white] (Lstm10) at ([xshift=1em]Lstm9.east)
{
\scriptsize
{
L
STM
}}
;
\node
[anchor=west,inner sep=4pt] (sep4) at ([xshift=
1
em]Lstm10.east)
{
\scriptsize
{
...
}}
;
\node
[anchor=west,inner sep=4pt] (sep4) at ([xshift=
0.5
em]Lstm10.east)
{
\scriptsize
{
...
}}
;
\node
[anchor=west,draw,inner sep=4pt,fill=blue!20!white] (Lstm11) at ([xshift=
1em]sep4.east)
{
\scriptsize
{
Lstm
}}
;
\node
[anchor=west,draw,inner sep=4pt,fill=blue!20!white] (Lstm11) at ([xshift=
0.5em]sep4.east)
{
\scriptsize
{
LSTM
}}
;
\node
[rectangle,rounded corners,draw=black!50,densely dashed,inner sep=0.4em] [fit = (Lstm6) (Lstm8) (Lstm9) (Lstm11)] (inputshadow)
{}
;
\node
[rectangle,rounded corners,draw=black!50,densely dashed,inner sep=0.4em] [fit = (Lstm6) (Lstm8) (Lstm9) (Lstm11)] (inputshadow)
{}
;
\draw
[->] ([xshift=0.1em]Lstm0.east) -- ([xshift=-0.1em]Lstm1.west);
\draw
[->] ([xshift=0.1em]Lstm0.east) -- ([xshift=-0.1em]Lstm1.west);
\draw
[->] ([xshift=0.1em]Lstm1.east) -- ([xshift=
-
0.1em]sep.west);
\draw
[->] ([xshift=0.1em]Lstm1.east) -- ([xshift=0.1em]sep.west);
\draw
[->] ([xshift=0.1em]sep.east) -- ([xshift=-0.1em]Lstm2.west);
\draw
[->] ([xshift=
-
0.1em]sep.east) -- ([xshift=-0.1em]Lstm2.west);
\draw
[->] ([xshift=0.1em]Lstm3.east) -- ([xshift=-0.1em]Lstm4.west);
\draw
[->] ([xshift=0.1em]Lstm3.east) -- ([xshift=-0.1em]Lstm4.west);
\draw
[->] ([xshift=0.1em]Lstm4.east) -- ([xshift=
-
0.1em]sep1.west);
\draw
[->] ([xshift=0.1em]Lstm4.east) -- ([xshift=0.1em]sep1.west);
\draw
[->] ([xshift=0.1em]sep1.east) -- ([xshift=-0.1em]Lstm5.west);
\draw
[->] ([xshift=
-
0.1em]sep1.east) -- ([xshift=-0.1em]Lstm5.west);
\draw
[->] ([yshift=0.1em]Lstm0.north) -- ([yshift=-0.1em]Lstm3.south);
\draw
[->] ([yshift=0.1em]Lstm0.north) -- ([yshift=-0.1em]Lstm3.south);
\draw
[->] ([yshift=0.1em]Lstm1.north) -- ([yshift=-0.1em]Lstm4.south);
\draw
[->] ([yshift=0.1em]Lstm1.north) -- ([yshift=-0.1em]Lstm4.south);
\draw
[->] ([yshift=0.1em]Lstm2.north) -- ([yshift=-0.1em]Lstm5.south);
\draw
[->] ([yshift=0.1em]Lstm2.north) -- ([yshift=-0.1em]Lstm5.south);
\draw
[->] ([xshift=0.1em]Lstm6.east) -- ([xshift=-0.1em]Lstm7.west);
\draw
[->] ([xshift=0.1em]Lstm6.east) -- ([xshift=-0.1em]Lstm7.west);
\draw
[->] ([xshift=0.1em]Lstm7.east) -- ([xshift=
-
0.1em]sep3.west);
\draw
[->] ([xshift=0.1em]Lstm7.east) -- ([xshift=0.1em]sep3.west);
\draw
[->] ([xshift=0.1em]sep3.east) -- ([xshift=-0.1em]Lstm8.west);
\draw
[->] ([xshift=
-
0.1em]sep3.east) -- ([xshift=-0.1em]Lstm8.west);
\draw
[->] ([xshift=0.1em]Lstm9.east) -- ([xshift=-0.1em]Lstm10.west);
\draw
[->] ([xshift=0.1em]Lstm9.east) -- ([xshift=-0.1em]Lstm10.west);
\draw
[->] ([xshift=0.1em]Lstm10.east) -- ([xshift=
-
0.1em]sep4.west);
\draw
[->] ([xshift=0.1em]Lstm10.east) -- ([xshift=0.1em]sep4.west);
\draw
[->] ([xshift=0.1em]sep4.east) -- ([xshift=-0.1em]Lstm11.west);
\draw
[->] ([xshift=
-
0.1em]sep4.east) -- ([xshift=-0.1em]Lstm11.west);
\draw
[->] ([yshift=0.1em]Lstm6.north) -- ([yshift=-0.1em]Lstm9.south);
\draw
[->] ([yshift=0.1em]Lstm6.north) -- ([yshift=-0.1em]Lstm9.south);
\draw
[->] ([yshift=0.1em]Lstm7.north) -- ([yshift=-0.1em]Lstm10.south);
\draw
[->] ([yshift=0.1em]Lstm7.north) -- ([yshift=-0.1em]Lstm10.south);
...
@@ -5108,35 +5108,38 @@ GPT-2 (Transformer) & Radford et al. & 2019 & 35.7
...
@@ -5108,35 +5108,38 @@ GPT-2 (Transformer) & Radford et al. & 2019 & 35.7
\begin{tikzpicture}
\begin{tikzpicture}
\begin{scope}
[scale=1.2]
\begin{scope}
[scale=1.2]
\node
[anchor=west,draw,inner sep=4pt,fill=blue!20!white,minimum width=3em] (Trm0) at (0,0)
{
\scriptsize
{
T
rm
}}
;
\node
[anchor=west,draw,inner sep=4pt,fill=blue!20!white,minimum width=3em] (Trm0) at (0,0)
{
\scriptsize
{
T
RM
}}
;
\node
[anchor=west,draw,inner sep=4pt,fill=blue!20!white,minimum width=3em] (Trm1) at ([xshift=1em]Trm0.east)
{
\scriptsize
{
T
rm
}}
;
\node
[anchor=west,draw,inner sep=4pt,fill=blue!20!white,minimum width=3em] (Trm1) at ([xshift=1em]Trm0.east)
{
\scriptsize
{
T
RM
}}
;
\node
[anchor=west,draw,inner sep=4pt,fill=blue!20!white,minimum width=3em] (Trm2) at ([xshift=1em]Trm1.east)
{
\scriptsize
{
T
rm
}}
;
\node
[anchor=west,draw,inner sep=4pt,fill=blue!20!white,minimum width=3em] (Trm2) at ([xshift=1em]Trm1.east)
{
\scriptsize
{
T
RM
}}
;
\node
[anchor=west,draw,inner sep=4pt,fill=blue!20!white,minimum width=3em] (Trm3) at ([xshift=1em]Trm2.east)
{
\scriptsize
{
T
rm
}}
;
\node
[anchor=west,draw,inner sep=4pt,fill=blue!20!white,minimum width=3em] (Trm3) at ([xshift=1em]Trm2.east)
{
\scriptsize
{
T
RM
}}
;
\node
[anchor=west,inner sep=4pt] (sep) at ([xshift=1em]Trm3.east)
{
\scriptsize
{
...
}}
;
\node
[anchor=west,inner sep=4pt] (sep) at ([xshift=1em]Trm3.east)
{
\scriptsize
{
...
}}
;
\node
[anchor=west,draw,inner sep=4pt,fill=blue!20!white,minimum width=3em] (Trm4) at ([xshift=1em]sep.east)
{
\scriptsize
{
T
rm
}}
;
\node
[anchor=west,draw,inner sep=4pt,fill=blue!20!white,minimum width=3em] (Trm4) at ([xshift=1em]sep.east)
{
\scriptsize
{
T
RM
}}
;
\node
[anchor=south,draw,inner sep=4pt,fill=blue!20!white,minimum width=3em] (Trm5) at ([yshift=1em]Trm0.north)
{
\scriptsize
{
T
rm
}}
;
\node
[anchor=south,draw,inner sep=4pt,fill=blue!20!white,minimum width=3em] (Trm5) at ([yshift=1em]Trm0.north)
{
\scriptsize
{
T
RM
}}
;
\node
[anchor=west,draw,inner sep=4pt,fill=blue!20!white,minimum width=3em] (Trm6) at ([xshift=1em]Trm5.east)
{
\scriptsize
{
T
rm
}}
;
\node
[anchor=west,draw,inner sep=4pt,fill=blue!20!white,minimum width=3em] (Trm6) at ([xshift=1em]Trm5.east)
{
\scriptsize
{
T
RM
}}
;
\node
[anchor=west,draw,inner sep=4pt,fill=blue!20!white,minimum width=3em] (Trm7) at ([xshift=1em]Trm6.east)
{
\scriptsize
{
T
rm
}}
;
\node
[anchor=west,draw,inner sep=4pt,fill=blue!20!white,minimum width=3em] (Trm7) at ([xshift=1em]Trm6.east)
{
\scriptsize
{
T
RM
}}
;
\node
[anchor=west,draw,inner sep=4pt,fill=blue!20!white,minimum width=3em] (Trm8) at ([xshift=1em]Trm7.east)
{
\scriptsize
{
T
rm
}}
;
\node
[anchor=west,draw,inner sep=4pt,fill=blue!20!white,minimum width=3em] (Trm8) at ([xshift=1em]Trm7.east)
{
\scriptsize
{
T
RM
}}
;
\node
[anchor=west,inner sep=4pt] (sep1) at ([xshift=1em]Trm8.east)
{
\scriptsize
{
...
}}
;
\node
[anchor=west,inner sep=4pt] (sep1) at ([xshift=1em]Trm8.east)
{
\scriptsize
{
...
}}
;
\node
[anchor=west,draw,inner sep=4pt,fill=blue!20!white,minimum width=3em] (Trm9) at ([xshift=1em]sep1.east)
{
\scriptsize
{
T
rm
}}
;
\node
[anchor=west,draw,inner sep=4pt,fill=blue!20!white,minimum width=3em] (Trm9) at ([xshift=1em]sep1.east)
{
\scriptsize
{
T
RM
}}
;
\node
[rectangle,rounded corners,draw=black!50,densely dashed,inner sep=0.4em] [fit = (Trm0) (Trm4) (Trm5) (Trm9)] (inputshadow)
{}
;
\node
[rectangle,rounded corners,draw=black!50,densely dashed,inner sep=0.4em] [fit = (Trm0) (Trm4) (Trm5) (Trm9)] (inputshadow)
{}
;
\node
[anchor=north,draw,inner sep=4pt,fill=ugreen!20!white,minimum width=2em] (e1) at ([yshift=-1em]Trm0.south)
{
\scriptsize
{$
E
_
1
$}}
;
\node
[anchor=north,draw,inner sep=4pt,fill=ugreen!20!white,minimum width=2em] (e1) at ([yshift=-1em]Trm0.south)
{
\scriptsize
{$
\textbf
{
x
}
_
1
$}}
;
\node
[anchor=north,draw,inner sep=4pt,fill=ugreen!20!white,minimum width=2em] (e2) at ([yshift=-1em]Trm1.south)
{
\scriptsize
{$
E
_
2
$}}
;
\node
[anchor=north,draw,inner sep=4pt,fill=ugreen!20!white,minimum width=2em] (e2) at ([yshift=-1em]Trm1.south)
{
\scriptsize
{$
\textbf
{
x
}
_
2
$}}
;
\node
[anchor=north,draw,inner sep=4pt,fill=ugreen!20!white,minimum width=2em] (e3) at ([yshift=-1em]Trm2.south)
{
\scriptsize
{$
E
_
3
$}}
;
\node
[anchor=north,draw,inner sep=4pt,fill=ugreen!20!white,minimum width=2em] (e3) at ([yshift=-1em]Trm2.south)
{
\scriptsize
{$
\textbf
{
x
}
_
3
$}}
;
\node
[anchor=north,draw,inner sep=4pt,fill=ugreen!20!white,minimum width=2em] (e4) at ([yshift=-1em]Trm3.south)
{
\scriptsize
{$
E
_
4
$}}
;
\node
[anchor=north,draw,inner sep=4pt,fill=ugreen!20!white,minimum width=2em] (e4) at ([yshift=-1em]Trm3.south)
{
\scriptsize
{$
\textbf
{
x
}
_
4
$}}
;
\node
[anchor=north,inner sep=4pt] (sep5) at ([yshift=-1em]sep.south)
{
\scriptsize
{
...
}}
;
\node
[anchor=north,inner sep=4pt] (sep5) at ([yshift=-1em]sep.south)
{
\scriptsize
{
...
}}
;
\node
[anchor=north,draw,inner sep=4pt,fill=ugreen!20!white,minimum width=2em] (e5) at ([yshift=-1em]Trm4.south)
{
\scriptsize
{$
E
_
5
$}}
;
\node
[anchor=north,draw,inner sep=4pt,fill=ugreen!20!white,minimum width=2em] (e5) at ([yshift=-1em]Trm4.south)
{
\scriptsize
{$
\textbf
{
x
}_
n
$}}
;
\node
[anchor=south,draw,inner sep=4pt,fill=yellow!30,minimum width=2em] (t1) at ([yshift=1em]Trm5.north)
{
\scriptsize
{$
T
_
1
$}}
;
\node
[anchor=south,draw,inner sep=4pt,fill=yellow!30,minimum width=2em] (t1) at ([yshift=1em]Trm5.north)
{
\scriptsize
{$
\textbf
{
h
}
_
1
$}}
;
\node
[anchor=south,draw,inner sep=4pt,fill=yellow!30,minimum width=2em] (t2) at ([yshift=1em]Trm6.north)
{
\scriptsize
{$
T
_
2
$}}
;
\node
[anchor=south,draw,inner sep=4pt,fill=yellow!30,minimum width=2em] (t2) at ([yshift=1em]Trm6.north)
{
\scriptsize
{$
\textbf
{
h
}
_
2
$}}
;
\node
[anchor=south,draw,inner sep=4pt,fill=yellow!30,minimum width=2em] (t3) at ([yshift=1em]Trm7.north)
{
\scriptsize
{$
T
_
3
$}}
;
\node
[anchor=south,draw,inner sep=4pt,fill=yellow!30,minimum width=2em] (t3) at ([yshift=1em]Trm7.north)
{
\scriptsize
{$
\textbf
{
h
}
_
3
$}}
;
\node
[anchor=south,draw,inner sep=4pt,fill=yellow!30,minimum width=2em] (t4) at ([yshift=1em]Trm8.north)
{
\scriptsize
{$
T
_
4
$}}
;
\node
[anchor=south,draw,inner sep=4pt,fill=yellow!30,minimum width=2em] (t4) at ([yshift=1em]Trm8.north)
{
\scriptsize
{$
\textbf
{
h
}
_
4
$}}
;
\node
[anchor=south,inner sep=4pt] (sep6) at ([yshift=1em]sep1.north)
{
\scriptsize
{
...
}}
;
\node
[anchor=south,inner sep=4pt] (sep6) at ([yshift=1em]sep1.north)
{
\scriptsize
{
...
}}
;
\node
[anchor=south,draw,inner sep=4pt,fill=yellow!30,minimum width=2em] (t5) at ([yshift=1em]Trm9.north)
{
\scriptsize
{$
T
_
5
$}}
;
\node
[anchor=south,draw,inner sep=4pt,fill=yellow!30,minimum width=2em] (t5) at ([yshift=1em]Trm9.north)
{
\scriptsize
{$
\textbf
{
h
}_
n
$}}
;
\node
[anchor=west,draw,inner sep=3pt,fill=blue!20!white,minimum width=1em] (Lt1) at ([yshift=1.5em]t1.west)
{
\tiny
{
TRM
}}
;
\node
[anchor=west] (Lt2) at ([xshift=-0.1em]Lt1.east)
{
\tiny
{
: Transformer
}}
;
\draw
[->] ([yshift=0.1em]e1.north) -- ([yshift=-0.1em]Trm0.south);
\draw
[->] ([yshift=0.1em]e1.north) -- ([yshift=-0.1em]Trm0.south);
\draw
[->] ([yshift=0.1em]e1.north) -- ([yshift=-0.1em]Trm1.south);
\draw
[->] ([yshift=0.1em]e1.north) -- ([yshift=-0.1em]Trm1.south);
...
@@ -5190,34 +5193,37 @@ GPT-2 (Transformer) & Radford et al. & 2019 & 35.7
...
@@ -5190,34 +5193,37 @@ GPT-2 (Transformer) & Radford et al. & 2019 & 35.7
\begin{scope}
[scale=1.2]
\begin{scope}
[scale=1.2]
\node
[anchor=west,draw,inner sep=4pt,fill=blue!20!white,minimum width=3em] (Trm0) at (0,0)
{
\scriptsize
{
Trm
}}
;
\node
[anchor=west,draw,inner sep=4pt,fill=blue!20!white,minimum width=3em] (Trm0) at (0,0)
{
\scriptsize
{
Trm
}}
;
\node
[anchor=west,draw,inner sep=4pt,fill=blue!20!white,minimum width=3em] (Trm1) at ([xshift=1em]Trm0.east)
{
\scriptsize
{
T
rm
}}
;
\node
[anchor=west,draw,inner sep=4pt,fill=blue!20!white,minimum width=3em] (Trm1) at ([xshift=1em]Trm0.east)
{
\scriptsize
{
T
RM
}}
;
\node
[anchor=west,draw,inner sep=4pt,fill=blue!20!white,minimum width=3em] (Trm2) at ([xshift=1em]Trm1.east)
{
\scriptsize
{
T
rm
}}
;
\node
[anchor=west,draw,inner sep=4pt,fill=blue!20!white,minimum width=3em] (Trm2) at ([xshift=1em]Trm1.east)
{
\scriptsize
{
T
RM
}}
;
\node
[anchor=west,draw,inner sep=4pt,fill=blue!20!white,minimum width=3em] (Trm3) at ([xshift=1em]Trm2.east)
{
\scriptsize
{
T
rm
}}
;
\node
[anchor=west,draw,inner sep=4pt,fill=blue!20!white,minimum width=3em] (Trm3) at ([xshift=1em]Trm2.east)
{
\scriptsize
{
T
RM
}}
;
\node
[anchor=west,inner sep=4pt] (sep) at ([xshift=1em]Trm3.east)
{
\scriptsize
{
...
}}
;
\node
[anchor=west,inner sep=4pt] (sep) at ([xshift=1em]Trm3.east)
{
\scriptsize
{
...
}}
;
\node
[anchor=west,draw,inner sep=4pt,fill=blue!20!white,minimum width=3em] (Trm4) at ([xshift=1em]sep.east)
{
\scriptsize
{
T
rm
}}
;
\node
[anchor=west,draw,inner sep=4pt,fill=blue!20!white,minimum width=3em] (Trm4) at ([xshift=1em]sep.east)
{
\scriptsize
{
T
RM
}}
;
\node
[anchor=south,draw,inner sep=4pt,fill=blue!20!white,minimum width=3em] (Trm5) at ([yshift=1em]Trm0.north)
{
\scriptsize
{
T
rm
}}
;
\node
[anchor=south,draw,inner sep=4pt,fill=blue!20!white,minimum width=3em] (Trm5) at ([yshift=1em]Trm0.north)
{
\scriptsize
{
T
RM
}}
;
\node
[anchor=west,draw,inner sep=4pt,fill=blue!20!white,minimum width=3em] (Trm6) at ([xshift=1em]Trm5.east)
{
\scriptsize
{
T
rm
}}
;
\node
[anchor=west,draw,inner sep=4pt,fill=blue!20!white,minimum width=3em] (Trm6) at ([xshift=1em]Trm5.east)
{
\scriptsize
{
T
RM
}}
;
\node
[anchor=west,draw,inner sep=4pt,fill=blue!20!white,minimum width=3em] (Trm7) at ([xshift=1em]Trm6.east)
{
\scriptsize
{
T
rm
}}
;
\node
[anchor=west,draw,inner sep=4pt,fill=blue!20!white,minimum width=3em] (Trm7) at ([xshift=1em]Trm6.east)
{
\scriptsize
{
T
RM
}}
;
\node
[anchor=west,draw,inner sep=4pt,fill=blue!20!white,minimum width=3em] (Trm8) at ([xshift=1em]Trm7.east)
{
\scriptsize
{
T
rm
}}
;
\node
[anchor=west,draw,inner sep=4pt,fill=blue!20!white,minimum width=3em] (Trm8) at ([xshift=1em]Trm7.east)
{
\scriptsize
{
T
RM
}}
;
\node
[anchor=west,inner sep=4pt] (sep1) at ([xshift=1em]Trm8.east)
{
\scriptsize
{
...
}}
;
\node
[anchor=west,inner sep=4pt] (sep1) at ([xshift=1em]Trm8.east)
{
\scriptsize
{
...
}}
;
\node
[anchor=west,draw,inner sep=4pt,fill=blue!20!white,minimum width=3em] (Trm9) at ([xshift=1em]sep1.east)
{
\scriptsize
{
T
rm
}}
;
\node
[anchor=west,draw,inner sep=4pt,fill=blue!20!white,minimum width=3em] (Trm9) at ([xshift=1em]sep1.east)
{
\scriptsize
{
T
RM
}}
;
\node
[rectangle,rounded corners,draw=black!50,densely dashed,inner sep=0.4em] [fit = (Trm0) (Trm4) (Trm5) (Trm9)] (inputshadow)
{}
;
\node
[rectangle,rounded corners,draw=black!50,densely dashed,inner sep=0.4em] [fit = (Trm0) (Trm4) (Trm5) (Trm9)] (inputshadow)
{}
;
\node
[anchor=north,draw,inner sep=4pt,fill=ugreen!20!white,minimum width=2em] (e1) at ([yshift=-1em]Trm0.south)
{
\scriptsize
{$
E
_
1
$}}
;
\node
[anchor=north,draw,inner sep=4pt,fill=ugreen!20!white,minimum width=2em] (e1) at ([yshift=-1em]Trm0.south)
{
\scriptsize
{$
\textbf
{
x
}
_
1
$}}
;
\node
[anchor=north,draw,inner sep=4pt,fill=ugreen!20!white,minimum width=2em] (e2) at ([yshift=-1em]Trm1.south)
{
\scriptsize
{$
E
_
2
$}}
;
\node
[anchor=north,draw,inner sep=4pt,fill=ugreen!20!white,minimum width=2em] (e2) at ([yshift=-1em]Trm1.south)
{
\scriptsize
{$
\textbf
{
x
}
_
2
$}}
;
\node
[anchor=north,draw,inner sep=4pt,fill=ugreen!20!white,minimum width=2em] (e3) at ([yshift=-1em]Trm2.south)
{
\scriptsize
{$
E
_
3
$}}
;
\node
[anchor=north,draw,inner sep=4pt,fill=ugreen!20!white,minimum width=2em] (e3) at ([yshift=-1em]Trm2.south)
{
\scriptsize
{$
\textbf
{
x
}
_
3
$}}
;
\node
[anchor=north,draw,inner sep=4pt,fill=ugreen!20!white,minimum width=2em] (e4) at ([yshift=-1em]Trm3.south)
{
\scriptsize
{$
E
_
4
$}}
;
\node
[anchor=north,draw,inner sep=4pt,fill=ugreen!20!white,minimum width=2em] (e4) at ([yshift=-1em]Trm3.south)
{
\scriptsize
{$
\textbf
{
x
}
_
4
$}}
;
\node
[anchor=north,inner sep=4pt] (sep5) at ([yshift=-1em]sep.south)
{
\scriptsize
{
...
}}
;
\node
[anchor=north,inner sep=4pt] (sep5) at ([yshift=-1em]sep.south)
{
\scriptsize
{
...
}}
;
\node
[anchor=north,draw,inner sep=4pt,fill=ugreen!20!white,minimum width=2em] (e5) at ([yshift=-1em]Trm4.south)
{
\scriptsize
{$
E
_
5
$}}
;
\node
[anchor=north,draw,inner sep=4pt,fill=ugreen!20!white,minimum width=2em] (e5) at ([yshift=-1em]Trm4.south)
{
\scriptsize
{$
\textbf
{
x
}_
n
$}}
;
\node
[anchor=south,draw,inner sep=4pt,fill=yellow!30,minimum width=2em] (t1) at ([yshift=1em]Trm5.north)
{
\scriptsize
{$
T
_
1
$}}
;
\node
[anchor=south,draw,inner sep=4pt,fill=yellow!30,minimum width=2em] (t1) at ([yshift=1em]Trm5.north)
{
\scriptsize
{$
\textbf
{
h
}
_
1
$}}
;
\node
[anchor=south,draw,inner sep=4pt,fill=yellow!30,minimum width=2em] (t2) at ([yshift=1em]Trm6.north)
{
\scriptsize
{$
T
_
2
$}}
;
\node
[anchor=south,draw,inner sep=4pt,fill=yellow!30,minimum width=2em] (t2) at ([yshift=1em]Trm6.north)
{
\scriptsize
{$
\textbf
{
h
}
_
2
$}}
;
\node
[anchor=south,draw,inner sep=4pt,fill=yellow!30,minimum width=2em] (t3) at ([yshift=1em]Trm7.north)
{
\scriptsize
{$
T
_
3
$}}
;
\node
[anchor=south,draw,inner sep=4pt,fill=yellow!30,minimum width=2em] (t3) at ([yshift=1em]Trm7.north)
{
\scriptsize
{$
\textbf
{
h
}
_
3
$}}
;
\node
[anchor=south,draw,inner sep=4pt,fill=yellow!30,minimum width=2em] (t4) at ([yshift=1em]Trm8.north)
{
\scriptsize
{$
T
_
4
$}}
;
\node
[anchor=south,draw,inner sep=4pt,fill=yellow!30,minimum width=2em] (t4) at ([yshift=1em]Trm8.north)
{
\scriptsize
{$
\textbf
{
h
}
_
4
$}}
;
\node
[anchor=south,inner sep=4pt] (sep6) at ([yshift=1em]sep1.north)
{
\scriptsize
{
...
}}
;
\node
[anchor=south,inner sep=4pt] (sep6) at ([yshift=1em]sep1.north)
{
\scriptsize
{
...
}}
;
\node
[anchor=south,draw,inner sep=4pt,fill=yellow!30,minimum width=2em] (t5) at ([yshift=1em]Trm9.north)
{
\scriptsize
{$
T
_
5
$}}
;
\node
[anchor=south,draw,inner sep=4pt,fill=yellow!30,minimum width=2em] (t5) at ([yshift=1em]Trm9.north)
{
\scriptsize
{$
\textbf
{
h
}_
n
$}}
;
\node
[anchor=west,draw,inner sep=3pt,fill=blue!20!white,minimum width=1em] (Lt1) at ([yshift=1.5em]t1.west)
{
\tiny
{
TRM
}}
;
\node
[anchor=west] (Lt2) at ([xshift=-0.1em]Lt1.east)
{
\tiny
{
: Transformer
}}
;
\draw
[->] ([yshift=0.1em]e1.north) -- ([yshift=-0.1em]Trm0.south);
\draw
[->] ([yshift=0.1em]e1.north) -- ([yshift=-0.1em]Trm0.south);
\draw
[->] ([yshift=0.1em]e1.north) -- ([yshift=-0.1em]Trm1.south);
\draw
[->] ([yshift=0.1em]e1.north) -- ([yshift=-0.1em]Trm1.south);
...
...
编写
预览
Markdown
格式
0%
重试
或
添加新文件
添加附件
取消
您添加了
0
人
到此讨论。请谨慎行事。
请先完成此评论的编辑！
取消
请
注册
或者
登录
后发表评论
