Skip to content
项目
群组
代码片段
帮助
当前项目
正在载入...
登录 / 注册
切换导航面板
M
mtbookv2
概览
Overview
Details
Activity
Cycle Analytics
版本库
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Charts
问题
0
Issues
0
列表
Board
标记
里程碑
合并请求
0
Merge Requests
0
CI / CD
CI / CD
流水线
作业
日程表
图表
维基
Wiki
代码片段
Snippets
成员
Collapse sidebar
Close sidebar
活动
图像
聊天
创建新问题
作业
提交
Issue Boards
Open sidebar
NiuTrans
mtbookv2
Commits
7a19b5c0
Commit
7a19b5c0
authored
Nov 14, 2020
by
zengxin
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
11
parent
56e2b861
显示空白字符变更
内嵌
并排
正在显示
3 个修改的文件
包含
12 行增加
和
12 行删除
+12
-12
Chapter11/Figures/figure-convolution-kernel.tex
+2
-2
Chapter11/Figures/figure-structural-comparison-b.tex
+8
-8
Chapter11/chapter11.tex
+2
-2
没有找到文件。
Chapter11/Figures/figure-convolution-kernel.tex
查看文件 @
7a19b5c0
...
@@ -51,8 +51,8 @@
...
@@ -51,8 +51,8 @@
%\node[minimum width = 1.8cm] (sub) at ([xshift=-5.5cm,yshift=2cm]num9_9.east) {};
%\node[minimum width = 1.8cm] (sub) at ([xshift=-5.5cm,yshift=2cm]num9_9.east) {};
\draw
[decorate,decoration={brace,mirror,raise=0pt,amplitude=0.3cm},black,thick]
([yshift=0.4cm,xshift=-0.1cm]num1
_
1.west) -- node[att,xshift=-0.5cm]
{$
q
$}
([yshift=-0.4cm,xshift=-0.1cm]num3
_
3.west);
\draw
[decorate,decoration={brace,mirror,raise=0pt,amplitude=0.3cm},black,thick]
([yshift=0.4cm,xshift=-0.1cm]num1
_
1.west) -- node[att,xshift=-0.5cm]
{$
k
$}
([yshift=-0.4cm,xshift=-0.1cm]num3
_
3.west);
\draw
[decorate,decoration={brace,raise=0pt,amplitude=0.3cm},black,thick]
([xshift=-0.4cm,yshift=0.1cm]num1.north) -- node[att,yshift=0.5cm]
{$
k
$}
([xshift=0.4cm,yshift=0.1cm]num7.north);
\draw
[decorate,decoration={brace,raise=0pt,amplitude=0.3cm},black,thick]
([xshift=-0.4cm,yshift=0.1cm]num1.north) -- node[att,yshift=0.5cm]
{$
q
$}
([xshift=0.4cm,yshift=0.1cm]num7.north);
\draw
[decorate,decoration={brace,mirror,raise=0pt,amplitude=0.3cm},black,thick]
([xshift=0.5cm,yshift=0.00cm]num9
_
9.south) -- node[att,xshift=0.5cm,yshift=-0.3cm]
{$
o
$}
([xshift=0.5cm,yshift=0.00cm]num9.south);
\draw
[decorate,decoration={brace,mirror,raise=0pt,amplitude=0.3cm},black,thick]
([xshift=0.5cm,yshift=0.00cm]num9
_
9.south) -- node[att,xshift=0.5cm,yshift=-0.3cm]
{$
o
$}
([xshift=0.5cm,yshift=0.00cm]num9.south);
...
...
Chapter11/Figures/figure-structural-comparison-b.tex
查看文件 @
7a19b5c0
...
@@ -2,7 +2,7 @@
...
@@ -2,7 +2,7 @@
\begin{tikzpicture}
[node distance = 0cm]
\begin{tikzpicture}
[node distance = 0cm]
\node
(num1
_
0)[num, fill = blue!40]
{
\text
bf
{
\textcolor
{
white
}
0
}}
;
\node
(num1
_
0)[num, fill = blue!40]
{
\text
color
{
white
}{$
\mathbi
{
0
}$
}}
;
\node
(num1
_
1)[num,right of = num1
_
0,xshift = 1.2cm]
{$
\mathbi
{
e
}_
1
$}
;
\node
(num1
_
1)[num,right of = num1
_
0,xshift = 1.2cm]
{$
\mathbi
{
e
}_
1
$}
;
\node
(num1
_
2)[num,right of = num1
_
1,xshift = 1.2cm]
{
\textcolor
{
blue!70
}{$
\mathbi
{
e
}_
2
$}}
;
\node
(num1
_
2)[num,right of = num1
_
1,xshift = 1.2cm]
{
\textcolor
{
blue!70
}{$
\mathbi
{
e
}_
2
$}}
;
\node
(num1
_
3)[num,right of = num1
_
2,xshift = 1.2cm]
{
\textcolor
{
blue!70
}{$
\mathbi
{
e
}_
3
$}}
;
\node
(num1
_
3)[num,right of = num1
_
2,xshift = 1.2cm]
{
\textcolor
{
blue!70
}{$
\mathbi
{
e
}_
3
$}}
;
...
@@ -12,11 +12,11 @@
...
@@ -12,11 +12,11 @@
\node
(num1
_
7)[num,right of = num1
_
6,xshift = 1.2cm]
{
\textcolor
{
blue!70
}{$
\mathbi
{
e
}_
7
$}}
;
\node
(num1
_
7)[num,right of = num1
_
6,xshift = 1.2cm]
{
\textcolor
{
blue!70
}{$
\mathbi
{
e
}_
7
$}}
;
\node
(num1
_
8)[num,right of = num1
_
7,xshift = 1.2cm]
{
\textcolor
{
blue!70
}{$
\mathbi
{
e
}_
8
$}}
;
\node
(num1
_
8)[num,right of = num1
_
7,xshift = 1.2cm]
{
\textcolor
{
blue!70
}{$
\mathbi
{
e
}_
8
$}}
;
\node
(num1
_
9)[num,right of = num1
_
8,xshift = 1.2cm]
{$
\mathbi
{
e
}_
9
$}
;
\node
(num1
_
9)[num,right of = num1
_
8,xshift = 1.2cm]
{$
\mathbi
{
e
}_
9
$}
;
\node
(num1
_
10)[num,right of = num1
_
9,xshift = 1.2cm, fill = blue!40]
{
\textbf
0
}
;
\node
(num1
_
10)[num,right of = num1
_
9,xshift = 1.2cm, fill = blue!40]
{
$
\mathbi
{
0
}$
}
;
\node
(A)[below of = num2,yshift = -0.6cm]
{
A
}
;
\node
(A)[below of = num2,yshift = -0.6cm]
{
A
}
;
\node
(B)[below of = num8,yshift = -0.6cm]
{
B
}
;
\node
(B)[below of = num8,yshift = -0.6cm]
{
B
}
;
\node
(num2
_
0)[num,above of = num1
_
0,yshift = 1.2cm, fill = blue!40]
{
\text
bf
{
\textcolor
{
white
}
0
}}
;
\node
(num2
_
0)[num,above of = num1
_
0,yshift = 1.2cm, fill = blue!40]
{
\text
color
{
white
}{$
\mathbi
{
0
}$
}}
;
\node
(num2
_
1)[num,right of = num2
_
0,xshift = 1.2cm]
{
\textbf
2
}
;
\node
(num2
_
1)[num,right of = num2
_
0,xshift = 1.2cm]
{
\textbf
2
}
;
\node
(num2
_
2)[num,right of = num2
_
1,xshift = 1.2cm]
{
\textbf
2
}
;
\node
(num2
_
2)[num,right of = num2
_
1,xshift = 1.2cm]
{
\textbf
2
}
;
\node
(num2
_
3)[num,right of = num2
_
2,xshift = 1.2cm]
{
\textbf
{
\textcolor
{
blue!70
}
2
}}
;
\node
(num2
_
3)[num,right of = num2
_
2,xshift = 1.2cm]
{
\textbf
{
\textcolor
{
blue!70
}
2
}}
;
...
@@ -26,9 +26,9 @@
...
@@ -26,9 +26,9 @@
\node
(num2
_
7)[num,right of = num2
_
6,xshift = 1.2cm]
{
\textbf
{
\textcolor
{
blue!70
}
2
}}
;
\node
(num2
_
7)[num,right of = num2
_
6,xshift = 1.2cm]
{
\textbf
{
\textcolor
{
blue!70
}
2
}}
;
\node
(num2
_
8)[num,right of = num2
_
7,xshift = 1.2cm]
{
\textbf
2
}
;
\node
(num2
_
8)[num,right of = num2
_
7,xshift = 1.2cm]
{
\textbf
2
}
;
\node
(num2
_
9)[num,right of = num2
_
8,xshift = 1.2cm]
{
\textbf
2
}
;
\node
(num2
_
9)[num,right of = num2
_
8,xshift = 1.2cm]
{
\textbf
2
}
;
\node
(num2
_
10)[num,right of = num2
_
9,xshift = 1.2cm, fill = blue!40]
{
\textbf
0
}
;
\node
(num2
_
10)[num,right of = num2
_
9,xshift = 1.2cm, fill = blue!40]
{
$
\mathbi
{
0
}$
}
;
\node
(num3
_
0)[num,above of = num2
_
0,yshift = 1.2cm, fill = blue!40]
{
\text
bf
{
\textcolor
{
white
}
0
}}
;
\node
(num3
_
0)[num,above of = num2
_
0,yshift = 1.2cm, fill = blue!40]
{
\text
color
{
white
}{$
\mathbi
{
0
}$
}}
;
\node
(num3
_
1)[num,right of = num3
_
0,xshift = 1.2cm]
{
\textbf
3
}
;
\node
(num3
_
1)[num,right of = num3
_
0,xshift = 1.2cm]
{
\textbf
3
}
;
\node
(num3
_
2)[num,right of = num3
_
1,xshift = 1.2cm]
{
\textbf
3
}
;
\node
(num3
_
2)[num,right of = num3
_
1,xshift = 1.2cm]
{
\textbf
3
}
;
\node
(num3
_
3)[num,right of = num3
_
2,xshift = 1.2cm]
{
\textbf
3
}
;
\node
(num3
_
3)[num,right of = num3
_
2,xshift = 1.2cm]
{
\textbf
3
}
;
...
@@ -38,9 +38,9 @@
...
@@ -38,9 +38,9 @@
\node
(num3
_
7)[num,right of = num3
_
6,xshift = 1.2cm]
{
\textbf
3
}
;
\node
(num3
_
7)[num,right of = num3
_
6,xshift = 1.2cm]
{
\textbf
3
}
;
\node
(num3
_
8)[num,right of = num3
_
7,xshift = 1.2cm]
{
\textbf
3
}
;
\node
(num3
_
8)[num,right of = num3
_
7,xshift = 1.2cm]
{
\textbf
3
}
;
\node
(num3
_
9)[num,right of = num3
_
8,xshift = 1.2cm]
{
\textbf
3
}
;
\node
(num3
_
9)[num,right of = num3
_
8,xshift = 1.2cm]
{
\textbf
3
}
;
\node
(num3
_
10)[num,right of = num3
_
9,xshift = 1.2cm, fill = blue!40]
{
\textbf
0
}
;
\node
(num3
_
10)[num,right of = num3
_
9,xshift = 1.2cm, fill = blue!40]
{
$
\mathbi
{
0
}$
}
;
\node
(num4
_
0)[num,above of = num3
_
0,yshift = 1.2cm, fill = blue!40]
{
\text
bf
{
\textcolor
{
white
}
0
}}
;
\node
(num4
_
0)[num,above of = num3
_
0,yshift = 1.2cm, fill = blue!40]
{
\text
color
{
white
}{$
\mathbi
{
0
}$
}}
;
\node
(num4
_
1)[num,right of = num4
_
0,xshift = 1.2cm]
{
\textbf
4
}
;
\node
(num4
_
1)[num,right of = num4
_
0,xshift = 1.2cm]
{
\textbf
4
}
;
\node
(num4
_
2)[num,right of = num4
_
1,xshift = 1.2cm]
{
\textbf
4
}
;
\node
(num4
_
2)[num,right of = num4
_
1,xshift = 1.2cm]
{
\textbf
4
}
;
\node
(num4
_
3)[num,right of = num4
_
2,xshift = 1.2cm]
{
\textbf
4
}
;
\node
(num4
_
3)[num,right of = num4
_
2,xshift = 1.2cm]
{
\textbf
4
}
;
...
@@ -50,7 +50,7 @@
...
@@ -50,7 +50,7 @@
\node
(num4
_
7)[num,right of = num4
_
6,xshift = 1.2cm]
{
\textbf
4
}
;
\node
(num4
_
7)[num,right of = num4
_
6,xshift = 1.2cm]
{
\textbf
4
}
;
\node
(num4
_
8)[num,right of = num4
_
7,xshift = 1.2cm]
{
\textbf
4
}
;
\node
(num4
_
8)[num,right of = num4
_
7,xshift = 1.2cm]
{
\textbf
4
}
;
\node
(num4
_
9)[num,right of = num4
_
8,xshift = 1.2cm]
{
\textbf
4
}
;
\node
(num4
_
9)[num,right of = num4
_
8,xshift = 1.2cm]
{
\textbf
4
}
;
\node
(num4
_
10)[num,right of = num4
_
9,xshift = 1.2cm, fill = blue!40]
{
\textbf
0
}
;
\node
(num4
_
10)[num,right of = num4
_
9,xshift = 1.2cm, fill = blue!40]
{
$
\mathbi
{
0
}$
}
;
\draw
[->, thick](num1
_
0.north)--([xshift=-0.1em,yshift=-0.1em]num2
_
1.south);
\draw
[->, thick](num1
_
0.north)--([xshift=-0.1em,yshift=-0.1em]num2
_
1.south);
\draw
[->, thick](num2
_
0.north)--([xshift=-0.1em,yshift=-0.1em]num3
_
1.south);
\draw
[->, thick](num2
_
0.north)--([xshift=-0.1em,yshift=-0.1em]num3
_
1.south);
...
...
Chapter11/chapter11.tex
查看文件 @
7a19b5c0
...
@@ -176,7 +176,7 @@
...
@@ -176,7 +176,7 @@
%\input{./Chapter11/Figures/figure-f }
%\input{./Chapter11/Figures/figure-f }
\subfigure
[循环神经网络的串行结构（$O(n)$）]
{
\input
{
./Chapter11/Figures/figure-structural-comparison-a
}}
\subfigure
[循环神经网络的串行结构（$O(n)$）]
{
\input
{
./Chapter11/Figures/figure-structural-comparison-a
}}
\subfigure
[卷积神经网络的层级结构（$O(n/k)$）]
{
\input
{
./Chapter11/Figures/figure-structural-comparison-b
}}
\subfigure
[卷积神经网络的层级结构（$O(n/k)$）]
{
\input
{
./Chapter11/Figures/figure-structural-comparison-b
}}
\caption
{
串行及层级结构对比（
$
\mathbi
{
e
}_
i
$
表示词嵌入，
0表示0
向量，2,3,4表示第几层）
}
\caption
{
串行及层级结构对比（
$
\mathbi
{
e
}_
i
$
表示词嵌入，
$
\mathbi
{
0
}$
表示
$
\mathbi
{
0
}$
向量，2,3,4表示第几层）
}
\label
{
fig:11-9
}
\label
{
fig:11-9
}
\end{figure}
\end{figure}
%----------------------------------------------
%----------------------------------------------
...
@@ -440,7 +440,7 @@
...
@@ -440,7 +440,7 @@
\section
{
局部模型的改进
}
\section
{
局部模型的改进
}
\parinterval
在序列建模中，卷积神经网络可以通过参数共享，高效地捕捉局部上下文特征，如图
\ref
{
fig:11-11
}
所示。但是通过进一步分析可以发现，在标准卷积操作中包括了不同词和不同通道之间两种信息的交互，每个卷积核都是对相邻词的不同通道进行卷积，参数量为
$
K
\times
O
$
。其中
$
K
$
为卷积核大小，
$
O
$
为输入的通道数，即单词表示的维度大小。因此
$
N
$
个卷积核总共的参数量为
$
K
\times
O
\times
N
$
。这里涉及卷积核大小、输入通道数和输出通道数三个维度，因此计算复杂度较高。为了进一步提升计算效率，降低参数量，一些研究人员提出
{
\small\bfnew
{
深度可分离卷积
}}
\index
{
深度可分离卷积
}
（Depthwise Separable Convolution）
\index
{
Depthwise Separable Convolution
}
，将空间维度和通道间的信息交互分离成深度卷积（也叫逐通道卷积，Depthwise Convolution）
\index
{
逐通道卷积，Depthwise Convolution
}
和
{
\small\bfnew
{
逐点卷积
}}
\index
{
逐点卷积
}
（Pointwise Convolution）
\index
{
Pointwise Convolution
}
两部分
\upcite
{
Chollet2017XceptionDL,Howard2017MobileNetsEC
}
。除了直接将深度可分离卷积应用到神经机器翻译中
\upcite
{
Kaiser2018DepthwiseSC
}
，研究人员提出使用更高效的
{
\small\bfnew
{
轻量卷积
}}
\index
{
轻量卷积
}
（Lightweight Convolution）
\index
{
Lightweight Convolution
}
和
{
\small\bfnew
{
动态卷积
}}
\index
{
动态卷积
}
（Dynamic
c
onvolution）
\index
{
Dynamic convolution
}
来进行不同词之间的特征提取
\upcite
{
Wu2019PayLA
}
。本节将主要介绍这些改进的卷积操作。在后续章节中也会看到这些模型在神经机器翻译中的应用。
\parinterval
在序列建模中，卷积神经网络可以通过参数共享，高效地捕捉局部上下文特征，如图
\ref
{
fig:11-11
}
所示。但是通过进一步分析可以发现，在标准卷积操作中包括了不同词和不同通道之间两种信息的交互，每个卷积核都是对相邻词的不同通道进行卷积，参数量为
$
K
\times
O
$
。其中
$
K
$
为卷积核大小，
$
O
$
为输入的通道数，即单词表示的维度大小。因此
$
N
$
个卷积核总共的参数量为
$
K
\times
O
\times
N
$
。这里涉及卷积核大小、输入通道数和输出通道数三个维度，因此计算复杂度较高。为了进一步提升计算效率，降低参数量，一些研究人员提出
{
\small\bfnew
{
深度可分离卷积
}}
\index
{
深度可分离卷积
}
（Depthwise Separable Convolution）
\index
{
Depthwise Separable Convolution
}
，将空间维度和通道间的信息交互分离成深度卷积（也叫逐通道卷积，Depthwise Convolution）
\index
{
逐通道卷积，Depthwise Convolution
}
和
{
\small\bfnew
{
逐点卷积
}}
\index
{
逐点卷积
}
（Pointwise Convolution）
\index
{
Pointwise Convolution
}
两部分
\upcite
{
Chollet2017XceptionDL,Howard2017MobileNetsEC
}
。除了直接将深度可分离卷积应用到神经机器翻译中
\upcite
{
Kaiser2018DepthwiseSC
}
，研究人员提出使用更高效的
{
\small\bfnew
{
轻量卷积
}}
\index
{
轻量卷积
}
（Lightweight Convolution）
\index
{
Lightweight Convolution
}
和
{
\small\bfnew
{
动态卷积
}}
\index
{
动态卷积
}
（Dynamic
C
onvolution）
\index
{
Dynamic convolution
}
来进行不同词之间的特征提取
\upcite
{
Wu2019PayLA
}
。本节将主要介绍这些改进的卷积操作。在后续章节中也会看到这些模型在神经机器翻译中的应用。
%----------------------------------------------------------------------------------------
%----------------------------------------------------------------------------------------
% NEW SUB-SECTION
% NEW SUB-SECTION
...
...
编写
预览
Markdown
格式
0%
重试
或
添加新文件
添加附件
取消
您添加了
0
人
到此讨论。请谨慎行事。
请先完成此评论的编辑！
取消
请
注册
或者
登录
后发表评论
