The initial implementation of dynamic encoding.

It only supports condensation based on the specific threshold then creates a new tensor.
We will merge it into the s2t_transformer.py finally.

egs/mustc/st/conf/dynamic.yaml

+arch: s2t_dynamic_transformer
+
+condensation-metric: ratio
+#condensation-metric: threshold
+condensation-mode: create
+##condensation-mode: mask
+#condensation-layers: 3,6,9
+condensation-threshold: 0.9
+condensation-ratio: 0.8
+
+share-ctc-and-embed: True
+interleaved-ctc-weight: 0.2
+interleaved-ctc-layers: 3,6,9
+share-interleaved-ctc: True

fairseq/models/speech_to_text/__init__.py

@@ -11,3 +11,4 @@ from .s2t_sate import *  # noqa
 from .s2t_dual import *  # noqa
 from .s2t_dual import *  # noqa
 from .s2t_ctc import *
 from .s2t_ctc import *
 from .s2t_multibranch import *
 from .s2t_multibranch import *
+from .s2t_dynamic_transformer import *