It must be said that some problems still confuse me:
1. Whether to scale in the input layer (I try to replace it with layer specification);
2. The detailed setting of weight sharing between output projection matrix and embedding matrix in the adapter (I notice that inconsistent variance will lead to bad results);
3. The biggest confusion is that the variance increases with the calculation layer by layer (I am not sure if this phenomenon is reasonable, I will compare the behavior on the latest code).
Finally, the detailed implementation is so important to the final performance, even if it is a subtle difference.

big update! I integrate the latest updates of shell scripts, optimize the implementation of sae and fix some bugs.

fix the implementation of the relative position encoding in conformer, optimize the code of ctc loss

I reformat the code, implement the intermedia ctc with adapter and the ctc decoding.

Cumulative updates. I mainly optimize the shell scripts and support the new benchmarks. It is more friendly to MT researchers (also including me). I also improve the code. Of course, old problems still remain and new problems arise. Just keep coding.

I have wanted to do this for a long time!

I optimize the implementation of the speech-to-text tasks. As always, I update the shell scripts and YAML configures for easy training.
There may be some bugs. So, the follow-up update is coming!