Name |
Last commit
|
Last Update |
---|---|---|
.. | ||
__init__.py | ||
eval_lm.py | ||
generate.py | ||
hydra_train.py | ||
interactive.py | ||
preprocess.py | ||
score.py | ||
train.py | ||
validate.py |
It must be said that some problems still confuse me: 1. Whether to scale in the input layer (I try to replace it with layer specification); 2. The detailed setting of weight sharing between output projection matrix and embedding matrix in the adapter (I notice that inconsistent variance will lead to bad results); 3. The biggest confusion is that the variance increases with the calculation layer by layer (I am not sure if this phenomenon is reasonable, I will compare the behavior on the latest code). Finally, the detailed implementation is so important to the final performance, even if it is a subtle difference.
Name |
Last commit
|
Last Update |
---|---|---|
.. | ||
__init__.py | 正在载入提交数据... | |
eval_lm.py | 正在载入提交数据... | |
generate.py | 正在载入提交数据... | |
hydra_train.py | 正在载入提交数据... | |
interactive.py | 正在载入提交数据... | |
preprocess.py | 正在载入提交数据... | |
score.py | 正在载入提交数据... | |
train.py | 正在载入提交数据... | |
validate.py | 正在载入提交数据... |