Commit f04dfc79 by xiaotong

sort samples by max length of source and target sequences

parent 12d11fab
...@@ -557,9 +557,12 @@ int T2TTrainer::LoadBuf(FILE * file, bool isSorted, int step) ...@@ -557,9 +557,12 @@ int T2TTrainer::LoadBuf(FILE * file, bool isSorted, int step)
node.offset = i; node.offset = i;
node.p = buf + offset; node.p = buf + offset;
node.size = 0; node.size = 0;
for(int j = 0; j < step; j++) int max = 0;
for (int j = 0; j < step; j++) {
node.size += seqLen[i + j]; node.size += seqLen[i + j];
node.value = seqLen[i]; max = MAX(max, seqLen[i + j]);
}
node.value = max;
node.key = rand(); node.key = rand();
count++; count++;
offset += node.size; offset += node.size;
......
Markdown 格式
0%
您添加了 0 到此讨论。请谨慎行事。
请先完成此评论的编辑!
注册 或者 后发表评论