Add support for tensor parallelism and add OLMo2-26B model config / train script #848
Job | Run time |
---|---|
9m 7s | |
23s | |
3m 56s | |
1m 19s | |
3m 16s | |
1m 7s | |
19s | |
32s | |
34s | |
42s | |
30s | |
0s | |
21m 45s |
Job | Run time |
---|---|
9m 7s | |
23s | |
3m 56s | |
1m 19s | |
3m 16s | |
1m 7s | |
19s | |
32s | |
34s | |
42s | |
30s | |
0s | |
21m 45s |