feat: Make tokenizer add_special_tokens
option configurable
#242
Job | Run time |
---|---|
4m 43s | |
5m 55s | |
1m 8s | |
11m 46s |
add_special_tokens
option configurable
#242
Job | Run time |
---|---|
4m 43s | |
5m 55s | |
1m 8s | |
11m 46s |