Skip to content

Commit

Permalink
readme
Browse files Browse the repository at this point in the history
  • Loading branch information
@ccdv-ai
ccdv-ai committed Jan 17, 2024
1 parent a9d3c4f commit d44ccb0
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ model.save_pretrained("my_new_model/")
new_tokenizer.save_pretrained("my_new_model/")
```

If you want to use a custom/different tokenizer (**experimental**), you may need to use the `custom_preprocessing` argument. \
To rely on a custom tokenizer (**experimental**), you may need to use the `custom_preprocessing` argument. \
Example using a RoBERTa (similar to Phi-2) style tokenizer for a CamemBERT model:

```python
Expand Down

0 comments on commit d44ccb0

Please sign in to comment.