Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: Jamba instruct tokenizer #84

Merged
merged 24 commits into from
Mar 28, 2024
Merged

feat: Jamba instruct tokenizer #84

merged 24 commits into from
Mar 28, 2024

Commits on Mar 26, 2024

  1. feat: added huggingface dependency to project

    @asafgardin
    asafgardin committed Mar 26, 2024
    Configuration menu
    Copy the full SHA
    e6fe0c8 View commit details
    Browse the repository at this point in the history

  2. feat: Added jamba tokenizer

    @asafgardin
    asafgardin committed Mar 26, 2024
    Configuration menu
    Copy the full SHA
    fe91a26 View commit details
    Browse the repository at this point in the history

  3. test: Added tests to factory

    @asafgardin
    asafgardin committed Mar 26, 2024
    Configuration menu
    Copy the full SHA
    5f41587 View commit details
    Browse the repository at this point in the history

  4. refactor: Removed unnecessary const

    @asafgardin
    asafgardin committed Mar 26, 2024
    Configuration menu
    Copy the full SHA
    247e68c View commit details
    Browse the repository at this point in the history

  5. refactor: Removed redundant code

    @asafgardin
    asafgardin committed Mar 26, 2024
    Configuration menu
    Copy the full SHA
    b9c0158 View commit details
    Browse the repository at this point in the history

  6. docs: Added comment

    @asafgardin
    asafgardin committed Mar 26, 2024
    Configuration menu
    Copy the full SHA
    7dba113 View commit details
    Browse the repository at this point in the history

  7. refactor: Added import to init

    @asafgardin
    asafgardin committed Mar 26, 2024
    Configuration menu
    Copy the full SHA
    7e37fd0 View commit details
    Browse the repository at this point in the history

  8. fix: path

    @asafgardin
    asafgardin committed Mar 26, 2024
    Configuration menu
    Copy the full SHA
    7e4c8a9 View commit details
    Browse the repository at this point in the history

Commits on Mar 27, 2024

  1. refactor: Moved config load to the init

    @asafgardin
    asafgardin committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    9993dcd View commit details
    Browse the repository at this point in the history

  2. fix: Removed python 3.7 support in tests

    @asafgardin
    asafgardin committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    a78053c View commit details
    Browse the repository at this point in the history

  3. refactor: Renamed file

    @asafgardin
    asafgardin committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    5dff07f View commit details
    Browse the repository at this point in the history

  4. test: Added skip

    @asafgardin
    asafgardin committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    2e6fabf View commit details
    Browse the repository at this point in the history

  5. test: Removed 3.12 (testing)

    @asafgardin
    asafgardin committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    dba73dd View commit details
    Browse the repository at this point in the history

  6. fix: python support change

    @asafgardin
    asafgardin committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    f28164d View commit details
    Browse the repository at this point in the history

  7. fix: Added JambaInstructTokenizer to main init

    @asafgardin
    asafgardin committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    b8dfc84 View commit details
    Browse the repository at this point in the history

  8. fix: Changed condition

    @asafgardin
    asafgardin committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    5a2f628 View commit details
    Browse the repository at this point in the history

  9. refactor: Moved methods

    @asafgardin
    asafgardin committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    6340fc0 View commit details
    Browse the repository at this point in the history

  10. fix: Added value error raise if tokenizer name not found

    @asafgardin
    asafgardin committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    6010ff4 View commit details
    Browse the repository at this point in the history

Commits on Mar 28, 2024

  1. fix: uses tokenizer instead

    @asafgardin
    asafgardin committed Mar 28, 2024
    Configuration menu
    Copy the full SHA
    6465b78 View commit details
    Browse the repository at this point in the history

  2. refactor: used tokenizers package instead of transformers

    @asafgardin
    asafgardin committed Mar 28, 2024
    Configuration menu
    Copy the full SHA
    8897733 View commit details
    Browse the repository at this point in the history

  3. fix: Added docstring and organized file

    @asafgardin
    asafgardin committed Mar 28, 2024
    Configuration menu
    Copy the full SHA
    5ccd655 View commit details
    Browse the repository at this point in the history

  4. test: Added unittests to jamba instruct tokenizer

    @asafgardin
    asafgardin committed Mar 28, 2024
    Configuration menu
    Copy the full SHA
    7755a1a View commit details
    Browse the repository at this point in the history

  5. feat: Used hf path to tokenizer

    @asafgardin
    asafgardin committed Mar 28, 2024
    Configuration menu
    Copy the full SHA
    9d193c9 View commit details
    Browse the repository at this point in the history

  6. fix: CR

    @asafgardin
    asafgardin committed Mar 28, 2024
    Configuration menu
    Copy the full SHA
    262afe8 View commit details
    Browse the repository at this point in the history