Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

add e2e tests for Unsloth qlora and test the builds #2093

Merged
merged 15 commits into from
Nov 30, 2024
Merged

add e2e tests for Unsloth qlora and test the builds #2093

merged 15 commits into from
Nov 30, 2024

Commits on Nov 21, 2024

  1. see if unsloth installs cleanly in ci

    @winglian
    winglian committed Nov 21, 2024
    Configuration menu
    Copy the full SHA
    c51df19 View commit details
    Browse the repository at this point in the history

  2. check unsloth install on regular tests, not sdist

    @winglian
    winglian committed Nov 21, 2024
    Configuration menu
    Copy the full SHA
    752d9f0 View commit details
    Browse the repository at this point in the history

  3. fix ampere check exception for ci

    @winglian
    winglian committed Nov 21, 2024
    Configuration menu
    Copy the full SHA
    035a504 View commit details
    Browse the repository at this point in the history

  4. use cached_property instead

    @winglian
    winglian committed Nov 21, 2024
    Configuration menu
    Copy the full SHA
    2c7dd74 View commit details
    Browse the repository at this point in the history

  5. add an e2e test for unsloth qlora

    @winglian
    winglian committed Nov 21, 2024
    Configuration menu
    Copy the full SHA
    51ef7cb View commit details
    Browse the repository at this point in the history

  6. reduce seq len and mbsz to prevent oom in ci

    @winglian
    winglian committed Nov 21, 2024
    Configuration menu
    Copy the full SHA
    5af4c21 View commit details
    Browse the repository at this point in the history

  7. add checks for fp16 and sdp_attention

    @winglian
    winglian committed Nov 21, 2024
    Configuration menu
    Copy the full SHA
    80f797c View commit details
    Browse the repository at this point in the history

Commits on Nov 22, 2024

  1. pin unsloth to a specific release

    @winglian
    winglian committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    a90e5cd View commit details
    Browse the repository at this point in the history

  2. add unsloth to docker image too

    @winglian
    winglian committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    356e0c3 View commit details
    Browse the repository at this point in the history

Commits on Nov 26, 2024

  1. fix flash attn xentropy patch

    @winglian
    winglian committed Nov 26, 2024
    Configuration menu
    Copy the full SHA
    5340c49 View commit details
    Browse the repository at this point in the history

  2. fix loss, add check for loss when using fa_xentropy

    @winglian
    winglian committed Nov 26, 2024
    Configuration menu
    Copy the full SHA
    35fae4b View commit details
    Browse the repository at this point in the history

  3. fix special tokens for test

    @winglian
    winglian committed Nov 26, 2024
    Configuration menu
    Copy the full SHA
    f14b42f View commit details
    Browse the repository at this point in the history

  4. typo

    @winglian
    winglian committed Nov 26, 2024
    Configuration menu
    Copy the full SHA
    e4a8407 View commit details
    Browse the repository at this point in the history

  5. test fa xentropy with and without gradient accum

    @winglian
    winglian committed Nov 26, 2024
    Configuration menu
    Copy the full SHA
    4c25ee9 View commit details
    Browse the repository at this point in the history

Commits on Nov 29, 2024

  1. pr feedback changes

    @winglian
    winglian committed Nov 29, 2024
    Configuration menu
    Copy the full SHA
    59a947c View commit details
    Browse the repository at this point in the history