Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

0.3.1 #54

Merged
merged 9 commits into from
Nov 11, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
55 changes: 55 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
# Contributing to TRAK

Thanks for your interest in contributing to TRAK! We welcome any sort of
feedback---bug fixes, questions, extensions, etc.

## Extending TRAK

If you have extended TRAK to your own task (e.g., by subclassing
`AbstractModelOutput` or `AbstractGradientComputer`), you can make a pull
request to add your extension to the `trak/contrib` directory. Then, users
can use your extension by importing it from `trak.contrib`. Below, we provide
guidelines for how to structure your extension.

1. Create a new directory in `trak/contrib` for your extension. For example,
if you are extending TRAK to work with diffusion models, you might create
a directory called `diffusion_models`.

2. Create a `README.md` file in your new directory. This file should contain
a description of your extension and a brief example of how to use it.

3. Add all modules that implement your extension.

4. If your extension requires any dependencies that are not already listed in
`setup.py`, add an entry to the `extras_require` dictionary in `setup.py`.
For example, if your extension requires `diffusers`, you might add the
following: `'diffusion_models': ['diffusers']`. Then, users can install your
extension's dependencies with `pip install traker[diffusion_models]`. Do
**not** add the dependencies to the `install_requires` list in `setup.py`.

5. Add any tests in a subdirectory of `tests/contrib` matching the name of your
extension's directory. For example, if your extension is in
`trak/contrib/diffusion_models`, add tests in
`tests/contrib/diffusion_models`. At a minimum, submit an integration test
that demonstrates how to use your extension. Ideally, also submit unit tests
that verify that your extension works as expected.

6. Once you are done, make a pull request to add your extension to TRAK.

## Bugs

If you observe a bug, make an issue with a code snippet that reproduces the
undesired behavior. Feel free to make pull requests that address the bug (see
below).

## Bug fixes

If you observe a bug, and you know how to fix it, feel free to make a pull
request that fixes the bug. Please include a unit test that demonstrates the
bug and verifies that your fix works. Additionally, please run the existing
tests to ensure that your fix does not break any existing functionality. You can
install the test dependencies with `pip install traker[tests]`.

Note that some of the tests are compute-intensive and may require a GPU to run.
So long as your fix does not interact at all with the functionality being tested
by these tests, you can skip them by running `pytest -m "not cuda"`.
12 changes: 9 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,11 +65,17 @@ for model_id, checkpoint in enumerate(checkpoints):

scores = traker.finalize_scores(exp_name='test')
```
Check out the [quickstart](https://trak.readthedocs.io/en/latest/quickstart.html) for a complete ready-to-run example notebook.
Then, you can use the compute TRAK scores to analyze your model's behavior. For example, here are the most (positively and negatively) impactful examples for a ResNet18 model trained on ImageNet for three targets from the ImageNet validation set:
![ImageNet Figure](assets/imagenet_figure.jpeg)

Check out the
[quickstart](https://trak.readthedocs.io/en/latest/quickstart.html) for a
complete ready-to-run example notebook. You can also find several end-to-end
examples in the `examples/` directory.

## Examples
You can find several end-to-end examples in the `examples/` directory.
## Contributing
We welcome contributions to this project! Please see our [contributing
guidelines](CONTRIBUTING.md) for more information.

## Citation
If you use this code in your work, please cite using the following BibTeX entry:
Expand Down
Binary file added assets/imagenet_figure.jpeg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
2 changes: 1 addition & 1 deletion docs/html/.buildinfo
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Sphinx build info version 1
# This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done.
config: e60030996305630cdf692b57404fa4b3
config: 746b6350ce2007ec99ab115c5cdc39c4
tags: 645f666f9bcd5a90fca523b33c5a78b7
Binary file modified docs/html/.doctrees/bert.doctree
Binary file not shown.
Binary file modified docs/html/.doctrees/clip.doctree
Binary file not shown.
Binary file modified docs/html/.doctrees/environment.pickle
Binary file not shown.
Binary file modified docs/html/.doctrees/index.doctree
Binary file not shown.
Binary file modified docs/html/.doctrees/install.doctree
Binary file not shown.
Binary file modified docs/html/.doctrees/modeloutput.doctree
Binary file not shown.
Binary file modified docs/html/.doctrees/quickstart.doctree
Binary file not shown.
Binary file modified docs/html/.doctrees/slurm.doctree
Binary file not shown.
Binary file modified docs/html/.doctrees/trak.doctree
Binary file not shown.
242 changes: 0 additions & 242 deletions docs/html/_modules/index.html

This file was deleted.

Loading