Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove Duplicate Code in Fairness #17

Open
danielz02 opened this issue Oct 25, 2023 · 1 comment
Open

Remove Duplicate Code in Fairness #17

danielz02 opened this issue Oct 25, 2023 · 1 comment
Assignees
Labels
enhancement New feature or request

Comments

@danielz02
Copy link
Member

Is your feature request related to a problem? Please describe.
fairness_evaluation.py and score_calculation_script.py have over 100 lines of mostly identical code. However, score_calculation_script.py calculates numbers differently from fairness_evaluation.py. An ordinary user might find this confusing. The two branches from if args.dry_run: are also repetitive.

Describe the solution you'd like

  • Refactor the score calculation part in fairness_evaluation.py into score_calculation_script.py.
  • get_path function is also weird. Can os.path.dirname do its job?
  • Merge the two branches from if args.dry_run: are also repetitive. Since fairness needs a special do_classification. I added a monkey patch (Line 173). Whether to activate the monkey patch or not can be determined based on whether we are running dry runs. This should be much cleaner once we fully adopt the return object, so we don't need to return a bunch of tuples...
@danielz02 danielz02 added the enhancement New feature or request label Oct 25, 2023
@kangmintong
Copy link
Member

  1. Rearranged the codes and removed the duplication. Now, we can compute the scores of existing scenarios without running any scenarios: dt-run +fairness=score_computation +model_config=hf
  2. Replaced get_path function with os.path.dirname
  3. Now we can run dry_run via dt-run +fairness=crime_br_0.0 ++dry_run=True ++model=openai/gpt-3.5-turbo-0301

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants