Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Speed up task resolution and planning logic #4132

Merged
merged 15 commits into from
Dec 14, 2024

Conversation

lihaoyi
Copy link
Member

@lihaoyi lihaoyi commented Dec 14, 2024

Fixes #4129 and #4026. We never really looked at performance at this stuff before, but in order to support selective.prepare __ (the default of selective.prepare) we need to improve these bottlenecks

  • Introduces the ResolveCore#Cache class that holds mutable dictionaries to memo-ize various expensive parts of the task resolution logic

  • Lots of micro-optimizations in various places: cache Segments#hashCode, avoid calling Segments#render for sorting or Segments#part for taking the last part, etc.

Brings down time ./mill plan __ from ~60s to ~3s

@lihaoyi lihaoyi marked this pull request as ready for review December 14, 2024 10:10
@lihaoyi lihaoyi merged commit 178a545 into com-lihaoyi:main Dec 14, 2024
26 checks passed
@lefou lefou added this to the 0.12.4 milestone Dec 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

./mill selective.prepare __ on the Mill codebase takes >60s
2 participants