treefmt-2.0.1 is 2x slower than treefmt-0.6.1 on a cold cache

[michaelpj]

Describe the bug

Sadly I don't have a good reproducer since the codebase is private, but running treefmt --no-cache takes ~30s with treefmt 0.6.1 and ~60s with treefmt 2.0.1. That's with the exact same treefmt.toml.

I'm slightly baffled how this is happening, since I would expect the time to be dominated by the executable. Possibly it's bad choice of batch size? I can try and debug if anyone has ideas.

A possibly related mystery: it outputs this:

matched 3495 files to formatters
formatted 51 files in 58.038069297s

If it matched 3495 files, why is it only choosing to format 51? And there is no way that formatting only 51 files took 60s. Is the output misleading?

[brianmcgee]

@michaelpj you can get more verbose logging by adding -v or -vv.

When treefmt says it matched a file, that means 3495 files matched at least one of the includes globs for a formatter. It then applied those formatters, and afterwards only 51 files had actually changed.

Here's an example run against nixpkgs using this test config:

❯ nix run .# -- --config-file ./test/examples/nixpkgs.toml --tree-root /home/brian/Development/com/github/nixos/nixpkgs -c -u debug
traversed 41932 files
emitted 41932 files for processing
matched 34771 files to formatters
formatted 20181 files in 24.579394143s

This means:

• it found 41932 files
• it emitted 41932 files for processing since the cache was cold, and it has to assume they've all changed
• 34771 files matched the various formatter rules
• 20181 files had actually changed after applying the formatters

If you run with more verbose logging we should be able to get to the bottom of this.

[michaelpj]

Argh. PEBKAC. I was trying to work out why our formatting was slow, and so I had built a profiling version of our Haskell formatter, which runs slower 🤦 🤦 No blame attached to treefmt at all.

Indeed, reverting back to the original formatter, new treefmt takes half the time of the old treefmt, which is exactly in line with the parallelism gains from doing 2 batches. Hooray!

Sorry for the noise!

[brianmcgee]

Glad to hear it 🙂 I'm curious what performance improvements you might see with smaller batches 🤔