Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clarify purpose and usefulness of timeline graphs #348

Open
rwstauner opened this issue Nov 20, 2024 · 1 comment
Open

Clarify purpose and usefulness of timeline graphs #348

rwstauner opened this issue Nov 20, 2024 · 1 comment

Comments

@rwstauner
Copy link
Contributor

The timeline links at the bottom (and the pages they link to) could use descriptions.

Image

For example, the "speed comparison results" one is useful for investigating changes
but for overall YJIT performance the "yjit speedup" is a much better graph.

The time measured for any given benchmark means very little by itself (and changes when we change hardware or benchmark code).
Rather the relative comparison to other benchmarks from the same run is the metric most people would want to watch.

@maximecb
Copy link
Contributor

I agree with everything you said.

Possibly we could just remove the "speed comparison results over time?" Like you said it's not very useful and it's kind of confusing to have multiple graphs that are similar but not the same. Better to have fewer graphs that are more likely to be what people should actually be looking at? Similar feelings about "YJIT Results-Over-Time Deep Dive".

Also, what the hell, total time in seconds to run the benchmarks? Summed across all iterations? But why...???

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants