gh-119786: add intro to adaptive.md. Add tier2.md. #127175
Conversation
iritkatriel
commented
Nov 22, 2024
•
iritkatriel
commented
- Issue: create an internals documentation folder in the cpython repo #119786
InternalDocs/README.md
Outdated
| - [Adaptive Instruction Families](adaptive.md) | ||
| - [The Specializing Interpreter](adaptive.md) | ||
|
|
||
| - [The Tier 2 Interpreter (coming soon)](tier2.md) |
There was a problem hiding this comment.
@markshannon, should we call this the "JIT interpreter"? That matches the general shift in naming we've been using, and matches the configure option --enable-experimental-jit=interpreter.
(It's beyond the scope of this PR, obviously, but we should probably go through and clean up all occurrences of r"[Tt][Ii][Ee][Rr][_ ]?(2|[Tt][Ww][Oo])" in the codebase.)
| - [The Tier 2 Interpreter (coming soon)](tier2.md) | |
| - [The JIT Interpreter (coming soon)](jit_interpreter.md) |
There was a problem hiding this comment.
I was thinking of writing two documents - "the tier 2 interpreter" describes the optimiser and executors mechanism.
Then another doc about the "experimental jit interpreter" which has the stencils business.
I think it will be more accessible broken up like this.
There was a problem hiding this comment.
I'd call the thing that runs with --enable-experimental-jit the "JIT compiler" and the thing that runs with --enable-experimental-jit=interpreter either the "tier two interpreter" or "jit interpreter". The former doesn't really run your code in an interpreter, so I think calling it that could be confusing.
But naming is hard, and the term "interpreter" is super overloaded already. Eh.
There was a problem hiding this comment.
The code under tier2_dispatch is not an interpreter?
There was a problem hiding this comment.
It is. But that code is #ifdef'd out unless the build is configured with --enable-experimental-jit=interpreter. Instead of compiling to machine code, it will interpret the micro-ops, which is useful for people debugging issues in tier 2 that aren't directly related to machine code (no user should use this interpreter, it's very slow).
When the build is configured with bare --enable-experimental-jit, _Py_JIT is defined, which makes sure that executors are compiled, and it redefines a couple of macros to jump into the compiled code instead of using the tier 2 interpreter.
More info on the difference here: https://docs.python.org/3/using/configure.html#cmdoption-enable-experimental-jit
There was a problem hiding this comment.
This is internals documentation, it's supposed to help people who want to work on the code get to grips with the how thing are implemented. Explaining Tier 2 first (as a uops interpreter) and then layering the jit compilation on top of that seems more didactic to me than describing it from a user's POV.
There was a problem hiding this comment.
Yeah, I think that approach makes sense too.
iritkatriel
changed the title
|
|
||
| The optimizer that `_PyOptimizer_Optimize()` runs is configurable | ||
| via the `_Py_SetTier2Optimizer()` function (this is used in test | ||
| via `_testinternalcapi.set_optimizer()`.) |
There was a problem hiding this comment.
Is this the API we plan to remove? #126599
There was a problem hiding this comment.
The PR that removes it should update the docs.
