Formed in 2009, the Archive Team (not to be confused with the archive.org Archive-It Team) is a rogue archivist collective dedicated to saving copies of rapidly dying or deleted websites for the sake of history and digital heritage. The group is 100% composed of volunteers and interested parties, and has expanded into a large amount of related projects for saving online and digital history.
GH-113464: A copy-and-patch JIT compiler #113465
Conversation
brandtbucher
added
performance
|
(FYI, the merge commit is broken because tier 2 isn't working on |
|
|
||
| ### Installing LLVM | ||
|
|
||
| The JIT compiler does not require end users to install any third-party dependencies, but part of it must be *built* using LLVM. You are *not* required to build the rest of CPython using LLVM, or the even the same version of LLVM (in fact, this is uncommon). |
There was a problem hiding this comment.
Could you perhaps put a few notes about what the blockers are for GCC support / what specifically is relied upon from LLVM?
There was a problem hiding this comment.
Looks like it might be https://gcc.gnu.org/bugzilla/show_bug.cgi?id=110899 and https://gcc.gnu.org/bugzilla/show_bug.cgi?id=83324.
| @@ -1579,6 +1579,26 @@ else | |||
| AC_MSG_RESULT([no]) | |||
| fi | |||
|
|
|||
| # Check for --enable-experimental-jit: | |||
There was a problem hiding this comment.
Should this look for the LLVM tools and allow overriding the used paths and such?
| #endif | ||
|
|
||
| #ifndef Py_BUILD_CORE | ||
| #error "this header requires Py_BUILD_CORE define" |
There was a problem hiding this comment.
| #error "this header requires Py_BUILD_CORE define" | |
| # error "this header requires Py_BUILD_CORE define" |
| replace = dataclasses.replace | ||
|
|
||
|
|
||
| S = typing.TypeVar("S", schema.COFFSection, schema.ELFSection, schema.MachOSection) |
There was a problem hiding this comment.
It is very common to name such variables as _S and _R
| self.disassembly.append(f"{offset:x}: {' '.join(['00'] * padding)}") | ||
| self.body.extend([0] * padding) | ||
|
|
||
| def emit_aarch64_trampoline(self, hole: Hole) -> typing.Generator[Hole, None, None]: |
There was a problem hiding this comment.
| def emit_aarch64_trampoline(self, hole: Hole) -> typing.Generator[Hole, None, None]: | |
| def emit_aarch64_trampoline(self, hole: Hole) -> typing.Iterable[Hole]: |
Most likely that is the only thing you care about in this contract.
|
|
||
|
|
||
| S = typing.TypeVar("S", schema.COFFSection, schema.ELFSection, schema.MachOSection) | ||
| R = typing.TypeVar( |
There was a problem hiding this comment.
Most likely you mean _R = TypeVar("_R", bound=schema.COFFRelocation | schema.ELFRelocation | schema.MachORelocation)
Using type var values seems not needed here.
| return dataclasses.replace(target, debug=debug, verbose=verbose) | ||
|
|
||
|
|
||
| def dump_header() -> typing.Generator[str, None, None]: |
There was a problem hiding this comment.
| def dump_header() -> typing.Generator[str, None, None]: | |
| def dump_header() -> typing.Iterable[str]: |
| yield "" | ||
|
|
||
|
|
||
| def dump_footer(opnames: list[str]) -> typing.Generator[str, None, None]: |
There was a problem hiding this comment.
| def dump_footer(opnames: list[str]) -> typing.Generator[str, None, None]: | |
| def dump_footer(opnames: list[str]) -> typing.Iterable[str]: |
There was a problem hiding this comment.
Iterable means that it's can be reused. I think here should be Iterator[str]. Same for dump & dump_header functions
There was a problem hiding this comment.
Both are fine :)
(but, mypy does not make this assumption about reusability: https://mypy-play.net/?mypy=latest&python=3.12&flags=strict&gist=c8d554830f0ab2731f4072c4ec6cb489)
| yield "}" | ||
|
|
||
|
|
||
| def dump(stencil_groups: dict[str, StencilGroup]) -> typing.Generator[str, None, None]: |
There was a problem hiding this comment.
| def dump(stencil_groups: dict[str, StencilGroup]) -> typing.Generator[str, None, None]: | |
| def dump(stencil_groups: dict[str, StencilGroup]) -> typing.Generator[str]: |
| try: | ||
| args = [name, "--version"] | ||
| if echo: | ||
| print(shlex.join(args)) |
There was a problem hiding this comment.
| try: | |
| args = [name, "--version"] | |
| if echo: | |
| print(shlex.join(args)) | |
| args = [name, "--version"] | |
| if echo: | |
| print(shlex.join(args)) | |
| try: |
| try: | ||
| args = ["brew", "--prefix", f"llvm@{LLVM_VERSION}"] | ||
| if echo: | ||
| print(shlex.join(args)) |
There was a problem hiding this comment.
| try: | |
| args = ["brew", "--prefix", f"llvm@{LLVM_VERSION}"] | |
| if echo: | |
| print(shlex.join(args)) | |
| args = ["brew", "--prefix", f"llvm@{LLVM_VERSION}"] | |
| if echo: | |
| print(shlex.join(args)) | |
| try: |
|
I'm curious, given the perf results reported in your talk, do you have any documented ideas on improving the generated code - either by tinkering with whatever gets generated (though I'm aware messing with it too much manually defeats the idea of having it work "magically"), or by improving the template for LLVM? Some of the things I see in generated code seem really pessimized, like the obligatory jump-to-continue in each op (with a jump-to-register too, probably enforced by |
| export QEMU_LD_PREFIX="/usr/$HOST" | ||
| ./configure --enable-experimental-jit ${{ matrix.debug && '--with-pydebug' || '--enable-optimizations --with-lto' }} --build=x86_64-linux-gnu --host="$HOST" --with-build-python=../build/bin/python3 --with-pkg-config=no ac_cv_buggy_getaddrinfo=no ac_cv_file__dev_ptc=no ac_cv_file__dev_ptmx=yes | ||
| make all --jobs 2 | ||
| ./python -m test --exclude ${{ matrix.exclude }} --multiprocess 0 --timeout 3600 --verbose2 --verbose3 |
There was a problem hiding this comment.
@hugovk Can we apply reusable workflow as same as free-threaded?
There was a problem hiding this comment.
I think longer term we should integrate into the main test matrix, but there's a big matrix here, and we might want to have some different trigger rules and required checks whilst this is still experimental.
Plus shorter term, we need to get it passing, re: #113465 (comment) and https://github.com/python/cpython/actions/runs/7320036782?pr=113465
| - uses: actions/checkout@v3 | ||
| - uses: actions/setup-python@v4 |
There was a problem hiding this comment.
| - uses: actions/checkout@v3 | |
| - uses: actions/setup-python@v4 | |
| - uses: actions/checkout@v4 | |
| - uses: actions/setup-python@v5 |
| - uses: actions/checkout@v3 | ||
| - uses: actions/setup-python@v4 | ||
| with: | ||
| python-version: '3.11' |
There was a problem hiding this comment.
| python-version: '3.11' | |
| python-version: '3.12' |
or for latest stable
| python-version: '3.11' | |
| python-version: '3.x' |
?
| - uses: actions/setup-python@v4 | ||
| with: | ||
| python-version: '3.11' | ||
| - name: Windows |
There was a problem hiding this comment.
Bit of space to help readability.
| - name: Windows | |
| - name: Windows |
| choco install llvm --allow-downgrade --version ${{ matrix.llvm }} | ||
| ./PCbuild/build.bat --experimental-jit ${{ matrix.debug && '-d' || '--pgo' }} -p ${{ matrix.architecture }} | ||
| ./PCbuild/rt.bat ${{ matrix.debug && '-d' }} -p ${{ matrix.architecture }} -q --exclude ${{ matrix.exclude }} --multiprocess 0 --timeout 3600 --verbose2 --verbose3 | ||
| - name: macOS |
There was a problem hiding this comment.
| - name: macOS | |
| - name: macOS |
| ./configure --enable-experimental-jit ${{ matrix.debug && '--with-pydebug' || '--enable-optimizations --with-lto' }} | ||
| make all --jobs 3 | ||
| ./python.exe -m test --exclude ${{ matrix.exclude }} --multiprocess 0 --timeout 3600 --verbose2 --verbose3 | ||
| - name: Native Linux |
There was a problem hiding this comment.
| - name: Native Linux | |
| - name: Native Linux |
| export QEMU_LD_PREFIX="/usr/$HOST" | ||
| ./configure --enable-experimental-jit ${{ matrix.debug && '--with-pydebug' || '--enable-optimizations --with-lto' }} --build=x86_64-linux-gnu --host="$HOST" --with-build-python=../build/bin/python3 --with-pkg-config=no ac_cv_buggy_getaddrinfo=no ac_cv_file__dev_ptc=no ac_cv_file__dev_ptmx=yes | ||
| make all --jobs 2 | ||
| ./python -m test --exclude ${{ matrix.exclude }} --multiprocess 0 --timeout 3600 --verbose2 --verbose3 |
There was a problem hiding this comment.
I think longer term we should integrate into the main test matrix, but there's a big matrix here, and we might want to have some different trigger rules and required checks whilst this is still experimental.
Plus shorter term, we need to get it passing, re: #113465 (comment) and https://github.com/python/cpython/actions/runs/7320036782?pr=113465
| .tp_itemsize = 0, | ||
| .tp_flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_DISALLOW_INSTANTIATION, | ||
| .tp_dealloc = uop_opt_dealloc, | ||
| }; | ||
|
|
||
| PyObject * | ||
| PyUnstable_Optimizer_NewUOpOptimizer(void) | ||
| PyUnstable_Optimizer_NewUOpOptimizer(int jit) |
There was a problem hiding this comment.
Hi, maybe jit parameter can be declared as bool type?
|
FWIW: unrelated to anything code, the "night before christmas" theme of the post is extremely well done! I cracked up. |
'Twas the night before Christmas, when all through the code
Not a core dev was merging, not even Guido;
The CI was spun on the PRs with care
In hopes that green check-markings soon would be there;
The buildbots were nestled all snug under desks,
Even PPC64 AIX;
Doc-writers, triage team, the Council of Steering,
Had just stashed every change and stopped engineering,
When in the "PRs" tab arose such a clatter,
They opened GitHub to see what was the matter.
Away to CPython they flew like a flash,
Towards sounds of
PROT_EXECand__builtin___clear_cache.First LLVM was downloaded, unzipped
Then the Actions were running a strange new build script,
When something appeared, they were stopped in their tracks,
jit_stencils.h, generated from hacks,With their spines all a-shiver, they muttered "Oh, shit...",
They knew in a moment it must be a JIT.
More rapid than interpretation it came
And it copied-and-patched every stencil by name:
"Now,
_LOAD_FAST! Now,_STORE_FAST!_BINARY_OP_ADD_INT!On,
_GUARD_DORV_VALUES_INST_ATTR_FROM_DICT!To the top of the loop! And down into the call!
Now cache away! Cache away! Cache away all!"
But why now? And how so? They needed a hint,
Thankfully, Brandt gave a great talk at the sprint;
So over to YouTube the reviewers flew,
They read the white paper, and the blog post too.
And then, after watching, they saw its appeal
Not writing the code themselves seemed so unreal.
And the platform support was almost too easy,
ARM64 Macs to 32-bit PCs.
There was some runtime C, not too much, just enough,
Basically a loader, relocating stuff;
It ran every test, one by one passed them all,
With not one runtime dependency to install.
Mostly build-time Python! With strict static typing!
For maintenance ease, and also nerd-sniping!
Though dispatch was faster, the JIT wasn't wise,
And the traces it used still should be optimized;
The code it was JIT'ing still needed some thinning,
With code models small, and some register pinning;
Or new calling conventions, shared stubs for paths slow,
Since this JIT was brand new, there was fruit hanging low.
It was awkwardly large, parsed straight out of the ELFs,
And they laughed when they saw it, in spite of themselves;
A
configureflag, and no merging this year,Soon gave them to know they had nothing to fear;
It wasn't much faster, at least it could work,
They knew that'd come later; no one was a jerk,
But they were still smart, and determined, and skilled,
They opened a shell, and configured the build;
--enable-experimental-jit, then made it,And away the JIT flew as their "+1"s okay'ed it.
But they heard it exclaim, as it traced out of sight,
"Happy JIT-mas to all, and to all a good night!"