[draft] register machine #100276
[draft] register machine #100276
Conversation
iritkatriel
commented
Dec 15, 2022
•
- Each instruction is written to the bytecode as two code units, one for opcode + oparg1, the second for oparg2 + oparg3. (oparg2,3 are 0 for now).
- Add a copy of the consts to the end of the fast locals array, which now consists of [localsplus, the stack, consts].
- Register versions of the UNARY_* ops, with tmp registers added to the localsplus as "$i".
|
I have a test failing (locally, will probably fail here too) due to the per-instruction stack depth calculating dipping below 0 (there's an assertion that it doesn't). I think it could be due to the stack effect of CALL being off by 1 - it depends on the value of is_meth which is not known to stack_effect(). |
That's not what it was - I saw a test pass the first time and then fail the stackdepth>0 assertion only second time I ran it. Now my local build is broken and nothing seems to help (I tried distclean/clean/touching all the stdlib/bumping magic number). |
|
The tests all pass on Mac now. On windows there's a problem with test_import's test_unencodable_filename, which runs in a subprocess. |
I can reproduce this on windows, it does: Fatal Python error: _Py_CheckSlotResult: Slot |= of type set failed without setting an exception I managed to get it to print: |
|
It enters set_ior with an exception set. I don't know if that's supposed to happen. |
|
I'm sure it's not supposed to. |
| c->c_regcode = false; | ||
| } | ||
| else { | ||
| c->c_regcode = strstr(f, "mytest"); |
There was a problem hiding this comment.
It seems to work now when I run a small test file, but if I set c_regcode to always be true then it seems to pick up some stale frozen bytecode or something like that.
There was a problem hiding this comment.
./_bootstrap_python ./Programs/_freeze_module.py abc ./Lib/abc.py Python/frozen_modules/abc.h
1138
_bootstrap_python: Python/generated_cases.c.h:52: _PyEval_EvalFrameDefault: Assertion `value != NULL' failed.
There was a problem hiding this comment.
Did I mess up something in marshal?
There was a problem hiding this comment.
Fixed a bug, and now I get a different error:
During
./_bootstrap_python ./Programs/_freeze_module.py abc ./Lib/abc.py Python/frozen_modules/abc.h
it calls PyFunction_NewWithQualName and code_obj is freed:
frame #4: 0x0000000100178715 _bootstrap_python`_PyEval_EvalFrameDefault(tstate=<unavailable>, frame=<unavailable>, throwflag=0) at generated_cases.c.h:3731:17 [opt]
3728 TARGET(MAKE_FUNCTION) {
3729 PyObject *codeobj = POP();
3730 PyFunctionObject *func = (PyFunctionObject *)
-> 3731 PyFunction_New(codeobj, GLOBALS());
3732
3733 Py_DECREF(codeobj);
3734 if (func == NULL) {
(lldb) p _PyObject_Dump((PyObject*)codeobj)
<object at 0x101274240 is freed>
(This is why I'm wondering about marshal.)
There was a problem hiding this comment.
I don't think it's marshal. I set c_regcode = true and got the same crash. But before it runs _bootstrap_python (which crashes) it runs
./Programs/_freeze_module importlib._bootstrap ./Lib/importlib/_bootstrap.py Python/frozen_modules/importlib._bootstrap.h
(and two similar ones), which doesn't crash, and the output file is the same as with c_regcode = false.
There was a problem hiding this comment.
I found an easier way to debug, at least bypassing the _bootstrap_python crashes:
c->c_regcode = !strstr(f, "import");
This basically bypasses the register VM for importlib.
Now I can run various "real" things and get it to crash in places where it's a little easier to debug (maybe).
I wonder if the culprit might be 32b35d0 "put the tmp registers after the stack"?
There was a problem hiding this comment.
But I see nothing wrong with that commit, nor with anything else. All I see is that some register points to freed memory (filled with 0xdddddddd) instead of being either NULL or a valid object.
There was a problem hiding this comment.
Maybe we should invest in some better infrastructure to debug these kinds of things. E.g. speed up the build by not deep-freezing anything (freeze only the importlib bootstrap modules), and ignore __pycache__ directories, so you don't have to worry about the magic number. And upon a crash, somehow dump out the instructions around the crash, registers and stack, etc.
