Most changes to Python require a NEWS entry.

Please add it using the blurb_it web app or the blurb command-line tool.

Most changes to Python require a NEWS entry.

Please add it using the blurb_it web app or the blurb command-line tool.

Thanks for taking this on!

I suggest you take a look at how the test_capi module is laid out; the C code lives in the Modules/ dir, and the test code lives in Lib/test/test_capi.py. The tests there can be split in two patterns:

1. Tests that are fully implemented in C, for example test_from_spec_metatype_inheritance:

2. Tests that are set up in C and tested in Python, for example test_instancemethod:

You'll also need to modify configure.ac, Makefile.pre.in, and Modules/Setup.stdlib for the new module; you can grep -i for _testcapi in those files to see what needs to be done.

You'll also need to modify configure.ac, Makefile.pre.in, and Modules/Setup.stdlib for the new module; you can grep -i for _testcapi in those files to see what needs to be done.

Thanks for your advise.
I have rewritten this commit and now it is able to run testcases written both in C or Python.
Could you take a look again?

Once the framework is ready, I will add more case into it.

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

@colorfulappl, are you planning to pursue this PR? If not, would you mind me taking over and driving it forward. I'd really like to have this merged; it is good work.

Sorry, I have been busy with other stuff these days so this PR is delayed. :(

I am planning to finish this in this week. Here I made some changes at your suggestions and am going to add more test cases.

The following patch fixes the refleaks:

diff --git a/Modules/_testclinic.c b/Modules/_testclinic.c
index 0d58bee7bb..77b9d2671d 100644
--- a/Modules/_testclinic.c
+++ b/Modules/_testclinic.c
@@ -50,9 +50,8 @@ pack_arguments_newref(int argc, ...)
  * Do not accept NULL arguments unless error occurs. */
 #define RETURN_PACKED_ARGS(argc, ...) do { \
         assert(!PyErr_Occurred()); \
-        PyObject *out[argc] = {NULL,}; \
+        PyObject *out[argc] = {__VA_ARGS__}; \
         for (int _i = 0; _i < argc; _i++) { \
-            out[_i] = ((PyObject *[]){__VA_ARGS__})[_i]; \
             assert(out[_i] || PyErr_Occurred()); \
             if (!out[_i]) { \
                 for (int _j = 0; _j < _i; _j++) { \

@kumaraditya303 ➜ /workspaces/cpython (test_clinic_functionality ✗) $ ./python -m test -R 3:3 test_clinic
0:00:00 load avg: 2.72 Run tests sequentially
0:00:00 load avg: 2.72 [1/1] test_clinic
beginning 6 repetitions
123456
......

== Tests result: SUCCESS ==

1 test OK.

Total duration: 986 ms
Tests result: SUCCESS

@kumaraditya303 Thank you very much for this revision.
This patch does resolve the reference leak problem.

After this patch,

RETURN_PACKED_ARGS(3,
                   PyLong_FromLong(a),
                   PyLong_FromLong(b),
                   PyLong_FromLong(c));

is expanded to

do {
    assert(!PyErr_Occurred());
    PyObject *out[3] = {PyLong_FromUnsignedLong(a),
                        PyLong_FromUnsignedLong(b),
                        PyLong_FromUnsignedLong(c)};
    for (int _i = 0; _i < 3; _i++) {
        assert(out[_i] || PyErr_Occurred());
        if (!out[_i]) {
            for (int _j = 0; _j < _i; _j++) {
                Py_DECREF(out[_j]);
            }
            return NULL;
        }
    }
    PyObject *tuple = PyTuple_New(3);
    if (!tuple) {
        for (int _i = 0; _i < 3; _i++) {
            Py_DECREF(out[_i]);
        }
        return NULL;
    }
    for (int _i = 0; _i < 3; _i++) {
        PyTuple_SET_ITEM(tuple, _i, out[_i]);
    }
    return tuple;
} while (0);

.

But since all the PyLong_FromUnsignedLong functions are invoked at the initialization of out, there is still the object leak problem which erlend-aasland mentioned:

• For Python C APIs that return PyObject *, we must check their return value immediately and bail if needed. For example, as in char_converter_impl, calling multiple PyLong_FromUnsignedLong without checking their return value may result in exception objects leaking.

There should be some way can we delay the evaluation of the arguments to where we actually use them.
Any good ideas?

Now RETURN_PACKED_ARGS macro accepts a wrapper function and invoke it inside a for loop.
This indeed handles the arguments one-by-one, though not in an elegant manner.

Also, I'll wait for @kumaraditya303's thumbs up before landing this.

@isidentical, do you want to have a look?

BTW, @colorfulappl, please resolve the conflicts in Modules/Setup.stdlib.in (it's because of my recent PRs for the _testcapi module).

Sure. It's done. :)

Note to self: Like with _testcapi, we should consider adding a short README (or C comment) regarding how to add new tests. We can do this in a follow-up PR.

I'm going to give Batuhan a chance to chime in before merging. If I haven't heard anything until Friday, I'll land this. Thanks for doing this, and thanks for your patience with our nitpicking, @colorfulappl :)

@erlend-aasland @kumaraditya303 Thank you very much for your patient guidance! 🎉

@colorfulappl: please go ahead with the various Argument Clinic bugfixes and their accompanying tests :)

@kumaraditya303 I think we should backport this to 3.11 and 3.10. CC. @pablogsal as RM.

There are multiple clinic bugfixes coming up; IMO we should backport those bugfixes. Backporting this will benefit such bugfix backports.

I think we should backport this to 3.11 and 3.10.

SGTM

GH-100230 is a backport of this pull request to the 3.11 branch.

GH-100232 is a backport of this pull request to the 3.10 branch.