The Wayback Machine - https://web.archive.org/web/20240106171008/https://github.com/python/cpython/issues/113322
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extract args from operator.attrgetter("xxx") #113322

Open
Conchylicultor opened this issue Dec 20, 2023 · 7 comments
Open

Extract args from operator.attrgetter("xxx") #113322

Conchylicultor opened this issue Dec 20, 2023 · 7 comments
Labels
type-feature A feature request or enhancement

Comments

@Conchylicultor
Copy link
Contributor

Conchylicultor commented Dec 20, 2023

Feature or enhancement

Proposal:

Given a operator.attrgetter instance, I would like to recover which attribute will be accessed. Something like:

import operator

op = operator.attrgetter('a')

assert op.__args__ == 'a'  # << How to do this ?


op = operator.attrgetter('a', 'b')

assert op.__args__ == ('a', 'b')  # << How to do this ?

After inspecting the object, it doesn't seems possible to recover the argument passed:

image

Currently hack includes parsing repr(op) or applying op(my_dummy_obj) to a custom object which store the __getattr__ access.

Has this already been discussed elsewhere?

This is a minor feature, which does not need previous discussion elsewhere

Links to previous discussion of this feature:

No response

@Conchylicultor Conchylicultor added the type-feature A feature request or enhancement label Dec 20, 2023
@brianschubert
Copy link

brianschubert commented Dec 20, 2023

FWIW, it's currently possible to recover the args from an operator.attrgetter via its __reduce__ method:

>>> operator.attrgetter('a').__reduce__()
(<class 'operator.attrgetter'>, ('a',))

>>> operator.attrgetter('a', 'b').__reduce__()
(<class 'operator.attrgetter'>, ('a', 'b'))

@Conchylicultor
Copy link
Contributor Author

Conchylicultor commented Dec 20, 2023

Oh, that's what I was looking for. Seems a little hacky but fine enough. Sorry for the noise

@Eclips4 Eclips4 closed this as not planned Won't fix, can't repro, duplicate, stale Dec 20, 2023
@TeamSpen210
Copy link

That does work, but __reduce__()'s result really isn't public. It could change at any time, so it'd be better to have an actual public API for this.

@Eclips4
Copy link
Member

Eclips4 commented Dec 20, 2023

That does work, but __reduce__()'s result really isn't public. It could change at any time, so it'd be better to have an actual public API for this.

You're right. Let's reopen this then.

@Eclips4 Eclips4 reopened this Dec 20, 2023
@serhiy-storchaka
Copy link
Member

Why do you need this?

@Conchylicultor
Copy link
Contributor Author

Conchylicultor commented Dec 22, 2023

Why do you need this?

I'm working on a config system based on https://github.com/google/ml_collections. They have a system of lazy-reference (FieldReference) to connect various parts of the config together (e.g. changing one value change another value somewhere else). This is done by storing the ops so they can be applied later on. Like: https://github.com/google/ml_collections/blob/20f226ef4e671e0567ca6155dd99af361a4ddfd2/ml_collections/config_dict/config_dict.py#L412

I would like to serialize to json the FieldReference, so I need to somehow extract the ops from the FieldReference instances, so I can deserialize them later. Something like:

f1 = ml_collections.FieldReference()
f2 = f1.some_attribute

assert f2._ops = operator.attrgetter('some_attribute')

assert serialize_ref(f2) == {
    'id': 1234,
    'ops': {
        'op_name': 'operator.attrgetter',
        'arg': 'some_attribute',
    },
}

I'm sure if this was designed from scratch, there would be better way to handle this but I don't really have time to redesign the FieldReference system, so I'm just working with what I already have.

@serhiy-storchaka
Copy link
Member

When you implement more or less general serialization protocol, it is common to use the parts of the pickle protocol. __reduce__() returns the constructing function and its arguments, exactly what you need. You can use it right now and do not wait for 3.13.

Now, if in future we add support of keyword arguments in attrgetter, it will change __reduce__() and the code that uses it will immediately fail (if it is strict enough). But if you use custom code that only saves the args attribute, you miss additional kwargs attribute (or whatever can be added) and produce incorrect output which gives wrong object when deserialize it. And you can not notice it for a long time if it only occurs in rare circumstances.

So, while using __reduce__() can be fragile solution, it can be more reliable because it will fail faster.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type-feature A feature request or enhancement
Projects
None yet
Development

No branches or pull requests

5 participants