The Wayback Machine - https://web.archive.org/web/20230306105400/https://github.com/python/cpython/issues/102440
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

os.path.realpath(symlink to DOS devices path that starts with '\\?\Some without ":"\') returns without prefix. #102440

Open
wintermaples opened this issue Mar 5, 2023 · 3 comments
Labels
expert-pathlib OS-windows type-bug An unexpected behavior, bug, or error

Comments

@wintermaples
Copy link

os.path.realpath(symblic link to DOS Device Paths) returns a path without \\?\ prefix even if patch that fix.

For example, there is below symlinks on C:\test.

C:\test>dir
03/05/2023  06:16 AM    <SYMLINKD>     media [\\?\ContainerMappedDirectories\CE94A662-0837-4F45-B403-55B3E57CE848]
03/05/2023  06:19 AM    <SYMLINKD>     to_c [\\?\C:\]

Then exexute python and call os.path.realpath and call some functions to inspect.

>>> import os
>>> from ntpath import normpath, normcase, devnull, join, isabs, _getfinalpathname, _getfinalpathname_nonstrict
>>>
>>> to_c_fn = r'to_c'
>>> media_fn = r'media'
>>> os.path.realpath(to_c_fn)
'C:\\'
>>> os.path.realpath(media_fn)
'\\ContainerMappedDirectories\\CE94A662-0837-4F45-B403-55B3E57CE848'
>>> _getfinalpathname(to_c_fn)
'\\\\?\\C:\\'
>>> _getfinalpathname(media_fn)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
FileNotFoundError: [WinError 2] The system cannot find the file specified: 'media'
>>> _getfinalpathname_nonstrict(to_c_fn)
'\\\\?\\C:\\'
>>> _getfinalpathname_nonstrict(media_fn)
'\\ContainerMappedDirectories\\CE94A662-0837-4F45-B403-55B3E57CE848'

As the log indicates, os.path.realpath returns without a prefix.

I think it is wrong that _getfinalpathname_nonstrict removes DOS devices paths prefix.
So to fix the problem, I think we need to fix _getfinalpathname function on "Modules/posixmodule.c".

Please some opinions.

@wintermaples wintermaples added the type-bug An unexpected behavior, bug, or error label Mar 5, 2023
@wintermaples wintermaples changed the title os.path.realpath(symlink to DOS devices path that starts with '\\?\') returns without prefix. os.path.realpath(symlink to DOS devices path that starts with '\\?\Some without ":"\') returns without prefix. Mar 5, 2023
@eryksun
Copy link
Contributor

eryksun commented Mar 5, 2023

I don't see how ntpath._getfinalpathname_nonstrict() is responsible for removing the "\\?\" prefix, unless maybe there's something wrong with the target path of the "media" symlink.

A symbolic link reparse point contains a substitute path that the system uses, plus an optional print path for display1. What you see in the shell is the print path, but os.readlink() returns the substitute path.

What is the result of os.readlink(media_fn)?

As the log indicates, os.path.realpath returns without a prefix.

It's a known issue (#89760) that ntpath.realpath() mistakenly strips the prefix from some device paths for which the prefix must be retained. But that's unrelated to the given example in which the prefix gets stripped by ntpath._getfinalpathname_nonstrict().

Footnotes

  1. The protocol specifies what the print path "SHOULD" be. Per RFC 2119, "SHOULD" means that an item is strongly recommended but optional.

@wintermaples
Copy link
Author

What is the result of os.readlink(media_fn)?

>>> os.readlink('media')
'\\ContainerMappedDirectories\\93AA221C-4061-460F-8C4E-A2EAC1BF3324'

I read a code of ntpath._getfinalpathname_nonstrict(), perhaps the bug is caused by os.readlink behaviors.

The code of ntpath.realpath() processes L702-L706(Python3.11.0 amd64) because _getfinalpathname() causes FileNotFoundError (The specified path is symlink to DOS devices path that starts with '\?\Some without ":"').

        except OSError as ex:
            if strict:
                raise
            initial_winerror = ex.winerror
            path = _getfinalpathname_nonstrict(path)

That calls _getfinalpathname_nonstrict() and _getfinalpathname_nonstrict() calls os.readlink().

It's a known issue (#89760) that ntpath.realpath() mistakenly strips the prefix from some device paths for which the prefix must be retained. But that's unrelated to the given example in which the prefix gets stripped by ntpath._getfinalpathname_nonstrict().

I know and read #89760 but the bug is remained even if patch #89760 (comment).

@eryksun
Copy link
Contributor

eryksun commented Mar 6, 2023

This appears to be a shortcoming in os.readlink(), but a problem that's uncommon in practice. Normal symlinks created by WinAPI CreateSymbolicLinkW() are restricted to target a volume device name in NT's "\??\" directory, which contains local and global aliases for native NT device names, such as "\??\C:" -> "\Device\HarddiskVolume2". However, when manually creating a symlink via FSCTL_SET_REPARSE_POINT, it's possible to target an arbitrary NT path instead of a "\??\" prefixed path.

If the reparse point is a symlink with SYMLINK_FLAG_RELATIVE set in Flags, and the substitute path is a rooted path such as "\spam", then readlink() has to return the rooted path "\spam". This path is relative to the volume device of the opened path of the symlink. We currently implement this, except we don't check for SYMLINK_FLAG_RELATIVE.

If the reparse point is a mountpoint or if it's a symlink without SYMLINK_FLAG_RELATIVE set in Flags, and the substitute path is a rooted path such as "\spam", then readlink() has to prepend the prefix "\\?\GLOBALROOT" to the substitute path. Currently this is not implemented.

Assuming the target of "media" really exists, then the following stat() call should work:

os.stat(r'\\?\GLOBALROOT\ContainerMappedDirectories\93AA221C-4061-460F-8C4E-A2EAC1BF3324')

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
expert-pathlib OS-windows type-bug An unexpected behavior, bug, or error
Projects
None yet
Development

No branches or pull requests

3 participants