bpo-44394: Update libexpat copy to 2.4.1 #26945

vstinner · 2021-06-29T00:56:13Z

Update the vendored copy of libexpat to 2.4.1 (from 2.2.8) to get the
fix for the CVE-2013-0340 "Billion Laughs" vulnerability. This copy
is most used on Windows and macOS.

https://bugs.python.org/issue44394

Update the vendored copy of libexpat to 2.4.1 (from 2.2.8) to get the fix for the CVE-2013-0340 "Billion Laughs" vulnerability. This copy is most used on Windows and macOS.

vstinner · 2021-06-29T00:57:07Z

cc @tiran: please have a look at the XML vulnerability documentation change. I'm not sure that pyexpat is used by all Python XML parsers.

vstinner · 2021-06-29T01:02:14Z

I used cpython_rebuild_expat_dir.sh script attached to https://bugs.python.org/issue44394 to created this PR, then I manually reverted the following change:

diff --git a/Modules/expat/expat_external.h b/Modules/expat/expat_external.h
index f2b75dda8e..8829f77091 100644
--- a/Modules/expat/expat_external.h
+++ b/Modules/expat/expat_external.h
-
-/* Namespace external symbols to allow multiple libexpat version to
-   co-exist. */
-#include "pyexpatns.h"
-

I tested this PR with the command: ./configure --cache-file=../python-config.cache --with-pydebug CFLAGS=-O0 --with-system-ffi && make clean && make && ./python -m test -v test_pyexpat.

test_pyexpat pass successfully.

Manual test to ensure that the Python pyexpat module is not linked to the system libexpat:

$ ldd $(./python -c 'import pyexpat; print(pyexpat.__file__)')
	linux-vdso.so.1 (0x00007ffd90b6b000)
	libpthread.so.0 => /lib64/libpthread.so.0 (0x00007fa78c10c000)
	libc.so.6 => /lib64/libc.so.6 (0x00007fa78bf3d000)
	/lib64/ld-linux-x86-64.so.2 (0x00007fa78c193000)

libexpat is not listed in the library dependencies, so it's ok.

tiran · 2021-06-29T05:08:01Z

Doc/library/xml.rst

@@ -63,19 +63,21 @@ the various modules are vulnerable to them.
 =========================  ==============   ===============   ==============   ==============   ==============
 kind                       sax              etree             minidom          pulldom          xmlrpc
 =========================  ==============   ===============   ==============   ==============   ==============
-billion laughs             **Vulnerable**   **Vulnerable**    **Vulnerable**   **Vulnerable**   **Vulnerable**
+billion laughs             Safe (1)         Safe (1)          Safe (1)         Safe (1)         Safe (1)


I'm against marking it safe until Python no longer supports libexpat <= 2.4.0.

That's a fair point. Any ideas how to best communicate it in this table?

Vulnerable or Safe depends on the libexpat version, that's what I wrote in the footnote (1). I explain how how to check manually if your Python is vulnerable or not.

@tiran How do you want to explain that it depends on the libexpat version in this table, if you are unhappy with "Safe (1)"?

Do **Vulnerable** (1) until all relevant Linux distros have fixed libexpat: all supported CentOS streams, Debian stables, RHELs, Ubuntu LTS, etc.

hartwork · 2021-06-29T13:06:42Z

@vstinner the two new API functions and the new error codes do not seem included, yet. Did you see https://bugs.python.org/msg395642 ?

hartwork · 2021-06-29T13:09:05Z

Manual test to ensure that the Python pyexpat module is not linked to the system libexpat:
$ ldd $(./python -c 'import pyexpat; print(pyexpat.__file__)')
	linux-vdso.so.1 (0x00007ffd90b6b000)
	libpthread.so.0 => /lib64/libpthread.so.0 (0x00007fa78c10c000)
	libc.so.6 => /lib64/libc.so.6 (0x00007fa78bf3d000)
	/lib64/ld-linux-x86-64.so.2 (0x00007fa78c193000)
libexpat is not listed in the library dependencies, so it's ok.

@vstinner I'm not entirely sure what the idea is with that test but libexpat is listed for me on Linux:

# ldd $(python -c 'import pyexpat; print(pyexpat.__file__)')
        linux-vdso.so.1 (0x00007ffca4502000)
        libexpat.so.1 => /usr/lib64/libexpat.so.1 (0x00007ffa92b10000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00007ffa92af1000)
        libc.so.6 => /lib64/libc.so.6 (0x00007ffa92938000)
        /lib64/ld-linux-x86-64.so.2 (0x00007ffa92b73000)

Doc/library/xml.rst

vstinner · 2021-06-29T16:37:13Z

@vstinner the two new API functions and the new error codes do not seem included, yet. Did you see https://bugs.python.org/msg395642 ?

That's a new feature, it cannot be backported to older Python versions. I'm not interested to write a PR to implement it.

This PR is restricted to updated libexpat so it can be backported to all Python versions which still accept security fixes.

hartwork · 2021-06-29T16:58:56Z

@vstinner the two new API functions and the new error codes do not seem included, yet. Did you see https://bugs.python.org/msg395642 ?

That's a new feature, it cannot be backported to older Python versions. I'm not interested to write a PR to implement it.

This PR is restricted to updated libexpat so it can be backported to all Python versions which still accept security fixes.

The new API and error codes are part of the security fix.

github-actions · 2021-07-30T00:04:30Z

This PR is stale because it has been open for 30 days with no activity.

…xplanation

miss-islington · 2021-08-29T14:08:26Z

Thanks @vstinner for the PR, and @ambv for merging it 🌮🎉.. I'm working now to backport this PR to: 3.6, 3.7, 3.8, 3.9, 3.10.
🐍🍒⛏🤖

Update the vendored copy of libexpat to 2.4.1 (from 2.2.8) to get the fix for the CVE-2013-0340 "Billion Laughs" vulnerability. This copy is most used on Windows and macOS. Co-authored-by: Łukasz Langa <[email protected]> (cherry picked from commit 3fc5d84) Co-authored-by: Victor Stinner <[email protected]>

bedevere-bot · 2021-08-29T14:08:40Z

GH-28031 is a backport of this pull request to the 3.10 branch.

bedevere-bot · 2021-08-29T14:08:44Z

GH-28032 is a backport of this pull request to the 3.9 branch.

Update the vendored copy of libexpat to 2.4.1 (from 2.2.8) to get the fix for the CVE-2013-0340 "Billion Laughs" vulnerability. This copy is most used on Windows and macOS. Co-authored-by: Łukasz Langa <[email protected]> (cherry picked from commit 3fc5d84) Co-authored-by: Victor Stinner <[email protected]>

bedevere-bot · 2021-08-29T14:08:50Z

GH-28033 is a backport of this pull request to the 3.8 branch.

miss-islington · 2021-08-29T14:08:51Z

Sorry, @vstinner and @ambv, I could not cleanly backport this to 3.7 due to a conflict.
Please backport using cherry_picker on command line.
cherry_picker 3fc5d84046ddbd66abac5b598956ea34605a4e5d 3.7

miss-islington · 2021-08-29T14:08:53Z

Sorry @vstinner and @ambv, I had trouble checking out the 3.6 backport branch.
Please backport using cherry_picker on command line.
cherry_picker 3fc5d84046ddbd66abac5b598956ea34605a4e5d 3.6

ambv · 2021-08-29T14:10:31Z

@ned-deily, this is marked as needing backport to 3.6 and 3.7 as well. Since there's conflicts, please let me know if I should work on that.

Update the vendored copy of libexpat to 2.4.1 (from 2.2.8) to get the fix for the CVE-2013-0340 "Billion Laughs" vulnerability. This copy is most used on Windows and macOS. Co-authored-by: Łukasz Langa <[email protected]> (cherry picked from commit 3fc5d84) Co-authored-by: Victor Stinner <[email protected]>

Update the vendored copy of libexpat to 2.4.1 (from 2.2.8) to get the fix for the CVE-2013-0340 "Billion Laughs" vulnerability. This copy is most used on Windows and macOS. Co-authored-by: Łukasz Langa <[email protected]>. (cherry picked from commit 3fc5d84) Co-authored-by: Victor Stinner <[email protected]>

bedevere-bot · 2021-08-29T15:17:31Z

GH-28042 is a backport of this pull request to the 3.7 branch.

vstinner · 2021-08-30T13:29:10Z

Thanks for the update @ambv! I failed to find time to update this PR this summer ;-)

Update the vendored copy of libexpat to 2.4.1 (from 2.2.8) to get the fix for the CVE-2013-0340 "Billion Laughs" vulnerability. This copy is most used on Windows and macOS. Co-authored-by: Victor Stinner <[email protected]> Co-authored-by: Łukasz Langa <[email protected]>. (cherry picked from commit 3fc5d84)

…onGH-28042) Update the vendored copy of libexpat to 2.4.1 (from 2.2.8) to get the fix for the CVE-2013-0340 "Billion Laughs" vulnerability. This copy is most used on Windows and macOS. Co-authored-by: Victor Stinner <[email protected]> Co-authored-by: Łukasz Langa <[email protected]>. (cherry picked from commit 3fc5d84)

…H-28080) Update the vendored copy of libexpat to 2.4.1 (from 2.2.8) to get the fix for the CVE-2013-0340 "Billion Laughs" vulnerability. This copy is most used on Windows and macOS. Co-authored-by: Victor Stinner <[email protected]> Co-authored-by: Łukasz Langa <[email protected]>. (cherry picked from commit 3fc5d84)

bpo-44394: Update libexpat copy to 2.4.1

278da18

Update the vendored copy of libexpat to 2.4.1 (from 2.2.8) to get the fix for the CVE-2013-0340 "Billion Laughs" vulnerability. This copy is most used on Windows and macOS.

the-knights-who-say-ni added the CLA signed label Jun 29, 2021

bedevere-bot added the awaiting core review label Jun 29, 2021

tiran reviewed Jun 29, 2021

View changes

hartwork reviewed Jun 29, 2021

View changes

Doc/library/xml.rst Outdated Show resolved Hide resolved

github-actions bot added the stale label Jul 30, 2021

ambv added needs backport to 3.6 needs backport to 3.7 needs backport to 3.8 needs backport to 3.9 needs backport to 3.10 labels Aug 29, 2021

Mark "billion laughs" and "quadratic blowup" as vulnerable (1) with e…

6d51dd7

…xplanation

ambv removed the stale label Aug 29, 2021

ambv merged commit 3fc5d84 into python:main Aug 29, 2021
12 checks passed

bedevere-bot removed the awaiting core review label Aug 29, 2021

bedevere-bot removed the needs backport to 3.10 label Aug 29, 2021

bedevere-bot removed the needs backport to 3.9 label Aug 29, 2021

bedevere-bot removed the needs backport to 3.8 label Aug 29, 2021

miss-islington assigned ambv Aug 29, 2021

bedevere-bot removed the needs backport to 3.7 label Aug 29, 2021

vstinner deleted the expat241 branch Aug 30, 2021

Sep	OCT	Dec
	09
2020	2021	2022

python / cpython Public

bpo-44394: Update libexpat copy to 2.4.1 #26945

bpo-44394: Update libexpat copy to 2.4.1 #26945

vstinner commented Jun 29, 2021 •

edited by bedevere-bot

vstinner commented Jun 29, 2021

vstinner commented Jun 29, 2021

tiran Jun 29, 2021

hartwork Jun 29, 2021

vstinner Jun 29, 2021

tiran Jun 29, 2021

hartwork commented Jun 29, 2021

hartwork commented Jun 29, 2021

vstinner commented Jun 29, 2021

hartwork commented Jun 29, 2021

github-actions bot commented Jul 30, 2021

miss-islington commented Aug 29, 2021

bedevere-bot commented Aug 29, 2021

bedevere-bot commented Aug 29, 2021

bedevere-bot commented Aug 29, 2021

miss-islington commented Aug 29, 2021

miss-islington commented Aug 29, 2021

ambv commented Aug 29, 2021

bedevere-bot commented Aug 29, 2021

vstinner commented Aug 30, 2021

python / cpython Public

bpo-44394: Update libexpat copy to 2.4.1 #26945

bpo-44394: Update libexpat copy to 2.4.1 #26945

Conversation

vstinner commented Jun 29, 2021 • edited by bedevere-bot

vstinner commented Jun 29, 2021

vstinner commented Jun 29, 2021

tiran Jun 29, 2021

hartwork Jun 29, 2021

vstinner Jun 29, 2021

tiran Jun 29, 2021

hartwork commented Jun 29, 2021

hartwork commented Jun 29, 2021

vstinner commented Jun 29, 2021

hartwork commented Jun 29, 2021

github-actions bot commented Jul 30, 2021

miss-islington commented Aug 29, 2021

bedevere-bot commented Aug 29, 2021

bedevere-bot commented Aug 29, 2021

bedevere-bot commented Aug 29, 2021

miss-islington commented Aug 29, 2021

miss-islington commented Aug 29, 2021

ambv commented Aug 29, 2021

bedevere-bot commented Aug 29, 2021

vstinner commented Aug 30, 2021

vstinner commented Jun 29, 2021 •

edited by bedevere-bot