Fix crash involving exhausted list operator

Additionally don't generate sbom (causes build to fail),
backport an accidentally lost _PyCFunctionFastWithKeywords name.
This commit is contained in:
Karolina Surma 2024-02-21 10:41:48 +01:00
commit c9e9f5fd4d
4 changed files with 205 additions and 1 deletions

View file

@ -0,0 +1,31 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Karolina Surma <ksurma@redhat.com>
Date: Fri, 19 Jan 2024 12:00:18 +0100
Subject: [PATCH] 00418: Don't generate sbom in make regen-all
The script and make target, modified in Python 3.13.0a4,
assumes the presence of expat and _decimal/libmpdec libraries, resulting in an
error and failed build when not found.
Reported upstream: https://github.com/python/cpython/issues/115663
---
Makefile.pre.in | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/Makefile.pre.in b/Makefile.pre.in
index 8252e6631c..3f4a89ad07 100644
--- a/Makefile.pre.in
+++ b/Makefile.pre.in
@@ -1625,10 +1625,11 @@ regen-unicodedata:
regen-all: regen-cases regen-typeslots \
regen-token regen-ast regen-keyword regen-sre regen-frozen \
regen-pegen-metaparser regen-pegen regen-test-frozenmain \
- regen-test-levenshtein regen-global-objects regen-sbom regen-jit
+ regen-test-levenshtein regen-global-objects regen-jit
@echo
@echo "Note: make regen-stdlib-module-names, make regen-limited-abi, "
- @echo "make regen-configure and make regen-unicodedata should be run manually"
+ @echo "make regen-configure, make regen-unicodedata and make regen-sbom "
+ @echo "should be run manually"
############################################################################
# Special rules for object files

View file

@ -0,0 +1,41 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Victor Stinner <vstinner@python.org>
Date: Mon, 19 Feb 2024 13:20:46 +0100
Subject: [PATCH] 00420: gh-114626: Add again _PyCFunctionFastWithKeywords name
(GH-115561)
Keep the old private _PyCFunctionFastWithKeywords name (Python 3.7)
as an alias to the new public name PyCFunctionFastWithKeywords
(Python 3.13a4).
_PyCFunctionWithKeywords doesn't exist in Python 3.13a3, whereas
_PyCFunctionFastWithKeywords was removed in Python 3.13a4.
---
Include/methodobject.h | 2 +-
.../next/C API/2024-02-16-15-56-53.gh-issue-114626.ie2esA.rst | 4 ++++
2 files changed, 5 insertions(+), 1 deletion(-)
create mode 100644 Misc/NEWS.d/next/C API/2024-02-16-15-56-53.gh-issue-114626.ie2esA.rst
diff --git a/Include/methodobject.h b/Include/methodobject.h
index 452f891a7a..39272815b1 100644
--- a/Include/methodobject.h
+++ b/Include/methodobject.h
@@ -31,7 +31,7 @@ typedef PyObject *(*PyCMethod)(PyObject *, PyTypeObject *, PyObject *const *,
// Note that the underscore-prefixed names were documented in public docs;
// people may be using them.
typedef PyCFunctionFast _PyCFunctionFast;
-typedef PyCFunctionWithKeywords _PyCFunctionWithKeywords;
+typedef PyCFunctionFastWithKeywords _PyCFunctionFastWithKeywords;
// Cast an function to the PyCFunction type to use it with PyMethodDef.
//
diff --git a/Misc/NEWS.d/next/C API/2024-02-16-15-56-53.gh-issue-114626.ie2esA.rst b/Misc/NEWS.d/next/C API/2024-02-16-15-56-53.gh-issue-114626.ie2esA.rst
new file mode 100644
index 0000000000..763f4cee6d
--- /dev/null
+++ b/Misc/NEWS.d/next/C API/2024-02-16-15-56-53.gh-issue-114626.ie2esA.rst
@@ -0,0 +1,4 @@
+Add again ``_PyCFunctionFastWithKeywords`` name, removed in Python 3.13
+alpha 4 by mistake. Keep the old private ``_PyCFunctionFastWithKeywords``
+name (Python 3.7) as an alias to the new public name
+``PyCFunctionFastWithKeywords`` (Python 3.13a4). Patch by Victor Stinner.

View file

@ -0,0 +1,105 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Sam Gross <colesbury@gmail.com>
Date: Tue, 20 Feb 2024 15:18:44 -0500
Subject: [PATCH] 00421: gh-115733: Fix crash involving exhausted list iterator
(#115740)
---
Lib/test/list_tests.py | 5 +++++
.../2024-02-20-18-49-02.gh-issue-115733.51Zb85.rst | 1 +
Objects/listobject.c | 10 +++++-----
Python/bytecodes.c | 3 ++-
Python/executor_cases.c.h | 1 +
Python/generated_cases.c.h | 2 +-
6 files changed, 15 insertions(+), 7 deletions(-)
create mode 100644 Misc/NEWS.d/next/Core and Builtins/2024-02-20-18-49-02.gh-issue-115733.51Zb85.rst
diff --git a/Lib/test/list_tests.py b/Lib/test/list_tests.py
index d9ab21d494..26118e14bb 100644
--- a/Lib/test/list_tests.py
+++ b/Lib/test/list_tests.py
@@ -562,3 +562,8 @@ def test_exhausted_iterator(self):
self.assertEqual(list(exhit), [])
self.assertEqual(list(empit), [9])
self.assertEqual(a, self.type2test([1, 2, 3, 9]))
+
+ # gh-115733: Crash when iterating over exhausted iterator
+ exhit = iter(self.type2test([1, 2, 3]))
+ for _ in exhit:
+ next(exhit, 1)
diff --git a/Misc/NEWS.d/next/Core and Builtins/2024-02-20-18-49-02.gh-issue-115733.51Zb85.rst b/Misc/NEWS.d/next/Core and Builtins/2024-02-20-18-49-02.gh-issue-115733.51Zb85.rst
new file mode 100644
index 0000000000..5cbb292065
--- /dev/null
+++ b/Misc/NEWS.d/next/Core and Builtins/2024-02-20-18-49-02.gh-issue-115733.51Zb85.rst
@@ -0,0 +1 @@
+Fix crash when calling ``next()`` on exhausted list iterators.
diff --git a/Objects/listobject.c b/Objects/listobject.c
index 96182a4230..7917b80824 100644
--- a/Objects/listobject.c
+++ b/Objects/listobject.c
@@ -3537,13 +3537,13 @@ listreviter_next(PyObject *self)
{
listreviterobject *it = (listreviterobject *)self;
assert(it != NULL);
+ Py_ssize_t index = LOAD_SSIZE(it->it_index);
+ if (index < 0) {
+ return NULL;
+ }
+
PyListObject *seq = it->it_seq;
assert(PyList_Check(seq));
-
- Py_ssize_t index = LOAD_SSIZE(it->it_index);
- if (index < 0) {
- return NULL;
- }
PyObject *item = list_get_item_ref(seq, index);
if (item != NULL) {
STORE_SSIZE(it->it_index, index - 1);
diff --git a/Python/bytecodes.c b/Python/bytecodes.c
index 6822e772e9..228d4c2f7a 100644
--- a/Python/bytecodes.c
+++ b/Python/bytecodes.c
@@ -2606,7 +2606,7 @@ dummy_func(
assert(Py_TYPE(iter) == &PyListIter_Type);
STAT_INC(FOR_ITER, hit);
PyListObject *seq = it->it_seq;
- if ((size_t)it->it_index >= (size_t)PyList_GET_SIZE(seq)) {
+ if (seq == NULL || (size_t)it->it_index >= (size_t)PyList_GET_SIZE(seq)) {
it->it_index = -1;
#ifndef Py_GIL_DISABLED
if (seq != NULL) {
@@ -2627,6 +2627,7 @@ dummy_func(
_PyListIterObject *it = (_PyListIterObject *)iter;
assert(Py_TYPE(iter) == &PyListIter_Type);
PyListObject *seq = it->it_seq;
+ DEOPT_IF(seq == NULL);
DEOPT_IF((size_t)it->it_index >= (size_t)PyList_GET_SIZE(seq));
}
diff --git a/Python/executor_cases.c.h b/Python/executor_cases.c.h
index 11e2a1fe85..bb0033514e 100644
--- a/Python/executor_cases.c.h
+++ b/Python/executor_cases.c.h
@@ -2201,6 +2201,7 @@
_PyListIterObject *it = (_PyListIterObject *)iter;
assert(Py_TYPE(iter) == &PyListIter_Type);
PyListObject *seq = it->it_seq;
+ if (seq == NULL) goto deoptimize;
if ((size_t)it->it_index >= (size_t)PyList_GET_SIZE(seq)) goto deoptimize;
break;
}
diff --git a/Python/generated_cases.c.h b/Python/generated_cases.c.h
index 6c19adc60c..094fe0336d 100644
--- a/Python/generated_cases.c.h
+++ b/Python/generated_cases.c.h
@@ -2541,7 +2541,7 @@
assert(Py_TYPE(iter) == &PyListIter_Type);
STAT_INC(FOR_ITER, hit);
PyListObject *seq = it->it_seq;
- if ((size_t)it->it_index >= (size_t)PyList_GET_SIZE(seq)) {
+ if (seq == NULL || (size_t)it->it_index >= (size_t)PyList_GET_SIZE(seq)) {
it->it_index = -1;
#ifndef Py_GIL_DISABLED
if (seq != NULL) {

View file

@ -17,7 +17,7 @@ URL: https://www.python.org/
%global prerel a4
%global upstream_version %{general_version}%{?prerel}
Version: %{general_version}%{?prerel:~%{prerel}}
Release: 1%{?dist}
Release: 2%{?dist}
License: Python-2.0.1
@ -385,6 +385,30 @@ Patch251: 00251-change-user-install-location.patch
# https://github.com/GrahamDumpleton/mod_wsgi/issues/730
Patch371: 00371-revert-bpo-1596321-fix-threading-_shutdown-for-the-main-thread-gh-28549-gh-28589.patch
# 00418 # 00835b95549d7107cb998a663631c22494db8529
# Don't generate sbom in make regen-all
#
# The script and make target, modified in Python 3.13.0a4,
# assumes the presence of expat and _decimal/libmpdec libraries, resulting in an
# error and failed build when not found.
# Reported upstream: https://github.com/python/cpython/issues/115663
Patch418: 00418-don-t-generate-sbom-in-make-regen-all.patch
# 00420 # 9493f930d1e64ecb947d77548ed9993bfb7cd7f0
# gh-114626: Add again _PyCFunctionFastWithKeywords name (GH-115561)
#
# Keep the old private _PyCFunctionFastWithKeywords name (Python 3.7)
# as an alias to the new public name PyCFunctionFastWithKeywords
# (Python 3.13a4).
#
# _PyCFunctionWithKeywords doesn't exist in Python 3.13a3, whereas
# _PyCFunctionFastWithKeywords was removed in Python 3.13a4.
Patch420: 00420-gh-114626-add-again-_pycfunctionfastwithkeywords-name-gh-115561.patch
# 00421 # 11c050c0763b7e67e783c1433e38dcc65070c60c
# gh-115733: Fix crash involving exhausted list iterator (#115740)
Patch421: 00421-gh-115733-fix-crash-involving-exhausted-list-iterator-115740.patch
# (New patches go here ^^^)
#
# When adding new patches to "python" and "python3" in Fedora, EL, etc.,
@ -1668,6 +1692,9 @@ CheckPython freethreading
# ======================================================
%changelog
* Wed Feb 21 2024 Karolina Surma <ksurma@redhat.com> - 3.13.0~a4-2
- Fix crash involving exhausted list iterator
* Fri Feb 16 2024 Karolina Surma <ksurma@redhat.com> - 3.13.0~a4-1
- Update to Python 3.13.0a4