Compare commits

..

No commits in common. "rawhide" and "f26" have entirely different histories.

8 changed files with 212 additions and 341 deletions

2
.gitignore vendored
View file

@ -1 +1 @@
/progress-*.tar.gz
/progress-1.2.tar.gz

View file

@ -1,138 +0,0 @@
Fix fast upload avg calculation
Upstream verigak/progress is broken for very fast uploads; e.g.
with 'copr build' command, the next() call is called so often that
the avg() calculation probably suffers from some small
floating-point numbers problems:
With 15MB/s => next() called for each 8096B => ~2000 calls/s
Since the upstream default window size is only of size 10 items
(by default), it calculates the average speed only for the last
~0.005s. We could enlarge the size of window (sma_window param),
but the algorithm is so naive that it would decrease the
performance (O(N): sum(queue)/len(queue)).
This has been discussed very extensively with upstream (PR 24 and
friends) but I neither was not able to explain the problem, nor I
was able to convince upstream to accept my patches.
This downstream patch - while it keeps the backward API
compatibility - changes the algorithm so the average speed is
calculation is fast enough, and much more stable (by default it
calculates speed for window of 2 seconds).
Fork with this patch backported is maintained in
https://github.com/python-progress/python-progress
diff --git a/progress/__init__.py b/progress/__init__.py
index b434b30..f26056d 100644
--- a/progress/__init__.py
+++ b/progress/__init__.py
@@ -30,19 +30,55 @@ HIDE_CURSOR = '\x1b[?25l'
SHOW_CURSOR = '\x1b[?25h'
+class _Window(object):
+ max_seconds = 2
+ max_items = None
+
+ def __init__(self, max_seconds=2, max_items=None):
+ self.max_seconds = max_seconds
+ self.max_items = max_items
+
+ stamp = monotonic()
+ self.last = stamp - 0.001
+ self.counter = 0
+ self.deque = deque()
+ self.next(0, stamp)
+
+ def pop(self):
+ item = self.deque.popleft()
+ self.counter -= item[1]
+
+ def clean(self):
+ if self.max_items:
+ while len(self.deque) > self.max_items:
+ self.pop()
+ while len(self.deque) > 2 and self.last - self.deque[1][0] > float(self.max_seconds):
+ self.pop()
+
+ def next(self, n, t):
+ self.clean()
+ self.deque.append((self.last, n))
+ self.last = t
+ self.counter += n
+
+ @property
+ def avg(self):
+ return self.counter / (self.last - self.deque[0][0])
+
+
class Infinite(object):
file = stderr
- sma_window = 10 # Simple Moving Average window
+ # Maximum number of next() calls to be held in Simple Moving Average
+ # window structure (in memory), default is unlimited.
+ sma_window_seconds = 2
+ sma_window = None
check_tty = True
hide_cursor = True
def __init__(self, message='', **kwargs):
self.index = 0
self.start_ts = monotonic()
- self.avg = 0
- self._avg_update_ts = self.start_ts
- self._ts = self.start_ts
- self._xput = deque(maxlen=self.sma_window)
+ self.window = _Window(self.sma_window_seconds, self.sma_window)
for key, val in kwargs.items():
setattr(self, key, val)
@@ -69,21 +105,17 @@ class Infinite(object):
def elapsed(self):
return int(monotonic() - self.start_ts)
+ @property
+ def avg(self):
+ speed = self.window.avg
+ if speed:
+ return 1/speed
+ return float("inf")
+
@property
def elapsed_td(self):
return timedelta(seconds=self.elapsed)
- def update_avg(self, n, dt):
- if n > 0:
- xput_len = len(self._xput)
- self._xput.append(dt / n)
- now = monotonic()
- # update when we're still filling _xput, then after every second
- if (xput_len < self.sma_window or
- now - self._avg_update_ts > 1):
- self.avg = sum(self._xput) / len(self._xput)
- self._avg_update_ts = now
-
def update(self):
pass
@@ -112,14 +144,10 @@ class Infinite(object):
try:
return self.file.isatty() if self.check_tty else True
except AttributeError:
- msg = "%s has no attribute 'isatty'. Try setting check_tty=False." % self
- raise AttributeError(msg)
+ raise AttributeError('\'{}\' object has no attribute \'isatty\'. Try setting parameter check_tty=False.'.format(self))
def next(self, n=1):
- now = monotonic()
- dt = now - self._ts
- self.update_avg(n, dt)
- self._ts = now
+ self.window.next(n, monotonic())
self.index = self.index + n
self.update()

View file

@ -0,0 +1,73 @@
From 1911decab746ad8046d00768f6125c7a98ed4fd3 Mon Sep 17 00:00:00 2001
From: Pavel Raiskup <praiskup@redhat.com>
Date: Mon, 18 Jan 2016 18:32:20 +0100
Subject: [PATCH 1/2] (possibly) enlarge dequeue() to give us better statistics
It happened on Fedora 23 x86_64 that if the next() was called with
constant window of n=8196 units (in Bytes, default with
MultipartEncoderMonitor) -- some cache/buffer on the road caused
that the sum of deltas started to be highly inaccurate (bar shown
like 40MB instead of 1MB _most_ of the time). This could be
solved by higher sma_window; but we would have to rather large
numbers there and that would result in high complexity.
So store into deque() also "actual" time() and count all units in
window in separate variable (self._in_window) to avoid sum()
complexity issues. That way we may count:
(actual_time - last_item_time) / items_in_window
This would allow us to raise the value of sma_window.
This resolves verigak/progress/pull/24.
---
progress/__init__.py | 19 ++++++++++++-------
1 file changed, 12 insertions(+), 7 deletions(-)
diff --git a/progress/__init__.py b/progress/__init__.py
index a860e2c..3130fe1 100644
--- a/progress/__init__.py
+++ b/progress/__init__.py
@@ -32,7 +32,8 @@ class Infinite(object):
self.index = 0
self.start_ts = time()
self._ts = self.start_ts
- self._dt = deque(maxlen=self.sma_window)
+ self._dt = deque()
+ self._in_window = 0
for key, val in kwargs.items():
setattr(self, key, val)
@@ -43,7 +44,9 @@ class Infinite(object):
@property
def avg(self):
- return sum(self._dt) / len(self._dt) if self._dt else 0
+ if not self._in_window:
+ return 0
+ return (self._ts - self._dt[0]['t']) / self._in_window
@property
def elapsed(self):
@@ -63,11 +66,13 @@ class Infinite(object):
pass
def next(self, n=1):
- if n > 0:
- now = time()
- dt = (now - self._ts) / n
- self._dt.append(dt)
- self._ts = now
+ self._ts = time()
+ self._dt.append({'t': self._ts, 'n': n})
+ self._in_window = self._in_window + n
+
+ if len(self._dt) > self.sma_window:
+ item = self._dt.popleft()
+ self._in_window = self._in_window - item['n']
self.index = self.index + n
self.update()
--
2.7.4

View file

@ -0,0 +1,87 @@
From 4691a1efdcd8cbe4bc66d586307daf6505f21db4 Mon Sep 17 00:00:00 2001
From: Pavel Raiskup <praiskup@redhat.com>
Date: Mon, 18 Jan 2016 19:05:25 +0100
Subject: [PATCH 2/2] make the progress bar more stable
Add new sma_delta parameter. That parameter represents time delta
(in seconds) for which _one item_ in deque() is valid until we
create new one. Practically, one item represents number of items
processed during 'sma_delta' time (by default 0.3s).
Taking into account sma_delta and sma_window values, progress bar
now returns statistics for the last (sma_delta*sma_window)
seconds, which is by default 3s.
Document a bit.
This resolves verigak/progress/pull/24.
---
progress/__init__.py | 43 +++++++++++++++++++++++++++++++++++++++++--
1 file changed, 41 insertions(+), 2 deletions(-)
diff --git a/progress/__init__.py b/progress/__init__.py
index 3130fe1..37b4547 100644
--- a/progress/__init__.py
+++ b/progress/__init__.py
@@ -24,9 +24,35 @@ from time import time
__version__ = '1.2'
+"""
+The moving average is calculated by this formula:
+
+ _in_window / (time() - oldest_timestamp_in_window)
+
+Because the frequency of calling next() callback might be rather very high,
+doing the naive next()/avg() implementation leads to issues (PR 23):
+
+ def next(N):
+ t = time()
+ delta = N / (t - last_timestamp) # very small number
+ _dt.append(delta)
+ _avg = sum(_dt)/len(_dt)
+ last_timestamp = t
+
+.. Even if 'len(_dt)' was 1000, the time frame can be very small (e.g. file
+download for next(N=8k in bytes) on 1Gbit network. With the mentioned formula
+the window size is limited by *time*, not by the next() callback frequency.
+
+The moving average is calculated -at most- for the last 3 seconds by default:
+
+ sma_window x sma_delta = 10 x 0.3 = 3s
+
+Users can change this default window size, when needed.
+"""
class Infinite(object):
file = stderr
- sma_window = 10
+ sma_window = 10 # Size of the window -- (max) number of sma_delta items.
+ sma_delta = 0.3 # Time-length of one item in window, in seconds.
def __init__(self, *args, **kwargs):
self.index = 0
@@ -67,7 +93,20 @@ class Infinite(object):
def next(self, n=1):
self._ts = time()
- self._dt.append({'t': self._ts, 'n': n})
+
+ item = {'t': self._ts, 'n': 0}
+ if len(self._dt):
+ old_item = self._dt.pop()
+ if self._ts > old_item['t'] + self.sma_delta:
+ # Already reached timeout, we are not going to
+ # touch this item. Return it back.
+ self._dt.append(old_item)
+ else:
+ item = old_item
+
+ item['n'] = item['n'] + n
+
+ self._dt.append(item)
self._in_window = self._in_window + n
if len(self._dt) > self.sma_window:
--
2.7.4

View file

@ -1 +0,0 @@
See also automatic push/pr [Copr builds](https://copr.fedorainfracloud.org/coprs/praiskup/enterprise-ci-python-progress/).

View file

@ -1,39 +0,0 @@
#! /bin/sh
# Prepare main branch, review the list of branches below and then execute this
# script.
branches='main epel8 epel9 epel10 f39 f40 f41'
exit_handler ()
{
git checkout main
}
trap exit_handler EXIT
set -e
set -x
koji hello
tasks=
for branch in $branches; do
if test $branch != main; then
git checkout "$branch"
git merge main
fi
git push
skip_nvr_check=
if test $branch = epel7; then
skip_nvr_check=--skip-nvr-check
fi
tasks="${tasks}`fedpkg build $skip_nvr_check --nowait | grep 'Created task' | cut -d: -f2`"
done
if test -n "$tasks"; then
koji watch-task $tasks
fi
git checkout main
fedpkg update

View file

@ -1,197 +1,86 @@
# Created by pyp2rpm-0.5.2
%global pypi_name progress
%global with_python3 0%{?fedora}
Name: python-%{pypi_name}
Version: 1.6
Release: 19%{?dist}
Version: 1.2
Release: 12%{?dist}
Summary: Easy to use progress bars
License: ISC
URL: http://github.com/verigak/progress/
Source0: %{pypi_source %{pypi_name}}
Source0: https://pypi.python.org/packages/source/p/%{pypi_name}/%{pypi_name}-%{version}.tar.gz
BuildArch: noarch
BuildRequires: python2-devel
BuildRequires: python-setuptools
%if %{?with_python3}
BuildRequires: python3-devel
%if %{defined el8}
BuildRequires: python3-setuptools
%endif
%endif # if with_python3
Patch1: 0001-fixup-moving-average-window.patch
Patch1: 0001-possibly-enlarge-dequeue-to-give-us-better-statistic.patch
Patch2: 0002-make-the-progress-bar-more-stable.patch
%global _description %{expand:
Collection of easy to use progress bars and spinners.}
%description %_description
%description
Collection of easy to use progress bars and spinners.
%if 0%{?with_python3}
%package -n python3-%{pypi_name}
Summary: Easy to use progress bars
%description -n python3-%{pypi_name} %_description
%description -n python3-%{pypi_name}
Collection of easy to use progress bars and spinners.
%endif # with_python3
%prep
%autosetup -p1 -n %{pypi_name}-%{version}
# Remove bundled egg-info
rm -rf %{pypi_name}.egg-info
%if %{undefined el8}
%generate_buildrequires
%pyproject_buildrequires
%endif
%if 0%{?with_python3}
rm -rf %{py3dir}
cp -a . %{py3dir}
find %{py3dir} -name '*.py' | xargs sed -i '1s|^#!python|#!%{__python3}|'
%endif # with_python3
%build
%if %{defined el8}
%py3_build
%else
%pyproject_wheel
%endif
%{__python2} setup.py build
%if 0%{?with_python3}
pushd %{py3dir}
%{__python3} setup.py build
popd
%endif # with_python3
%install
%if %{defined el8}
%py3_install
%else
%pyproject_install
%pyproject_save_files -l %{pypi_name}
%endif
# Must do the subpackages' install first because the scripts in /usr/bin are
# overwritten with every setup.py install (and we want the python2 version
# to be the default for now).
%if 0%{?with_python3}
pushd %{py3dir}
%{__python3} setup.py install --skip-build --root %{buildroot}
popd
%endif # with_python3
%{__python2} setup.py install --skip-build --root %{buildroot}
%check
%if %{defined el8}
%py3_check_import %{pypi_name}
%else
%pyproject_check_import
%endif
%files
%doc README.rst LICENSE
%{python2_sitelib}/%{pypi_name}
%{python2_sitelib}/%{pypi_name}-%{version}-py?.?.egg-info
%files -n python3-%{pypi_name} %{!?el8:-f %{pyproject_files}}
%doc README.rst
%if %{defined el8}
%license LICENSE
%if 0%{?with_python3}
%files -n python3-%{pypi_name}
%doc README.rst LICENSE
%{python3_sitelib}/%{pypi_name}
%{python3_sitelib}/%{pypi_name}-%{version}-py%{python3_version}.egg-info
%endif
%{python3_sitelib}/%{pypi_name}-%{version}-py?.?.egg-info
%endif # with_python3
%changelog
* Fri Sep 19 2025 Python Maint <python-maint@redhat.com> - 1.6-19
- Rebuilt for Python 3.14.0rc3 bytecode
* Fri Aug 15 2025 Python Maint <python-maint@redhat.com> - 1.6-18
- Rebuilt for Python 3.14.0rc2 bytecode
* Fri Jul 25 2025 Fedora Release Engineering <releng@fedoraproject.org> - 1.6-17
- Rebuilt for https://fedoraproject.org/wiki/Fedora_43_Mass_Rebuild
* Mon Jun 02 2025 Python Maint <python-maint@redhat.com> - 1.6-16
- Rebuilt for Python 3.14
* Sat Jan 18 2025 Fedora Release Engineering <releng@fedoraproject.org> - 1.6-15
- Rebuilt for https://fedoraproject.org/wiki/Fedora_42_Mass_Rebuild
* Tue Sep 24 2024 Carl George <carlwgeorge@fedoraproject.org> - 1.6-14
- Convert to pyproject macros
* Fri Jul 19 2024 Fedora Release Engineering <releng@fedoraproject.org> - 1.6-13
- Rebuilt for https://fedoraproject.org/wiki/Fedora_41_Mass_Rebuild
* Fri Jun 07 2024 Python Maint <python-maint@redhat.com> - 1.6-12
- Rebuilt for Python 3.13
* Fri Jan 26 2024 Fedora Release Engineering <releng@fedoraproject.org> - 1.6-11
- Rebuilt for https://fedoraproject.org/wiki/Fedora_40_Mass_Rebuild
* Mon Jan 22 2024 Fedora Release Engineering <releng@fedoraproject.org> - 1.6-10
- Rebuilt for https://fedoraproject.org/wiki/Fedora_40_Mass_Rebuild
* Fri Jul 21 2023 Fedora Release Engineering <releng@fedoraproject.org> - 1.6-9
- Rebuilt for https://fedoraproject.org/wiki/Fedora_39_Mass_Rebuild
* Tue Jun 13 2023 Python Maint <python-maint@redhat.com> - 1.6-8
- Rebuilt for Python 3.12
* Fri Jan 20 2023 Fedora Release Engineering <releng@fedoraproject.org> - 1.6-7
- Rebuilt for https://fedoraproject.org/wiki/Fedora_38_Mass_Rebuild
* Fri Jul 22 2022 Pavel Raiskup <praiskup@redhat.com> - 1.6-6
- use time.monotonic(), bar.avg is infinity for zero speed
* Fri Jul 22 2022 Fedora Release Engineering <releng@fedoraproject.org> - 1.6-5
- Rebuilt for https://fedoraproject.org/wiki/Fedora_37_Mass_Rebuild
* Sat Jul 02 2022 Pavel Raiskup <praiskup@redhat.com> - 1.6-4
- fix for very frequent next() call, rhbz#2103093
* Mon Jun 13 2022 Python Maint <python-maint@redhat.com> - 1.6-3
- Rebuilt for Python 3.11
* Fri Jan 21 2022 Fedora Release Engineering <releng@fedoraproject.org> - 1.6-2
- Rebuilt for https://fedoraproject.org/wiki/Fedora_36_Mass_Rebuild
* Fri Jul 23 2021 Fedora Release Engineering <releng@fedoraproject.org> - 1.5-10
- Rebuilt for https://fedoraproject.org/wiki/Fedora_35_Mass_Rebuild
* Fri Jun 04 2021 Python Maint <python-maint@redhat.com> - 1.5-9
- Rebuilt for Python 3.10
* Wed Jan 27 2021 Fedora Release Engineering <releng@fedoraproject.org> - 1.5-8
- Rebuilt for https://fedoraproject.org/wiki/Fedora_34_Mass_Rebuild
* Wed Jul 29 2020 Fedora Release Engineering <releng@fedoraproject.org> - 1.5-7
- Rebuilt for https://fedoraproject.org/wiki/Fedora_33_Mass_Rebuild
* Tue May 26 2020 Miro Hrončok <mhroncok@redhat.com> - 1.5-6
- Rebuilt for Python 3.9
* Thu Jan 30 2020 Fedora Release Engineering <releng@fedoraproject.org> - 1.5-5
- Rebuilt for https://fedoraproject.org/wiki/Fedora_32_Mass_Rebuild
* Thu Oct 03 2019 Miro Hrončok <mhroncok@redhat.com> - 1.5-4
- Rebuilt for Python 3.8.0rc1 (#1748018)
* Mon Aug 19 2019 Miro Hrončok <mhroncok@redhat.com> - 1.5-3
- Rebuilt for Python 3.8
* Fri Jul 26 2019 Fedora Release Engineering <releng@fedoraproject.org> - 1.5-2
- Rebuilt for https://fedoraproject.org/wiki/Fedora_31_Mass_Rebuild
* Mon Mar 18 2019 Pavel Raiskup <praiskup@redhat.com> - 1.5-1
- the latest upstream release
* Sat Feb 02 2019 Fedora Release Engineering <releng@fedoraproject.org> - 1.4-3
- Rebuilt for https://fedoraproject.org/wiki/Fedora_30_Mass_Rebuild
* Tue Dec 11 2018 Pavel Raiskup <praiskup@redhat.com> - 1.4-2
- followup for previous commit, PR#3
* Wed Dec 05 2018 Gwyn Ciesla <limburgher@gmail.com> - 1.4-1
- 1.4
* Wed Oct 03 2018 Pavel Raiskup <praiskup@redhat.com> - 1.2-19
- no python2 in f30+ (rhbz#1634951)
- fix el6 build (rhbz#1310704)
* Sat Jul 14 2018 Fedora Release Engineering <releng@fedoraproject.org> - 1.2-18
- Rebuilt for https://fedoraproject.org/wiki/Fedora_29_Mass_Rebuild
* Tue Jun 19 2018 Miro Hrončok <mhroncok@redhat.com> - 1.2-17
- Rebuilt for Python 3.7
* Fri Feb 09 2018 Fedora Release Engineering <releng@fedoraproject.org> - 1.2-16
- Rebuilt for https://fedoraproject.org/wiki/Fedora_28_Mass_Rebuild
* Sat Jan 27 2018 Iryna Shcherbina <ishcherb@redhat.com> - 1.2-15
- Update Python 2 dependency declarations to new packaging standards
(See https://fedoraproject.org/wiki/FinalizingFedoraSwitchtoPython3)
* Sat Aug 19 2017 Zbigniew Jędrzejewski-Szmek <zbyszek@in.waw.pl> - 1.2-14
- Python 2 binary package renamed to python2-progress
See https://fedoraproject.org/wiki/FinalizingFedoraSwitchtoPython3
* Thu Jul 27 2017 Fedora Release Engineering <releng@fedoraproject.org> - 1.2-13
- Rebuilt for https://fedoraproject.org/wiki/Fedora_27_Mass_Rebuild
* Sat Feb 11 2017 Fedora Release Engineering <releng@fedoraproject.org> - 1.2-12
- Rebuilt for https://fedoraproject.org/wiki/Fedora_26_Mass_Rebuild

View file

@ -1 +1 @@
SHA512 (progress-1.6.tar.gz) = 58a614bba5a7273a42ba5e9607b3a965fec17f26a4f24563d4c13679ea32b23dda7509c5da81a2a34bebeeedd91154d0457ab8dac95cc7a092add2567249cc94
c1dbf49a41e80408d3874d976ba894cc progress-1.2.tar.gz