This should finally ensure that the fedora-toolbox image doesn't have
any package that had its content, such as documentation or translations,
stripped out by the fedora base image.
Until now, missing-docs had a hand-maintained list of packages that had
their content stripped out by the fedora base image. These packages are
reinstalled when building the fedora-toolbox image to restore the lost
content. Unfortunately, this list was incomplete because it was only
updated when someone noticed that something is missing.
Now, the list is generated with:
$ rpm --all --query --state --queryformat "PACKAGE: %{NAME}\n"
... to ensure that it's always complete.
The existing built-in test to ensure that the desired files are actually
present in the final image was extended to cover some of those that were
absent. A new built-in test, based on the above rpm(1) command, was
added as a fallback to ensure that the final image doesn't have any
package with missing content.
As suggested by Brian Campbell.
Note that the fedora-toolbox OCI image for Fedora 39 onwards is no
longer built using OpenShift Build Service from the Dockerfile here.
It's now being built using Image Factory from fedora-kickstarts and
pungi-fedora [1], as part of the ToolbxReleaseBlocker Change [2] for
Fedora 39.
Hence this is only for the sake of completeness.
[1] https://pagure.io/fedora-kickstarts/https://pagure.io/pungi-fedora/
[2] https://fedoraproject.org/wiki/Changes/ToolbxReleaseBlockerhttps://github.com/containers/toolbox/issues/603
The shadow-utils package has always been part of the fedora base image.
It's explicitly listed in extra-packages as a safeguard against losing
useradd(8) and usermod(8) by mistake because they are needed by the
entry point of a Toolbx container [1]. Hence, the need to restore the
shadow-utils documentation that was stripped out in the base image.
Note that the fedora-toolbox OCI image for Fedora 39 onwards is no
longer built using OpenShift Build Service from the Dockerfile here.
It's now being built using Image Factory from fedora-kickstarts and
pungi-fedora [2], as part of the ToolbxReleaseBlocker Change [3] for
Fedora 39.
Hence this is only for the sake of completeness.
[1] Toolbx commit c6772f0f112e8004
c6772f0f11
[2] https://pagure.io/fedora-kickstarts/https://pagure.io/pungi-fedora/
[3] https://fedoraproject.org/wiki/Changes/ToolbxReleaseBlockerhttps://github.com/containers/toolbox/pull/1394
It's currently being pulled in as a dependency of iproute. However,
since it's explicitly mentioned in the list of default packages on
Fedora Silverblue and Workstation [1], it should be mentioned here too.
The psmisc package marks the translations for its manuals with %lang().
Therefore, it's a very good example for testing that the fedora-toolbox
image is localized just like Fedora Silverblue and Workstation.
This is unlike the xz package, whose translations for manuals were added
to the tests recently [2]. The xz package doesn't mark its translated
manuals with %lang() [3], which means that they are going to get
installed regardless of whether RPM has been configured to not install
localization files or not. eg., through the %_install_langs macro. So,
they aren't a good candidate for the tests until this is fixed.
Note that the fedora-toolbox OCI image for Fedora 39 onwards is no
longer built using OpenShift Build Service from the Dockerfile here.
It's now being built using Image Factory from fedora-kickstarts and
pungi-fedora [4], as part of the ToolbxReleaseBlocker Change [5] for
Fedora 39.
Hence this is only for the sake of completeness.
[1] fedora-comps commit e4ed54dfcc497fd0
https://pagure.io/fedora-comps/c/e4ed54dfcc497fd0https://pagure.io/fedora-comps/pull-request/379
[2] Toolbx commit 20188a097a1a7a16
20188a097ahttps://github.com/containers/toolbox/pull/1384
[3] https://src.fedoraproject.org/rpms/xz/pull-request/10
[4] https://pagure.io/fedora-kickstarts/https://pagure.io/pungi-fedora/
[5] https://fedoraproject.org/wiki/Changes/ToolbxReleaseBlockerhttps://github.com/containers/toolbox/pull/1390
Until now, only the packages that are present in the fedora base image,
and had their documentation stripped out, were being tested for the
availability of documentation. There were no tests for the extra
packages that get added to the base image to form the fedora-toolbox
image.
The util-linux and xz packages were picked as examples for these new
tests. The xz package is a particularly good example because it has
translations for its manuals. It can help test that the fedora-toolbox
image is localized just like Fedora Silverblue and Workstation.
Note that the fedora-toolbox OCI image for Fedora 39 onwards is no
longer built using OpenShift Build Service from the Dockerfile here.
It's now being built using Image Factory from fedora-kickstarts and
pungi-fedora [1], as part of the ToolbxReleaseBlocker Change [2] for
Fedora 39.
Hence this is only for the sake of completeness.
[1] https://pagure.io/fedora-kickstarts/https://pagure.io/pungi-fedora/
[2] https://fedoraproject.org/wiki/Changes/ToolbxReleaseBlockerhttps://github.com/containers/toolbox/pull/1384
... because the image failed to build with:
Error in plugin orchestrate_build:
{"x86_64": {"docker_api": "ARG requires exactly one argument"},
"aarch64": {"docker_api": "Dockerfile parse error line 4: ARG
requires exactly one argument"}}.
Fallout from 77bb135f27
It turns out that at least since Fedora 30 [1], the gnupg2 package has
been part of the fedora base image, because it's required by the dnf
package:
dnf -> python3-dnf -> python3-libdnf -> libdnf -> gpgme -> gnupg2
Hence, the need to restore the gnupg2 documentation that was stripped
out in the base image.
[1] It's difficult to find out if the gnupg2 package wasn't part of the
fedora base image before Fedora 30, because those images are no
longer available from registry.fedoraproject.org.
https://github.com/containers/toolbox/pull/1228
Building an OCI image leads to so much spew that it's hard to notice if
something unexpected happened, and as seen in the previous commit [1],
unexpected things do happen.
Therefore, this adds a built-in test to ensure that the desired files
are actually present in the final image. Right now it only checks the
presence of some representative manuals to ensure that the packages
listed in the 'missing-docs' file really do get reinstalled, and the
documentation that was stripped out in the base image really does get
restored.
[1] Commit 6d4ecac69fhttps://github.com/containers/toolbox/pull/1226https://github.com/containers/toolbox/pull/1226
The RPM packages in the base 'fedora' image can be older than the those
currently available in the DNF 'updates' repository [1], but at the same
time newer than those available in the DNF 'fedora' repository [1]. The
first part happens because the base image isn't updated as often as the
individual packages, so the 'updates' repository can have newer RPMs.
The second part happens because the base image does get updated after a
stable Fedora has been released, and hence can have newer RPMs than the
'fedora' repository.
This is complicated by the fact that packages can get pulled directly
from Fedora's Koji build system into the base 'fedora' image before
they make it to one of the well-known repositories like 'fedora' or
'updates' [1]. These packages are marked as having come from the
koji-override-0 repository.
All that combined can lead to unexpected behaviour when DNF is invoked
to reinstall or swap the RPM packages in the base image. Some examples
below.
The base fedora:36 image contains glibc-minimal-langpack-2.35-20.fc36
that came from koji-override-0, while 'fedora' and 'updates' have
glibc-all-langpacks-2.35-4.fc36 and glibc-all-langpacks-2.35-22.fc36
respectively. This leads to:
STEP 8/15: RUN dnf -y swap glibc-minimal-langpack glibc-all-langpacks
Last metadata expiration check: 0:00:03 ago on Wed Feb 1 12:37:04...
Dependencies resolved.
======================================================================
Package Arch Version Repository
======================================================================
Installing:
glibc-all-langpacks x86_64 2.35-4.fc36 fedora
Removing:
glibc-minimal-langpack x86_64 2.35-20.fc36 @koji-override-0
Downgrading:
glibc x86_64 2.35-4.fc36 fedora
glibc-common x86_64 2.35-4.fc36 fedora
That's unexpected. Instead of upgrading all the glibc sub-packages to
the latest version from 'updates', it's downgrading them to the older
version from 'fedora'.
Similarly, the base fedora:36 image has bash-5.2.9-2.fc36.x86_64 from
koji-override-0, and there is bash-5.2.15-1.fc36.x86_64 in 'updates'.
This leads to:
STEP 10/15: RUN dnf -y reinstall $(<missing-docs)
Last metadata expiration check: 0:00:06 ago on Wed Feb 1 12:37:04...
Package acl available, but not installed.
No match for argument: acl
Installed package bash-5.2.9-2.fc36.x86_64 (from koji-override-0) not
available.
That's unexpected. Instead of upgrading bash to the latest version from
'updates', it's simply skipping the 'reinstall', which means that the
documentation that was stripped out in the base image doesn't get
restored.
Updating all the RPM packages in the base 'fedora' image to match the
contents of the 'updates' repository before making any changes to the
image's package set will avoid such unexpected behaviour.
[1] https://docs.fedoraproject.org/en-US/quick-docs/repositories/https://github.com/containers/toolbox/pull/1226
Note that this doesn't restore the translations that were stripped out
from the base fedora image. It only ensures that subsequent RPM
transactions retain the translations.
https://github.com/containers/toolbox/issues/60
Currently, the entry point of a Toolbox container runs updatedb(8) on
start-up, which can be very I/O intensive. This might be a hindrance
when troubleshooting performance problems on a host, or when
re-creating containers somewhat more frequently.
Users can install the mlocate RPM and restart their containers to
enable locate(1).
https://github.com/containers/toolbox/pull/938
There's no need to specify a CMD in a Toolbox image because it's
specified by 'toolbox create', through 'podman create', when creating a
container.
A CMD was specified [1] because the Fedora Container Guidelines
requires it [2]. The idea behind the guidelines is that the right
thing should happen when one runs:
$ podman run <image>
However, that only makes sense for images targeting single service
containers. Toolbox containers and images are different - they are not
meant to be used like that to run a single one-off service.
Conceptually, 'running' a Toolbox container is expected to provide the
user with a reasonable interactive command line experience. Arguably,
that means offering something like /bin/bash, not /bin/sh.
Also, note that when the CMD was introduced [1], Toolbox containers
were actually created, through 'podman create', with /bin/sh as their
entry points. So, it did make some sense. However, things have changed
since then [3]. The entry point is now 'toolbox init-container'. It's
not possible to mention it in the Toolbox image because the
/usr/bin/toolbox binary isn't present in the image, and it's not meant
to be present.
Therefore, today, /bin/sh is simply not the right fit for a Toolbox
image's CMD. A better option would be /bin/bash.
Note that the fedora base images have their CMD set to /bin/bash, which
is inherited by the fedora-toolbox images.
So, there are two options. Either repeat the same CMD in the
fedora-toolbox images and satisfy the guidelines, or take some
liberties and let the CMD be inherited from the fedora base images.
This commit takes the latter option. People tend to use the
fedora-toolbox images as the starting point for other custom Toolbox
images, sometimes for other operating system distributions. It's
better to keep them minimal to avoid implying extra requirements. In
this case, the CMD is an abstract concept, and the actual entry point
is 'toolbox init-container' as specified by 'toolbox create'.
Specifying /bin/bash might discourage people from creating custom
images that are only meant to have /bin/zsh.
Also, note that the current CMD was actually '/bin/sh -c /bin/sh', not
/bin/sh. Unless a CMD is specified as an array of command line
arguments, it's passed as a single argument to '/bin/sh -c' [4]. So,
this:
CMD foo bar
... is the same as:
CMD [ "/bin/sh", "-c", "foo bar" ]
[1] Toolbox commit 5cc2678a3677af44
5cc2678a36
[2] https://docs.fedoraproject.org/en-US/containers/guidelines/creation/
[3] Toolbox commit 8b84b5e4604921fa
https://github.com/containers/toolbox/pull/160
[4] https://docs.docker.com/engine/reference/builder/#cmdhttps://github.com/containers/toolbox/issues/885
The util-linux package was added to ensure the presence of the mount(8)
command. Currently the package is already pulled in by various
dependencies. Therefore, it doesn't increase the size of the image, but
serves as a safeguard against any inadvertent changes.
Note that starting from Fedora 35 onwards, the fedora base images no
longer have mount(8), which increases the importance of this change.
https://github.com/containers/toolbox/issues/929
It's true that the fedora base images no longer come with
coreutils-single, but they used to, and the ubi base images still do.
Therefore, it's worth being extra defensive about this.
It's better to make the build system execute one extra redundant
command than expose users to a bug because of a change that snuck in
unnoticed.
This reverts commit a2171d8742.
https://github.com/containers/toolbox/pull/931
Since Fedora 33, `nano` is the default editor[0]. It needs to be
included in the fedora-toolbox image to have the standard Fedora
experience inside the container.
https://fedoraproject.org/wiki/Changes/UseNanoByDefault).
Currently the images are named as "f<version>/fedora-toolbox". This is
troublesome for new users of toolbox (even those with some background to
containers) because everywhere the image is advertised or talked about
as "fedora-toolbox". This is taken care of by Toolbox CLI but has no
effect on Podman itself (or any other tool capable of working with OCI
images).
Another pain point is in the Fedora registry[0] all "fedora-toolbox"
images get a different entry for every version of Fedora. There is no
single place for all "fedora-toolbox" images.
With this change I propose to only use "fedora-toolbox" as the name of
the container and make use of VERSION to distinguish between versions of
Fedora. Currently when you go to the Fedora registry and find an entry
for "fedora-toolbox" you'll see all previous images. I believe that with
this change that "feature" will be lost. But I personally find that
"feature" to be rather confusing because what usually a user wants the
latest version of a container (I partially base this statement on the
fact that most images are versioned this way; e.g. Ubuntu on Docker
Hub[1]).
[0] https://registry.fedoraproject.org/
[1] https://hub.docker.com/_/ubuntu
The nss-mdns plugin for the GNU Name Service Switch (or NSS)
functionality of the GNU C Library is necessary to resolve the .local
mDNS domain. The plugin talks to the Avahi daemon running on the host
to resolve the names.
https://github.com/containers/toolbox/issues/209
If an X11 client is started inside a 'su -' session, then xauth(1)
needs to be present so that pam_xauth.so can add a new XAUTHORITY
environment variable to the 'su -' session.
https://github.com/containers/toolbox/pull/572
The gvfs-client package is necessary for GIO-based processes inside
toolbox containers to use the GVfs backend and volume monitor daemons,
and it comes preinstalled on Fedora Silverblue and Workstation.
https://github.com/containers/toolbox/pull/466
Currently, krb5-libs is pulled in by various other packages that are
already part of the fedora-toolbox OCI image. Unless someone is keeping
a close eye on the contents of the image, a change in the package
dependencies or the contents of the base fedora OCI image can cause
krb5-libs to go missing from the fedora-toolbox image.
This would be undesirable because toolbox(1) relies on the presence
of the /etc/krb5.conf file and the /etc/krb5.conf.d directory to set
up Kerberos inside a toolbox container, and those are provided by the
krb5-libs package.
Therefore, explicitly listing krb5-libs prevents us from accidentally
losing Kerberos integration in toolbox containers, and doesn't cost us
anything because it's already part of the image anyway.
This reverts commit f552b51ec2.
atomic_reactor.util - Package chkconfig available, but not installed.
atomic_reactor.util - No match for argument: chkconfig
atomic_reactor.util - Package dbus-daemon available, but not installed.
atomic_reactor.util - No match for argument: dbus-daemon
atomic_reactor.util - Package rpm-plugin-systemd-inhibit available, but not installed.
atomic_reactor.util - No match for argument: rpm-plugin-systemd-inhibit
:
:
atomic_reactor.util - DEBUG - Running scriptlet: gawk-5.0.1-7.fc32.x86_64 38/38
atomic_reactor.util - DEBUG - Removing intermediate container a533251cf472
atomic_reactor.util - ERROR - {'errorDetail': {'code': 143, 'message': "The command '/bin/sh -c dnf -y reinstall $(<missing-docs)' returned a non-zero code: 143"}, 'error': "The command '/bin/sh -c dnf -y reinstall $(<missing-docs)' returned a non-zero code: 143"}
The shadow-utils package was added to the base toolbox images to ensure
the presence of the useradd(8) command. Currently the package is
already pulled in by various dependencies. Therefore, it doesn't
increase the size of the base image, but serves as a safeguard against
any inadvertent changes.
Currently the toolbox script identifies toolbox images and containers
by checking whether the com.redhat.component label matches
"fedora-toolbox". However, as per the Fedora Container Guidelines [1],
the com.redhat.com label should match the Red Hat Bugzilla component
name where bugs against the image should be reported. This means that
images derived from the base fedora-toolbox image would likely end up
overwriting it.
One option would've been to mandate that all toolbox images have the
"fedora-toolbox-" prefix in their names. However, it's better to avoid
putting limitations on how images can be named. The "fedora" name
wouldn't anyway work for images based on other distributions, and not
all images are going to use the Red Hat bugzilla for tracking bugs.
It's better to use a tag that's uniquely associated with the toolbox
project, and isn't tied to a particular distribution or bug tracker.
[1] https://fedoraproject.org/wiki/Container:Guidelines
For a locally built squashed fedora-toolbox:29 image [1], the size
reported by 'podman images' goes from 630 MB to 425 MB.
[1] Using: buildah bud --squash ...
The krb5-libs package was added to the base toolbox images to ensure
the presence of the /etc/krb5.conf.d directory with the correct
permissions. Currently, the package is already pulled in by various
dependencies. Therefore, it doesn't increase the size of the base
image, but serves as a safeguard against any inadvertent changes.
https://github.com/debarshiray/toolbox/pull/74
