Correct generated files

Signed-off-by: Petr "Stone" Hracek <phracek@redhat.com>
This commit is contained in:
phracek 2021-02-16 14:42:16 +00:00
commit 9c6def25bf
16 changed files with 757 additions and 319 deletions

270
README.md
View file

@ -1,270 +1,4 @@
Python 3.7 container image
===================
=========================
This container image includes Python 3.7 as a [S2I](https://github.com/openshift/source-to-image) base image for your Python 3.7 applications.
Users can choose between RHEL and CentOS based builder images.
The RHEL images are available in the [Red Hat Container Catalog](https://access.redhat.com/containers/),
the CentOS images are available on [Docker Hub](https://hub.docker.com/r/centos/),
and the Fedora images are available in [Fedora Registry](https://registry.fedoraproject.org/).
The resulting image can be run using [podman](https://github.com/containers/libpod) or
[docker](http://docker.io).
Note: while the examples in this README are calling `podman`, you can replace any such calls by `docker` with the same arguments
Description
-----------
Python 3.7 available as container is a base platform for
building and running various Python 3.7 applications and frameworks.
Python is an easy to learn, powerful programming language. It has efficient high-level
data structures and a simple but effective approach to object-oriented programming.
Python's elegant syntax and dynamic typing, together with its interpreted nature,
make it an ideal language for scripting and rapid application development in many areas
on most platforms.
This container image includes an npm utility
(see [base image repository](https://github.com/sclorg/s2i-base-container/tree/master/base)),
so users can use it to install JavaScript
modules for their web applications. There is no guarantee for any specific npm or nodejs
version, that is included in the image; those versions can be changed anytime and
the nodejs itself is included just to make the npm work.
Usage
---------------------
For this, we will assume that you are using the supported image, available via `python:3.7` imagestream tag in Openshift.
Building a simple [python-sample-app](https://github.com/sclorg/s2i-python-container/tree/master/3.7/test/setup-test-app) application
in Openshift can be achieved with the following step:
```
oc new-app python:3.7~https://github.com/sclorg/s2i-python-container.git --context-dir=3.7/test/setup-test-app/
```
The same application can also be built using the standalone [S2I](https://github.com/openshift/source-to-image) application on systems that have it available:
```
$ s2i build https://github.com/sclorg/s2i-python-container.git --context-dir=3.7/test/setup-test-app/ <image_name> python-sample-app
```
Where `<image_name>` is the s2i-python image you [downloaded from RHEL, Centos or Fedora registry](../README.md#Download) or [built](../README.md#Build) from these sources. For example ubi8/python-36, centos/python-36-centos7 or f31/python3.
**Accessing the application:**
```
$ curl 127.0.0.1:8080
```
Environment variables
---------------------
To set these environment variables, you can place them as a key value pair into a `.s2i/environment`
file inside your source code repository.
* **APP_SCRIPT**
Used to run the application from a script file.
This should be a path to a script file (defaults to `app.sh` unless set to null) that will be
run to start the application.
* **APP_FILE**
Used to run the application from a Python script.
This should be a path to a Python file (defaults to `app.py` unless set to null) that will be
passed to the Python interpreter to start the application.
* **APP_MODULE**
Used to run the application with Gunicorn, as documented
[here](http://docs.gunicorn.org/en/latest/run.html#gunicorn).
This variable specifies a WSGI callable with the pattern
`MODULE_NAME:VARIABLE_NAME`, where `MODULE_NAME` is the full dotted path
of a module, and `VARIABLE_NAME` refers to a WSGI callable inside the
specified module.
Gunicorn will look for a WSGI callable named `application` if not specified.
If `APP_MODULE` is not provided, the `run` script will look for a `wsgi.py`
file in your project and use it if it exists.
If using `setup.py` for installing the application, the `MODULE_NAME` part
can be read from there. For an example, see
[setup-test-app](https://github.com/sclorg/s2i-python-container/tree/master/3.7/test/setup-test-app).
* **APP_HOME**
This variable can be used to specify a sub-directory in which the application to be run is contained.
The directory pointed to by this variable needs to contain `wsgi.py` (for Gunicorn) or `manage.py` (for Django).
If `APP_HOME` is not provided, the `assemble` and `run` scripts will use the application's root
directory.
* **APP_CONFIG**
Path to a valid Python file with a
[Gunicorn configuration](http://docs.gunicorn.org/en/latest/configure.html#configuration-file) file.
* **DISABLE_MIGRATE**
Set this variable to a non-empty value to inhibit the execution of 'manage.py migrate'
when the produced image is run. This only affects Django projects. See
"Handling Database Migrations" section of [Django blogpost on OpenShift blog](
https://blog.openshift.com/migrating-django-applications-openshift-3/) on suggestions
how/when to run DB migrations in OpenShift environment. Most importantly,
note that running DB migrations from two or more pods might corrupt your database.
* **DISABLE_COLLECTSTATIC**
Set this variable to a non-empty value to inhibit the execution of
'manage.py collectstatic' during the build. This only affects Django projects.
* **DISABLE_SETUP_PY_PROCESSING**
Set this to a non-empty value to skip processing of setup.py script if you
use `-e .` in requirements.txt to trigger its processing or you don't want
your application to be installed into site-packages directory.
* **ENABLE_PIPENV**
Set this variable to use [Pipenv](https://github.com/pypa/pipenv),
the higher-level Python packaging tool, to manage dependencies of the application.
This should be used only if your project contains properly formated Pipfile
and Pipfile.lock.
* **PIN_PIPENV_VERSION**
Set this variable together with `ENABLE_PIPENV` to use a specific version of Pipenv.
If not set, the latest stable version from PyPI is installed.
For example `PIN_PIPENV_VERSION=2018.11.26` installs `pipenv==2018.11.26`.
* **ENABLE_MICROPIPENV**
Set this variable to use [micropipenv](https://github.com/thoth-station/micropipenv),
a lightweight wrapper for pip to support requirements.txt, Pipenv and Poetry lock
files or converting them to pip-tools compatible output. Designed for containerized Python applications.
Available only for Python 3 images.
* **ENABLE_INIT_WRAPPER**
Set this variable to a non-empty value to make use of an init wrapper.
This is useful for servers that are not capable of reaping zombie
processes, such as Django development server or Tornado. This option can
be used together with **APP_SCRIPT** or **APP_FILE**. It never applies
to Gunicorn used through **APP_MODULE** as Gunicorn reaps zombie
processes correctly.
* **PIP_INDEX_URL**
Set this variable to use a custom index URL or mirror to download required packages
during build process. This only affects packages listed in requirements.txt.
Pipenv ignores this variable.
* **UPGRADE_PIP_TO_LATEST**
Set this variable to a non-empty value to have the 'pip' program and related
python packages (setuptools and wheel) be upgraded to the most recent version
before any Python packages are installed. If not set it will use whatever
the default version is included by the platform for the Python version being used.
* **WEB_CONCURRENCY**
Set this to change the default setting for the number of
[workers](http://docs.gunicorn.org/en/stable/settings.html#workers). By
default, this is set to the number of available cores times 2, capped
at 12.
Source repository layout
------------------------
You do not need to change anything in your existing Python project's repository.
However, if these files exist they will affect the behavior of the build process:
* **requirements.txt**
List of dependencies to be installed with `pip`. The format is documented
[here](https://pip.pypa.io/en/latest/user_guide.html#requirements-files).
* **Pipfile**
The replacement for requirements.txt, project is currently under active
design and development, as documented [here](https://github.com/pypa/pipfile).
Set `ENABLE_PIPENV` environment variable to true in order to process this file.
* **setup.py**
Configures various aspects of the project, including installation of
dependencies, as documented
[here](https://packaging.python.org/en/latest/distributing.html#setup-py).
For most projects, it is sufficient to simply use `requirements.txt` or
`Pipfile`. Set `DISABLE_SETUP_PY_PROCESSING` environment variable to true
in order to skip processing of this file.
Run strategies
--------------
The container image produced by s2i-python executes your project in one of the
following ways, in precedence order:
* **Gunicorn**
The Gunicorn WSGI HTTP server is used to serve your application in the case that it
is installed. It can be installed by listing it either in the `requirements.txt`
file or in the `install_requires` section of the `setup.py` file.
If a file named `wsgi.py` is present in your repository, it will be used as
the entry point to your application. This can be overridden with the
environment variable `APP_MODULE`.
This file is present in Django projects by default.
If you have both Django and Gunicorn in your requirements, your Django project
will automatically be served using Gunicorn.
* **Django development server**
If you have Django in your requirements but don't have Gunicorn, then your
application will be served using Django's development web server. However, this is not
recommended for production environments.
* **Python script**
This would be used where you provide a Python code file for running you
application. It will be used in the case where you specify a path to a
Python script via the `APP_FILE` environment variable, defaulting to a
file named `app.py` if it exists. The script is passed to a regular
Python interpreter to launch your application.
* **Application script file**
This is the most general way of executing your application. It will be
used in the case where you specify a path to an executable script file
via the `APP_SCRIPT` environment variable, defaulting to a file named
`app.sh` if it exists. The script is executed directly to launch your
application.
Hot deploy
---------------------
If you are using Django, hot deploy will work out of the box.
To enable hot deploy while using Gunicorn, make sure you have a Gunicorn
configuration file inside your repository with the
[`reload`](https://gunicorn-docs.readthedocs.org/en/latest/settings.html#reload)
option set to `true`. Make sure to specify your config via the `APP_CONFIG`
environment variable.
To change your source code in running container, use podman's (or docker's)
[exec](https://github.com/containers/libpod/blob/master/docs/podman-exec.1.md) command:
```
podman exec -it <CONTAINER_ID> /bin/bash
```
After you enter into the running container, your current directory is set
to `/opt/app-root/src`, where the source code is located.
See also
--------
Dockerfile and other sources are available on https://github.com/sclorg/s2i-python-container.
In that repository you also can find another versions of Python environment Dockerfiles.
Dockerfile for CentOS is called `Dockerfile`, Dockerfile for RHEL7 is called `Dockerfile.rhel7`,
for RHEL8 it's `Dockerfile.rhel8` and the Fedora Dockerfile is called `Dockerfile.fedora`.
**The Python 3.7 image based on Fedora 31 is deprecated.**

View file

@ -27,10 +27,10 @@ function install_tool() {
# have your own PyPI mirror, it will take it from there. If this try fails, try it
# again with --isolated which ignores external pip settings (env vars, config file)
# and installs the tool from PyPI (needs internet connetion).
$VENV_DIR/bin/pip install -U $1$2 # Combines package name with [extras] or version specifier if is defined as $2```
if [ $? -ne 0 ]; then
# $1$2 combines package name with [extras] or version specifier if is defined as $2```
if ! $VENV_DIR/bin/pip install -U $1$2; then
echo "WARNING: Installation of $1 failed, trying again from official PyPI with pip --isolated install"
$VENV_DIR/bin/pip install -U $1$2 # Combines package name with [extras] or version specifier if is defined as $2```
$VENV_DIR/bin/pip install --isolated -U $1$2 # Combines package name with [extras] or version specifier if is defined as $2```
fi
mkdir -p $HOME/.local/bin
ln -s $VENV_DIR/bin/$1 $HOME/.local/bin/$1
@ -58,11 +58,17 @@ fix-permissions /opt/app-root -P
# * pip < 19.3 does not support manylinux2014 wheels. Only manylinux2014 wheels
# support platforms like ppc64le, aarch64 or armv7
echo "---> Upgrading pip to version 19.3.1 ..."
pip install -U "pip==19.3.1"
if ! pip install -U "pip==19.3.1"; then
echo "WARNING: Installation of 'pip==19.3.1' failed, trying again from official PyPI with pip --isolated install"
pip install --isolated -U "pip==19.3.1"
fi
if [[ ! -z "$UPGRADE_PIP_TO_LATEST" ]]; then
echo "---> Upgrading pip to latest version ..."
pip install -U pip setuptools wheel
if ! pip install -U pip setuptools wheel; then
echo "WARNING: Installation of the latest pip,setuptools and wheel failed, trying again from official PyPI with pip --isolated install"
pip install --isolated -U pip setuptools wheel
fi
fi
if [[ ! -z "$ENABLE_PIPENV" ]]; then

View file

@ -6,13 +6,13 @@ NAMESPACE=centos
cat <<EOF
This is a S2I python-3.7 ${DISTRO} base image:
To use it, install S2I: https://github.com/openshift/source-to-image
Sample invocation:
s2i build https://github.com/sclorg/s2i-python-container.git --context-dir=3.7/test/setup-test-app/ ${NAMESPACE}/python-37-${DISTRO}7 python-sample-app
There are multiple ways how to run the image, see documentation at:
https://github.com/sclorg/s2i-python-container/blob/master/3.7/README.md
To use it in Openshift, run:
oc new-app python:3.7~https://github.com/sclorg/s2i-python-container.git --context-dir=3.7/test/setup-test-app/
You can then run the resulting image via:
podman run -p 8080:8080 python-sample-app
oc get pods
oc exec <pod> -- curl 127.0.0.1:8080
EOF

73
test/check_imagestreams.py Executable file
View file

@ -0,0 +1,73 @@
#!/bin/env python3
import sys
import json
import logging
import os
from pathlib import Path
from typing import Dict
IMAGESTREAMS_DIR: str = "imagestreams"
class ImageStreamChecker(object):
version: str = ""
results: Dict = {}
def __init__(self, version: str):
self.version = version
def load_json_file(self, filename: Path):
with open(str(filename)) as f:
return json.load(f)
def check_version(self, json_dict: Dict):
res = []
for tags in json_dict["spec"]["tags"]:
# The name can be"<stream>" or "<stream>-elX" or "<stream>-ubiX"
if (tags["name"] == self.version or
tags["name"].startswith(self.version + '-')):
res.append(tags)
return res
def check_latest_tag(self, json_dict: Dict):
latest_tag_correct: bool = False
for tags in json_dict["spec"]["tags"]:
if tags["name"] != "latest":
continue
# The latest can link to either "<stream>" or "<stream>-elX" or "<stream>-ubiX"
if tags["from"]["name"] == self.version or tags["from"]["name"].startswith(self.version + '-'):
latest_tag_correct = True
return latest_tag_correct
def check_imagestreams(self):
p = Path(".")
json_files = p.glob(f"{IMAGESTREAMS_DIR}/*.json")
if not json_files:
print(f"No json files present in {IMAGESTREAMS_DIR}.")
return 0
for f in json_files:
if os.environ.get("TARGET") in ("rhel7", "centos7") and "aarch64" in str(f):
print("Imagestream aarch64 is not supported on rhel7")
continue
print(f"Checking file {str(f)}.")
json_dict = self.load_json_file(f)
if not (self.check_version(json_dict) and self.check_latest_tag(json_dict)):
print(f"The latest version is not present in {str(f)} or in latest tag.")
self.results[f] = False
if self.results:
return 1
print("Imagestreams contains the latest version.")
return 0
if __name__ == "__main__":
if len(sys.argv) != 2:
logging.fatal("%s: %s", sys.argv[0], "VERSION as an argument was not provided")
sys.exit(1)
print(f"Version to check is {sys.argv[1]}.")
isc = ImageStreamChecker(version=sys.argv[1])
sys.exit(isc.check_imagestreams())

View file

@ -0,0 +1,14 @@
FROM #IMAGE_NAME# # Replaced by sed in tests, see test_from_dockerfile in test/run
# Add application sources to a directory that the assemble script expects them
# and set permissions so that the container runs without root access
USER 0
ADD app-src /tmp/src
RUN /usr/bin/fix-permissions /tmp/src
USER 1001
# Install the dependencies
RUN /usr/libexec/s2i/assemble
# Set the default command for the resulting image
CMD /usr/libexec/s2i/run

View file

@ -0,0 +1,16 @@
FROM #IMAGE_NAME# # Replaced by sed in tests, see test_from_dockerfile in test/run
# Add application sources with correct permissions for OpenShift
USER 0
ADD app-src .
RUN chown -R 1001:0 ./
USER 1001
# Install the dependencies
RUN pip install -U "pip>=19.3.1{% if spec.version == "2.7" %},<21.0{% endif %}" && \
pip install -r requirements.txt && \
python manage.py collectstatic --noinput && \
python manage.py migrate
# Run the application
CMD python manage.py runserver 0.0.0.0:8080

View file

@ -0,0 +1,153 @@
{
"kind": "ImageStream",
"apiVersion": "v1",
"metadata": {
"name": "python",
"annotations": {
"openshift.io/display-name": "Python"
}
},
"spec": {
"tags": [
{
"name": "latest",
"annotations": {
"openshift.io/display-name": "Python (Latest)",
"openshift.io/provider-display-name": "Red Hat, Inc.",
"description": "Build and run Python applications on UBI. For more information about using this builder image, including OpenShift considerations, see https://github.com/sclorg/s2i-python-container/blob/master/3.8/README.md.\n\nWARNING: By selecting this tag, your application will automatically update to use the latest version of Python available on OpenShift, including major version updates.",
"iconClass": "icon-python",
"tags": "builder,python",
"supports":"python",
"sampleRepo": "https://github.com/sclorg/django-ex.git"
},
"from": {
"kind": "ImageStreamTag",
"name": "3.8-ubi8"
},
"referencePolicy": {
"type": "Local"
}
},
{
"name": "3.8-ubi8",
"annotations": {
"openshift.io/display-name": "Python 3.8 (UBI 8)",
"openshift.io/provider-display-name": "Red Hat, Inc.",
"description": "Build and run Python 3.8 applications on UBI 8. For more information about using this builder image, including OpenShift considerations, see https://github.com/sclorg/s2i-python-container/blob/master/3.8/README.md.",
"iconClass": "icon-python",
"tags": "builder,python",
"supports":"python:3.8,python",
"version": "3.8",
"sampleRepo": "https://github.com/sclorg/django-ex.git"
},
"from": {
"kind": "DockerImage",
"name": "registry.access.redhat.com/ubi8/python-38:latest"
},
"referencePolicy": {
"type": "Local"
}
},
{
"name": "3.8-ubi7",
"annotations": {
"openshift.io/display-name": "Python 3.8 (UBI 7)",
"openshift.io/provider-display-name": "Red Hat, Inc.",
"description": "Build and run Python 3.8 applications on UBI 7. For more information about using this builder image, including OpenShift considerations, see https://github.com/sclorg/s2i-python-container/blob/master/3.8/README.md.",
"iconClass": "icon-python",
"tags": "builder,python",
"supports":"python:3.8,python",
"version": "3.8",
"sampleRepo": "https://github.com/sclorg/django-ex.git"
},
"from": {
"kind": "DockerImage",
"name": "registry.access.redhat.com/ubi7/python-38:latest"
},
"referencePolicy": {
"type": "Local"
}
},
{
"name": "3.6-ubi8",
"annotations": {
"openshift.io/display-name": "Python 3.6 (UBI 8)",
"openshift.io/provider-display-name": "Red Hat, Inc.",
"description": "Build and run Python 3.6 applications on UBI 8. For more information about using this builder image, including OpenShift considerations, see https://github.com/sclorg/s2i-python-container/blob/master/3.6/README.md.",
"iconClass": "icon-python",
"tags": "builder,python",
"supports":"python:3.6,python",
"version": "3.6",
"sampleRepo": "https://github.com/sclorg/django-ex.git"
},
"from": {
"kind": "DockerImage",
"name": "registry.access.redhat.com/ubi8/python-36:latest"
},
"referencePolicy": {
"type": "Local"
}
},
{
"name": "2.7-ubi8",
"annotations": {
"openshift.io/display-name": "Python 2.7 (UBI 8)",
"openshift.io/provider-display-name": "Red Hat, Inc.",
"description": "Build and run Python 2.7 applications on UBI 8. For more information about using this builder image, including OpenShift considerations, see https://github.com/sclorg/s2i-python-container/blob/master/2.7/README.md.",
"iconClass": "icon-python",
"tags": "builder,python",
"supports":"python:2.7,python",
"version": "2.7",
"sampleRepo": "https://github.com/sclorg/django-ex.git"
},
"from": {
"kind": "DockerImage",
"name": "registry.access.redhat.com/ubi8/python-27:latest"
},
"referencePolicy": {
"type": "Local"
}
},
{
"name": "2.7-ubi7",
"annotations": {
"openshift.io/display-name": "Python 2.7 (UBI 7)",
"openshift.io/provider-display-name": "Red Hat, Inc.",
"description": "Build and run Python 2.7 applications on UBI 7. For more information about using this builder image, including OpenShift considerations, see https://github.com/sclorg/s2i-python-container/blob/master/2.7/README.md.",
"iconClass": "icon-python",
"tags": "builder,python",
"supports":"python:2.7,python",
"version": "2.7",
"sampleRepo": "https://github.com/sclorg/django-ex.git"
},
"from": {
"kind": "DockerImage",
"name": "registry.access.redhat.com/ubi7/python-27:latest"
},
"referencePolicy": {
"type": "Local"
}
},
{
"name": "2.7",
"annotations": {
"openshift.io/display-name": "Python 2.7",
"openshift.io/provider-display-name": "Red Hat, Inc.",
"description": "Build and run Python 2.7 applications on CentOS 7. For more information about using this builder image, including OpenShift considerations, see https://github.com/sclorg/s2i-python-container/blob/master/2.7/README.md.",
"iconClass": "icon-python",
"tags": "builder,python,hidden",
"supports":"python:2.7,python",
"version": "2.7",
"sampleRepo": "https://github.com/sclorg/django-ex.git"
},
"from": {
"kind": "DockerImage",
"name": "docker.io/centos/python-27-centos7:latest"
},
"referencePolicy": {
"type": "Local"
}
}
]
}
}

View file

@ -0,0 +1,93 @@
{
"kind": "ImageStream",
"apiVersion": "v1",
"metadata": {
"name": "python",
"annotations": {
"openshift.io/display-name": "Python"
}
},
"spec": {
"tags": [
{
"name": "latest",
"annotations": {
"openshift.io/display-name": "Python (Latest)",
"openshift.io/provider-display-name": "Red Hat, Inc.",
"description": "Build and run Python applications on UBI. For more information about using this builder image, including OpenShift considerations, see https://github.com/sclorg/s2i-python-container/blob/master/3.8/README.md.\n\nWARNING: By selecting this tag, your application will automatically update to use the latest version of Python available on OpenShift, including major version updates.",
"iconClass": "icon-python",
"tags": "builder,python",
"supports":"python",
"sampleRepo": "https://github.com/sclorg/django-ex.git"
},
"from": {
"kind": "ImageStreamTag",
"name": "3.8-ubi8"
},
"referencePolicy": {
"type": "Local"
}
},
{
"name": "3.8-ubi8",
"annotations": {
"openshift.io/display-name": "Python 3.8 (UBI 8)",
"openshift.io/provider-display-name": "Red Hat, Inc.",
"description": "Build and run Python 3.8 applications on UBI 8. For more information about using this builder image, including OpenShift considerations, see https://github.com/sclorg/s2i-python-container/blob/master/3.8/README.md.",
"iconClass": "icon-python",
"tags": "builder,python",
"supports":"python:3.8,python",
"version": "3.8",
"sampleRepo": "https://github.com/sclorg/django-ex.git"
},
"from": {
"kind": "DockerImage",
"name": "registry.redhat.io/ubi8/python-38:latest"
},
"referencePolicy": {
"type": "Local"
}
},
{
"name": "3.6-ubi8",
"annotations": {
"openshift.io/display-name": "Python 3.6 (UBI 8)",
"openshift.io/provider-display-name": "Red Hat, Inc.",
"description": "Build and run Python 3.6 applications on UBI 8. For more information about using this builder image, including OpenShift considerations, see https://github.com/sclorg/s2i-python-container/blob/master/3.6/README.md.",
"iconClass": "icon-python",
"tags": "builder,python",
"supports":"python:3.6,python",
"version": "3.6",
"sampleRepo": "https://github.com/sclorg/django-ex.git"
},
"from": {
"kind": "DockerImage",
"name": "registry.redhat.io/ubi8/python-36:latest"
},
"referencePolicy": {
"type": "Local"
}
},
{
"name": "2.7-ubi8",
"annotations": {
"openshift.io/display-name": "Python 2.7 (UBI 8)",
"openshift.io/provider-display-name": "Red Hat, Inc.",
"description": "Build and run Python 2.7 applications on UBI 8. For more information about using this builder image, including OpenShift considerations, see https://github.com/sclorg/s2i-python-container/blob/master/2.7/README.md.",
"iconClass": "icon-python",
"tags": "builder,python",
"supports":"python:2.7,python",
"version": "2.7",
"sampleRepo": "https://github.com/sclorg/django-ex.git"
},
"from": {
"kind": "DockerImage",
"name": "registry.redhat.io/ubi8/python-27:latest"
},
"referencePolicy": {
"type": "Local"
}
}
]
}
}

View file

@ -0,0 +1,173 @@
{
"kind": "ImageStream",
"apiVersion": "v1",
"metadata": {
"name": "python",
"annotations": {
"openshift.io/display-name": "Python"
}
},
"spec": {
"tags": [
{
"name": "latest",
"annotations": {
"openshift.io/display-name": "Python (Latest)",
"openshift.io/provider-display-name": "Red Hat, Inc.",
"description": "Build and run Python applications on UBI. For more information about using this builder image, including OpenShift considerations, see https://github.com/sclorg/s2i-python-container/blob/master/3.8/README.md.\n\nWARNING: By selecting this tag, your application will automatically update to use the latest version of Python available on OpenShift, including major version updates.",
"iconClass": "icon-python",
"tags": "builder,python",
"supports":"python",
"sampleRepo": "https://github.com/sclorg/django-ex.git"
},
"from": {
"kind": "ImageStreamTag",
"name": "3.8-ubi8"
},
"referencePolicy": {
"type": "Local"
}
},
{
"name": "3.8-ubi8",
"annotations": {
"openshift.io/display-name": "Python 3.8 (UBI 8)",
"openshift.io/provider-display-name": "Red Hat, Inc.",
"description": "Build and run Python 3.8 applications on UBI 8. For more information about using this builder image, including OpenShift considerations, see https://github.com/sclorg/s2i-python-container/blob/master/3.8/README.md.",
"iconClass": "icon-python",
"tags": "builder,python",
"supports":"python:3.8,python",
"version": "3.8",
"sampleRepo": "https://github.com/sclorg/django-ex.git"
},
"from": {
"kind": "DockerImage",
"name": "registry.redhat.io/ubi8/python-38:latest"
},
"referencePolicy": {
"type": "Local"
}
},
{
"name": "3.8-ubi7",
"annotations": {
"openshift.io/display-name": "Python 3.8 (UBI 7)",
"openshift.io/provider-display-name": "Red Hat, Inc.",
"description": "Build and run Python 3.8 applications on UBI 7. For more information about using this builder image, including OpenShift considerations, see https://github.com/sclorg/s2i-python-container/blob/master/3.8/README.md.",
"iconClass": "icon-python",
"tags": "builder,python",
"supports":"python:3.8,python",
"version": "3.8",
"sampleRepo": "https://github.com/sclorg/django-ex.git"
},
"from": {
"kind": "DockerImage",
"name": "registry.redhat.io/ubi7/python-38:latest"
},
"referencePolicy": {
"type": "Local"
}
},
{
"name": "3.8",
"annotations": {
"openshift.io/display-name": "Python 3.8",
"openshift.io/provider-display-name": "Red Hat, Inc.",
"description": "Build and run Python 3.8 applications on RHEL 7. For more information about using this builder image, including OpenShift considerations, see https://github.com/sclorg/s2i-python-container/blob/master/3.8/README.md.",
"iconClass": "icon-python",
"tags": "builder,python,hidden",
"supports":"python:3.8,python",
"version": "3.8",
"sampleRepo": "https://github.com/sclorg/django-ex.git"
},
"from": {
"kind": "DockerImage",
"name": "registry.redhat.io/rhscl/python-38-rhel7:latest"
},
"referencePolicy": {
"type": "Local"
}
},
{
"name": "3.6-ubi8",
"annotations": {
"openshift.io/display-name": "Python 3.6 (UBI 8)",
"openshift.io/provider-display-name": "Red Hat, Inc.",
"description": "Build and run Python 3.6 applications on UBI 8. For more information about using this builder image, including OpenShift considerations, see https://github.com/sclorg/s2i-python-container/blob/master/3.6/README.md.",
"iconClass": "icon-python",
"tags": "builder,python",
"supports":"python:3.6,python",
"version": "3.6",
"sampleRepo": "https://github.com/sclorg/django-ex.git"
},
"from": {
"kind": "DockerImage",
"name": "registry.redhat.io/ubi8/python-36:latest"
},
"referencePolicy": {
"type": "Local"
}
},
{
"name": "2.7-ubi8",
"annotations": {
"openshift.io/display-name": "Python 2.7 (UBI 8)",
"openshift.io/provider-display-name": "Red Hat, Inc.",
"description": "Build and run Python 2.7 applications on UBI 8. For more information about using this builder image, including OpenShift considerations, see https://github.com/sclorg/s2i-python-container/blob/master/2.7/README.md.",
"iconClass": "icon-python",
"tags": "builder,python",
"supports":"python:2.7,python",
"version": "2.7",
"sampleRepo": "https://github.com/sclorg/django-ex.git"
},
"from": {
"kind": "DockerImage",
"name": "registry.redhat.io/ubi8/python-27:latest"
},
"referencePolicy": {
"type": "Local"
}
},
{
"name": "2.7-ubi7",
"annotations": {
"openshift.io/display-name": "Python 2.7 (UBI 7)",
"openshift.io/provider-display-name": "Red Hat, Inc.",
"description": "Build and run Python 2.7 applications on UBI 7. For more information about using this builder image, including OpenShift considerations, see https://github.com/sclorg/s2i-python-container/blob/master/2.7/README.md.",
"iconClass": "icon-python",
"tags": "builder,python",
"supports":"python:2.7,python",
"version": "2.7",
"sampleRepo": "https://github.com/sclorg/django-ex.git"
},
"from": {
"kind": "DockerImage",
"name": "registry.redhat.io/ubi7/python-27:latest"
},
"referencePolicy": {
"type": "Local"
}
},
{
"name": "2.7",
"annotations": {
"openshift.io/display-name": "Python 2.7",
"openshift.io/provider-display-name": "Red Hat, Inc.",
"description": "Build and run Python 2.7 applications on RHEL 7. For more information about using this builder image, including OpenShift considerations, see https://github.com/sclorg/s2i-python-container/blob/master/2.7/README.md.",
"iconClass": "icon-python",
"tags": "builder,python,hidden",
"supports":"python:2.7,python",
"version": "2.7",
"sampleRepo": "https://github.com/sclorg/django-ex.git"
},
"from": {
"kind": "DockerImage",
"name": "registry.redhat.io/rhscl/python-27-rhel7:latest"
},
"referencePolicy": {
"type": "Local"
}
}
]
}
}

View file

@ -1,2 +1,5 @@
ENABLE_MICROPIPENV=true
DISABLE_SETUP_PY_PROCESSING=true
# This tests second try to install micropipenv with --isolated
# because the first one won't work with following setting
PIP_INDEX_URL=https://example.com/

View file

@ -13,7 +13,7 @@ packages=("pip" "setuptools" "wheel")
for pkg in ${packages[@]}; do
# grep returns exit code 1 if the output contains only one line starting
# with "Requirement already …" which means that the package is updated
python -m pip install -U --no-python-version-warning $pkg 2>&1 | grep -v "^Requirement already up-to-date: "
python -m pip install -U --no-deps --no-python-version-warning $pkg 2>&1 | grep -Ev "^Requirement already (up-to-date|satisfied): "
if [ $? -eq 0 ]; then
echo "ERROR: Failed to upgrade '$pkg' to the latest version."
exit 1

View file

@ -187,6 +187,12 @@ test_application() {
cleanup_app
}
test_from_dockerfile(){
info "Test from Dockerfile"
sed "s@#IMAGE_NAME#@${IMAGE_NAME}@" $test_dir/from-dockerfile/Dockerfile.tpl > $test_dir/from-dockerfile/Dockerfile
ct_test_app_dockerfile $test_dir/from-dockerfile/Dockerfile 'https://github.com/sclorg/django-ex.git' 'Welcome to your Django application on OpenShift' app-src
check_result $?
}
# Since we built the candidate image locally, we don't want S2I attempt to pull
# it from Docker hub
@ -228,4 +234,20 @@ for app in ${@:-${WEB_APPS[@]}}; do
cleanup ${app}
done
if [ "$OS" == "rhel7" ] || [ "$OS" == "centos7" ]; then
# autocleanup only enabled here as only the following tests so far use it
CID_FILE_DIR=$(mktemp -d)
ct_enable_cleanup
info "Testing variable presence during \`docker exec\`"
ct_check_exec_env_vars
check_result $?
info "Checking if all scl variables are defined in Dockerfile"
ct_check_scl_enable_vars
check_result $?
fi
test_from_dockerfile
info "All tests finished successfully."

View file

@ -7,6 +7,7 @@
# The image has to be available before this script is executed.
THISDIR=$(dirname ${BASH_SOURCE[0]})
test_dir="$(readlink -f $(dirname ${BASH_SOURCE[0]}))"
source "${THISDIR}/test-lib.sh"
source "${THISDIR}/test-lib-openshift.sh"
@ -16,6 +17,14 @@ set -eo nounset
trap ct_os_cleanup EXIT SIGINT
test_latest_imagestreams() {
info "Testing the latest version in imagestreams"
# Switch to root directory of a container
pushd "${test_dir}/../.." >/dev/null
ct_check_latest_imagestreams
popd >/dev/null
}
ct_os_check_compulsory_vars
ct_os_enable_print_logs
@ -44,6 +53,9 @@ done
# Check the imagestream
test_python_imagestream
# check if latest imagestream version is correct
test_latest_imagestreams
OS_TESTSUITE_RESULT=0
ct_os_cluster_down

View file

@ -145,7 +145,7 @@ function ct_os_get_pod_status() {
# Arguments: pod_prefix - prefix or whole ID of the pod
function ct_os_get_build_pod_status() {
local pod_prefix="${1}" ; shift
local query="custom-columns=NAME:.metadata.name,Ready:status.containerStatuses[0].state.terminated.reason"
local query="custom-columns=NAME:.metadata.name,Ready:status.phase"
oc get pods -o "$query" | grep -e "${pod_prefix}" | grep -E "\-build\s" \
| sort -u | awk '{print $2}' | tail -n 1
}
@ -193,7 +193,7 @@ function ct_os_wait_pod_ready() {
if ct_os_get_all_pods_name | grep -E "${pod_prefix}.*-build"; then
SECONDS=0
echo -n "Waiting for ${pod_prefix} build pod to finish ..."
while ! [ "$(ct_os_get_build_pod_status "${pod_prefix}")" == "Completed" ] ; do
while ! [ "$(ct_os_get_build_pod_status "${pod_prefix}")" == "Succeeded" ] ; do
echo -n "."
[ "${SECONDS}" -gt "${timeout}0" ] && echo " FAIL" && return 1
sleep 3
@ -219,7 +219,7 @@ function ct_os_wait_rc_ready() {
local pod_prefix="${1}" ; shift
local timeout="${1}" ; shift
SECONDS=0
echo -n "Waiting for ${pod_prefix} pod becoming ready ..."
echo -n "Waiting for ${pod_prefix} having desired numbers of replicas ..."
while ! test "$( (oc get --no-headers statefulsets; oc get --no-headers rc) 2>/dev/null \
| grep "^${pod_prefix}" | awk '$2==$3 {print "ready"}')" == "ready" ; do
echo -n "."
@ -413,6 +413,7 @@ function ct_os_install_in_centos() {
bash-completion origin-clients docker origin-clients
}
# ct_os_cluster_up [DIR, IS_PUBLIC, CLUSTER_VERSION]
# --------------------
# Runs the local OpenShift cluster using 'oc cluster up' and logs in as developer.
@ -604,8 +605,7 @@ function ct_os_test_s2i_app_func() {
local check_command=${4}
local oc_args=${5:-}
local image_name_no_namespace=${image_name##*/}
local service_name="${image_name_no_namespace}-testing"
local image_tagged="${image_name_no_namespace}:${VERSION}"
local service_name="${image_name_no_namespace%%:*}-testing"
local namespace
if [ $# -lt 4 ] || [ -z "${1}" ] || [ -z "${2}" ] || [ -z "${3}" ] || [ -z "${4}" ]; then
@ -618,16 +618,20 @@ function ct_os_test_s2i_app_func() {
namespace=${CT_NAMESPACE:-"$(oc project -q)"}
# Create a specific imagestream tag for the image so that oc cannot use anything else
if [ "${CT_SKIP_UPLOAD_IMAGE:-false}" == 'true' ] ; then
echo "Importing image ${image_name} as ${namespace}/${image_tagged}"
# Use --reference-policy=local to pull remote image content to the cluster
# Works around the issue of builder pods not having access to registry.redhat.io
oc tag --source=docker "${image_name}" "${namespace}/${image_tagged}" --insecure=true --reference-policy=local
ct_os_wait_stream_ready "${image_tagged}" "${namespace}"
if [ "${CT_EXTERNAL_REGISTRY:-false}" == 'true' ] ; then
ct_os_import_image_ocp4 "${image_name}" "${image_name_no_namespace}"
else
echo "Uploading image ${image_name} as ${image_tagged}"
ct_os_upload_image "${image_name}" "${image_tagged}"
# Create a specific imagestream tag for the image so that oc cannot use anything else
if [ "${CT_SKIP_UPLOAD_IMAGE:-false}" == 'true' ] ; then
echo "Importing image ${image_name} as ${namespace}/${image_name_no_namespace}"
# Use --reference-policy=local to pull remote image content to the cluster
# Works around the issue of builder pods not having access to registry.redhat.io
oc tag --source=docker "${image_name}" "${namespace}/${image_name_no_namespace}" --insecure=true --reference-policy=local
ct_os_wait_stream_ready "${image_name_no_namespace}" "${namespace}"
else
echo "Uploading image ${image_name} as ${image_name_no_namespace}"
ct_os_upload_image "${image_name}" "${image_name_no_namespace}"
fi
fi
local app_param="${app}"
@ -638,7 +642,7 @@ function ct_os_test_s2i_app_func() {
fi
# shellcheck disable=SC2086
ct_os_deploy_s2i_image "${image_tagged}" "${app_param}" \
ct_os_deploy_s2i_image "${image_name_no_namespace}" "${app_param}" \
--context-dir="${context_dir}" \
--name "${service_name}" \
${oc_args}
@ -752,27 +756,35 @@ function ct_os_test_template_app_func() {
ct_os_new_project
namespace=${CT_NAMESPACE:-"$(oc project -q)"}
# Create a specific imagestream tag for the image so that oc cannot use anything else
if [ "${CT_SKIP_UPLOAD_IMAGE:-false}" == 'true' ] ; then
echo "Importing image ${image_name} as ${image_tagged}"
# Use --reference-policy=local to pull remote image content to the cluster
# Works around the issue of builder pods not having access to registry.redhat.io
oc tag --source=docker "${image_name}" "${namespace}/${image_tagged}" --insecure=true --reference-policy=local
ct_os_wait_stream_ready "${image_tagged}" "${namespace}"
if [ "${CT_EXTERNAL_REGISTRY:-false}" == 'true' ] ; then
ct_os_import_image_ocp4 "${image_name}" "${image_tagged}"
else
echo "Uploading image ${image_name} as ${image_tagged}"
ct_os_upload_image "${image_name}" "${image_tagged}"
# upload also other images, that template might need (list of pairs in the format <image>|<tag>
local image_tag_a
local i_t
for i_t in ${other_images} ; do
echo "${i_t}"
IFS='|' read -ra image_tag_a <<< "${i_t}"
docker pull "${image_tag_a[0]}"
ct_os_upload_image "${image_tag_a[0]}" "${image_tag_a[1]}"
done
if [ "${CT_SKIP_UPLOAD_IMAGE:-false}" == 'true' ] ; then
echo "Importing image ${image_name} as ${image_tagged}"
# Use --reference-policy=local to pull remote image content to the cluster
# Works around the issue of builder pods not having access to registry.redhat.io
oc tag --source=docker "${image_name}" "${namespace}/${image_tagged}" --insecure=true --reference-policy=local
ct_os_wait_stream_ready "${image_tagged}" "${namespace}"
else
echo "Uploading image ${image_name} as ${image_tagged}"
ct_os_upload_image "${image_name}" "${image_tagged}"
fi
fi
if [ "${CT_SKIP_UPLOAD_IMAGE:-false}" == 'false' ] ; then
# upload also other images, that template might need (list of pairs in the format <image>|<tag>
local image_tag_a
local i_t
for i_t in ${other_images} ; do
echo "${i_t}"
IFS='|' read -ra image_tag_a <<< "${i_t}"
docker pull "${image_tag_a[0]}"
if [ "${CT_EXTERNAL_REGISTRY:-false}" == 'true' ] ; then
ct_os_import_image_ocp4 "${image_tag_a[0]}" "${image_tag_a[1]}"
else
ct_os_upload_image "${image_tag_a[0]}" "${image_tag_a[1]}"
fi
done
fi
# get the template file from remote or local location; if not found, it is

View file

@ -18,7 +18,7 @@ function test_python_imagestream() {
*) echo "Imagestream testing not supported for $OS environment." ; return 0 ;;
esac
ct_os_test_image_stream_quickstart "${THISDIR}/imagestreams/python-${OS}.json" \
ct_os_test_image_stream_quickstart "${THISDIR}/imagestreams/python-${OS%[0-9]*}.json" \
'https://raw.githubusercontent.com/sclorg/django-ex/master/openshift/templates/django-postgresql.json' \
"${IMAGE_NAME}" \
'python' \

View file

@ -25,7 +25,9 @@ EXPECTED_EXIT_CODE=0
# Uses: $CID_FILE_DIR - path to directory containing cid_files
# Uses: $EXPECTED_EXIT_CODE - expected container exit code
function ct_cleanup() {
ct_show_resources
for cid_file in "$CID_FILE_DIR"/* ; do
[ -f "$cid_file" ] || continue
local container
container=$(cat "$cid_file")
@ -68,6 +70,7 @@ function ct_check_envs_set {
loop_envs=$1; shift
env_format=${1:-"*VALUE*"}
while read -r variable; do
[ -z "$variable" ] && continue
var_name=$(echo "$variable" | awk -F= '{ print $1 }')
stripped=$(echo "$variable" | awk -F= '{ print $2 }')
filtered_envs=$(echo "$check_envs" | grep "^$var_name=")
@ -81,11 +84,12 @@ function ct_check_envs_set {
if [ -n "${filtered_envs##${env_format//VALUE/$value}}" ]; then
echo " Value $value is missing from variable $var_name"
echo "$filtered_envs"
IFS=$old_IFS
return 1
fi
done
IFS=$old_IFS
done <<< "$(echo "$loop_envs" | grep "$env_filter" | grep -v "^PWD=")"
IFS=$old_IFS
}
# ct_get_cid [name]
@ -339,7 +343,7 @@ function ct_binary_found_from_df() {
# Create Dockerfile that looks for the binary
cat <<EOF >"$tmpdir/Dockerfile"
FROM $IMAGE_NAME
RUN which $binary | grep "$binary_path"
RUN command -v $binary | grep "$binary_path"
EOF
# Build an image, looking for expected path in the output
if ! docker build -f "$tmpdir/Dockerfile" --no-cache "$tmpdir"; then
@ -557,7 +561,7 @@ ct_registry_from_os() {
registry=registry.redhat.io
;;
*)
registry=docker.io
registry=quay.io
;;
esac
echo "$registry"
@ -583,7 +587,9 @@ ct_get_public_image_name() {
elif [ "x$os" == "xrhel8" ]; then
public_image_name=$registry/rhel8/$base_image_name-${version//./}
elif [ "x$os" == "xcentos7" ]; then
public_image_name=$registry/centos/$base_image_name-${version//./}-centos7
public_image_name=$registry/centos7/$base_image_name-${version//./}-centos7
elif [ "x$os" == "xcentos8" ]; then
public_image_name=$registry/centos8/$base_image_name-${version//./}-centos8
fi
echo "$public_image_name"
@ -773,4 +779,125 @@ ct_check_image_availability() {
fi
}
# ct_check_latest_imagestreams
# -----------------------------
# Check if the latest version present in Makefile in the variable VERSIONS
# is present in all imagestreams.
# Also the latest tag in the imagestreams has to contain the latest version
ct_check_latest_imagestreams() {
local latest_version=
local test_lib_dir=
# We only maintain imagestreams for RHEL and CentOS (Community)
if [[ "$OS" =~ ^fedora.* ]] ; then
echo "Imagestreams for Fedora are not maintained, skipping ct_check_latest_imagestreams"
return 0
fi
# Check only lines which starts with VERSIONS
latest_version=$(grep '^VERSIONS' Makefile | rev | cut -d ' ' -f 1 | rev )
# Fall back to previous version if the latest is excluded for this OS
[ -f "$latest_version/.exclude-$OS" ] && latest_version=$(grep '^VERSIONS' Makefile | rev | cut -d ' ' -f 2 | rev )
# Only test the imagestream once, when the version matches
# ignore the SC warning, $VERSION is always available
# shellcheck disable=SC2153
if [ "$latest_version" == "$VERSION" ]; then
test_lib_dir=$(dirname "$(readlink -f "$0")")
python3 "${test_lib_dir}/check_imagestreams.py" "$latest_version"
else
echo "Image version $VERSION is not latest, skipping ct_check_latest_imagestreams"
fi
}
# ct_show_resources
# ----------------
# Prints the available resources
ct_show_resources()
{
echo "Resources info:"
echo "Memory:"
free -h
echo "Storage:"
df -h
echo "CPU"
lscpu
}
# ct_test_app_dockerfile
# -----------------------------
# Argument: dockerfile - path to a Dockerfile that will be used for building an image
# (must work with an application directory called 'app-src')
# Argument: app_url - git URI with a testing application
# Argument: body_regexp - PCRE regular expression that must match the response body
# Argument: app_dir - name of the application directory that is used in the Dockerfile
# Argument: port - Optional port number (default: 8080)
ct_test_app_dockerfile() {
local dockerfile=$1
local app_url=$2
local expected_text=$3
local app_dir=$4 # this is a directory that must match with the name in the Dockerfile
local port=${5:-8080}
local app_image_name=myapp
local ret
local cname=app_dockerfile
if [ -z "$app_dir" ] ; then
echo "ERROR: Option app_dir not set. Terminating the Dockerfile build."
return 1
fi
if ! [ -r "${dockerfile}" ] || ! [ -s "${dockerfile}" ] ; then
echo "ERROR: Dockerfile ${dockerfile} does not exist or is empty."
echo "Terminating the Dockerfile build."
return 1
fi
CID_FILE_DIR=${CID_FILE_DIR:-$(mktemp -d)}
local dockerfile_abs
dockerfile_abs=$(readlink -f "${dockerfile}")
tmpdir=$(mktemp -d)
pushd "$tmpdir" >/dev/null
cp "${dockerfile_abs}" Dockerfile
# Rewrite the source image to what we test
sed -i -e "s|^FROM.*$|FROM $IMAGE_NAME|" Dockerfile
# a bit more verbose, but should help debugging failures
echo "Using this Dockerfile:"
cat Dockerfile
if ! git clone "${app_url}" "${app_dir}" ; then
echo "ERROR: Git repository ${app_url} cannot be cloned into ${app_dir}."
echo "Terminating the Dockerfile build."
return 1
fi
echo "Building '${app_image_name}' image using docker build"
if ! docker build --no-cache=true -t "${app_image_name}" . ; then
echo "ERROR: The image cannot be built from ${dockerfile} and application ${app_url}."
echo "Terminating the Dockerfile build."
return 1
fi
if ! docker run -d --cidfile="${CID_FILE_DIR}/app_dockerfile" --rm "${app_image_name}" ; then
echo "ERROR: The image ${app_image_name} cannot be run for ${dockerfile} and application ${app_url}."
echo "Terminating the Dockerfile build."
return 1
fi
echo "Waiting for ${app_image_name} to start"
ct_wait_for_cid "${CID_FILE_DIR}/app_dockerfile"
ip="$(ct_get_cip "${cname}")"
ct_test_response "http://$ip:${port}" 200 "${expected_text}"
ret=$?
# cleanup
docker kill "$(ct_get_cid "${cname}")"
sleep 2
docker rmi "${app_image_name}"
popd >/dev/null
rm -rf "${tmpdir}"
rm -f "${CID_FILE_DIR}/${cname}"
return $ret
}
# vim: set tabstop=2:shiftwidth=2:expandtab: