Pull changes from upstream repository
This commit is contained in:
parent
228023ff21
commit
f409f0a74f
46 changed files with 5661 additions and 0 deletions
363
root/help.1
Normal file
363
root/help.1
Normal file
|
|
@ -0,0 +1,363 @@
|
|||
.TH PostgreSQL 10 SQL Database Server container image
|
||||
.PP
|
||||
This container image includes PostgreSQL 10 SQL database server for OpenShift and general usage.
|
||||
Users can choose between RHEL and CentOS based images.
|
||||
The RHEL image is available in the Red Hat Container Catalog
|
||||
\[la]https://access.redhat.com/containers/#/registry.access.redhat.com/rhscl/postgresql-10-rhel7\[ra]
|
||||
as registry.access.redhat.com/rhscl/postgresql\-10\-rhel7.
|
||||
The CentOS image is then available on Docker Hub
|
||||
\[la]https://hub.docker.com/r/centos/postgresql-10-centos7/\[ra]
|
||||
as centos/postgresql\-10\-centos7.
|
||||
|
||||
.SH Description
|
||||
.PP
|
||||
This container image provides a containerized packaging of the PostgreSQL postgres daemon
|
||||
and client application. The postgres server daemon accepts connections from clients
|
||||
and provides access to content from PostgreSQL databases on behalf of the clients.
|
||||
You can find more information on the PostgreSQL project from the project Web site
|
||||
(
|
||||
\[la]https://www.postgresql.org/\[ra]).
|
||||
|
||||
.SH Usage
|
||||
.PP
|
||||
For this, we will assume that you are using the \fB\fCrhscl/postgresql\-10\-rhel7\fR image.
|
||||
If you want to set only the mandatory environment variables and not store the database
|
||||
in a host directory, execute the following command:
|
||||
|
||||
.PP
|
||||
.RS
|
||||
|
||||
.nf
|
||||
$ docker run \-d \-\-name postgresql\_database \-e POSTGRESQL\_USER=user \-e POSTGRESQL\_PASSWORD=pass \-e POSTGRESQL\_DATABASE=db \-p 5432:5432 rhscl/postgresql\-10\-rhel7
|
||||
|
||||
.fi
|
||||
.RE
|
||||
|
||||
.PP
|
||||
This will create a container named \fB\fCpostgresql\_database\fR running PostgreSQL with
|
||||
database \fB\fCdb\fR and user with credentials \fB\fCuser:pass\fR\&. Port 5432 will be exposed
|
||||
and mapped to the host. If you want your database to be persistent across container
|
||||
executions, also add a \fB\fC\-v /host/db/path:/var/lib/pgsql/data\fR argument (see
|
||||
below). This will be the PostgreSQL database cluster directory.
|
||||
|
||||
.PP
|
||||
If the database cluster directory is not initialized, the entrypoint script will
|
||||
first run \fB\fCinitdb\fR
|
||||
\[la]http://www.postgresql.org/docs/10/static/app-initdb.html\[ra]
|
||||
and setup necessary database users and passwords. After the database is initialized,
|
||||
or if it was already present, \fB\fCpostgres\fR
|
||||
\[la]http://www.postgresql.org/docs/10/static/app-postgres.html\[ra]
|
||||
is executed and will run as PID 1. You can stop the detached container by running
|
||||
\fB\fCdocker stop postgresql\_database\fR\&.
|
||||
|
||||
.SH Environment variables and volumes
|
||||
.PP
|
||||
The image recognizes the following environment variables that you can set during
|
||||
initialization by passing \fB\fC\-e VAR=VALUE\fR to the Docker run command.
|
||||
|
||||
.PP
|
||||
\fB\fB\fCPOSTGRESQL\_USER\fR\fP
|
||||
.br
|
||||
User name for PostgreSQL account to be created
|
||||
|
||||
.PP
|
||||
\fB\fB\fCPOSTGRESQL\_PASSWORD\fR\fP
|
||||
.br
|
||||
Password for the user account
|
||||
|
||||
.PP
|
||||
\fB\fB\fCPOSTGRESQL\_DATABASE\fR\fP
|
||||
.br
|
||||
Database name
|
||||
|
||||
.PP
|
||||
\fB\fB\fCPOSTGRESQL\_ADMIN\_PASSWORD\fR\fP
|
||||
.br
|
||||
Password for the \fB\fCpostgres\fR admin account (optional)
|
||||
|
||||
.PP
|
||||
Alternatively, the following options are related to migration scenario:
|
||||
|
||||
.PP
|
||||
\fB\fB\fCPOSTGRESQL\_MIGRATION\_REMOTE\_HOST\fR\fP
|
||||
.br
|
||||
Hostname/IP to migrate from
|
||||
|
||||
.PP
|
||||
\fB\fB\fCPOSTGRESQL\_MIGRATION\_ADMIN\_PASSWORD\fR\fP
|
||||
.br
|
||||
Password for the remote 'postgres' admin user
|
||||
|
||||
.PP
|
||||
\fB\fB\fCPOSTGRESQL\_MIGRATION\_IGNORE\_ERRORS (optional, default 'no')\fR\fP
|
||||
.br
|
||||
Set to 'yes' to ignore sql import errors
|
||||
|
||||
.PP
|
||||
The following environment variables influence the PostgreSQL configuration file. They are all optional.
|
||||
|
||||
.PP
|
||||
\fB\fB\fCPOSTGRESQL\_MAX\_CONNECTIONS (default: 100)\fR\fP
|
||||
.br
|
||||
The maximum number of client connections allowed
|
||||
|
||||
.PP
|
||||
\fB\fB\fCPOSTGRESQL\_MAX\_PREPARED\_TRANSACTIONS (default: 0)\fR\fP
|
||||
.br
|
||||
Sets the maximum number of transactions that can be in the "prepared" state. If you are using prepared transactions, you will probably want this to be at least as large as max\_connections
|
||||
|
||||
.PP
|
||||
\fB\fB\fCPOSTGRESQL\_SHARED\_BUFFERS (default: 32M)\fR\fP
|
||||
.br
|
||||
Sets how much memory is dedicated to PostgreSQL to use for caching data
|
||||
|
||||
.PP
|
||||
\fB\fB\fCPOSTGRESQL\_EFFECTIVE\_CACHE\_SIZE (default: 128M)\fR\fP
|
||||
.br
|
||||
Set to an estimate of how much memory is available for disk caching by the operating system and within the database itself
|
||||
|
||||
.PP
|
||||
You can also set the following mount points by passing the \fB\fC\-v /host/dir:/container/dir:Z\fR flag to Docker.
|
||||
|
||||
.PP
|
||||
\fB\fB\fC/var/lib/pgsql/data\fR\fP
|
||||
.br
|
||||
PostgreSQL database cluster directory
|
||||
|
||||
.PP
|
||||
\fBNotice: When mouting a directory from the host into the container, ensure that the mounted
|
||||
directory has the appropriate permissions and that the owner and group of the directory
|
||||
matches the user UID or name which is running inside the container.\fP
|
||||
|
||||
.PP
|
||||
Typically (unless you use \fB\fCdocker run \-u\fR option) processes in container
|
||||
run under UID 26, so \-\- on GNU/Linux \-\- you can fix the datadir permissions
|
||||
for example by:
|
||||
|
||||
.PP
|
||||
.RS
|
||||
|
||||
.nf
|
||||
$ setfacl \-m u:26:\-wx /your/data/dir
|
||||
$ docker run <...> \-v /your/data/dir:/var/lib/pgsql/data:Z <...>
|
||||
|
||||
.fi
|
||||
.RE
|
||||
|
||||
.SH Data migration
|
||||
.PP
|
||||
PostgreSQL container supports migration of data from remote PostgreSQL server.
|
||||
You can run it like:
|
||||
|
||||
.PP
|
||||
.RS
|
||||
|
||||
.nf
|
||||
$ docker run \-d \-\-name postgresql\_database \\
|
||||
\-e POSTGRESQL\_MIGRATION\_REMOTE\_HOST=172.17.0.2 \\
|
||||
\-e POSTGRESQL\_MIGRATION\_ADMIN\_PASSWORD=remoteAdminP@ssword \\
|
||||
[ OPTIONAL\_CONFIGURATION\_VARIABLES ]
|
||||
openshift/postgresql\-92\-centos7
|
||||
|
||||
.fi
|
||||
.RE
|
||||
|
||||
.PP
|
||||
The migration is done the \fBdump and restore\fP way (running \fB\fCpg\_dumpall\fR against
|
||||
remote cluster and importing the dump locally by \fB\fCpsql\fR). Because the process
|
||||
is streamed (unix pipeline), there are no intermediate dump files created during
|
||||
this process to not waste additional storage space.
|
||||
|
||||
.PP
|
||||
If some SQL commands fail during applying, the default behavior
|
||||
of the migration script is to fail as well to ensure the \fBall\fP or \fBnothing\fP
|
||||
result of scripted, unattended migration. In most common cases, successful
|
||||
migration is expected (but not guaranteed!), given you migrate from
|
||||
a previous version of PostgreSQL server container, that is created using
|
||||
the same principles as this one (e.g. migration from
|
||||
\fB\fCopenshift/postgresql\-92\-centos7\fR to \fB\fCcentos/postgresql\-95\-centos7\fR).
|
||||
Migration from a different kind of PostgreSQL container can likely fail.
|
||||
|
||||
.PP
|
||||
If this \fBall\fP or \fBnothing\fP principle is inadequate for you, and you know
|
||||
what you are doing, there's optional \fB\fCPOSTGRESQL\_MIGRATION\_IGNORE\_ERRORS\fR option
|
||||
which does \fBbest effort\fP migration (some data might be lost, it is up to user
|
||||
to review the standard error output and fix the issues manually in
|
||||
post\-migration time).
|
||||
|
||||
.PP
|
||||
Please keep in mind that the container image provides help for users'
|
||||
convenience, but fully automatic migration is not guaranteed. Thus, before you
|
||||
start proceeding with the database migration, get prepared to perform manual
|
||||
steps in order to get all your data migrated.
|
||||
|
||||
.PP
|
||||
Note that you might not use variables like \fB\fCPOSTGRESQL\_USER\fR in migration
|
||||
scenario, all the data (including info about databases, roles or passwords are
|
||||
copied from old cluster). Ensure that you use the same
|
||||
\fB\fCOPTIONAL\_CONFIGURATION\_VARIABLES\fR as you used for initialization of the old
|
||||
PostgreSQL container. If some non\-default configuration is done on remote
|
||||
cluster, you might need to copy the configuration files manually, too.
|
||||
|
||||
.PP
|
||||
Security warning: Note that the IP communication between old and new PostgreSQL
|
||||
clusters is not encrypted by default, it is up to user to configure SSL on
|
||||
remote cluster or ensure security via different means.
|
||||
|
||||
.SH PostgreSQL auto\-tuning
|
||||
.PP
|
||||
When the PostgreSQL image is run with the \fB\fC\-\-memory\fR parameter set and if there
|
||||
are no values provided for \fB\fCPOSTGRESQL\_SHARED\_BUFFERS\fR and
|
||||
\fB\fCPOSTGRESQL\_EFFECTIVE\_CACHE\_SIZE\fR those values are automatically calculated
|
||||
based on the value provided in the \fB\fC\-\-memory\fR parameter.
|
||||
|
||||
.PP
|
||||
The values are calculated based on the
|
||||
upstream
|
||||
\[la]https://wiki.postgresql.org/wiki/Tuning_Your_PostgreSQL_Server\[ra]
|
||||
formulas. For the \fB\fCshared\_buffers\fR we use 1/4 of given memory and for the
|
||||
\fB\fCeffective\_cache\_size\fR we set the value to 1/2 of the given memory.
|
||||
|
||||
.SH PostgreSQL admin account
|
||||
.PP
|
||||
The admin account \fB\fCpostgres\fR has no password set by default, only allowing local
|
||||
connections. You can set it by setting the \fB\fCPOSTGRESQL\_ADMIN\_PASSWORD\fR environment
|
||||
variable when initializing your container. This will allow you to login to the
|
||||
\fB\fCpostgres\fR account remotely. Local connections will still not require a password.
|
||||
|
||||
.SH Changing passwords
|
||||
.PP
|
||||
Since passwords are part of the image configuration, the only supported method
|
||||
to change passwords for the database user (\fB\fCPOSTGRESQL\_USER\fR) and \fB\fCpostgres\fR
|
||||
admin user is by changing the environment variables \fB\fCPOSTGRESQL\_PASSWORD\fR and
|
||||
\fB\fCPOSTGRESQL\_ADMIN\_PASSWORD\fR, respectively.
|
||||
|
||||
.PP
|
||||
Changing database passwords through SQL statements or any way other than through
|
||||
the environment variables aforementioned will cause a mismatch between the
|
||||
values stored in the variables and the actual passwords. Whenever a database
|
||||
container starts it will reset the passwords to the values stored in the
|
||||
environment variables.
|
||||
|
||||
.SH Upgrading database (by switching to newer PostgreSQL image version)
|
||||
.PP
|
||||
** Warning! Please, before you decide to do the data directory upgrade, always
|
||||
ensure that you've carefully backed up all your data and that you are OK with
|
||||
potential manual rollback! **
|
||||
|
||||
.PP
|
||||
This image supports automatic upgrade of data directory created by
|
||||
the PostgreSQL server version 9.6 (and \fIonly\fP this version) \- provided by sclorg
|
||||
image. The upgrade process is designed so that you should be able to just
|
||||
switch from \fIimage A\fP to \fIimage B\fP, and set the \fB\fC$POSTGRESQL\_UPGRADE\fR variable
|
||||
appropriately to explicitly request the database data transformation.
|
||||
|
||||
.PP
|
||||
The upgrade process is internally implemented via \fB\fCpg\_upgrade\fR binary, and for
|
||||
that purpose the container needs to contain two versions of PostgreSQL server
|
||||
(have a look at \fB\fCman pg\_upgrade\fR for more info).
|
||||
|
||||
.PP
|
||||
For the \fB\fCpg\_upgrade\fR process \- and the new server version, we need to initialize
|
||||
a brand new data directory. That's data directory is created automatically by
|
||||
container tooling under /var/lib/pgsql/data, which is usually external
|
||||
bind\-mountpoint. The \fB\fCpg\_upgrade\fR execution is then similar to dump\&restore
|
||||
approach \-\- it starts both old and new PostgreSQL servers (within container) and
|
||||
"dumps" the old datadir while and at the same time it "restores" it into new
|
||||
datadir. This operation requires a lot of data files copying, so you can decide
|
||||
what type of upgrade you'll do by setting \fB\fC$POSTGRESQL\_UPGRADE\fR appropriately:
|
||||
|
||||
.PP
|
||||
\fB\fB\fCcopy\fR\fP
|
||||
.br
|
||||
The data files are copied from old datadir to new datadir. This option has low risk of data loss in case of some upgrade failure.
|
||||
|
||||
.PP
|
||||
\fB\fB\fChardlink\fR\fP
|
||||
.br
|
||||
Data files are hard\-linked from old to the new data directory, which brings performance optimization \- but the old directory becomes unusable, even in case of failure.
|
||||
|
||||
.PP
|
||||
Note that because we copy data directory, you need to make sure that you have
|
||||
enough space for the copy; upgrade failure because of not enough space might
|
||||
lead to data loss.
|
||||
|
||||
.SH Extending image
|
||||
.PP
|
||||
This image can be extended using
|
||||
source\-to\-image
|
||||
\[la]https://github.com/openshift/source-to-image\[ra]\&.
|
||||
|
||||
.PP
|
||||
For example to build customized image \fB\fCnew\-postgresql\fR
|
||||
with configuration in \fB\fC\~/image\-configuration/\fR run:
|
||||
|
||||
.PP
|
||||
.RS
|
||||
|
||||
.nf
|
||||
$ s2i build \~/image\-configuration/ postgresql new\-postgresql
|
||||
|
||||
.fi
|
||||
.RE
|
||||
|
||||
.PP
|
||||
The directory passed to \fB\fCs2i build\fR should contain one or more of the
|
||||
following directories:
|
||||
|
||||
.SS \fB\fCpostgresql\-pre\-start/\fR
|
||||
.PP
|
||||
Source all \fB\fC*.sh\fR files from this directory during early start of the
|
||||
container. There's no PostgreSQL daemon running on background.
|
||||
|
||||
.SS \fB\fCpostgresql\-cfg/\fR
|
||||
.PP
|
||||
Contained configuration files (\fB\fC*.conf\fR) will be included at the end of image
|
||||
postgresql.conf file.
|
||||
|
||||
.SS \fB\fCpostgresql\-init/\fR
|
||||
.PP
|
||||
Contained shell scripts (\fB\fC*.sh\fR) are sourced when the database is freshly
|
||||
initialized (after successful initdb run which made the data directory
|
||||
non\-empty). At the time of sourcing these scripts, the local PostgreSQL
|
||||
server is running. For re\-deployments scenarios with persistent data
|
||||
directory, the scripts are not sourced (no\-op).
|
||||
|
||||
.SS \fB\fCpostgresql\-start/\fR
|
||||
.PP
|
||||
Same sematics as \fB\fCpostgresql\-init/\fR, except that these scripts are
|
||||
always sourced (after \fB\fCpostgresql\-init/\fR scripts, if they exist).
|
||||
|
||||
.ti 0
|
||||
\l'\n(.lu'
|
||||
|
||||
.PP
|
||||
During \fB\fCs2i build\fR all provided files are copied into \fB\fC/opt/app\-root/src\fR
|
||||
directory in the new image. Only one
|
||||
file with the same name can be used for customization and user provided files
|
||||
are preferred over default files in \fB\fC/usr/share/container\-scripts/\fR\-
|
||||
so it is possible to overwrite them.
|
||||
|
||||
.SH Troubleshooting
|
||||
.PP
|
||||
At first the postgres daemon writes its logs to the standard output, so these are available in the container log. The log can be examined by running:
|
||||
|
||||
.PP
|
||||
.RS
|
||||
|
||||
.nf
|
||||
docker logs <container>
|
||||
|
||||
.fi
|
||||
.RE
|
||||
|
||||
.PP
|
||||
Then log output is redirected to logging collector process and will appear in directory "pg\_log".
|
||||
|
||||
.SH See also
|
||||
.PP
|
||||
Dockerfile and other sources for this container image are available on
|
||||
|
||||
\[la]https://github.com/sclorg/postgresql-container\[ra]\&.
|
||||
In that repository, Dockerfile for CentOS is called Dockerfile, Dockerfile
|
||||
for RHEL is called Dockerfile.rhel7.
|
||||
92
root/usr/bin/cgroup-limits
Executable file
92
root/usr/bin/cgroup-limits
Executable file
|
|
@ -0,0 +1,92 @@
|
|||
#!/usr/bin/python
|
||||
|
||||
"""
|
||||
Script for parsing cgroup information
|
||||
|
||||
This script will read some limits from the cgroup system and parse
|
||||
them, printing out "VARIABLE=VALUE" on each line for every limit that is
|
||||
successfully read. Output of this script can be directly fed into
|
||||
bash's export command. Recommended usage from a bash script:
|
||||
|
||||
set -o errexit
|
||||
export_vars=$(cgroup-limits) ; export $export_vars
|
||||
|
||||
Variables currently supported:
|
||||
MAX_MEMORY_LIMIT_IN_BYTES
|
||||
Maximum possible limit MEMORY_LIMIT_IN_BYTES can have. This is
|
||||
currently constant value of 9223372036854775807.
|
||||
MEMORY_LIMIT_IN_BYTES
|
||||
Maximum amount of user memory in bytes. If this value is set
|
||||
to the same value as MAX_MEMORY_LIMIT_IN_BYTES, it means that
|
||||
there is no limit set. The value is taken from
|
||||
/sys/fs/cgroup/memory/memory.limit_in_bytes
|
||||
NUMBER_OF_CORES
|
||||
Number of detected CPU cores that can be used. This value is
|
||||
calculated from /sys/fs/cgroup/cpuset/cpuset.cpus
|
||||
NO_MEMORY_LIMIT
|
||||
Set to "true" if MEMORY_LIMIT_IN_BYTES is so high that the caller
|
||||
can act as if no memory limit was set. Undefined otherwise.
|
||||
"""
|
||||
|
||||
from __future__ import print_function
|
||||
import sys
|
||||
|
||||
|
||||
def _read_file(path):
|
||||
try:
|
||||
with open(path, 'r') as f:
|
||||
return f.read().strip()
|
||||
except IOError:
|
||||
return None
|
||||
|
||||
|
||||
def get_memory_limit():
|
||||
"""
|
||||
Read memory limit, in bytes.
|
||||
"""
|
||||
|
||||
limit = _read_file('/sys/fs/cgroup/memory/memory.limit_in_bytes')
|
||||
if limit is None or not limit.isdigit():
|
||||
print("Warning: Can't detect memory limit from cgroups",
|
||||
file=sys.stderr)
|
||||
return None
|
||||
return int(limit)
|
||||
|
||||
|
||||
def get_number_of_cores():
|
||||
"""
|
||||
Read number of CPU cores.
|
||||
"""
|
||||
|
||||
core_count = 0
|
||||
|
||||
line = _read_file('/sys/fs/cgroup/cpuset/cpuset.cpus')
|
||||
if line is None:
|
||||
print("Warning: Can't detect number of CPU cores from cgroups",
|
||||
file=sys.stderr)
|
||||
return None
|
||||
|
||||
for group in line.split(','):
|
||||
core_ids = list(map(int, group.split('-')))
|
||||
if len(core_ids) == 2:
|
||||
core_count += core_ids[1] - core_ids[0] + 1
|
||||
else:
|
||||
core_count += 1
|
||||
|
||||
return core_count
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
env_vars = {
|
||||
"MAX_MEMORY_LIMIT_IN_BYTES": 9223372036854775807,
|
||||
"MEMORY_LIMIT_IN_BYTES": get_memory_limit(),
|
||||
"NUMBER_OF_CORES": get_number_of_cores()
|
||||
}
|
||||
|
||||
env_vars = {k: v for k, v in env_vars.items() if v is not None}
|
||||
|
||||
if env_vars.get("MEMORY_LIMIT_IN_BYTES", 0) >= 92233720368547:
|
||||
env_vars["NO_MEMORY_LIMIT"] = "true"
|
||||
|
||||
for key, value in env_vars.items():
|
||||
print("{0}={1}".format(key, value))
|
||||
3
root/usr/bin/container-entrypoint
Executable file
3
root/usr/bin/container-entrypoint
Executable file
|
|
@ -0,0 +1,3 @@
|
|||
#!/bin/bash
|
||||
|
||||
exec "$@"
|
||||
47
root/usr/bin/run-postgresql
Executable file
47
root/usr/bin/run-postgresql
Executable file
|
|
@ -0,0 +1,47 @@
|
|||
#!/bin/bash
|
||||
|
||||
export ENABLE_REPLICATION=${ENABLE_REPLICATION:-false}
|
||||
|
||||
set -eu
|
||||
export_vars=$(cgroup-limits) ; export $export_vars
|
||||
|
||||
source "${CONTAINER_SCRIPTS_PATH}/common.sh"
|
||||
|
||||
set_pgdata
|
||||
|
||||
process_extending_files \
|
||||
"${APP_DATA}/src/postgresql-pre-start" \
|
||||
"${CONTAINER_SCRIPTS_PATH}/pre-start"
|
||||
|
||||
check_env_vars
|
||||
generate_passwd_file
|
||||
generate_postgresql_config
|
||||
|
||||
# Is this brand new data volume?
|
||||
PG_INITIALIZED=false
|
||||
|
||||
if [ ! -f "$PGDATA/postgresql.conf" ]; then
|
||||
initialize_database
|
||||
PG_INITIALIZED=:
|
||||
else
|
||||
try_pgupgrade
|
||||
fi
|
||||
|
||||
pg_ctl -w start -o "-h ''"
|
||||
if $PG_INITIALIZED ; then
|
||||
process_extending_files \
|
||||
"${APP_DATA}/src/postgresql-init" \
|
||||
"${CONTAINER_SCRIPTS_PATH}/init"
|
||||
migrate_db
|
||||
create_users
|
||||
fi
|
||||
|
||||
process_extending_files \
|
||||
"${APP_DATA}/src/postgresql-start" \
|
||||
"${CONTAINER_SCRIPTS_PATH}/start"
|
||||
|
||||
pg_ctl stop
|
||||
|
||||
unset_env_vars
|
||||
echo "Starting server..."
|
||||
exec postgres "$@"
|
||||
5
root/usr/bin/run-postgresql-master
Executable file
5
root/usr/bin/run-postgresql-master
Executable file
|
|
@ -0,0 +1,5 @@
|
|||
#!/bin/bash
|
||||
|
||||
export ENABLE_REPLICATION=true
|
||||
|
||||
exec run-postgresql "$@"
|
||||
37
root/usr/bin/run-postgresql-slave
Executable file
37
root/usr/bin/run-postgresql-slave
Executable file
|
|
@ -0,0 +1,37 @@
|
|||
#!/bin/bash
|
||||
|
||||
export ENABLE_REPLICATION=true
|
||||
|
||||
set -eu
|
||||
export_vars=$(cgroup-limits) ; export $export_vars
|
||||
|
||||
source "$CONTAINER_SCRIPTS_PATH"/common.sh
|
||||
|
||||
set_pgdata
|
||||
|
||||
function initialize_replica() {
|
||||
echo "Initializing PostgreSQL slave ..."
|
||||
# TODO: Validate and reuse existing data?
|
||||
rm -rf $PGDATA
|
||||
PGPASSWORD="${POSTGRESQL_MASTER_PASSWORD}" pg_basebackup -X fetch --no-password --pgdata ${PGDATA} --host=${MASTER_FQDN} --port=5432 -U "${POSTGRESQL_MASTER_USER}"
|
||||
|
||||
# PostgreSQL recovery configuration.
|
||||
generate_postgresql_recovery_config
|
||||
cat >> "$PGDATA/recovery.conf" <<EOF
|
||||
|
||||
# Custom OpenShift recovery configuration:
|
||||
include '${POSTGRESQL_RECOVERY_FILE}'
|
||||
EOF
|
||||
}
|
||||
|
||||
check_env_vars
|
||||
generate_passwd_file
|
||||
generate_postgresql_config
|
||||
|
||||
wait_for_postgresql_master
|
||||
export MASTER_FQDN=$(postgresql_master_addr)
|
||||
initialize_replica
|
||||
|
||||
unset_env_vars
|
||||
echo "Starting server..."
|
||||
exec postgres "$@"
|
||||
4
root/usr/bin/usage
Executable file
4
root/usr/bin/usage
Executable file
|
|
@ -0,0 +1,4 @@
|
|||
#!/bin/bash
|
||||
|
||||
cat /usr/share/container-scripts/postgresql/README.md
|
||||
|
||||
39
root/usr/libexec/check-container
Executable file
39
root/usr/libexec/check-container
Executable file
|
|
@ -0,0 +1,39 @@
|
|||
#! /bin/sh
|
||||
|
||||
# Try whether the PostgreSQL in container accepts connections.
|
||||
#
|
||||
# With --live, be tolerant to starting PG server. If the /bin/postgres binary
|
||||
# has not been executed yet (the shell script is initializing the container),
|
||||
# wait for it (this script might run forever, we expect that the timeout is
|
||||
# maintained externally).
|
||||
|
||||
test -z "$ENABLED_COLLECTIONS" || . scl_source enable $ENABLED_COLLECTIONS
|
||||
|
||||
check=ready
|
||||
case $1 in
|
||||
--live) check=live ;;
|
||||
esac
|
||||
|
||||
while true; do
|
||||
# Wait until the /bin/postgres is executed.
|
||||
case $(readlink -f /proc/1/exe) in
|
||||
*bash) sleep 0.2; continue ;;
|
||||
*) break ;;
|
||||
esac
|
||||
done
|
||||
|
||||
# Timeout is infinite (openshift template has the timeout pre-set)
|
||||
pg_isready -q \
|
||||
-h 127.0.0.1 \
|
||||
${POSTGRESQL_USER+-U "$POSTGRESQL_USER"} \
|
||||
${POSTGRESQL_DATABASE+-d "$POSTGRESQL_DATABASE"} \
|
||||
--timeout 0
|
||||
rc=$?
|
||||
|
||||
# we are ready only if the db accepts connections
|
||||
test $rc -eq 0 && exit 0
|
||||
|
||||
# container is live also during pg cluster start-up
|
||||
test $rc -eq 1 && test $check = live && exit 0
|
||||
|
||||
exit 1
|
||||
7
root/usr/libexec/fix-permissions
Executable file
7
root/usr/libexec/fix-permissions
Executable file
|
|
@ -0,0 +1,7 @@
|
|||
#!/bin/sh
|
||||
# Fix permissions on the given directory to allow group read/write of
|
||||
# regular files and execute of directories.
|
||||
find "$1" -exec chown postgres {} \;
|
||||
find "$1" -exec chgrp 0 {} \;
|
||||
find "$1" -exec chmod g+rw {} \;
|
||||
find "$1" -type d -exec chmod g+x {} +
|
||||
306
root/usr/share/container-scripts/postgresql/README.md
Normal file
306
root/usr/share/container-scripts/postgresql/README.md
Normal file
|
|
@ -0,0 +1,306 @@
|
|||
PostgreSQL 10 SQL Database Server container image
|
||||
===============================================
|
||||
|
||||
This container image includes PostgreSQL 10 SQL database server for OpenShift and general usage.
|
||||
Users can choose between RHEL and CentOS based images.
|
||||
The RHEL image is available in the [Red Hat Container Catalog](https://access.redhat.com/containers/#/registry.access.redhat.com/rhscl/postgresql-10-rhel7)
|
||||
as registry.access.redhat.com/rhscl/postgresql-10-rhel7.
|
||||
The CentOS image is then available on [Docker Hub](https://hub.docker.com/r/centos/postgresql-10-centos7/)
|
||||
as centos/postgresql-10-centos7.
|
||||
|
||||
|
||||
Description
|
||||
-----------
|
||||
|
||||
This container image provides a containerized packaging of the PostgreSQL postgres daemon
|
||||
and client application. The postgres server daemon accepts connections from clients
|
||||
and provides access to content from PostgreSQL databases on behalf of the clients.
|
||||
You can find more information on the PostgreSQL project from the project Web site
|
||||
(https://www.postgresql.org/).
|
||||
|
||||
|
||||
Usage
|
||||
-----
|
||||
|
||||
For this, we will assume that you are using the `rhscl/postgresql-10-rhel7` image.
|
||||
If you want to set only the mandatory environment variables and not store the database
|
||||
in a host directory, execute the following command:
|
||||
|
||||
```
|
||||
$ docker run -d --name postgresql_database -e POSTGRESQL_USER=user -e POSTGRESQL_PASSWORD=pass -e POSTGRESQL_DATABASE=db -p 5432:5432 rhscl/postgresql-10-rhel7
|
||||
```
|
||||
|
||||
This will create a container named `postgresql_database` running PostgreSQL with
|
||||
database `db` and user with credentials `user:pass`. Port 5432 will be exposed
|
||||
and mapped to the host. If you want your database to be persistent across container
|
||||
executions, also add a `-v /host/db/path:/var/lib/pgsql/data` argument (see
|
||||
below). This will be the PostgreSQL database cluster directory.
|
||||
|
||||
If the database cluster directory is not initialized, the entrypoint script will
|
||||
first run [`initdb`](http://www.postgresql.org/docs/10/static/app-initdb.html)
|
||||
and setup necessary database users and passwords. After the database is initialized,
|
||||
or if it was already present, [`postgres`](http://www.postgresql.org/docs/10/static/app-postgres.html)
|
||||
is executed and will run as PID 1. You can stop the detached container by running
|
||||
`docker stop postgresql_database`.
|
||||
|
||||
|
||||
|
||||
Environment variables and volumes
|
||||
---------------------------------
|
||||
|
||||
The image recognizes the following environment variables that you can set during
|
||||
initialization by passing `-e VAR=VALUE` to the Docker run command.
|
||||
|
||||
**`POSTGRESQL_USER`**
|
||||
User name for PostgreSQL account to be created
|
||||
|
||||
**`POSTGRESQL_PASSWORD`**
|
||||
Password for the user account
|
||||
|
||||
**`POSTGRESQL_DATABASE`**
|
||||
Database name
|
||||
|
||||
**`POSTGRESQL_ADMIN_PASSWORD`**
|
||||
Password for the `postgres` admin account (optional)
|
||||
|
||||
|
||||
Alternatively, the following options are related to migration scenario:
|
||||
|
||||
**`POSTGRESQL_MIGRATION_REMOTE_HOST`**
|
||||
Hostname/IP to migrate from
|
||||
|
||||
**`POSTGRESQL_MIGRATION_ADMIN_PASSWORD`**
|
||||
Password for the remote 'postgres' admin user
|
||||
|
||||
**`POSTGRESQL_MIGRATION_IGNORE_ERRORS (optional, default 'no')`**
|
||||
Set to 'yes' to ignore sql import errors
|
||||
|
||||
|
||||
The following environment variables influence the PostgreSQL configuration file. They are all optional.
|
||||
|
||||
**`POSTGRESQL_MAX_CONNECTIONS (default: 100)`**
|
||||
The maximum number of client connections allowed
|
||||
|
||||
**`POSTGRESQL_MAX_PREPARED_TRANSACTIONS (default: 0)`**
|
||||
Sets the maximum number of transactions that can be in the "prepared" state. If you are using prepared transactions, you will probably want this to be at least as large as max_connections
|
||||
|
||||
**`POSTGRESQL_SHARED_BUFFERS (default: 32M)`**
|
||||
Sets how much memory is dedicated to PostgreSQL to use for caching data
|
||||
|
||||
**`POSTGRESQL_EFFECTIVE_CACHE_SIZE (default: 128M)`**
|
||||
Set to an estimate of how much memory is available for disk caching by the operating system and within the database itself
|
||||
|
||||
|
||||
You can also set the following mount points by passing the `-v /host/dir:/container/dir:Z` flag to Docker.
|
||||
|
||||
**`/var/lib/pgsql/data`**
|
||||
PostgreSQL database cluster directory
|
||||
|
||||
|
||||
**Notice: When mouting a directory from the host into the container, ensure that the mounted
|
||||
directory has the appropriate permissions and that the owner and group of the directory
|
||||
matches the user UID or name which is running inside the container.**
|
||||
|
||||
Typically (unless you use `docker run -u` option) processes in container
|
||||
run under UID 26, so -- on GNU/Linux -- you can fix the datadir permissions
|
||||
for example by:
|
||||
|
||||
```
|
||||
$ setfacl -m u:26:-wx /your/data/dir
|
||||
$ docker run <...> -v /your/data/dir:/var/lib/pgsql/data:Z <...>
|
||||
```
|
||||
|
||||
|
||||
Data migration
|
||||
----------------------
|
||||
|
||||
PostgreSQL container supports migration of data from remote PostgreSQL server.
|
||||
You can run it like:
|
||||
|
||||
```
|
||||
$ docker run -d --name postgresql_database \
|
||||
-e POSTGRESQL_MIGRATION_REMOTE_HOST=172.17.0.2 \
|
||||
-e POSTGRESQL_MIGRATION_ADMIN_PASSWORD=remoteAdminP@ssword \
|
||||
[ OPTIONAL_CONFIGURATION_VARIABLES ]
|
||||
openshift/postgresql-92-centos7
|
||||
```
|
||||
|
||||
The migration is done the **dump and restore** way (running `pg_dumpall` against
|
||||
remote cluster and importing the dump locally by `psql`). Because the process
|
||||
is streamed (unix pipeline), there are no intermediate dump files created during
|
||||
this process to not waste additional storage space.
|
||||
|
||||
If some SQL commands fail during applying, the default behavior
|
||||
of the migration script is to fail as well to ensure the **all** or **nothing**
|
||||
result of scripted, unattended migration. In most common cases, successful
|
||||
migration is expected (but not guaranteed!), given you migrate from
|
||||
a previous version of PostgreSQL server container, that is created using
|
||||
the same principles as this one (e.g. migration from
|
||||
`openshift/postgresql-92-centos7` to `centos/postgresql-95-centos7`).
|
||||
Migration from a different kind of PostgreSQL container can likely fail.
|
||||
|
||||
If this **all** or **nothing** principle is inadequate for you, and you know
|
||||
what you are doing, there's optional `POSTGRESQL_MIGRATION_IGNORE_ERRORS` option
|
||||
which does **best effort** migration (some data might be lost, it is up to user
|
||||
to review the standard error output and fix the issues manually in
|
||||
post-migration time).
|
||||
|
||||
Please keep in mind that the container image provides help for users'
|
||||
convenience, but fully automatic migration is not guaranteed. Thus, before you
|
||||
start proceeding with the database migration, get prepared to perform manual
|
||||
steps in order to get all your data migrated.
|
||||
|
||||
Note that you might not use variables like `POSTGRESQL_USER` in migration
|
||||
scenario, all the data (including info about databases, roles or passwords are
|
||||
copied from old cluster). Ensure that you use the same
|
||||
`OPTIONAL_CONFIGURATION_VARIABLES` as you used for initialization of the old
|
||||
PostgreSQL container. If some non-default configuration is done on remote
|
||||
cluster, you might need to copy the configuration files manually, too.
|
||||
|
||||
Security warning: Note that the IP communication between old and new PostgreSQL
|
||||
clusters is not encrypted by default, it is up to user to configure SSL on
|
||||
remote cluster or ensure security via different means.
|
||||
|
||||
PostgreSQL auto-tuning
|
||||
--------------------
|
||||
|
||||
When the PostgreSQL image is run with the `--memory` parameter set and if there
|
||||
are no values provided for `POSTGRESQL_SHARED_BUFFERS` and
|
||||
`POSTGRESQL_EFFECTIVE_CACHE_SIZE` those values are automatically calculated
|
||||
based on the value provided in the `--memory` parameter.
|
||||
|
||||
The values are calculated based on the
|
||||
[upstream](https://wiki.postgresql.org/wiki/Tuning_Your_PostgreSQL_Server)
|
||||
formulas. For the `shared_buffers` we use 1/4 of given memory and for the
|
||||
`effective_cache_size` we set the value to 1/2 of the given memory.
|
||||
|
||||
PostgreSQL admin account
|
||||
------------------------
|
||||
The admin account `postgres` has no password set by default, only allowing local
|
||||
connections. You can set it by setting the `POSTGRESQL_ADMIN_PASSWORD` environment
|
||||
variable when initializing your container. This will allow you to login to the
|
||||
`postgres` account remotely. Local connections will still not require a password.
|
||||
|
||||
|
||||
Changing passwords
|
||||
------------------
|
||||
|
||||
Since passwords are part of the image configuration, the only supported method
|
||||
to change passwords for the database user (`POSTGRESQL_USER`) and `postgres`
|
||||
admin user is by changing the environment variables `POSTGRESQL_PASSWORD` and
|
||||
`POSTGRESQL_ADMIN_PASSWORD`, respectively.
|
||||
|
||||
Changing database passwords through SQL statements or any way other than through
|
||||
the environment variables aforementioned will cause a mismatch between the
|
||||
values stored in the variables and the actual passwords. Whenever a database
|
||||
container starts it will reset the passwords to the values stored in the
|
||||
environment variables.
|
||||
|
||||
|
||||
Upgrading database (by switching to newer PostgreSQL image version)
|
||||
-------------------------------------------------------------------
|
||||
|
||||
** Warning! Please, before you decide to do the data directory upgrade, always
|
||||
ensure that you've carefully backed up all your data and that you are OK with
|
||||
potential manual rollback! **
|
||||
|
||||
This image supports automatic upgrade of data directory created by
|
||||
the PostgreSQL server version 9.6 (and _only_ this version) - provided by sclorg
|
||||
image. The upgrade process is designed so that you should be able to just
|
||||
switch from *image A* to *image B*, and set the `$POSTGRESQL_UPGRADE` variable
|
||||
appropriately to explicitly request the database data transformation.
|
||||
|
||||
The upgrade process is internally implemented via `pg_upgrade` binary, and for
|
||||
that purpose the container needs to contain two versions of PostgreSQL server
|
||||
(have a look at `man pg_upgrade` for more info).
|
||||
|
||||
For the `pg_upgrade` process - and the new server version, we need to initialize
|
||||
a brand new data directory. That's data directory is created automatically by
|
||||
container tooling under /var/lib/pgsql/data, which is usually external
|
||||
bind-mountpoint. The `pg_upgrade` execution is then similar to dump&restore
|
||||
approach -- it starts both old and new PostgreSQL servers (within container) and
|
||||
"dumps" the old datadir while and at the same time it "restores" it into new
|
||||
datadir. This operation requires a lot of data files copying, so you can decide
|
||||
what type of upgrade you'll do by setting `$POSTGRESQL_UPGRADE` appropriately:
|
||||
|
||||
**`copy`**
|
||||
The data files are copied from old datadir to new datadir. This option has low risk of data loss in case of some upgrade failure.
|
||||
|
||||
**`hardlink`**
|
||||
Data files are hard-linked from old to the new data directory, which brings performance optimization - but the old directory becomes unusable, even in case of failure.
|
||||
|
||||
|
||||
Note that because we copy data directory, you need to make sure that you have
|
||||
enough space for the copy; upgrade failure because of not enough space might
|
||||
lead to data loss.
|
||||
|
||||
|
||||
Extending image
|
||||
----------------
|
||||
|
||||
This image can be extended using
|
||||
[source-to-image](https://github.com/openshift/source-to-image).
|
||||
|
||||
For example to build customized image `new-postgresql`
|
||||
with configuration in `~/image-configuration/` run:
|
||||
|
||||
|
||||
```
|
||||
$ s2i build ~/image-configuration/ postgresql new-postgresql
|
||||
```
|
||||
|
||||
The directory passed to `s2i build` should contain one or more of the
|
||||
following directories:
|
||||
|
||||
|
||||
##### `postgresql-pre-start/`
|
||||
|
||||
Source all `*.sh` files from this directory during early start of the
|
||||
container. There's no PostgreSQL daemon running on background.
|
||||
|
||||
|
||||
##### `postgresql-cfg/`
|
||||
|
||||
Contained configuration files (`*.conf`) will be included at the end of image
|
||||
postgresql.conf file.
|
||||
|
||||
|
||||
##### `postgresql-init/`
|
||||
|
||||
Contained shell scripts (`*.sh`) are sourced when the database is freshly
|
||||
initialized (after successful initdb run which made the data directory
|
||||
non-empty). At the time of sourcing these scripts, the local PostgreSQL
|
||||
server is running. For re-deployments scenarios with persistent data
|
||||
directory, the scripts are not sourced (no-op).
|
||||
|
||||
|
||||
##### `postgresql-start/`
|
||||
|
||||
Same sematics as `postgresql-init/`, except that these scripts are
|
||||
always sourced (after `postgresql-init/` scripts, if they exist).
|
||||
|
||||
|
||||
----------------------------------------------
|
||||
|
||||
During `s2i build` all provided files are copied into `/opt/app-root/src`
|
||||
directory in the new image. Only one
|
||||
file with the same name can be used for customization and user provided files
|
||||
are preferred over default files in `/usr/share/container-scripts/`-
|
||||
so it is possible to overwrite them.
|
||||
|
||||
|
||||
Troubleshooting
|
||||
---------------
|
||||
At first the postgres daemon writes its logs to the standard output, so these are available in the container log. The log can be examined by running:
|
||||
|
||||
docker logs <container>
|
||||
|
||||
Then log output is redirected to logging collector process and will appear in directory "pg_log".
|
||||
|
||||
|
||||
See also
|
||||
--------
|
||||
Dockerfile and other sources for this container image are available on
|
||||
https://github.com/sclorg/postgresql-container.
|
||||
In that repository, Dockerfile for CentOS is called Dockerfile, Dockerfile
|
||||
for RHEL is called Dockerfile.rhel7.
|
||||
431
root/usr/share/container-scripts/postgresql/common.sh
Normal file
431
root/usr/share/container-scripts/postgresql/common.sh
Normal file
|
|
@ -0,0 +1,431 @@
|
|||
# Configuration settings.
|
||||
export POSTGRESQL_MAX_CONNECTIONS=${POSTGRESQL_MAX_CONNECTIONS:-100}
|
||||
export POSTGRESQL_MAX_PREPARED_TRANSACTIONS=${POSTGRESQL_MAX_PREPARED_TRANSACTIONS:-0}
|
||||
|
||||
# Perform auto-tuning based on the container cgroups limits (only when the
|
||||
# limits are set).
|
||||
# Users can still override this by setting the POSTGRESQL_SHARED_BUFFERS
|
||||
# and POSTGRESQL_EFFECTIVE_CACHE_SIZE variables.
|
||||
if [[ "${NO_MEMORY_LIMIT:-}" == "true" || -z "${MEMORY_LIMIT_IN_BYTES:-}" ]]; then
|
||||
export POSTGRESQL_SHARED_BUFFERS=${POSTGRESQL_SHARED_BUFFERS:-32MB}
|
||||
export POSTGRESQL_EFFECTIVE_CACHE_SIZE=${POSTGRESQL_EFFECTIVE_CACHE_SIZE:-128MB}
|
||||
else
|
||||
# Use 1/4 of given memory for shared buffers
|
||||
shared_buffers_computed="$(($MEMORY_LIMIT_IN_BYTES/1024/1024/4))MB"
|
||||
# Setting effective_cache_size to 1/2 of total memory would be a normal conservative setting,
|
||||
effective_cache="$(($MEMORY_LIMIT_IN_BYTES/1024/1024/2))MB"
|
||||
export POSTGRESQL_SHARED_BUFFERS=${POSTGRESQL_SHARED_BUFFERS:-$shared_buffers_computed}
|
||||
export POSTGRESQL_EFFECTIVE_CACHE_SIZE=${POSTGRESQL_EFFECTIVE_CACHE_SIZE:-$effective_cache}
|
||||
fi
|
||||
|
||||
export POSTGRESQL_RECOVERY_FILE=$HOME/openshift-custom-recovery.conf
|
||||
export POSTGRESQL_CONFIG_FILE=$HOME/openshift-custom-postgresql.conf
|
||||
|
||||
postinitdb_actions=
|
||||
|
||||
# match . files when moving userdata below
|
||||
shopt -s dotglob
|
||||
# extglob enables the !(userdata) glob pattern below.
|
||||
shopt -s extglob
|
||||
|
||||
function usage() {
|
||||
if [ $# == 1 ]; then
|
||||
echo >&2 "error: $1"
|
||||
fi
|
||||
|
||||
cat >&2 <<EOF
|
||||
For general container run, you must either specify the following environment
|
||||
variables:
|
||||
POSTGRESQL_USER POSTGRESQL_PASSWORD POSTGRESQL_DATABASE
|
||||
Or the following environment variable:
|
||||
POSTGRESQL_ADMIN_PASSWORD
|
||||
Or both.
|
||||
|
||||
To migrate data from different PostgreSQL container:
|
||||
POSTGRESQL_MIGRATION_REMOTE_HOST (hostname or IP address)
|
||||
POSTGRESQL_MIGRATION_ADMIN_PASSWORD (password of remote 'postgres' user)
|
||||
And optionally:
|
||||
POSTGRESQL_MIGRATION_IGNORE_ERRORS=yes (default is 'no')
|
||||
|
||||
Optional settings:
|
||||
POSTGRESQL_MAX_CONNECTIONS (default: 100)
|
||||
POSTGRESQL_MAX_PREPARED_TRANSACTIONS (default: 0)
|
||||
POSTGRESQL_SHARED_BUFFERS (default: 32MB)
|
||||
|
||||
For more information see /usr/share/container-scripts/postgresql/README.md
|
||||
within the container or visit https://github.com/sclorg/postgresql-container.
|
||||
EOF
|
||||
exit 1
|
||||
}
|
||||
|
||||
function check_env_vars() {
|
||||
if [[ -v POSTGRESQL_USER || -v POSTGRESQL_PASSWORD || -v POSTGRESQL_DATABASE ]]; then
|
||||
# one var means all three must be specified
|
||||
[[ -v POSTGRESQL_USER && -v POSTGRESQL_PASSWORD && -v POSTGRESQL_DATABASE ]] || usage
|
||||
|
||||
[ ${#POSTGRESQL_USER} -le 63 ] || usage "PostgreSQL username too long (maximum 63 characters)"
|
||||
[ ${#POSTGRESQL_DATABASE} -le 63 ] || usage "Database name too long (maximum 63 characters)"
|
||||
postinitdb_actions+=",simple_db"
|
||||
fi
|
||||
|
||||
if [ -v POSTGRESQL_ADMIN_PASSWORD ]; then
|
||||
postinitdb_actions+=",admin_pass"
|
||||
fi
|
||||
|
||||
if [ -v POSTGRESQL_MIGRATION_REMOTE_HOST -a \
|
||||
-v POSTGRESQL_MIGRATION_ADMIN_PASSWORD ]; then
|
||||
postinitdb_actions+=",migration"
|
||||
fi
|
||||
|
||||
case "$postinitdb_actions" in
|
||||
,simple_db,admin_pass) ;;
|
||||
,migration|,simple_db|,admin_pass) ;;
|
||||
*) usage ;;
|
||||
esac
|
||||
}
|
||||
|
||||
# Make sure env variables don't propagate to PostgreSQL process.
|
||||
function unset_env_vars() {
|
||||
unset POSTGRESQL_{DATABASE,USER,PASSWORD,ADMIN_PASSWORD}
|
||||
}
|
||||
|
||||
# postgresql_master_addr lookups the 'postgresql-master' DNS and get list of the available
|
||||
# endpoints. Each endpoint is a PostgreSQL container with the 'master' PostgreSQL running.
|
||||
function postgresql_master_addr() {
|
||||
local service_name=${POSTGRESQL_MASTER_SERVICE_NAME:-postgresql-master}
|
||||
local endpoints=$(dig ${service_name} A +search | grep ";${service_name}" | cut -d ';' -f 2 2>/dev/null)
|
||||
# FIXME: This is for debugging (docker run)
|
||||
if [ -v POSTGRESQL_MASTER_IP ]; then
|
||||
endpoints=${POSTGRESQL_MASTER_IP:-}
|
||||
fi
|
||||
if [ -z "$endpoints" ]; then
|
||||
>&2 echo "Failed to resolve PostgreSQL master IP address"
|
||||
exit 3
|
||||
fi
|
||||
echo -n "$(echo $endpoints | cut -d ' ' -f 1)"
|
||||
}
|
||||
|
||||
# New config is generated every time a container is created. It only contains
|
||||
# additional custom settings and is included from $PGDATA/postgresql.conf.
|
||||
function generate_postgresql_config() {
|
||||
envsubst \
|
||||
< "${CONTAINER_SCRIPTS_PATH}/openshift-custom-postgresql.conf.template" \
|
||||
> "${POSTGRESQL_CONFIG_FILE}"
|
||||
|
||||
if [ "${ENABLE_REPLICATION}" == "true" ]; then
|
||||
envsubst \
|
||||
< "${CONTAINER_SCRIPTS_PATH}/openshift-custom-postgresql-replication.conf.template" \
|
||||
>> "${POSTGRESQL_CONFIG_FILE}"
|
||||
fi
|
||||
|
||||
(
|
||||
shopt -s nullglob
|
||||
for conf in "${APP_DATA}"/src/postgresql-cfg/*.conf; do
|
||||
echo include \'${conf}\' >> "${POSTGRESQL_CONFIG_FILE}"
|
||||
done
|
||||
)
|
||||
}
|
||||
|
||||
function generate_postgresql_recovery_config() {
|
||||
envsubst \
|
||||
< "${CONTAINER_SCRIPTS_PATH}/openshift-custom-recovery.conf.template" \
|
||||
> "${POSTGRESQL_RECOVERY_FILE}"
|
||||
}
|
||||
|
||||
# Generate passwd file based on current uid
|
||||
function generate_passwd_file() {
|
||||
export USER_ID=$(id -u)
|
||||
export GROUP_ID=$(id -g)
|
||||
grep -v ^postgres /etc/passwd > "$HOME/passwd"
|
||||
echo "postgres:x:${USER_ID}:${GROUP_ID}:PostgreSQL Server:${HOME}:/bin/bash" >> "$HOME/passwd"
|
||||
export LD_PRELOAD=libnss_wrapper.so
|
||||
export NSS_WRAPPER_PASSWD=${HOME}/passwd
|
||||
export NSS_WRAPPER_GROUP=/etc/group
|
||||
}
|
||||
|
||||
initdb_wrapper ()
|
||||
{
|
||||
# Initialize the database cluster with utf8 support enabled by default.
|
||||
# This might affect performance, see:
|
||||
# http://www.postgresql.org/docs/10/static/locale.html
|
||||
LANG=${LANG:-en_US.utf8} "$@"
|
||||
}
|
||||
|
||||
function initialize_database() {
|
||||
initdb_wrapper initdb
|
||||
|
||||
# PostgreSQL configuration.
|
||||
cat >> "$PGDATA/postgresql.conf" <<EOF
|
||||
|
||||
# Custom OpenShift configuration:
|
||||
include '${POSTGRESQL_CONFIG_FILE}'
|
||||
EOF
|
||||
|
||||
# Access control configuration.
|
||||
# FIXME: would be nice-to-have if we could allow connections only from
|
||||
# specific hosts / subnet
|
||||
cat >> "$PGDATA/pg_hba.conf" <<EOF
|
||||
|
||||
#
|
||||
# Custom OpenShift configuration starting at this point.
|
||||
#
|
||||
|
||||
# Allow connections from all hosts.
|
||||
host all all all md5
|
||||
|
||||
# Allow replication connections from all hosts.
|
||||
host replication all all md5
|
||||
EOF
|
||||
}
|
||||
|
||||
function create_users() {
|
||||
if [[ ",$postinitdb_actions," = *,simple_db,* ]]; then
|
||||
createuser "$POSTGRESQL_USER"
|
||||
createdb --owner="$POSTGRESQL_USER" "$POSTGRESQL_DATABASE"
|
||||
fi
|
||||
|
||||
if [ -v POSTGRESQL_MASTER_USER ]; then
|
||||
createuser "$POSTGRESQL_MASTER_USER"
|
||||
fi
|
||||
}
|
||||
|
||||
migrate_db ()
|
||||
{
|
||||
test "$postinitdb_actions" = ",migration" || return 0
|
||||
|
||||
# Migration path.
|
||||
(
|
||||
if [ ${POSTGRESQL_MIGRATION_IGNORE_ERRORS-no} = no ]; then
|
||||
echo '\set ON_ERROR_STOP on'
|
||||
fi
|
||||
# initdb automatically creates 'postgres' role; creating it again would
|
||||
# fail the whole migration so we drop it here
|
||||
PGPASSWORD="$POSTGRESQL_MIGRATION_ADMIN_PASSWORD" \
|
||||
pg_dumpall -h "$POSTGRESQL_MIGRATION_REMOTE_HOST" \
|
||||
| grep -v '^CREATE ROLE postgres;'
|
||||
) | psql
|
||||
}
|
||||
|
||||
function set_pgdata ()
|
||||
{
|
||||
export PGDATA=$HOME/data/userdata
|
||||
# create a subdirectory that the user owns
|
||||
mkdir -p "$PGDATA"
|
||||
# backwards compatibility case, we used to put the data here,
|
||||
# move it into our new expected location (userdata)
|
||||
if [ -e ${HOME}/data/PG_VERSION ]; then
|
||||
pushd "${HOME}/data"
|
||||
# move everything except the userdata directory itself, into the userdata directory.
|
||||
mv !(userdata) "userdata"
|
||||
popd
|
||||
fi
|
||||
# ensure sane perms for postgresql startup
|
||||
chmod 700 "$PGDATA"
|
||||
}
|
||||
|
||||
function wait_for_postgresql_master() {
|
||||
while true; do
|
||||
master_fqdn=$(postgresql_master_addr)
|
||||
echo "Waiting for PostgreSQL master (${master_fqdn}) to accept connections ..."
|
||||
if [ -v POSTGRESQL_ADMIN_PASSWORD ]; then
|
||||
PGPASSWORD=${POSTGRESQL_ADMIN_PASSWORD} psql "postgresql://postgres@${master_fqdn}" -c "SELECT 1;" && return 0
|
||||
else
|
||||
PGPASSWORD=${POSTGRESQL_PASSWORD} psql "postgresql://${POSTGRESQL_USER}@${master_fqdn}/${POSTGRESQL_DATABASE}" -c "SELECT 1;" && return 0
|
||||
fi
|
||||
sleep 1
|
||||
done
|
||||
}
|
||||
|
||||
|
||||
run_pgupgrade ()
|
||||
(
|
||||
optimized=false
|
||||
old_raw_version=${POSTGRESQL_PREV_VERSION//\./}
|
||||
new_raw_version=${POSTGRESQL_VERSION//\./}
|
||||
|
||||
if test "$old_raw_version" = 92; then
|
||||
old_collection=postgresql92
|
||||
else
|
||||
old_collection=rh-postgresql$old_raw_version
|
||||
fi
|
||||
|
||||
old_pgengine=/opt/rh/$old_collection/root/usr/bin
|
||||
new_pgengine=/opt/rh/rh-postgresql${new_raw_version}/root/usr/bin
|
||||
PGDATA_new="${PGDATA}-new"
|
||||
|
||||
printf >&2 "\n========== \$PGDATA upgrade: %s -> %s ==========\n\n" \
|
||||
"$POSTGRESQL_PREV_VERSION" \
|
||||
"$POSTGRESQL_VERSION"
|
||||
|
||||
info_msg () { printf >&2 "\n===> $*\n\n" ;}
|
||||
|
||||
# pg_upgrade writes logs to cwd, so go to the persistent storage first
|
||||
cd "$HOME"/data
|
||||
|
||||
# disable this because of scl_source, 'set +u' just makes the code ugly
|
||||
# anyways
|
||||
set +u
|
||||
|
||||
# we need to have the old SCL enabled, otherwise the $old_pgengine is not
|
||||
# working. The scl_source script doesn't pay attention to non-zero exit
|
||||
# statuses, so use 'set +e'.
|
||||
set +e
|
||||
source scl_source enable $old_collection
|
||||
set -e
|
||||
|
||||
case $POSTGRESQL_UPGRADE in
|
||||
copy) # we accept this
|
||||
;;
|
||||
hardlink)
|
||||
optimized=:
|
||||
;;
|
||||
*)
|
||||
echo >&2 "Unsupported value: \$POSTGRESQL_UPGRADE=$POSTGRESQL_UPGRADE"
|
||||
false
|
||||
;;
|
||||
esac
|
||||
|
||||
# Ensure $PGDATA_new doesn't exist yet, so we can immediately remove it if
|
||||
# there's some problem.
|
||||
test ! -e "$PGDATA_new"
|
||||
|
||||
# initialize the database
|
||||
info_msg "Initialize new data directory; we will migrate to that."
|
||||
initdb_cmd=( initdb_wrapper "$new_pgengine"/initdb "$PGDATA_new" )
|
||||
eval "\${initdb_cmd[@]} ${POSTGRESQL_UPGRADE_INITDB_OPTIONS-}" || \
|
||||
{ rm -rf "$PGDATA_new" ; false ; }
|
||||
|
||||
upgrade_cmd=(
|
||||
"$new_pgengine"/pg_upgrade
|
||||
"--old-bindir=$old_pgengine"
|
||||
"--new-bindir=$new_pgengine"
|
||||
"--old-datadir=$PGDATA"
|
||||
"--new-datadir=$PGDATA_new"
|
||||
)
|
||||
|
||||
# Dangerous --link option, we loose $DATADIR if something goes wrong.
|
||||
! $optimized || upgrade_cmd+=(--link)
|
||||
|
||||
# User-specififed options for pg_upgrade.
|
||||
eval "upgrade_cmd+=(${POSTGRESQL_UPGRADE_PGUPGRADE_OPTIONS-})"
|
||||
|
||||
# the upgrade
|
||||
info_msg "Starting the pg_upgrade process."
|
||||
|
||||
# Once we stop support for PostgreSQL 9.4, we don't need
|
||||
# REDHAT_PGUPGRADE_FROM_RHEL hack as we don't upgrade from 9.2 -- that means
|
||||
# that we don't need to fiddle with unix_socket_director{y,ies} option.
|
||||
REDHAT_PGUPGRADE_FROM_RHEL=1 \
|
||||
"${upgrade_cmd[@]}" || { rm -rf "$PGDATA_new" && false ; }
|
||||
|
||||
# Move the important configuration and remove old data. This is highly
|
||||
# careless, but we can't do more for this over-automatized process.
|
||||
info_msg "Swap the old and new PGDATA and cleanup."
|
||||
mv "$PGDATA"/*.conf "$PGDATA_new"
|
||||
rm -rf "$PGDATA"
|
||||
mv "$PGDATA_new" "$PGDATA"
|
||||
|
||||
info_msg "Upgrade DONE."
|
||||
)
|
||||
|
||||
|
||||
# Run right after container startup, when the data volume is already initialized
|
||||
# (not initialized by this container run) and thus there exists a chance that
|
||||
# the data was generated by incompatible PostgreSQL major version.
|
||||
try_pgupgrade ()
|
||||
{
|
||||
local versionfile="$PGDATA"/PG_VERSION version upgrade_available
|
||||
|
||||
# This file always exists.
|
||||
test -f "$versionfile"
|
||||
version=$(cat "$versionfile")
|
||||
|
||||
# If we don't support pg_upgrade, skip.
|
||||
test -z "${POSTGRESQL_PREV_VERSION-}" && return 0
|
||||
|
||||
if test "$POSTGRESQL_VERSION" = "$version"; then
|
||||
# No need to call pg_upgrade.
|
||||
|
||||
# Mistakenly requests upgrade? If not, just start the DB.
|
||||
test -z "${POSTGRESQL_UPGRADE-}" && return 0
|
||||
|
||||
# Make _sure_ we have this safety-belt here, otherwise our users would
|
||||
# just specify '-e POSTGRESQL_UPGRADE=hardlink' permanently, even for
|
||||
# re-deployment cases when upgrade is not needed. Setting such
|
||||
# unfortunate default could mean that pg_upgrade might (after some user
|
||||
# mistake) migrate (or even destruct, especially with --link) the old data
|
||||
# directory with limited rollback options, if any.
|
||||
echo >&2
|
||||
echo >&2 "== WARNING!! =="
|
||||
echo >&2 "PostgreSQL server version matches the datadir PG_VERSION."
|
||||
echo >&2 "The \$POSTGRESQL_UPGRADE makes no sense and you probably"
|
||||
echo >&2 "made some mistake, keeping the variable set you might"
|
||||
echo >&2 "risk a data loss in future!"
|
||||
echo >&2 "==============="
|
||||
echo >&2
|
||||
|
||||
# Exit here, but allow _really explicit_ foot-shot.
|
||||
${POSTGRESQL_UPGRADE_FORCE-false}
|
||||
return 0
|
||||
fi
|
||||
|
||||
# At this point in code we know that PG_VERSION doesn't match the PostgreSQL
|
||||
# server major version; this might mean that user either (a) mistakenly
|
||||
# deploys from a bad image, or (b) user wants to perform upgrade. For the
|
||||
# upgrade we require explicit request -- just to avoid disasters in (a)-cases.
|
||||
|
||||
if test -z "${POSTGRESQL_UPGRADE-}"; then
|
||||
echo >&2 "Incompatible data directory. This container image provides"
|
||||
echo >&2 "PostgreSQL '$POSTGRESQL_VERSION', but data directory is of"
|
||||
echo >&2 "version '$version'."
|
||||
echo >&2
|
||||
echo >&2 "This image supports automatic data directory upgrade from"
|
||||
echo >&2 "'$POSTGRESQL_PREV_VERSION', please _carefully_ consult image documentation"
|
||||
echo >&2 "about how to use the '\$POSTGRESQL_UPGRADE' startup option."
|
||||
# We could wait for postgresql startup failure (there's no risk of data dir
|
||||
# corruption), but fail rather early.
|
||||
false
|
||||
fi
|
||||
|
||||
# We support pg_upgrade process only from previous version of this container
|
||||
# (upgrade to N to N+1 is possible, so e.g. 9.4 to 9.5).
|
||||
if test "$POSTGRESQL_PREV_VERSION" != "$version"; then
|
||||
echo >&2 "With this container image you can only upgrade from data directory"
|
||||
echo >&2 "of version '$POSTGRESQL_PREV_VERSION', not '$version'."
|
||||
false
|
||||
fi
|
||||
|
||||
run_pgupgrade
|
||||
}
|
||||
|
||||
# get_matched_files PATTERN DIR [DIR ...]
|
||||
# ---------------------------------------
|
||||
# Print all basenames for files matching PATTERN in DIRs.
|
||||
get_matched_files ()
|
||||
{
|
||||
local pattern=$1 dir
|
||||
shift
|
||||
for dir; do
|
||||
test -d "$dir" || continue
|
||||
find "$dir" -maxdepth 1 -type f -name "$pattern" -printf "%f\n"
|
||||
done
|
||||
}
|
||||
|
||||
# process_extending_files DIR [DIR ...]
|
||||
# -------------------------------------
|
||||
# Source all *.sh files in DIRs in alphabetical order, but if the file exists in
|
||||
# more then one DIR, source only the first occurrence (first found wins).
|
||||
process_extending_files()
|
||||
{
|
||||
local filename dir
|
||||
while read filename ; do
|
||||
for dir in "$@"; do
|
||||
local file="$dir/$filename"
|
||||
if test -f "$file"; then
|
||||
echo "=> sourcing $file ..."
|
||||
source "$file"
|
||||
break
|
||||
fi
|
||||
done
|
||||
done <<<"$(get_matched_files '*.sh' "$@" | sort -u)"
|
||||
}
|
||||
|
|
@ -0,0 +1,7 @@
|
|||
# required on master for replication
|
||||
wal_level = hot_standby # minimal, archive, hot_standby, or logical
|
||||
max_wal_senders = 6 # max number of walsender processes
|
||||
wal_keep_segments = 400 # in logfile segments, 16MB each; 0 disables
|
||||
|
||||
# required on replicas for replication
|
||||
hot_standby = on
|
||||
|
|
@ -0,0 +1,21 @@
|
|||
#
|
||||
# Custom OpenShift configuration.
|
||||
#
|
||||
# NOTE: This file is rewritten every time the container is started!
|
||||
# Changes to this file will be overwritten.
|
||||
#
|
||||
|
||||
# Listen on all interfaces.
|
||||
listen_addresses = '*'
|
||||
|
||||
# Determines the maximum number of concurrent connections to the database server. Default: 100
|
||||
max_connections = ${POSTGRESQL_MAX_CONNECTIONS}
|
||||
|
||||
# Allow each connection to use a prepared transaction
|
||||
max_prepared_transactions = ${POSTGRESQL_MAX_PREPARED_TRANSACTIONS}
|
||||
|
||||
# Sets the amount of memory the database server uses for shared memory buffers. Default: 32MB
|
||||
shared_buffers = ${POSTGRESQL_SHARED_BUFFERS}
|
||||
|
||||
# Sets the planner's assumption about the effective size of the disk cache that is available to a single query
|
||||
effective_cache_size = ${POSTGRESQL_EFFECTIVE_CACHE_SIZE}
|
||||
|
|
@ -0,0 +1,9 @@
|
|||
#
|
||||
# Custom OpenShift configuration.
|
||||
#
|
||||
# NOTE: This file is rewritten every time the container is started!
|
||||
# Changes to this file will be overwritten.
|
||||
#
|
||||
|
||||
standby_mode = on
|
||||
primary_conninfo = 'host=${MASTER_FQDN} port=5432 user=${POSTGRESQL_MASTER_USER} password=${POSTGRESQL_MASTER_PASSWORD}'
|
||||
3
root/usr/share/container-scripts/postgresql/scl_enable
Normal file
3
root/usr/share/container-scripts/postgresql/scl_enable
Normal file
|
|
@ -0,0 +1,3 @@
|
|||
# This will make scl collection binaries work out of box.
|
||||
unset BASH_ENV PROMPT_COMMAND ENV
|
||||
source scl_source enable $ENABLED_COLLECTIONS
|
||||
|
|
@ -0,0 +1,23 @@
|
|||
#!/bin/bash
|
||||
|
||||
_psql () { psql --set ON_ERROR_STOP=1 "$@" ; }
|
||||
|
||||
if [[ ",$postinitdb_actions," = *,simple_db,* ]]; then
|
||||
_psql --set=username="$POSTGRESQL_USER" \
|
||||
--set=password="$POSTGRESQL_PASSWORD" \
|
||||
<<< "ALTER USER :\"username\" WITH ENCRYPTED PASSWORD :'password';"
|
||||
fi
|
||||
|
||||
if [ -v POSTGRESQL_MASTER_USER ]; then
|
||||
_psql --set=masteruser="$POSTGRESQL_MASTER_USER" \
|
||||
--set=masterpass="$POSTGRESQL_MASTER_PASSWORD" \
|
||||
<<'EOF'
|
||||
ALTER USER :"masteruser" WITH REPLICATION;
|
||||
ALTER USER :"masteruser" WITH ENCRYPTED PASSWORD :'masterpass';
|
||||
EOF
|
||||
fi
|
||||
|
||||
if [ -v POSTGRESQL_ADMIN_PASSWORD ]; then
|
||||
_psql --set=adminpass="$POSTGRESQL_ADMIN_PASSWORD" \
|
||||
<<<"ALTER USER \"postgres\" WITH ENCRYPTED PASSWORD :'adminpass';"
|
||||
fi
|
||||
Loading…
Add table
Add a link
Reference in a new issue