Compare commits

...
Sign in to create a new pull request.

4 commits

Author SHA1 Message Date
Packit
955bf4f9ee [packit] 1.12.2 upstream release
Upstream tag: v1.12.2
Upstream commit: 497e95b3

Signed-off-by: Packit <hello@packit.dev>
2022-12-02 10:21:21 +00:00
Tibor Dudlák
6eb69814eb
Add spec readme and sources upstream version 1.12.1
Signed-off-by: Tibor Dudlák <tdudlak@redhat.com>
2022-11-29 15:02:24 +01:00
Tibor Dudlák
8e7299cf20
Add spec readme and sources upstream version 1.12.0
Signed-off-by: Tibor Dudlák <tdudlak@redhat.com>
2022-11-21 09:43:41 +01:00
Tibor Dudlák
e853f0116d
Add spec readme and sources upstream version 1.11.0
Signed-off-by: Tibor Dudlák <tdudlak@redhat.com>
2022-11-04 12:56:31 +01:00
6 changed files with 636 additions and 1 deletions

4
.gitignore vendored Normal file
View file

@ -0,0 +1,4 @@
/mrack-1.11.0.tar.gz
/mrack-1.12.0.tar.gz
/mrack-1.12.1.tar.gz
/mrack-1.12.2.tar.gz

51
.packit.yaml Normal file
View file

@ -0,0 +1,51 @@
---
# See the documentation for more information:
# https://packit.dev/docs/configuration/
specfile_path: mrack.spec
# add or remove files that should be synced
files_to_sync:
- mrack.spec
- README.md
- .packit.yaml
# name in upstream package repository/registry (e.g. in PyPI)
upstream_package_name: mrack
# downstream (Fedora) RPM package name
downstream_package_name: mrack
# mrack uses tag v(ver_number) and we need to make sure packit know what to expect
# see: https://packit.dev/docs/configuration/#upstream_tag_template
upstream_tag_template: "v{version}"
actions:
create-archive:
- "python3 setup.py sdist --dist-dir ."
- "sh -c 'echo mrack-$(python3 setup.py --version).tar.gz'"
get-current-version:
- "python3 setup.py --version"
jobs:
- job: copr_build
trigger: pull_request
targets: [fedora-all]
- job: copr_build
trigger: release
owner: "@freeipa"
project: neoave
targets: [fedora-all]
- job: propose_downstream
trigger: release
dist_git_branches:
- fedora-all
- job: koji_build
trigger: commit
dist_git_branches:
- fedora-all
- job: bodhi_update
trigger: commit
dist_git_branches:
- fedora-branched # rawhide updates are created automatically

303
README.md
View file

@ -1,3 +1,304 @@
# mrack
The mrack package
![pypi_badge](https://img.shields.io/pypi/v/mrack?label=PyPI&logo=pypi) ![readthedocs_badge](https://img.shields.io/readthedocs/mrack?label=Read%20the%20Docs&logo=read-the-docs) ![badge](https://copr.fedorainfracloud.org/coprs/g/freeipa/neoave/package/mrack/status_image/last_build.png)
Provisioning library for CI and local multi-host testing supporting multiple
provisioning providers e.g. OpenStack, libvirt, containers, Beaker).
But in comparison to multi-cloud libraries, the aim is to be able to describe
host from application perspective. E.g.:
```yaml
network: IPv4
hosts:
- name: master.testdomain.test
role: master
group: ipaserver
os: fedora-31
- name: client.testdomain.test
role: client
group: ipaclient
os: fedora-30
```
This is then combined with provisioning configuration where each provider has
defined meaning for `role`, `group`, `os` params which e.g. translates to
flavors, images, ...
```yaml
provider: openstack # default provider
openstack:
images:
fedora-30: Fedora-Cloud-Base-30-compose-latest
fedora-31: Fedora-Cloud-Base-31
flavors:
ipaserver: ci.m1.medium
ipaclient: ci.m1.micro
networks:
IPv4: net_ci_4
IPv6: net_ipv6_only
dual: net_cci_4_6
keypair: ipa_key
beaker:
distros:
fedora-30: FEDORA-30%
fodora-31: FEDORA-31%
```
So then user can quickly change provisioning location just by changing provider
name without needing to redefine the provider/cloud specifics. This is
especially useful for a lot of jobs sharing the same app specific provisioner
configuration.
```yaml
provider: beaker
```
Or each host can use a different provider:
```yaml
hosts:
- name: master.testdomain.test
role: master
group: ipaserver
os: fedora-31
provider: openstack
network: dual
- name: client.testdomain.test
role: client
group: ipaclient
os: fedora-30
provider: beaker
```
The output is then Ansible inventory with correct group mapping and information
which allows to SSH to the machines.
```yaml
all:
children:
ipaserver:
hosts:
f30-1.mrack.test: {}
f33-2.mrack.test: {}
hosts:
f30-1.mrack.test:
ansible_host: 10.0.154.21
ansible_python_interpreter: /usr/bin/python3
ansible_ssh_private_key_file: config/id_rsa
ansible_user: fedora
meta_dc_record: DC=mrack,DC=test
meta_domain: mrack.test
meta_fqdn: f30-1.mrack.test
meta_ip: 10.0.154.21
meta_os: fedora-30
meta_provider_id: 7c3c28f9-4674-4f7f-b413-00bc0b00d711
meta_restraint_id: 9
meta_role: master
f33-2.mrack.test:
ansible_host: ec2-13-15-16-1.eu-central-1.compute.amazonaws.com
ansible_python_interpreter: /usr/bin/python3
ansible_ssh_private_key_file: config/id_rsa
ansible_user: fedora
meta_dc_record: DC=mrack,DC=test
meta_domain: mrack.test
meta_fqdn: f33-2.mrack.test
meta_ip: 13.15.16.1
meta_os: fedora-33
meta_provider_id: i-08c0d4a86c4b7f7ef
meta_restraint_id: 1
meta_role: master
```
## Installation
mrack can be installed via pip, from [PyPI](https://pypi.org/project/mrack/):
```
pip install mrack
```
It is also available for Fedora 32+ via [COPR](https://copr.fedorainfracloud.org/coprs/g/freeipa/neoave/package/mrack/):
```
# enable the copr at your system
sudo dnf copr enable @freeipa/neoave
# install mrack package with all supported providers and cli
sudo dnf install mrack
# install mrack command and e.g. AWS provider
sudo dnf install mrack-cli python3-mrack-aws
# install mrack library to be later used as an import with support of OpenStack provider:
sudo dnf install python3-mrack-openstack
```
---
**NOTE**
Above command will install `mrack` package which depends on (contains) following packages:
- mrack-cli
- python3-mracklib
- python3-mrack-aws
- python3-mrack-beaker
- python3-mrack-openstack
- python3-mrack-podman
- python3-mrack-virt
The `python3-mracklib` is mandatory as it contains core mrack library.
To install only some of `mrack` providers you should install `python3-mrack-{provider_name}`.
To install mrack command a package `mrack-cli` is required.
---
## Run mrack
In order to use the mrack utility a mrack.conf (e.g. [mrack.conf] from the repository(repo/blob/main/src/mrack/data/mrack.conf)) is needed.
mrack looks for the config file in following order:
- `./` actual directory
- `~/.mrack/` home directory
- `/etc/mrack/` system directory
Values from the configuration file could be overriden using mrack utility
options `--mrack-config` `--provisioning-config` `--db` (for more see `mrack --help`).
```
Usage: mrack [OPTIONS] COMMAND [ARGS]...
Multihost human friendly provisioner.
Options:
-c, --mrack-config PATH
-p, --provisioning-config PATH
-d, --db PATH
--debug
--help Show this message and exit.
Commands:
destroy Destroy provisioned hosts.
list List host tracked by.
output Create outputs - such as Ansible inventory.
ssh SSH to host.
up Provision hosts.
```
### Provisioning with mrack
To provision system from the metadata.yaml either run:
```
mrack up
```
or use `up` command's option `--metadata`/`-m` to override path to the metadata file.
```
mrack up --metadata other-metadata.yaml
```
To return resources using mrack run:
```
mrack destroy
```
or use `destroy` command's option `--metadata`/`-m` to override path to the metadata file.
```
mrack destroy --metadata other-metadata.yaml
```
### mrack as python library
```python
# first set up authentication for each provider
# $ export AWS_CONFIG_FILE=`readlink -f ./aws.key`
from mrack.providers import providers
# import all supported providers:
from mrack.providers.aws import PROVISIONER_KEY as AWS
from mrack.providers.aws import AWSProvider
from mrack.providers.beaker import PROVISIONER_KEY as BEAKER
from mrack.providers.beaker import BeakerProvider
from mrack.providers.openstack import PROVISIONER_KEY as OPENSTACK
from mrack.providers.openstack import OpenStackProvider
from mrack.providers.podman import PROVISIONER_KEY as PODMAN
from mrack.providers.podman import PodmanProvider
from mrack.providers.static import PROVISIONER_KEY as STATIC
from mrack.providers.static import StaticProvider
from mrack.providers.virt import PROVISIONER_KEY as VIRT
from mrack.providers.virt import VirtProvider
# register all supported providers:
providers.register(AWS, AWSProvider)
providers.register(OPENSTACK, OpenStackProvider)
providers.register(BEAKER, BeakerProvider)
providers.register(PODMAN, PodmanProvider)
providers.register(STATIC, StaticProvider)
providers.register(VIRT, VirtProvider)
# use global context class
import mrack
global_context = mrack.context.global_context
# init global context with paths to files
mrack_config = "mrack.conf"
provisioning_config = "provisioning-config.yaml"
db_file = "mrackdb.json"
global_context.init(mrack_config, provisioning_config, db_file)
# load the metadata to global_context
metadata = "./metadata-f34.yaml"
global_context.init_metadata(metadata)
# pick default provider:
provider = "aws"
# create Up action
from mrack.actions.up import Up
up_action = Up()
await up_action.init(provider)
# provision machines:
await up_action.provision()
# store the output to db or just use the db later on
from mrack.actions.output import Output
output_action = Output()
await output_action.generate_outputs()
# work with machines...
# cleanup the machines:
from mrack.actions.destroy import Destroy
destroy_action = Destroy()
await destroy_action.destroy()
```
## Contribute
Projects is using [black](https://github.com/psf/black) formater and [isort](https://github.com/PyCQA/isort) to keep consistent
formatting, [flake8](https://flake8.pycqa.org/en/latest/) and
[pydocstyle](http://pycodestyle.pycqa.org/en/latest/intro.html) to ensure following
Python good practices.
Contributions (Pull Requests) are welcome. It is expected that they will pass tox tests and code checkers.
Inclusion of the unit tests for the new code is recommended.
Because of that we have configured [pre-commit](https://pre-commit.com/) hook.
Please enable the feature on your local system and use it before sending a patch.
It could save us lot of re-pushing to the PR.
### Black formatting and isort
Expected formatting can be achieved by running:
```
$ make format
```
Look into [black](https://github.com/psf/black) documentation for possible integration
in editors/IDEs.
### Testing
Just run tox to execute all tests and linters
```
$ tox
# or use make
$ make test
```

3
README.packit Normal file
View file

@ -0,0 +1,3 @@
This repository is maintained by packit.
https://packit.dev/
The file was generated using packit 0.63.2.dev8+g33fd64a.

275
mrack.spec Normal file
View file

@ -0,0 +1,275 @@
Name: mrack
Version: 1.12.2
Release: 1%{?dist}
Summary: Multicloud use-case based multihost async provisioner
License: Apache-2.0
URL: https://github.com/neoave/mrack
Source0: %{URL}/releases/download/v%{version}/%{name}-%{version}.tar.gz
BuildArch: noarch
BuildRequires: python3-devel
BuildRequires: python3-click
BuildRequires: python3-pyyaml
BuildRequires: python3-setuptools
# coma separated list of provider plugins
%global provider_plugins aws,beaker,openstack,podman,virt
Requires: %{name}-cli = %{version}-%{release}
Requires: python3-%{name}lib = %{version}-%{release}
Requires: python3-%{name}-aws = %{version}-%{release}
Requires: python3-%{name}-beaker = %{version}-%{release}
Requires: python3-%{name}-openstack = %{version}-%{release}
Requires: python3-%{name}-podman = %{version}-%{release}
Requires: python3-%{name}-virt = %{version}-%{release}
# We filter out the asyncopenstackclient dependency of this package
# so it is not forcing installation of missing dependencies in Fedora
# Once python3-AsyncOpenStackClient is in fedora we can drop this line
%global __requires_exclude asyncopenstackclient
%{?python_disable_dependency_generator}
%description
mrack is a provisioning tool and a library for CI and local multi-host
testing supporting multiple provisioning providers (e.g. AWS, Beaker,
Openstack). But in comparison to other multi-cloud libraries,
the aim is to be able to describe host from application perspective.
%package cli
Summary: Command line interface for mrack
Requires: python3-%{name}lib = %{version}-%{release}
Requires: python3-click
%package -n python3-%{name}lib
Summary: Core mrack libraries
Requires: python3-pyyaml
Recommends: python3-gssapi
Requires: sshpass
%{?python_provide:%python_provide python3-%{name}lib}
%package -n python3-%{name}-aws
Summary: AWS provider plugin for mrack
Requires: python3-%{name}lib = %{version}-%{release}
Requires: python3-boto3
Requires: python3-botocore
%{?python_provide:%python_provide python3-%{name}-aws}
%package -n python3-%{name}-beaker
Summary: Beaker provider plugin for mrack
Requires: python3-%{name}lib = %{version}-%{release}
Requires: beaker-client
%{?python_provide:%python_provide python3-%{name}-beaker}
%package -n python3-%{name}-openstack
Summary: Openstack provider plugin for mrack
Requires: python3-%{name}lib = %{version}-%{release}
Recommends: python3-AsyncOpenStackClient
%{?python_provide:%python_provide python3-%{name}-openstack}
%package -n python3-%{name}-podman
Summary: Podman provider plugin for mrack
Requires: python3-%{name}lib = %{version}-%{release}
Requires: podman
%{?python_provide:%python_provide python3-%{name}-podman}
%package -n python3-%{name}-virt
Summary: Virtualization provider plugin for mrack using testcloud
Requires: python3-%{name}lib = %{version}-%{release}
Requires: testcloud
%{?python_provide:%python_provide python3-%{name}-virt}
%description cli
%{name}-cli contains mrack command which functionality
can be extended by installing mrack plugins
%description -n python3-%{name}lib
python3-%{name}lib contains core mrack functionalities
and static provider which can be used as a library
%description -n python3-%{name}-aws
%{name}-aws is an additional plugin with AWS provisioning
library extending mrack package
%description -n python3-%{name}-beaker
%{name}-beaker is an additional plugin with Beaker provisioning
library extending mrack package
%description -n python3-%{name}-openstack
%{name}-openstack is an additional plugin with OpenStack provisioning
library extending mrack package
%description -n python3-%{name}-podman
%{name}-podman is an additional plugin with Podman provisioning
library extending mrack package
%description -n python3-%{name}-virt
%{name}-virt is an additional plugin with Virualization provisioning
library extending mrack package using testcloud
%prep
%autosetup -p1 -n %{name}-%{version}
# Remove bundled egg-info
rm -r src/%{name}.egg-info
%build
%py3_build
%install
%py3_install
%files
%license LICENSE
%doc README.md
%doc CHANGELOG.md
%files cli
# the mrack man page RFE: https://github.com/neoave/mrack/issues/197
%license LICENSE
%doc README.md
%doc CHANGELOG.md
%{_bindir}/%{name}
%{python3_sitelib}/%{name}/{,__pycache__/}run.*
%files -n python3-%{name}lib
%license LICENSE
%doc README.md
%doc CHANGELOG.md
%{python3_sitelib}/%{name}
%{python3_sitelib}/%{name}-%{version}-py%{python3_version}.egg-info
%exclude %{python3_sitelib}/%{name}/{,__pycache__/}run.*
%exclude %{python3_sitelib}/%{name}/providers/utils/{,__pycache__/}osapi.*
%exclude %{python3_sitelib}/%{name}/providers/utils/{,__pycache__/}testcloud.*
%exclude %{python3_sitelib}/%{name}/providers/utils/{,__pycache__/}podman.*
%exclude %{python3_sitelib}/%{name}/providers/{,__pycache__/}{%{provider_plugins}}.*
%exclude %{python3_sitelib}/%{name}/transformers/{,__pycache__/}{%{provider_plugins}}.*
%files -n python3-%{name}-aws
%{python3_sitelib}/%{name}/transformers/{,__pycache__/}aws.*
%{python3_sitelib}/%{name}/providers/{,__pycache__/}aws.*
%files -n python3-%{name}-beaker
%{python3_sitelib}/%{name}/transformers/{,__pycache__/}beaker.*
%{python3_sitelib}/%{name}/providers/{,__pycache__/}beaker.*
%files -n python3-%{name}-openstack
%{python3_sitelib}/%{name}/transformers/{,__pycache__/}openstack.*
%{python3_sitelib}/%{name}/providers/{,__pycache__/}openstack.*
%{python3_sitelib}/%{name}/providers/utils/{,__pycache__/}osapi.*
%files -n python3-%{name}-podman
%{python3_sitelib}/%{name}/transformers/{,__pycache__/}podman.*
%{python3_sitelib}/%{name}/providers/{,__pycache__/}podman.*
%{python3_sitelib}/%{name}/providers/utils/{,__pycache__/}podman.*
%files -n python3-%{name}-virt
%{python3_sitelib}/%{name}/transformers/{,__pycache__/}virt.*
%{python3_sitelib}/%{name}/providers/{,__pycache__/}virt.*
%{python3_sitelib}/%{name}/providers/utils/{,__pycache__/}testcloud.*
%changelog
* Fri Dec 02 2022 Packit <hello@packit.dev> - 1.12.2-1
- chore: Release version 1.12.2 (github-actions)
- chore: Use python 3.10 in GH actions (Tibor Dudlák)
- refactor: pylint fixes related to Python 3.10 (Tibor Dudlák)
- test: Fix test_utils.py to be included in pytest run (Tibor Dudlák)
- chore(pytest): add missing python_path when using pytest >=7.0.0 (Tibor Dudlák)
- test: Add test for value_to_bool util function (Tibor Dudlák)
- fix: Owner requirement boolean parsing from string (Tibor Dudlák)
- chore(Packit): Add upstream_tag_template to .packit.yaml (Tibor Dudlák)
* Thu Nov 24 2022 Tibor Dudlák <tdudlak@redhat.com> - 1.12.1-1
- Released upstream version 1.12.1
* Mon Nov 14 2022 Tibor Dudlák <tdudlak@redhat.com> - 1.12.0-1
- Released upstream version 1.12.0
* Thu Nov 03 2022 Tibor Dudlák <tdudlak@redhat.com> - 1.11.0-1
- Released upstream version 1.11.0
* Wed Oct 26 2022 Tibor Dudlák <tdudlak@redhat.com> - 1.10.0-1
- Released upstream version 1.10.0
* Thu Oct 20 2022 Tibor Dudlák <tdudlak@redhat.com> - 1.9.1-1
- Released upstream version 1.9.1
* Wed Oct 12 2022 Tibor Dudlák <tdudlak@redhat.com> - 1.9.0-1
- Released upstream version 1.9.0
* Mon Oct 10 2022 Tibor Dudlák <tdudlak@redhat.com> - 1.8.1-1
- Released upstream version 1.8.1
* Mon Oct 10 2022 Tibor Dudlák <tdudlak@redhat.com> - 1.8.0-1
- Released upstream version 1.8.0
* Tue Sep 20 2022 Tibor Dudlák <tdudlak@redhat.com> - 1.7.0-1
- Released upstream version 1.7.0
* Wed Jul 27 2022 Tibor Dudlák <tdudlak@redhat.com> - 1.6.0-1
- Released upstream version 1.6.0
* Fri Jul 08 2022 Tibor Dudlák <tdudlak@redhat.com> - 1.5.0-1
- Released upstream version 1.5.0
* Fri Jun 17 2022 David Pascual Hernandez <davherna@redhat.com> - 1.4.1-1
- Released upstream version 1.4.1
* Thu May 05 2022 Tibor Dudlák <tdudlak@redhat.com> - 1.4.0-1
- Released upstream version 1.4.0
* Tue Apr 05 2022 Tibor Dudlák <tdudlak@redhat.com> - 1.3.1-1
- Released upstream version 1.3.1
* Fri Apr 01 2022 David Pascual Hernandez <davherna@redhat.com> - 1.3.0-1
- Released upstream version 1.3.0
* Wed Dec 15 2021 Tibor Dudlák <tdudlak@redhat.com> - 1.2.0-1
- Released upstream version 1.2.0
* Thu Nov 25 2021 Tibor Dudlák <tdudlak@redhat.com> - 1.1.1-1
- Released upstream version 1.1.1
* Tue Nov 23 2021 Tibor Dudlák <tdudlak@redhat.com> - 1.1.0-1
- Released upstream version 1.1.0
* Fri Sep 03 2021 Tibor Dudlák <tdudlak@redhat.com> - 1.0.0-1
- Released upstream version 1.0.0
* Thu Jul 01 2021 Tibor Dudlák <tdudlak@redhat.com> - 0.14.0-1
- Released upstream version 0.14.0
* Tue Jun 08 2021 Francisco Triviño <ftrivino@redhat.com> - 0.13.0-1
- Released upstream version 0.13.0
* Thu May 13 2021 Tibor Dudlák <tdudlak@redhat.com> - 0.12.0-1
- Released upstream version 0.12.0
* Fri May 07 2021 Tibor Dudlák <tdudlak@redhat.com> - 0.11.0-1
- Released upstream version 0.11.0
* Fri Apr 30 2021 Bhavik Bhavsar <bbhavsar@redhat.com> - 0.10.0-1
- Released upstream version 0.10.0
* Mon Apr 19 2021 Armando Neto <abiagion@redhat.com> - 0.9.0-1
- Released upstream version 0.9.0
* Thu Apr 15 2021 Armando Neto <abiagion@redhat.com> - 0.8.0-1
- Released upstream version 0.8.0
* Tue Mar 23 2021 Armando Neto <abiagion@redhat.com> - 0.7.1-1
- Released upstream version 0.7.1
* Mon Mar 22 2021 Tibor Dudlák <tdudlak@redhat.com> - 0.7.0-1
- Released upstream version 0.7.0
* Thu Feb 04 2021 Armando Neto <abiagion@redhat.com> - 0.6.0-1
- Initial package.

1
sources Normal file
View file

@ -0,0 +1 @@
SHA512 (mrack-1.12.2.tar.gz) = 7ac6ec24398d58cb092e4a12c15c75758485a5e430c74cbe3312a48dde905f0350029b70245d8b21e6bb060b3022a16657b25bd080ced9de1f193159a2752405