From d7482d0008b3ba5e63c6739f48ca7d2e92903c90 Mon Sep 17 00:00:00 2001 From: Daniel Bodky Date: Tue, 21 Nov 2023 13:23:15 +0100 Subject: [PATCH 01/39] Updates available roles --- doc/getting-started.md | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/doc/getting-started.md b/doc/getting-started.md index 6fd1ce35..17ab2c68 100644 --- a/doc/getting-started.md +++ b/doc/getting-started.md @@ -1,11 +1,19 @@ ### Getting Started -The collection includes two roles in the current version. +The collection includes six roles in the current version. * icinga.repos: Role to manage repositories * [Documentation: doc/role-repos](role-repos/role-repos.md) * icinga.icinga2: Role to install and manage Icinga 2 instances. * [Documentation: doc/role-icinga2](role-icinga2/role-icinga2.md) +* icinga.icingadb: Role to install and manage IcingaDB, Icinga2's new data backend. + * [Documentation: doc/role-icingadb](role-icingadb/role-icingadb.md) +* icinga.icingadb_redis: Role to install and manage Redis, IcingaDB's cache backend. + * [Documentation: doc/role-icingadb_redis](role-icingadb_redis/role-icingadb_redis.md) +* icinga.icingaweb2: Role to install and manage Icinga Web 2. + * [Documentation: doc/role-icingaweb2](role-icingaweb2/role-icingaweb2.md) +* icinga.monitoring_plugins: Role to install and manage Icinga2 compatible monitoring plugins. + * [Documentation: doc/role-monitoring_plugins](role-monitoring_plugins/role-monitoring_plugins.md) --- From 6a8b34fd815ee45e3247ea14b9d8406ea3a82c2a Mon Sep 17 00:00:00 2001 From: Daniel Bodky Date: Tue, 21 Nov 2023 13:39:54 +0100 Subject: [PATCH 02/39] Adds references to the database examples to relevant parts of the docs --- doc/getting-started.md | 30 ++++++++++++++++++++++++++ doc/role-icingadb/role-icingadb.md | 7 +++++- doc/role-icingaweb2/role-icingaweb2.md | 4 ++++ 3 files changed, 40 insertions(+), 1 deletion(-) diff --git a/doc/getting-started.md b/doc/getting-started.md index 17ab2c68..851df2a7 100644 --- a/doc/getting-started.md +++ b/doc/getting-started.md @@ -47,6 +47,36 @@ ansible-galaxy collection build ansible-collection-icinga ansible-galaxy collection install icinga-icinga-0.3.0.tar.gz ``` +## Databases + +Icinga2 relies on relational databases for many parts of its functionality. **None** of those databases gets installed by the roles. You need to install and configure them yourself. For doing so, there are many ways available, e.g. the Ansible role [geerlingguy.mysql](https://galaxy.ansible.com/geerlingguy/mysql) for MySQL flavour (both MySQL and MariaDB) or [geerlingguy.postgresql](https://galaxy.ansible.com/geerlingguy/postgresql) PostGresQL: + +```yaml +- name: Configure databases for Icinga2 + hosts: database + vars: + mysql_databases: + - name: icingadb + - name: icingaweb + - name: vspheredb + encoding: utf8mb4 + collation: utf8mb4_unicode_ci + - name: director + mysql_users: + - name: icingadb-user + host: localhost + password: icingadb-password + priv: "icingadb.*:ALL" + [...] + roles: + - role: geerlingguy.mysql +``` + +> [!NOTE] +> Schema migrations needed for the respective Icinga components to work will be handled either by the respective roles or by the Icinga components themselves. + + + ## Example Playbooks This is an example on how to install an Icinga 2 server/master instance. diff --git a/doc/role-icingadb/role-icingadb.md b/doc/role-icingadb/role-icingadb.md index 5d66a32a..e30742ab 100644 --- a/doc/role-icingadb/role-icingadb.md +++ b/doc/role-icingadb/role-icingadb.md @@ -5,7 +5,12 @@ This role installs and configures the IcingaDB daemon. In addition it can also i It serves as the official, more performant successor to Icinga IDO. More information about its purpose and design can be found [in the official documentation](https://icinga.com/docs/icinga-db/latest/doc/01-About/). -> :information_source: In many scenarios you want to install the [icingadb_redis role](../role-icingadb_redis/) together with this role. It is part of this collection, too. +> [!TIP] +> In many scenarios you want to install the [icingadb_redis role](../role-icingadb_redis/) together with this role. It is part of this collection, too. + +## Database + +IcingaDB relies on a relational database to persist received data. This database **won't** be created by this role - you need to deploy and configure one in advance. For more information, see the [Databases](../getting-started.md#databases) section in the getting started guide. ## Variables diff --git a/doc/role-icingaweb2/role-icingaweb2.md b/doc/role-icingaweb2/role-icingaweb2.md index 4faf1ae7..bff00b7c 100644 --- a/doc/role-icingaweb2/role-icingaweb2.md +++ b/doc/role-icingaweb2/role-icingaweb2.md @@ -7,6 +7,10 @@ The role icingaweb2 installs and configures Icinga Web 2 and its modules. * [IcingaDB](./module-icingadb.md) * [Monitoring](./module-monitoring.md) +## Databases + +Icingaweb2 and some of its modules rely on a relational database to persist data. These databases **won't** be created by this role - you need to deploy and configure them in advance. For more information, see the [Databases](../getting-started.md#databases) section in the getting started guide. + ## Variables ### Icinga Web 2 DB Configuration From cfdf26f79a9974143efc1962301da71389919f09 Mon Sep 17 00:00:00 2001 From: Daniel Bodky Date: Tue, 21 Nov 2023 13:52:59 +0100 Subject: [PATCH 03/39] Fixes typos --- doc/getting-started.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/getting-started.md b/doc/getting-started.md index 851df2a7..8a5b6767 100644 --- a/doc/getting-started.md +++ b/doc/getting-started.md @@ -49,7 +49,7 @@ ansible-galaxy collection install icinga-icinga-0.3.0.tar.gz ## Databases -Icinga2 relies on relational databases for many parts of its functionality. **None** of those databases gets installed by the roles. You need to install and configure them yourself. For doing so, there are many ways available, e.g. the Ansible role [geerlingguy.mysql](https://galaxy.ansible.com/geerlingguy/mysql) for MySQL flavour (both MySQL and MariaDB) or [geerlingguy.postgresql](https://galaxy.ansible.com/geerlingguy/postgresql) PostGresQL: +Icinga2 relies on relational databases for many parts of its functionality. **None** of those databases get installed by the roles. You need to install and configure them yourself. For doing so, there are many ways available, e.g. the Ansible role [geerlingguy.mysql](https://galaxy.ansible.com/geerlingguy/mysql) for MySQL flavours (both MySQL and MariaDB) or [geerlingguy.postgresql](https://galaxy.ansible.com/geerlingguy/postgresql) for PostGresQL: ```yaml - name: Configure databases for Icinga2 From 462dcc3876f187701208446ad5301744ff75a4aa Mon Sep 17 00:00:00 2001 From: Thilo W Date: Thu, 30 Nov 2023 09:53:42 +0100 Subject: [PATCH 04/39] Fix variable matching with ansible_lsb.id Ref #224 --- roles/repos/tasks/main.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/roles/repos/tasks/main.yml b/roles/repos/tasks/main.yml index d0bd32d8..2cee1fe1 100644 --- a/roles/repos/tasks/main.yml +++ b/roles/repos/tasks/main.yml @@ -7,7 +7,7 @@ - "{{ ansible_os_family }}-{{ ansible_distribution }}-{{ ansible_distribution_major_version }}.yml" - "{{ ansible_os_family }}-{{ ansible_distribution_major_version }}.yml" - "{{ ansible_os_family }}-{{ ansible_distribution }}.yml" - - "{{ ansible_os_family }}-{{ ansible_lsb.id }}.yml" + - "{{ ansible_os_family }}-{{ ansible_lsb.id if ansible_lsb.id is defined else ansible_distribution }}.yml" - "{{ ansible_os_family }}.yml" - default.yml paths: From b1c66bd0353de3102f545733db487fee7a3b545e Mon Sep 17 00:00:00 2001 From: Thilo W Date: Thu, 30 Nov 2023 17:11:03 +0100 Subject: [PATCH 05/39] add changelog --- CHANGELOG.rst | 12 ++++++++++-- changelogs/changelog.yaml | 9 +++++++++ 2 files changed, 19 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.rst b/CHANGELOG.rst index 2ba6d5be..c76f6a46 100644 --- a/CHANGELOG.rst +++ b/CHANGELOG.rst @@ -5,13 +5,21 @@ Icinga.Icinga Release Notes .. contents:: Topics -v0.3.1 +v0.3.2 ====== Release Summary --------------- -This is a bugfix release +Bugfix Release + +Bugfixes +-------- + +- Role repos: Fix bug in variable search - thanks to @gianmarco-mameli #224 + +v0.3.1 +====== Major Changes ------------- diff --git a/changelogs/changelog.yaml b/changelogs/changelog.yaml index 98f3c3c8..cda5e461 100644 --- a/changelogs/changelog.yaml +++ b/changelogs/changelog.yaml @@ -110,3 +110,12 @@ releases: - trivial_naming_tasks.yml - trivial_wrong_variable_name.yml release_date: '2023-11-21' + 0.3.2: + changes: + bugfixes: + - 'Role repos: Fix bug in variable search - thanks to @gianmarco-mameli #224' + release_summary: Bugfix Release + fragments: + - bugfix_variable_search.yml + - release.yml + release_date: '2023-11-30' From c49eb792b095af53fc551bd085de74e829509a49 Mon Sep 17 00:00:00 2001 From: Thilo W Date: Thu, 30 Nov 2023 17:11:19 +0100 Subject: [PATCH 06/39] remove old fragments From 0afe3a5ae3e6029ec9424b6dc62bd96c5bfb8fad Mon Sep 17 00:00:00 2001 From: Thilo W Date: Thu, 30 Nov 2023 17:12:14 +0100 Subject: [PATCH 07/39] bump to version 0.3.2 --- galaxy.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/galaxy.yml b/galaxy.yml index 445591a3..d1d8d8fb 100644 --- a/galaxy.yml +++ b/galaxy.yml @@ -1,6 +1,6 @@ namespace: icinga name: icinga -version: 0.3.1 +version: 0.3.2 readme: README.md authors: - Lennart Betz From 4a1ad942ae670866d661bb1f0bbec3493c432a9c Mon Sep 17 00:00:00 2001 From: Daniel Bodky Date: Thu, 7 Dec 2023 14:38:28 +0100 Subject: [PATCH 08/39] Updates documentation for IcingaDB schema management --- doc/role-icingadb/role-icingadb.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/doc/role-icingadb/role-icingadb.md b/doc/role-icingadb/role-icingadb.md index e30742ab..c8c3604b 100644 --- a/doc/role-icingadb/role-icingadb.md +++ b/doc/role-icingadb/role-icingadb.md @@ -25,7 +25,7 @@ For more information on the respective settings please see [the official documen | `icingadb_database_ca` | `String` | Defines the path to the certificate authority for the TLS connection. | **n/a** | | `icingadb_database_cert` | `String` | Defines the path to the certificate for client key authentication. | **n/a** | | `icingadb_database_host` | `String` | Defines database address to connect to. | `127.0.0.1` | -| `icingadb_database_import_schema` | `bool` | Defines whether to import the schema into the database or not. | `false` | +| `icingadb_database_import_schema` | `bool` | Defines whether to import the schema into the database or not. **Needs `icingadb_database_type` to be set**. | `false` | | `icingadb_database_key` | `String` | Defines the path to the certificate key for client key authentication. | **n/a** | | `icingadb_database_name` | `String` | Defines the database to connect to. | `icingadb` | | `icingadb_database_password` | `String` | Defines the database password to connect with. | `icingadb` | @@ -81,6 +81,7 @@ This play installs IcingaDB with on the same host as its connected MysQL databas become: true vars: icingadb_database_import_schema: true # Import the schema into the database + icingadb_database_type: mysql # needed by the schema import roles: - role: icinga.icinga.icingadb From 1e0d3528709d2ee552483ea995dd362e1a877a80 Mon Sep 17 00:00:00 2001 From: Daniel Bodky Date: Wed, 22 Nov 2023 17:16:36 +0100 Subject: [PATCH 09/39] Adjusts installation of director to work from source --- roles/icingaweb2/tasks/modules/director.yml | 45 ++++++++++++++++----- 1 file changed, 36 insertions(+), 9 deletions(-) diff --git a/roles/icingaweb2/tasks/modules/director.yml b/roles/icingaweb2/tasks/modules/director.yml index e68d7698..0edc1a67 100644 --- a/roles/icingaweb2/tasks/modules/director.yml +++ b/roles/icingaweb2/tasks/modules/director.yml @@ -1,3 +1,4 @@ +--- - name: Module Director | Ensure config directory ansible.builtin.file: state: directory @@ -18,7 +19,7 @@ - kickstart - config -- name: Module Director | Check for pending migrations +- name: Module Director | Check for pending migrations # noqa: command-instead-of-shell ansible.builtin.shell: cmd: icingacli director migration pending register: _pending @@ -26,12 +27,12 @@ failed_when: _pending.stdout|length > 0 when: vars['icingaweb2_modules']['director']['import_schema'] is defined and vars['icingaweb2_modules']['director']['import_schema'] and vars['icingaweb2_modules']['director']['config'] is defined -- name: Module Director | Apply pending migrations +- name: Module Director | Apply pending migrations # noqa: command-instead-of-shell ansible.builtin.shell: cmd: icingacli director migration run when: vars['icingaweb2_modules']['director']['import_schema'] is defined and vars['icingaweb2_modules']['director']['import_schema'] and vars['icingaweb2_modules']['director']['config'] is defined and _pending.rc|int == 0 -- name: Module Director | Check if kickstart is required +- name: Module Director | Check if kickstart is required # noqa: command-instead-of-shell ansible.builtin.shell: cmd: icingacli director kickstart required register: _required @@ -39,13 +40,39 @@ failed_when: _required.rc|int >= 2 when: vars['icingaweb2_modules']['director']['run_kickstart'] is defined and vars['icingaweb2_modules']['director']['run_kickstart'] and vars['icingaweb2_modules']['director']['kickstart'] is defined -- name: Module Director | Check if kickstart is required +- name: Module Director | Check if kickstart is required # noqa: command-instead-of-shell ansible.builtin.shell: cmd: icingacli director kickstart run when: vars['icingaweb2_modules']['director']['run_kickstart'] is defined and vars['icingaweb2_modules']['director']['run_kickstart'] and vars['icingaweb2_modules']['director']['kickstart'] is defined and _required.rc|int == 0 -- name: Module Director | Ensure daemon is running - ansible.builtin.service: - name: "{{ icingaweb2_director_service }}" - state: started - enabled: yes +- name: Module Director | Ensure installation from source is complete + when: icingaweb2_modules['director']['source'] == 'git' + block: + - name: Module Director | Ensure daemon user exists + ansible.builtin.user: + name: icingadirector + state: present + shell: /bin/nologin + system: yes + home: /var/lib/icingadirector + group: "{{ icingaweb2_group }}" + when: icingaweb2_modules['director']['source'] == 'git' # Only required for installations from source + + - name: Module Director | Ensure home directory exists + ansible.builtin.file: + state: directory + dest: /var/lib/icingadirector + owner: icingadirector + group: "{{ icingaweb2_group }}" + mode: "0750" + when: icingaweb2_modules['director']['source'] == 'git' # Only required for installations from source + + - name: Module Director | Ensure systemd unit file exists + ansible.builtin.copy: + src: "{{ icingaweb2_config.global.module_path }}/director/contrib/systemd/icinga-director.service" + dest: /etc/systemd/system/icingadirector.service + owner: root + group: root + mode: "0644" + remote_src: yes + when: icingaweb2_modules['director']['source'] == 'git' # Only required for installations from source From d16a87a6795d9ef9cd70309e4732da0ae5d77b64 Mon Sep 17 00:00:00 2001 From: Daniel Bodky Date: Wed, 22 Nov 2023 17:25:20 +0100 Subject: [PATCH 10/39] Adds changelog --- .../fragments/feature_adjust_director_source_installation.yml | 3 +++ 1 file changed, 3 insertions(+) create mode 100644 changelogs/fragments/feature_adjust_director_source_installation.yml diff --git a/changelogs/fragments/feature_adjust_director_source_installation.yml b/changelogs/fragments/feature_adjust_director_source_installation.yml new file mode 100644 index 00000000..b80bb855 --- /dev/null +++ b/changelogs/fragments/feature_adjust_director_source_installation.yml @@ -0,0 +1,3 @@ +--- +minor_change: + - Adjusted the installation of the director module when using the source installation. From 814d701658df6b49fed2b1bcb6351a364fb09134 Mon Sep 17 00:00:00 2001 From: Daniel Bodky Date: Thu, 7 Dec 2023 15:05:53 +0100 Subject: [PATCH 11/39] Removes redundant 'when' entries --- roles/icingaweb2/tasks/modules/director.yml | 3 --- 1 file changed, 3 deletions(-) diff --git a/roles/icingaweb2/tasks/modules/director.yml b/roles/icingaweb2/tasks/modules/director.yml index 0edc1a67..e60159d4 100644 --- a/roles/icingaweb2/tasks/modules/director.yml +++ b/roles/icingaweb2/tasks/modules/director.yml @@ -56,7 +56,6 @@ system: yes home: /var/lib/icingadirector group: "{{ icingaweb2_group }}" - when: icingaweb2_modules['director']['source'] == 'git' # Only required for installations from source - name: Module Director | Ensure home directory exists ansible.builtin.file: @@ -65,7 +64,6 @@ owner: icingadirector group: "{{ icingaweb2_group }}" mode: "0750" - when: icingaweb2_modules['director']['source'] == 'git' # Only required for installations from source - name: Module Director | Ensure systemd unit file exists ansible.builtin.copy: @@ -75,4 +73,3 @@ group: root mode: "0644" remote_src: yes - when: icingaweb2_modules['director']['source'] == 'git' # Only required for installations from source From 96dccf7456b7d5d47d303902755339ffe47954e0 Mon Sep 17 00:00:00 2001 From: Daniel Bodky Date: Thu, 23 Nov 2023 14:34:00 +0100 Subject: [PATCH 12/39] Adds possibility to delegate ticket creation to satellites --- .../feature_add_satellite_delegation.yml | 3 +++ doc/role-icinga2/features/feature-api.md | 15 +++++++++++++-- roles/icinga2/tasks/features/api.yml | 3 ++- 3 files changed, 18 insertions(+), 3 deletions(-) create mode 100644 changelogs/fragments/feature_add_satellite_delegation.yml diff --git a/changelogs/fragments/feature_add_satellite_delegation.yml b/changelogs/fragments/feature_add_satellite_delegation.yml new file mode 100644 index 00000000..f67ad698 --- /dev/null +++ b/changelogs/fragments/feature_add_satellite_delegation.yml @@ -0,0 +1,3 @@ +--- +minor_changes: + - Added possibility to delegate ticket creation to satellites diff --git a/doc/role-icinga2/features/feature-api.md b/doc/role-icinga2/features/feature-api.md index 3e3327de..85ebf785 100644 --- a/doc/role-icinga2/features/feature-api.md +++ b/doc/role-icinga2/features/feature-api.md @@ -52,13 +52,24 @@ Create Signing Request to get a certificate managed by the parameter `ca_host` a set to the master/server hostname, FQDN or IP, the node setup tries to connect via API an retrieve the trusted certificate. -> **_NOTE:_** Ansible will delegate the ticket creation to the CA host. You can change this behaviour by setting 'icinga2_delegate_host' to match another Ansible alias. +> [!INFO] +> Ansible will delegate the ticket creation to the CA host. You can change this behaviour by setting 'icinga2_delegate_host' to match another Ansible alias. -``` +```yaml ca_host: icinga-server.localdomain ca_host_port: 5665 ``` +> [!INFO] +> In case your agent can't connect to the CA host, you can use the variables `icinga2_delegate_host` +> and `ticket_salt` to delegate ticket creation to one of your satellites instead. + +```yaml +ca_host: icinga-server.localdomain +icinga2_delegate_host: icinga-satellite.localdomain +ticket_salt: "{{ icinga2_constants.ticket_salt }}" +``` + By default the FQDN is used as certificate common name, to put a name yourself: diff --git a/roles/icinga2/tasks/features/api.yml b/roles/icinga2/tasks/features/api.yml index 7480487c..6a34b451 100644 --- a/roles/icinga2/tasks/features/api.yml +++ b/roles/icinga2/tasks/features/api.yml @@ -12,6 +12,7 @@ icinga2_ssl_cert: "{{ icinga2_dict_features.api.ssl_cert | default(omit) }}" icinga2_ssl_cacert: "{{ icinga2_dict_features.api.ssl_cacert | default(omit) }}" icinga2_ssl_key: "{{ icinga2_dict_features.api.ssl_key | default(omit) }}" + icinga2_ticket_salt: "{{ icinga2_dict_features.api.ticket_salt | default(omit) }}" - assert: that: ((icinga2_ssl_cacert is defined and icinga2_ssl_cert is defined and icinga2_ssl_key is defined) or (icinga2_ssl_cacert is undefined and icinga2_ssl_cert is undefined and icinga2_ssl_key is undefined and icinga2_ca_host is defined)) @@ -135,7 +136,7 @@ {% if icinga2_ca_host != 'none' %} --cert "{{ icinga2_cert_path }}/{{ icinga2_cert_name }}.crt" {% else %} --csr "{{ icinga2_cert_path }}/{{ icinga2_cert_name }}.csr" {%- endif %} - name: delegate ticket request to master - shell: icinga2 pki ticket --cn "{{ icinga2_cert_name }}" + shell: icinga2 pki ticket --cn "{{ icinga2_cert_name }}{% if icinga2_ticket_salt is defined %} --salt {{ icinga2_ticket_salt }}{% endif %}}" delegate_to: "{{ icinga2_delegate_host | default(icinga2_ca_host) }}" register: icinga2_ticket when: icinga2_ca_host != 'none' From bd3dc250e47da637622f91974b1c14e28a693505 Mon Sep 17 00:00:00 2001 From: Daniel Bodky Date: Thu, 7 Dec 2023 15:51:57 +0100 Subject: [PATCH 13/39] Prepare release v0.3.2 --- CHANGELOG.rst | 8 +++++++- changelogs/changelog.yaml | 7 ++++++- changelogs/fragments/feature_add_satellite_delegation.yml | 3 --- .../feature_adjust_director_source_installation.yml | 3 --- 4 files changed, 13 insertions(+), 8 deletions(-) delete mode 100644 changelogs/fragments/feature_add_satellite_delegation.yml delete mode 100644 changelogs/fragments/feature_adjust_director_source_installation.yml diff --git a/CHANGELOG.rst b/CHANGELOG.rst index c76f6a46..cd088bab 100644 --- a/CHANGELOG.rst +++ b/CHANGELOG.rst @@ -11,7 +11,13 @@ v0.3.2 Release Summary --------------- -Bugfix Release +This is a bugfix release, bringing two QOL features and a fix for the installation process of some of the roles which broke with v0.3.1. + +Minor Changes +------------- + +- Added possibility to delegate ticket creation to satellites +- Adjusted the installation of the director module when using the source installation. Bugfixes -------- diff --git a/changelogs/changelog.yaml b/changelogs/changelog.yaml index cda5e461..266657ac 100644 --- a/changelogs/changelog.yaml +++ b/changelogs/changelog.yaml @@ -114,8 +114,13 @@ releases: changes: bugfixes: - 'Role repos: Fix bug in variable search - thanks to @gianmarco-mameli #224' + minor_changes: + - Added possibility to delegate ticket creation to satellites + - Adjusted the installation of the director module when using the source installation. release_summary: Bugfix Release fragments: - bugfix_variable_search.yml + - feature_add_satellite_delegation.yml + - feature_adjust_director_source_installation.yml - release.yml - release_date: '2023-11-30' + release_date: '2023-12-07' diff --git a/changelogs/fragments/feature_add_satellite_delegation.yml b/changelogs/fragments/feature_add_satellite_delegation.yml deleted file mode 100644 index f67ad698..00000000 --- a/changelogs/fragments/feature_add_satellite_delegation.yml +++ /dev/null @@ -1,3 +0,0 @@ ---- -minor_changes: - - Added possibility to delegate ticket creation to satellites diff --git a/changelogs/fragments/feature_adjust_director_source_installation.yml b/changelogs/fragments/feature_adjust_director_source_installation.yml deleted file mode 100644 index b80bb855..00000000 --- a/changelogs/fragments/feature_adjust_director_source_installation.yml +++ /dev/null @@ -1,3 +0,0 @@ ---- -minor_change: - - Adjusted the installation of the director module when using the source installation. From 5f2e504189e2ab2e0e4ac684d31b5dc21ffa138e Mon Sep 17 00:00:00 2001 From: Daniel Bodky Date: Tue, 21 Nov 2023 13:40:51 +0100 Subject: [PATCH 14/39] add installation for x509 module (#214) * add module x509 and mysql imports task * Add documentation for x509 module * Add documentation about database imports * Continues working on x509 module installation --------- Co-authored-by: Thilo W --- .../feature_add_x509_module_installation.yml | 3 + doc/role-icingaweb2/module-x509.md | 95 +++++++++++++++++++ roles/icingaweb2/tasks/main.yml | 8 ++ .../icingaweb2/tasks/manage_mysql_imports.yml | 38 ++++++++ roles/icingaweb2/tasks/modules/x509.yml | 66 +++++++++++++ roles/icingaweb2/vars/main.yml | 3 +- 6 files changed, 212 insertions(+), 1 deletion(-) create mode 100644 changelogs/fragments/feature_add_x509_module_installation.yml create mode 100644 doc/role-icingaweb2/module-x509.md create mode 100644 roles/icingaweb2/tasks/manage_mysql_imports.yml create mode 100644 roles/icingaweb2/tasks/modules/x509.yml diff --git a/changelogs/fragments/feature_add_x509_module_installation.yml b/changelogs/fragments/feature_add_x509_module_installation.yml new file mode 100644 index 00000000..82c1f981 --- /dev/null +++ b/changelogs/fragments/feature_add_x509_module_installation.yml @@ -0,0 +1,3 @@ +--- +major_changes: + - Added Installation of x509 certificate monitoring model diff --git a/doc/role-icingaweb2/module-x509.md b/doc/role-icingaweb2/module-x509.md new file mode 100644 index 00000000..684000fd --- /dev/null +++ b/doc/role-icingaweb2/module-x509.md @@ -0,0 +1,95 @@ +## Module x509 + +### Variables and Configuration + +The general module parameter like `enabled` and `source` can be applied here. + +| Variable | Value | +|----------|------------| +| enabled | true/false | +| source | package | + +#### Section configuration + +The backend database for the module needs to be available and configured at the `icingaweb2_resources` variable. + +``` +icingaweb2_modules: + x509: + source: package + enabled: true + config: + backend: + resource: x509 +``` + +#### Configure SNI Names. + +To configure SNIs for a IP address, use the dictionary `sni`. + +Example: + +``` +icingaweb2_modules: + x509: + source: package + enabled: true + config: + backend: + resource: x509 + sni: + 192.168.56.213: + hostnames: + - icinga.com + - test2.icinga.com +``` + +#### Import Certificates + +To import certificates use the **list** `certificate_files` all files need to be +available locally beforehand. + +``` +icingaweb2_modules: + x509: + source: package + enabled: true + config: + backend: + resource: x509 + certificate_files: + - /etc/ssl/certs/ca-certificates.crt +``` + +#### Database Schema Setup + +To import the database schema use `database` dictionary with the following variables. + +| Variable | Type | Description | Default | +|----------|------|-------------|---------| +| `import_schema` | `Boolean` | Defines wether the schema will be imported or not. | false | +| `host` | `String` | Defines database address to connect to. | `localhost` | +| `port` | `int` | Defines the database port to connect to. | `3306` or `5432` | +| `user` | `string` | Defines database user | `x509` | +| `name` | `String` | Defines the database to connect to. | `x509` | +| `password` | `String` | Defines the database password to connect with. | OMITTED | +| `ssl_mode` | `String` | Clients attempt to connect using encryption, falling back to an unencrypted connection if an encrypted connection cannot be established |**n/a** | +|`ssl_ca`| `String`| Defines the path to the ca certificate for client authentication. | **n/a** | +|`ssl_cert`|`String`| Defines the path to the certificate for client authentication. | **n/a** | +|`ssl_key`| `String` | Defines the path to the certificate key for client key authentication. | **n/a** | +|`ssl_cipher`|`String`| Ciphers for the client authentication. | **n/a** | +|`ssl_extra_options`|`String`| Extra options for the client authentication. | **n/a** | + + +``` +icingaweb2_modules: + x509: + source: package + enabled: true + database: + import_schema: true + host: localhost + port: 3306 + user: x509 + password: secret +``` diff --git a/roles/icingaweb2/tasks/main.yml b/roles/icingaweb2/tasks/main.yml index e6bd7b6b..a020d4cd 100644 --- a/roles/icingaweb2/tasks/main.yml +++ b/roles/icingaweb2/tasks/main.yml @@ -43,3 +43,11 @@ force: yes when: icingaweb2_modules is defined loop: "{{ icingaweb2_modules | dict2items }}" + +# Many daemons fail before e.g. the resource is set up or the schema hasn't been migrated. This is a workaround. +- name: Manage enabled module daemons + ansible.builtin.service: + name: "icinga-{{ item.key }}" + state: restarted + when: icingaweb2_modules is defined and item.value.enabled|bool == true and item.key in ['vspheredb', 'x509'] + loop: "{{ icingaweb2_modules | dict2items }}" diff --git a/roles/icingaweb2/tasks/manage_mysql_imports.yml b/roles/icingaweb2/tasks/manage_mysql_imports.yml new file mode 100644 index 00000000..676c6df2 --- /dev/null +++ b/roles/icingaweb2/tasks/manage_mysql_imports.yml @@ -0,0 +1,38 @@ +--- +- name: Check Database Credentials + ansible.builtin.assert: + that: + - _db['user'] is defined + - _db['password'] is defined + fail_msg: "No database credentials defined." + +- name: Build mysql command + ansible.builtin.set_fact: + _tmp_mysqlcmd: >- + mysql {% if _db['host'] | default('localhost') != 'localhost' %} -h "{{ _db['host'] }}" {%- endif %} + {% if _db['port'] is defined %} -P "{{ _db['port'] }}" {%- endif %} + {% if _db['ssl_mode'] is defined %} --ssl-mode "{{ _db['ssl_mode'] }}" {%- endif %} + {% if _db['ssl_ca'] is defined %} --ssl-ca "{{ _db['ssl_ca'] }}" {%- endif %} + {% if _db['ssl_cert'] is defined %} --ssl-cert "{{ _db['ssl_cert'] }}" {%- endif %} + {% if _db['ssl_key'] is defined %} --ssl-key "{{ _db['ssl_key'] }}" {%- endif %} + {% if _db['ssl_cipher'] is defined %} --ssl-cipher "{{ _db['ssl_cipher'] }}" {%- endif %} + {% if _db['ssl_extra_options'] is defined %} {{ _db['ssl_extra_options'] }} {%- endif %} + -u "{{ _db['user'] }}" + -p"{{ _db['password'] }}" + "{{ _db['name'] }}" + +- name: MySQL check for db schema + ansible.builtin.shell: > + {{ _tmp_mysqlcmd }} + -Ns -e "{{ _db['select_query'] }}" + failed_when: false + changed_when: false + check_mode: false + register: _db_schema + +- name: MySQL import db schema + ansible.builtin.shell: > + {{ _tmp_mysqlcmd }} + < {{ _db['schema_path'] }} + when: _db_schema.rc != 0 + run_once: yes diff --git a/roles/icingaweb2/tasks/modules/x509.yml b/roles/icingaweb2/tasks/modules/x509.yml new file mode 100644 index 00000000..a0bc7e25 --- /dev/null +++ b/roles/icingaweb2/tasks/modules/x509.yml @@ -0,0 +1,66 @@ +- name: Module x509 | Ensure config directory + ansible.builtin.file: + state: directory + dest: "{{ icingaweb2_modules_config_dir }}/{{ _module }}" + owner: "{{ icingaweb2_httpd_user }}" + group: "{{ icingaweb2_group }}" + mode: "2770" + vars: + _module: "{{ item.key }}" + +- name: Module x509 | Manage config files + ansible.builtin.include_tasks: manage_module_config.yml + loop: "{{ _files }}" + loop_control: + loop_var: _file + when: vars['icingaweb2_modules'][_module][_file] is defined + vars: + _module: "{{ item.key }}" + _files: + - config + - sni + +- name: Module x509 | Manage Schema + block: + - name: Module x509 | Prepare _db informations + ansible.builtin.set_fact: + _db: + host: "{{ vars['icingaweb2_modules'][_module]['database']['host'] | default('localhost') }}" + port: "{{ vars['icingaweb2_modules'][_module]['database']['port'] | default('3306') }}" + user: "{{ vars['icingaweb2_modules'][_module]['database']['user'] | default('x509') }}" + password: "{{ vars['icingaweb2_modules'][_module]['database']['password'] | default(omit) }}" + name: "{{ vars['icingaweb2_modules'][_module]['database']['name'] | default('x509') }}" + ssl_mode: "{{ vars['icingaweb2_modules'][_module]['database']['ssl_mode'] | default(omit) }}" + ssl_ca: "{{ vars['icingaweb2_modules'][_module]['database']['ssl_ca'] | default(omit) }}" + ssl_cert: "{{ vars['icingaweb2_modules'][_module]['database']['ssl_cert'] | default(omit) }}" + ssl_key: "{{ vars['icingaweb2_modules'][_module]['database']['ssl_key'] | default(omit) }}" + ssl_cipher: "{{ vars['icingaweb2_modules'][_module]['database']['ssl_cipher'] | default(omit) }}" + ssl_extra_options: "{{ vars['icingaweb2_modules'][_module]['database']['ssl_extra_options'] | default(omit) }}" + schema_path: /usr/share/icingaweb2/modules/x509/schema/mysql.schema.sql + select_query: "select * from x509_certificate" + when: vars['icingaweb2_modules'][_module]['database']['type'] | default('mysql') == 'mysql' + + - ansible.builtin.fail: + fail_msg: "The Database type select is not supported, {{ vars['icingaweb2_modules'][_module]['database']['type'] }} [Supported=mysql]" + when: vars['icingaweb2_modules'][_module]['database']['type'] is defined and vars['icingaweb2_modules'][_module]['database']['type'] != 'mysql' + + - name: Module x509 | Import Schema + ansible.builtin.include_tasks: ../manage_mysql_imports.yml + + - name: Module x509 | empty _db var + ansible.builtin.set_fact: + _db: {} + when: vars['icingaweb2_modules'][_module]['database']['import_schema'] | default(false) + vars: + _module: "{{ item.key }}" + +- name: Module x509 | Import Certificates + ansible.builtin.shell: > + icingacli {{ _module }} import --file {{ _file }} + loop: "{{ vars['icingaweb2_modules'][_module]['certificate_files'] }}" + loop_control: + loop_var: _file + vars: + _module: "{{ item.key }}" + when: vars['icingaweb2_modules'][_module]['certificate_files'] is defined + changed_when: false diff --git a/roles/icingaweb2/vars/main.yml b/roles/icingaweb2/vars/main.yml index 8092fd97..588f0d6d 100644 --- a/roles/icingaweb2/vars/main.yml +++ b/roles/icingaweb2/vars/main.yml @@ -2,4 +2,5 @@ icingaweb2_module_packages: icingadb: icingadb-web director: icinga-director - businessprocess: icinga-businessprocess \ No newline at end of file + x509: icinga-x509 + businessprocess: icinga-businessprocess From 16a9aab13d73b4735248c54106dd9e3304168e30 Mon Sep 17 00:00:00 2001 From: rolatsch Date: Thu, 14 Dec 2023 15:39:46 +0100 Subject: [PATCH 15/39] remove superfluous curly brace (#246) --- roles/icinga2/tasks/features/api.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/roles/icinga2/tasks/features/api.yml b/roles/icinga2/tasks/features/api.yml index 6a34b451..0767de69 100644 --- a/roles/icinga2/tasks/features/api.yml +++ b/roles/icinga2/tasks/features/api.yml @@ -136,7 +136,7 @@ {% if icinga2_ca_host != 'none' %} --cert "{{ icinga2_cert_path }}/{{ icinga2_cert_name }}.crt" {% else %} --csr "{{ icinga2_cert_path }}/{{ icinga2_cert_name }}.csr" {%- endif %} - name: delegate ticket request to master - shell: icinga2 pki ticket --cn "{{ icinga2_cert_name }}{% if icinga2_ticket_salt is defined %} --salt {{ icinga2_ticket_salt }}{% endif %}}" + shell: icinga2 pki ticket --cn "{{ icinga2_cert_name }}{% if icinga2_ticket_salt is defined %} --salt {{ icinga2_ticket_salt }}{% endif %}" delegate_to: "{{ icinga2_delegate_host | default(icinga2_ca_host) }}" register: icinga2_ticket when: icinga2_ca_host != 'none' From b8c4b1615426760670824ac25924c2b142a92ecb Mon Sep 17 00:00:00 2001 From: Daniel Bodky Date: Thu, 21 Dec 2023 14:07:10 +0100 Subject: [PATCH 16/39] Adjusts variable lookups (#248) * Fixes lookup of passwords * Adjusts variable lookup to only use vars[...] when needed * Adds changelog fragment --- .../minor_change_adjust_vars_lookup.yml | 3 ++ .../tasks/modules/businessprocess.yml | 2 +- roles/icingaweb2/tasks/modules/director.yml | 8 ++--- .../tasks/modules/manage_module_config.yml | 2 +- roles/icingaweb2/tasks/modules/x509.yml | 34 +++++++++---------- 5 files changed, 26 insertions(+), 23 deletions(-) create mode 100644 changelogs/fragments/minor_change_adjust_vars_lookup.yml diff --git a/changelogs/fragments/minor_change_adjust_vars_lookup.yml b/changelogs/fragments/minor_change_adjust_vars_lookup.yml new file mode 100644 index 00000000..9ba14a47 --- /dev/null +++ b/changelogs/fragments/minor_change_adjust_vars_lookup.yml @@ -0,0 +1,3 @@ +--- +bugfixes: + - Adjusted the way variables get looked up from `vars['varname']` to `varname` in most places. diff --git a/roles/icingaweb2/tasks/modules/businessprocess.yml b/roles/icingaweb2/tasks/modules/businessprocess.yml index 1bcee37a..72c1f5b1 100644 --- a/roles/icingaweb2/tasks/modules/businessprocess.yml +++ b/roles/icingaweb2/tasks/modules/businessprocess.yml @@ -21,7 +21,7 @@ src: "files/{{ _file.src_path }}" dest: "{{ icingaweb2_modules_config_dir }}/{{ item.key }}/processes/{{ _file.name }}" when: vars['icingaweb2_modules'][_module]['custom_process_files'] is defined - loop: "{{ vars['icingaweb2_modules'][_module]['custom_process_files'] }}" + loop: "{{ icingaweb2_modules[_module].custom_process_files }}" loop_control: loop_var: _file vars: diff --git a/roles/icingaweb2/tasks/modules/director.yml b/roles/icingaweb2/tasks/modules/director.yml index e60159d4..d3f0bf18 100644 --- a/roles/icingaweb2/tasks/modules/director.yml +++ b/roles/icingaweb2/tasks/modules/director.yml @@ -25,12 +25,12 @@ register: _pending changed_when: _pending.rc|int == 0 failed_when: _pending.stdout|length > 0 - when: vars['icingaweb2_modules']['director']['import_schema'] is defined and vars['icingaweb2_modules']['director']['import_schema'] and vars['icingaweb2_modules']['director']['config'] is defined + when: vars['icingaweb2_modules']['director']['import_schema'] is defined and icingaweb2_modules.director.import_schema and vars['icingaweb2_modules']['director']['config'] is defined - name: Module Director | Apply pending migrations # noqa: command-instead-of-shell ansible.builtin.shell: cmd: icingacli director migration run - when: vars['icingaweb2_modules']['director']['import_schema'] is defined and vars['icingaweb2_modules']['director']['import_schema'] and vars['icingaweb2_modules']['director']['config'] is defined and _pending.rc|int == 0 + when: vars['icingaweb2_modules']['director']['import_schema'] is defined and icingaweb2_modules.director.import_schema and vars['icingaweb2_modules']['director']['config'] is defined and _pending.rc|int == 0 - name: Module Director | Check if kickstart is required # noqa: command-instead-of-shell ansible.builtin.shell: @@ -38,12 +38,12 @@ register: _required changed_when: _required.rc|int == 0 failed_when: _required.rc|int >= 2 - when: vars['icingaweb2_modules']['director']['run_kickstart'] is defined and vars['icingaweb2_modules']['director']['run_kickstart'] and vars['icingaweb2_modules']['director']['kickstart'] is defined + when: vars['icingaweb2_modules']['director']['run_kickstart'] is defined and icingaweb2_modules.director.run_kickstart and vars['icingaweb2_modules']['director']['kickstart'] is defined - name: Module Director | Check if kickstart is required # noqa: command-instead-of-shell ansible.builtin.shell: cmd: icingacli director kickstart run - when: vars['icingaweb2_modules']['director']['run_kickstart'] is defined and vars['icingaweb2_modules']['director']['run_kickstart'] and vars['icingaweb2_modules']['director']['kickstart'] is defined and _required.rc|int == 0 + when: vars['icingaweb2_modules']['director']['run_kickstart'] is defined and icingaweb2_modules.director.run_kickstart and vars['icingaweb2_modules']['director']['kickstart'] is defined and _required.rc|int == 0 - name: Module Director | Ensure installation from source is complete when: icingaweb2_modules['director']['source'] == 'git' diff --git a/roles/icingaweb2/tasks/modules/manage_module_config.yml b/roles/icingaweb2/tasks/modules/manage_module_config.yml index c2f774d3..1570d52e 100644 --- a/roles/icingaweb2/tasks/modules/manage_module_config.yml +++ b/roles/icingaweb2/tasks/modules/manage_module_config.yml @@ -1,6 +1,6 @@ - name: Module {{ _module }} | Set file content as hash ansible.builtin.set_fact: - _i2_config_hash: "{{ lookup('list', vars['icingaweb2_modules'][_module][_file]) }}" + _i2_config_hash: "{{ lookup('list', icingaweb2_modules[_module][_file]) }}" - name: Module {{ _module }} | Write config file {{ _file }}.ini ansible.builtin.template: diff --git a/roles/icingaweb2/tasks/modules/x509.yml b/roles/icingaweb2/tasks/modules/x509.yml index a0bc7e25..fa49a8c3 100644 --- a/roles/icingaweb2/tasks/modules/x509.yml +++ b/roles/icingaweb2/tasks/modules/x509.yml @@ -25,24 +25,24 @@ - name: Module x509 | Prepare _db informations ansible.builtin.set_fact: _db: - host: "{{ vars['icingaweb2_modules'][_module]['database']['host'] | default('localhost') }}" - port: "{{ vars['icingaweb2_modules'][_module]['database']['port'] | default('3306') }}" - user: "{{ vars['icingaweb2_modules'][_module]['database']['user'] | default('x509') }}" - password: "{{ vars['icingaweb2_modules'][_module]['database']['password'] | default(omit) }}" - name: "{{ vars['icingaweb2_modules'][_module]['database']['name'] | default('x509') }}" - ssl_mode: "{{ vars['icingaweb2_modules'][_module]['database']['ssl_mode'] | default(omit) }}" - ssl_ca: "{{ vars['icingaweb2_modules'][_module]['database']['ssl_ca'] | default(omit) }}" - ssl_cert: "{{ vars['icingaweb2_modules'][_module]['database']['ssl_cert'] | default(omit) }}" - ssl_key: "{{ vars['icingaweb2_modules'][_module]['database']['ssl_key'] | default(omit) }}" - ssl_cipher: "{{ vars['icingaweb2_modules'][_module]['database']['ssl_cipher'] | default(omit) }}" - ssl_extra_options: "{{ vars['icingaweb2_modules'][_module]['database']['ssl_extra_options'] | default(omit) }}" + host: "{{ icingaweb2_modules[_module].database.host | default('localhost') }}" + port: "{{ icingaweb2_modules[_module].database.port | default('3306') }}" + user: "{{ icingaweb2_modules[_module].database.user | default('x509') }}" + password: "{{ icingaweb2_modules[_module].database.password | default(omit) }}" + name: "{{ icingaweb2_modules[_module].database.name | default('x509') }}" + ssl_mode: "{{ icingaweb2_modules[_module].database.ssl_mode | default(omit) }}" + ssl_ca: "{{ icingaweb2_modules[_module].database.ssl_ca | default(omit) }}" + ssl_cert: "{{ icingaweb2_modules[_module].database.ssl_cert | default(omit) }}" + ssl_key: "{{ icingaweb2_modules[_module].database.ssl_key | default(omit) }}" + ssl_cipher: "{{ icingaweb2_modules[_module].database.ssl_cipher | default(omit) }}" + ssl_extra_options: "{{ icingaweb2_modules[_module].database.ssl_extra_options | default(omit) }}" schema_path: /usr/share/icingaweb2/modules/x509/schema/mysql.schema.sql select_query: "select * from x509_certificate" - when: vars['icingaweb2_modules'][_module]['database']['type'] | default('mysql') == 'mysql' + when: icingaweb2_modules[_module].database.type | default('mysql') == 'mysql' - ansible.builtin.fail: - fail_msg: "The Database type select is not supported, {{ vars['icingaweb2_modules'][_module]['database']['type'] }} [Supported=mysql]" - when: vars['icingaweb2_modules'][_module]['database']['type'] is defined and vars['icingaweb2_modules'][_module]['database']['type'] != 'mysql' + fail_msg: "The Database type select is not supported, {{ icingaweb2_modules[_module].database.type }} [Supported=mysql]" + when: vars['icingaweb2_modules'][_module]['database']['type'] is defined and icingaweb2_modules[_module].database.type != 'mysql' - name: Module x509 | Import Schema ansible.builtin.include_tasks: ../manage_mysql_imports.yml @@ -50,17 +50,17 @@ - name: Module x509 | empty _db var ansible.builtin.set_fact: _db: {} - when: vars['icingaweb2_modules'][_module]['database']['import_schema'] | default(false) + when: icingaweb2_modules[_module].database.import_schema | default(false) vars: _module: "{{ item.key }}" - name: Module x509 | Import Certificates ansible.builtin.shell: > icingacli {{ _module }} import --file {{ _file }} - loop: "{{ vars['icingaweb2_modules'][_module]['certificate_files'] }}" + loop: "{{ icingaweb2_modules[_module].certificate_files }}" loop_control: loop_var: _file vars: _module: "{{ item.key }}" - when: vars['icingaweb2_modules'][_module]['certificate_files'] is defined + when: icingaweb2_modules[_module].certificate_files is defined changed_when: false From 69f708bfaef75e3ba21ffd1c50f8f630fc7e16db Mon Sep 17 00:00:00 2001 From: Johannes Kastl Date: Fri, 22 Dec 2023 08:01:22 +0100 Subject: [PATCH 17/39] icingaweb2: run pqslcmd with LANG=C to ensure the output is in english * fix run pqslcmd with LANG=C to ensure the output is in english and we can match for '(0 rows)' in the next task (#241) --- roles/icingaweb2/tasks/manage_icingaweb_pgsql_db.yml | 1 + 1 file changed, 1 insertion(+) diff --git a/roles/icingaweb2/tasks/manage_icingaweb_pgsql_db.yml b/roles/icingaweb2/tasks/manage_icingaweb_pgsql_db.yml index 44c7252f..6a6f1461 100644 --- a/roles/icingaweb2/tasks/manage_icingaweb_pgsql_db.yml +++ b/roles/icingaweb2/tasks/manage_icingaweb_pgsql_db.yml @@ -45,6 +45,7 @@ block: - name: PostgreSQL check for icingaweb admin user ansible.builtin.shell: > + LANG=C {{ _tmp_pgsqlcmd }} -w -c "select name from icingaweb_user where name like '{{ icingaweb2_admin_username }}'" failed_when: false From ff0fcfdc70b41887f959ec80cc7aff41d97ad90e Mon Sep 17 00:00:00 2001 From: Thilo W Date: Fri, 22 Dec 2023 14:52:33 +0100 Subject: [PATCH 18/39] Add changelog fragments --- changelogs/fragments/minor_changes.yml | 4 ++++ 1 file changed, 4 insertions(+) create mode 100644 changelogs/fragments/minor_changes.yml diff --git a/changelogs/fragments/minor_changes.yml b/changelogs/fragments/minor_changes.yml new file mode 100644 index 00000000..a75f2b57 --- /dev/null +++ b/changelogs/fragments/minor_changes.yml @@ -0,0 +1,4 @@ +--- +bugfixes: + - "icingaweb2: run pqslcmd with LANG=C to ensure the output is in english." + - remove superfluous curly brace (#246) From 21e8d3b8adbf8d1820a8ca532a7d8ae0a5588ea5 Mon Sep 17 00:00:00 2001 From: Fl0w Date: Fri, 29 Dec 2023 11:47:25 +0100 Subject: [PATCH 19/39] added missing quotes to ticket delegation --- roles/icinga2/tasks/features/api.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/roles/icinga2/tasks/features/api.yml b/roles/icinga2/tasks/features/api.yml index 0767de69..e4c85da7 100644 --- a/roles/icinga2/tasks/features/api.yml +++ b/roles/icinga2/tasks/features/api.yml @@ -136,7 +136,7 @@ {% if icinga2_ca_host != 'none' %} --cert "{{ icinga2_cert_path }}/{{ icinga2_cert_name }}.crt" {% else %} --csr "{{ icinga2_cert_path }}/{{ icinga2_cert_name }}.csr" {%- endif %} - name: delegate ticket request to master - shell: icinga2 pki ticket --cn "{{ icinga2_cert_name }}{% if icinga2_ticket_salt is defined %} --salt {{ icinga2_ticket_salt }}{% endif %}" + shell: icinga2 pki ticket --cn "{{ icinga2_cert_name }}" {% if icinga2_ticket_salt is defined %} --salt "{{ icinga2_ticket_salt }}"{% endif %} delegate_to: "{{ icinga2_delegate_host | default(icinga2_ca_host) }}" register: icinga2_ticket when: icinga2_ca_host != 'none' From bf024ca46a6e2d2a5f369ec7f06b9b5dcd5a0fb6 Mon Sep 17 00:00:00 2001 From: Fl0w Date: Fri, 29 Dec 2023 15:17:00 +0000 Subject: [PATCH 20/39] added changelog fragment --- changelogs/fragments/fix_missing_quotes_delegate_ticket.yml | 3 +++ 1 file changed, 3 insertions(+) create mode 100644 changelogs/fragments/fix_missing_quotes_delegate_ticket.yml diff --git a/changelogs/fragments/fix_missing_quotes_delegate_ticket.yml b/changelogs/fragments/fix_missing_quotes_delegate_ticket.yml new file mode 100644 index 00000000..1951fdf9 --- /dev/null +++ b/changelogs/fragments/fix_missing_quotes_delegate_ticket.yml @@ -0,0 +1,3 @@ +--- +bugfixes: + - "icinga2 feature api: fixed missing quotes in delegate ticket command for satellites or second master nodes." \ No newline at end of file From b93d1462e57b31d2791c49605c1c97f0e63d1cf5 Mon Sep 17 00:00:00 2001 From: Thilo W Date: Thu, 4 Jan 2024 16:34:36 +0100 Subject: [PATCH 21/39] Fix/advanced ldap filters (#252) * Fix quoting in template and quote if "!" is in chars * add icingaweb2_groups syntax for testing * added molecule test for ini_templating * added pyinilint as ini validator --- .github/workflows/build.yml | 3 +- .../test_icingaweb2_ini_template.yml | 62 +++++++++++++++++++ .../fragments/fix_advanced_ldap_filters.yml | 3 + changelogs/fragments/minor_changes.yml | 3 + .../ini-configuration-tests/collections.yml | 4 ++ molecule/ini-configuration-tests/converge.yml | 46 ++++++++++++++ molecule/ini-configuration-tests/molecule.yml | 26 ++++++++ .../tests/integration/test_ini_config.py | 29 +++++++++ molecule/local-default/converge.yml | 6 ++ roles/icingaweb2/templates/ini_template.j2 | 4 +- 10 files changed, 182 insertions(+), 4 deletions(-) create mode 100644 .github/workflows/test_icingaweb2_ini_template.yml create mode 100644 changelogs/fragments/fix_advanced_ldap_filters.yml create mode 100644 molecule/ini-configuration-tests/collections.yml create mode 100644 molecule/ini-configuration-tests/converge.yml create mode 100644 molecule/ini-configuration-tests/molecule.yml create mode 100644 molecule/ini-configuration-tests/tests/integration/test_ini_config.py diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index 692a967c..1fe79b42 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -12,8 +12,7 @@ on: paths: - roles/** - plugins/** - - molecule/** - - tests/** + - molecule/default/** pull_request: branches: - 'feature/**' diff --git a/.github/workflows/test_icingaweb2_ini_template.yml b/.github/workflows/test_icingaweb2_ini_template.yml new file mode 100644 index 00000000..ff12ab85 --- /dev/null +++ b/.github/workflows/test_icingaweb2_ini_template.yml @@ -0,0 +1,62 @@ +name: Icingaweb2 Templates +on: + push: + tags: + - '*' + branches: + - main + - 'feature/**' + - 'fix/**' + - '!doc/**' + paths: + - 'roles/icingaweb2/templates/**' + - 'molecule/ini-configuration-tests/**' + pull_request: + branches: + - 'feature/**' + - 'fix/**' + - '!doc/**' + +jobs: + test_ini_template: + runs-on: ubuntu-latest + + env: + COLLECTION_NAMESPACE: icinga + COLLECTION_NAME: icinga + + strategy: + fail-fast: false + max-parallel: 1 + matrix: + distro: [ubuntu2204] + python: ['3.9', '3.10'] + ansible: ['2.13.10', '2.14.7'] + scenario: [ini-configuration-tests] + + steps: + - name: Check out code + uses: actions/checkout@v4 + + - name: Set up Python + uses: actions/setup-python@v4 + with: + python-version: ${{ matrix.python }} + + - name: Install dependencies ansible + run: | + python3 -m pip install --upgrade pip + python3 -m pip install -r requirements-test-${{ matrix.ansible }}.txt + + - name: Install collection + run: | + mkdir -p ~/.ansible/collections/ansible_collections/$COLLECTION_NAMESPACE + cp -a ../ansible-collection-$COLLECTION_NAME ~/.ansible/collections/ansible_collections/$COLLECTION_NAMESPACE/$COLLECTION_NAME + + - name: Test with molecule + run: | + ansible --version + molecule --version + molecule test -s ${{ matrix.scenario }} + env: + MOLECULE_DISTRO: ${{ matrix.distro }} diff --git a/changelogs/fragments/fix_advanced_ldap_filters.yml b/changelogs/fragments/fix_advanced_ldap_filters.yml new file mode 100644 index 00000000..2a44c7aa --- /dev/null +++ b/changelogs/fragments/fix_advanced_ldap_filters.yml @@ -0,0 +1,3 @@ +bugfixes: + - "Fix quoting for ! in templating Issue #208" + - "Replaced quote filter from ini template" diff --git a/changelogs/fragments/minor_changes.yml b/changelogs/fragments/minor_changes.yml index a75f2b57..72380758 100644 --- a/changelogs/fragments/minor_changes.yml +++ b/changelogs/fragments/minor_changes.yml @@ -2,3 +2,6 @@ bugfixes: - "icingaweb2: run pqslcmd with LANG=C to ensure the output is in english." - remove superfluous curly brace (#246) +minor_changes: + - added tests for icingaweb2 ini template + - added pyinilint as ini validator after templates diff --git a/molecule/ini-configuration-tests/collections.yml b/molecule/ini-configuration-tests/collections.yml new file mode 100644 index 00000000..bed6991a --- /dev/null +++ b/molecule/ini-configuration-tests/collections.yml @@ -0,0 +1,4 @@ +collections: + - name: community.docker + - name: community.general + version: ">=2,<3" diff --git a/molecule/ini-configuration-tests/converge.yml b/molecule/ini-configuration-tests/converge.yml new file mode 100644 index 00000000..66498112 --- /dev/null +++ b/molecule/ini-configuration-tests/converge.yml @@ -0,0 +1,46 @@ +--- +- name: Converge + hosts: all + vars: + test_cases: + - name: string + _i2_config_hash: + section: + test: string + - name: number + _i2_config_hash: + section: + test: 10 + - name: advanced_filter + _i2_config_hash: + section: + test: '!(objectClass=user)' + test2: "!(objectClass=user)" + test3: "!attribute" + - name: list + _i2_config_hash: + section: + test: + - "foo" + - bar + - 'baz' + - name: equal_sign + _i2_config_hash: + section: + test: equal=sign + + + collections: + - icinga.icinga + tasks: + - ansible.builtin.pip: + name: pyinilint + state: present + + - ansible.builtin.template: + src: "{{ lookup('ansible.builtin.env', 'MOLECULE_PROJECT_DIRECTORY', default=Undefined) }}/roles/icingaweb2/templates/modules_config.ini.j2" + dest: "/tmp/{{ item.name }}" + validate: pyinilint %s + loop: "{{ test_cases }}" + vars: + _i2_config_hash: "{{ item._i2_config_hash }}" diff --git a/molecule/ini-configuration-tests/molecule.yml b/molecule/ini-configuration-tests/molecule.yml new file mode 100644 index 00000000..3f13ab2d --- /dev/null +++ b/molecule/ini-configuration-tests/molecule.yml @@ -0,0 +1,26 @@ +--- +dependency: + name: galaxy +driver: + name: docker +platforms: + - name: icinga-default + image: "geerlingguy/docker-${MOLECULE_DISTRO:-ubuntu2204}-ansible:latest" + command: ${MOLECULE_DOCKER_COMMAND:-""} + volumes: + - /sys/fs/cgroup:/sys/fs/cgroup:rw + cgroupns_mode: host + privileged: true + pre_build_image: true +provisioner: + name: ansible + inventory: + link: + host_vars: host_vars/ +verifier: + name: testinfra + directory: tests/integration/ +lint: | + set -e + yamllint --no-warnings roles/ + ansible-lint roles/ diff --git a/molecule/ini-configuration-tests/tests/integration/test_ini_config.py b/molecule/ini-configuration-tests/tests/integration/test_ini_config.py new file mode 100644 index 00000000..f138ce08 --- /dev/null +++ b/molecule/ini-configuration-tests/tests/integration/test_ini_config.py @@ -0,0 +1,29 @@ +def test_string(host): + i2_file = host.file("/tmp/string") + print(i2_file.content_string) + assert i2_file.is_file + assert i2_file.content_string == "\n[section]\ntest = string\n" + +def test_number(host): + i2_file = host.file("/tmp/number") + print(i2_file.content_string) + assert i2_file.is_file + assert i2_file.content_string == '\n[section]\ntest = "10"\n' + +def test_list(host): + i2_file = host.file("/tmp/list") + print(i2_file.content_string) + assert i2_file.is_file + assert i2_file.content_string == '\n[section]\ntest = "foo, bar, baz"\n' + +def test_advanced_filter(host): + i2_file = host.file("/tmp/advanced_filter") + print(i2_file.content_string) + assert i2_file.is_file + assert i2_file.content_string == "\n[section]\ntest = '!(objectClass=user)'\ntest2 = '!(objectClass=user)'\ntest3 = '!attribute'\n" + +def test_equal_sign(host): + i2_file = host.file("/tmp/equal_sign") + print(i2_file.content_string) + assert i2_file.is_file + assert i2_file.content_string == "\n[section]\ntest = 'equal=sign'\n" diff --git a/molecule/local-default/converge.yml b/molecule/local-default/converge.yml index b1b2da74..de6656ef 100644 --- a/molecule/local-default/converge.yml +++ b/molecule/local-default/converge.yml @@ -5,6 +5,12 @@ ansible.builtin.apt: cache_valid_time: 3600 vars: + icingaweb2_groups: + icingaweb2_group_ldap: + resource: icingaweb2_ldap + user_backend: icingaweb2_user_ldap + group_class: group + group_filter: '!(objectClass=user)' icingaweb2_resources: director_db: type: db diff --git a/roles/icingaweb2/templates/ini_template.j2 b/roles/icingaweb2/templates/ini_template.j2 index 9e80e30d..21b44bc2 100644 --- a/roles/icingaweb2/templates/ini_template.j2 +++ b/roles/icingaweb2/templates/ini_template.j2 @@ -7,8 +7,8 @@ {{ option }} = "{{ value }}" {% elif value is iterable and (value is not string and value is not mapping) %} {{ option }} = "{{ value | join(', ') }}" -{% elif value is string and "=" in value %} -{{ option }} = "{{ value | quote }}" +{% elif ( value is string and ( "=" in value or "!" in value ) )%} +{{ option }} = '{{ value }}' {% else %} {{ option }} = {{ value }} {% endif %} From 68ba3065af8dd45fec60874f85580af4457d2286 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Thu, 4 Jan 2024 15:35:20 +0000 Subject: [PATCH 22/39] Bump actions/setup-python from 4 to 5 Bumps [actions/setup-python](https://github.com/actions/setup-python) from 4 to 5. - [Release notes](https://github.com/actions/setup-python/releases) - [Commits](https://github.com/actions/setup-python/compare/v4...v5) --- updated-dependencies: - dependency-name: actions/setup-python dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] --- .github/workflows/build.yml | 2 +- .github/workflows/python-test.yml | 4 ++-- .github/workflows/test_icingaweb2_ini_template.yml | 2 +- 3 files changed, 4 insertions(+), 4 deletions(-) diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index 1fe79b42..03ba1fe4 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -41,7 +41,7 @@ jobs: uses: actions/checkout@v4 - name: Set up Python - uses: actions/setup-python@v4 + uses: actions/setup-python@v5 with: python-version: ${{ matrix.python }} diff --git a/.github/workflows/python-test.yml b/.github/workflows/python-test.yml index 953a801c..cb2f519e 100644 --- a/.github/workflows/python-test.yml +++ b/.github/workflows/python-test.yml @@ -29,7 +29,7 @@ jobs: uses: actions/checkout@v4 - name: Set up Python - uses: actions/setup-python@v4 + uses: actions/setup-python@v5 with: python-version: ${{ matrix.python }} - name: Install dependencies @@ -54,7 +54,7 @@ jobs: uses: actions/checkout@v4 - name: Set up Python - uses: actions/setup-python@v4 + uses: actions/setup-python@v5 with: python-version: ${{ matrix.python }} diff --git a/.github/workflows/test_icingaweb2_ini_template.yml b/.github/workflows/test_icingaweb2_ini_template.yml index ff12ab85..54f3b887 100644 --- a/.github/workflows/test_icingaweb2_ini_template.yml +++ b/.github/workflows/test_icingaweb2_ini_template.yml @@ -39,7 +39,7 @@ jobs: uses: actions/checkout@v4 - name: Set up Python - uses: actions/setup-python@v4 + uses: actions/setup-python@v5 with: python-version: ${{ matrix.python }} From d17d7adfcf3b264ca663d1db510f910f5bb21e2a Mon Sep 17 00:00:00 2001 From: Thilo W Date: Thu, 4 Jan 2024 16:40:01 +0100 Subject: [PATCH 23/39] changed default image in molecule build --- molecule/default/molecule.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/molecule/default/molecule.yml b/molecule/default/molecule.yml index 734354fd..3f13ab2d 100644 --- a/molecule/default/molecule.yml +++ b/molecule/default/molecule.yml @@ -5,7 +5,7 @@ driver: name: docker platforms: - name: icinga-default - image: "geerlingguy/docker-${MOLECULE_DISTRO:-centos7}-ansible:latest" + image: "geerlingguy/docker-${MOLECULE_DISTRO:-ubuntu2204}-ansible:latest" command: ${MOLECULE_DOCKER_COMMAND:-""} volumes: - /sys/fs/cgroup:/sys/fs/cgroup:rw From 343415d98446a81f07ce247236b420e24e6cc1c2 Mon Sep 17 00:00:00 2001 From: Thilo W Date: Thu, 4 Jan 2024 16:42:25 +0100 Subject: [PATCH 24/39] add some local-default test infos [skip ci] --- molecule/local-default/ansible.cfg | 2 ++ .../files/graphite_templates/test.ini | 30 +++++++++++++++++++ 2 files changed, 32 insertions(+) create mode 100644 molecule/local-default/ansible.cfg create mode 100644 molecule/local-default/files/graphite_templates/test.ini diff --git a/molecule/local-default/ansible.cfg b/molecule/local-default/ansible.cfg new file mode 100644 index 00000000..c58fea3c --- /dev/null +++ b/molecule/local-default/ansible.cfg @@ -0,0 +1,2 @@ +[ssh_connection] +pipelining = True diff --git a/molecule/local-default/files/graphite_templates/test.ini b/molecule/local-default/files/graphite_templates/test.ini new file mode 100644 index 00000000..a7755e80 --- /dev/null +++ b/molecule/local-default/files/graphite_templates/test.ini @@ -0,0 +1,30 @@ +[hostalive-rta.graph] +check_command = "hostalive" + +[hostalive-rta.metrics_filters] +rta.value = "$host_name_template$.perfdata.rta.value" + +[hostalive-rta.urlparams] +areaAlpha = "0.5" +areaMode = "all" +min = "0" +yUnitSystem = "none" + +[hostalive-rta.functions] +rta.value = "alias(color(scale($metric$, 1000), '#1a7dd7'), 'Round trip time (ms)')" + + +[hostalive-pl.graph] +check_command = "hostalive" + +[hostalive-pl.metrics_filters] +pl.value = "$host_name_template$.perfdata.pl.value" + +[hostalive-pl.urlparams] +areaAlpha = "0.5" +areaMode = "all" +min = "0" +yUnitSystem = "none" + +[hostalive-pl.functions] +pl.value = "alias(color($metric$, '#1a7dd7'), 'Packet loss (%)')" From fca0a4792667807e84c8708da83a5715a7068af3 Mon Sep 17 00:00:00 2001 From: Thilo W Date: Fri, 5 Jan 2024 10:57:38 +0100 Subject: [PATCH 25/39] Fix module configuration task order (#254) * changed order, to enable/disable modules before configuration. * added bugfix changelog #225 --- .../fragments/fix_change_order_of_module_tasks.yml | 3 +++ roles/icingaweb2/tasks/main.yml | 14 ++++++++------ 2 files changed, 11 insertions(+), 6 deletions(-) create mode 100644 changelogs/fragments/fix_change_order_of_module_tasks.yml diff --git a/changelogs/fragments/fix_change_order_of_module_tasks.yml b/changelogs/fragments/fix_change_order_of_module_tasks.yml new file mode 100644 index 00000000..7f9ffbff --- /dev/null +++ b/changelogs/fragments/fix_change_order_of_module_tasks.yml @@ -0,0 +1,3 @@ +--- +bugfixes: + - "Icingaweb2: Change order of module state and configuration tasks #225" diff --git a/roles/icingaweb2/tasks/main.yml b/roles/icingaweb2/tasks/main.yml index a020d4cd..ea8dbe43 100644 --- a/roles/icingaweb2/tasks/main.yml +++ b/roles/icingaweb2/tasks/main.yml @@ -28,12 +28,7 @@ ansible.builtin.include_tasks: "manage_icingaweb_{{ icingaweb2_db.type }}_db.yml" when: icingaweb2_db is defined -- name: Configure modules - ansible.builtin.include_tasks: "modules/{{ item.key }}.yml" - when: icingaweb2_modules is defined - loop: "{{ icingaweb2_modules | dict2items }}" - -- name: Manage enabled/disabled modules +- name: Manage module states ansible.builtin.file: src: "{{ icingaweb2_config.global.module_path + '/' + item.key if item.value.enabled|bool == true else omit }}" dest: "{{ icingaweb2_config_dir }}/enabledModules/{{ item.key }}" @@ -43,6 +38,13 @@ force: yes when: icingaweb2_modules is defined loop: "{{ icingaweb2_modules | dict2items }}" + loop_control: + label: "Ensure {{ item.key }} is {{ 'enabled' if item.value.enabled|bool == true else 'disabled' }}" + +- name: Configure modules + ansible.builtin.include_tasks: "modules/{{ item.key }}.yml" + when: icingaweb2_modules is defined + loop: "{{ icingaweb2_modules | dict2items }}" # Many daemons fail before e.g. the resource is set up or the schema hasn't been migrated. This is a workaround. - name: Manage enabled module daemons From 9397d1825a537c854199ffd6619fc290df544831 Mon Sep 17 00:00:00 2001 From: Thilo W Date: Fri, 5 Jan 2024 12:29:35 +0100 Subject: [PATCH 26/39] Ensure ansible backwards compatibility with bool filter --- roles/icingaweb2/tasks/main.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/roles/icingaweb2/tasks/main.yml b/roles/icingaweb2/tasks/main.yml index ea8dbe43..d38ec237 100644 --- a/roles/icingaweb2/tasks/main.yml +++ b/roles/icingaweb2/tasks/main.yml @@ -16,7 +16,7 @@ ansible.builtin.set_fact: icingaweb2_packages: "{{ icingaweb2_packages + [ icingaweb2_module_packages[item.key] ] }}" loop: "{{ icingaweb2_modules | dict2items }}" - when: icingaweb2_modules is defined and icingaweb2_module_packages[item.key] is defined and item.value.enabled is true and item.value.source == "package" + when: icingaweb2_modules is defined and icingaweb2_module_packages[item.key] is defined and item.value.enabled | bool == true and item.value.source == "package" - name: Include OS specific installation ansible.builtin.include_tasks: "install_on_{{ ansible_os_family | lower }}.yml" From 0bba33d74e4930c8098106dc7759eea22ca4f57c Mon Sep 17 00:00:00 2001 From: Thilo W Date: Fri, 5 Jan 2024 12:35:45 +0100 Subject: [PATCH 27/39] added changelog fragment for issue #218 --- changelogs/fragments/minor_changes.yml | 1 + 1 file changed, 1 insertion(+) diff --git a/changelogs/fragments/minor_changes.yml b/changelogs/fragments/minor_changes.yml index 72380758..06c12f7c 100644 --- a/changelogs/fragments/minor_changes.yml +++ b/changelogs/fragments/minor_changes.yml @@ -5,3 +5,4 @@ bugfixes: minor_changes: - added tests for icingaweb2 ini template - added pyinilint as ini validator after templates + - ensure backwards compatibility with bool filter (#218) From 5c976ce0b8b6d025fb88af8e50425c922f6f713c Mon Sep 17 00:00:00 2001 From: Thilo W Date: Fri, 5 Jan 2024 14:11:58 +0100 Subject: [PATCH 28/39] added test for icingaweb2 role with defaults --- .github/workflows/role-icingaweb2.yml | 63 +++++++++++++++++++ molecule/role-icingaweb2/collections.yml | 5 ++ molecule/role-icingaweb2/converge.yml | 12 ++++ molecule/role-icingaweb2/dependency.yml | 4 ++ molecule/role-icingaweb2/files/test.conf | 14 +++++ .../host_vars/icinga-default.yaml | 0 molecule/role-icingaweb2/molecule.yml | 26 ++++++++ molecule/role-icingaweb2/prepare.yml | 11 ++++ molecule/role-icingaweb2/requirements.yml | 2 + molecule/role-icingaweb2/verify.yml | 9 +++ requirements-test-2.16.2.txt | 5 ++ 11 files changed, 151 insertions(+) create mode 100644 .github/workflows/role-icingaweb2.yml create mode 100644 molecule/role-icingaweb2/collections.yml create mode 100644 molecule/role-icingaweb2/converge.yml create mode 100644 molecule/role-icingaweb2/dependency.yml create mode 100644 molecule/role-icingaweb2/files/test.conf create mode 100644 molecule/role-icingaweb2/host_vars/icinga-default.yaml create mode 100644 molecule/role-icingaweb2/molecule.yml create mode 100644 molecule/role-icingaweb2/prepare.yml create mode 100644 molecule/role-icingaweb2/requirements.yml create mode 100644 molecule/role-icingaweb2/verify.yml create mode 100644 requirements-test-2.16.2.txt diff --git a/.github/workflows/role-icingaweb2.yml b/.github/workflows/role-icingaweb2.yml new file mode 100644 index 00000000..67bd2149 --- /dev/null +++ b/.github/workflows/role-icingaweb2.yml @@ -0,0 +1,63 @@ +--- +name: Empty role-icingaweb2 +on: + push: + tags: + - '*' + branches: + - main + - 'feature/**' + - 'fix/**' + - '!doc/**' + paths: + - roles/icingaweb2/** + - molecule/role-icingaweb2/** + pull_request: + branches: + - 'feature/**' + - 'fix/**' + - '!doc/**' + +jobs: + icingaweb2-ansible-latest: + runs-on: ubuntu-latest + + env: + COLLECTION_NAMESPACE: icinga + COLLECTION_NAME: icinga + + strategy: + fail-fast: false + max-parallel: 1 + matrix: + distro: [ubuntu2204] + python: ['3.10'] + ansible: ['2.16.2'] + scenario: [role-icingaweb2] + + steps: + - name: Check out code + uses: actions/checkout@v4 + + - name: Set up Python + uses: actions/setup-python@v4 + with: + python-version: ${{ matrix.python }} + + - name: Install dependencies ansible + run: | + python3 -m pip install --upgrade pip + python3 -m pip install -r requirements-test-${{ matrix.ansible }}.txt + + - name: Install collection + run: | + mkdir -p ~/.ansible/collections/ansible_collections/$COLLECTION_NAMESPACE + cp -a ../ansible-collection-$COLLECTION_NAME ~/.ansible/collections/ansible_collections/$COLLECTION_NAMESPACE/$COLLECTION_NAME + + - name: Test with molecule + run: | + ansible --version + molecule --version + molecule test -s ${{ matrix.scenario }} + env: + MOLECULE_DISTRO: ${{ matrix.distro }} diff --git a/molecule/role-icingaweb2/collections.yml b/molecule/role-icingaweb2/collections.yml new file mode 100644 index 00000000..66cb0953 --- /dev/null +++ b/molecule/role-icingaweb2/collections.yml @@ -0,0 +1,5 @@ +collections: + - name: community.mysql + - name: community.docker + - name: community.general + version: ">=2,<3" diff --git a/molecule/role-icingaweb2/converge.yml b/molecule/role-icingaweb2/converge.yml new file mode 100644 index 00000000..63ac7133 --- /dev/null +++ b/molecule/role-icingaweb2/converge.yml @@ -0,0 +1,12 @@ +--- + +- name: Converge + hosts: all + collections: + - icinga.icinga + pre_tasks: + - ansible.builtin.include_role: + name: repos + post_tasks: + - ansible.builtin.include_role: + name: icingaweb2 diff --git a/molecule/role-icingaweb2/dependency.yml b/molecule/role-icingaweb2/dependency.yml new file mode 100644 index 00000000..9810d54a --- /dev/null +++ b/molecule/role-icingaweb2/dependency.yml @@ -0,0 +1,4 @@ +dependency: + name: galaxy + options: + role-file: requirements.yml diff --git a/molecule/role-icingaweb2/files/test.conf b/molecule/role-icingaweb2/files/test.conf new file mode 100644 index 00000000..62690e4e --- /dev/null +++ b/molecule/role-icingaweb2/files/test.conf @@ -0,0 +1,14 @@ +### Business Process Config File ### +# +# Title : test +# Description : test +# Owner : icinga +# AddToMenu : yes +# Statetype : soft +# +################################### + +test2 = +display 1;test2;test2 +webserver = +display 1;webserver;webserver diff --git a/molecule/role-icingaweb2/host_vars/icinga-default.yaml b/molecule/role-icingaweb2/host_vars/icinga-default.yaml new file mode 100644 index 00000000..e69de29b diff --git a/molecule/role-icingaweb2/molecule.yml b/molecule/role-icingaweb2/molecule.yml new file mode 100644 index 00000000..3f13ab2d --- /dev/null +++ b/molecule/role-icingaweb2/molecule.yml @@ -0,0 +1,26 @@ +--- +dependency: + name: galaxy +driver: + name: docker +platforms: + - name: icinga-default + image: "geerlingguy/docker-${MOLECULE_DISTRO:-ubuntu2204}-ansible:latest" + command: ${MOLECULE_DOCKER_COMMAND:-""} + volumes: + - /sys/fs/cgroup:/sys/fs/cgroup:rw + cgroupns_mode: host + privileged: true + pre_build_image: true +provisioner: + name: ansible + inventory: + link: + host_vars: host_vars/ +verifier: + name: testinfra + directory: tests/integration/ +lint: | + set -e + yamllint --no-warnings roles/ + ansible-lint roles/ diff --git a/molecule/role-icingaweb2/prepare.yml b/molecule/role-icingaweb2/prepare.yml new file mode 100644 index 00000000..cc5bbc56 --- /dev/null +++ b/molecule/role-icingaweb2/prepare.yml @@ -0,0 +1,11 @@ +--- +- name: Prepare + hosts: all + tasks: + - name: Install requirements for Debian + apt: + name: + - gpg + - apt-transport-https + update_cache: yes + when: ansible_os_family == "Debian" diff --git a/molecule/role-icingaweb2/requirements.yml b/molecule/role-icingaweb2/requirements.yml new file mode 100644 index 00000000..cf94e2e2 --- /dev/null +++ b/molecule/role-icingaweb2/requirements.yml @@ -0,0 +1,2 @@ +roles: + - geerlingguy.mysql diff --git a/molecule/role-icingaweb2/verify.yml b/molecule/role-icingaweb2/verify.yml new file mode 100644 index 00000000..3c4ed28d --- /dev/null +++ b/molecule/role-icingaweb2/verify.yml @@ -0,0 +1,9 @@ +--- + +- name: Verify + hosts: all + tasks: + - name: Check for running icinga2 + service: + name: icinga2 + state: started diff --git a/requirements-test-2.16.2.txt b/requirements-test-2.16.2.txt new file mode 100644 index 00000000..9332828e --- /dev/null +++ b/requirements-test-2.16.2.txt @@ -0,0 +1,5 @@ +ansible-core==2.16.2 +ansible-lint +molecule +molecule-docker +pytest-testinfra From a586add8a2bff945141f05ee15f85cf0b1cff13e Mon Sep 17 00:00:00 2001 From: Thilo W Date: Fri, 5 Jan 2024 15:25:44 +0100 Subject: [PATCH 29/39] added more quick default role tests icingadb_redis --- .github/workflows/role-icingadb_redis.yml | 66 +++++++++++++++++++ .github/workflows/role-icingaweb2.yml | 11 ++-- molecule/role-icingadb_redis/collections.yml | 5 ++ molecule/role-icingadb_redis/converge.yml | 12 ++++ molecule/role-icingadb_redis/dependency.yml | 4 ++ .../host_vars/icinga-default.yaml | 0 molecule/role-icingadb_redis/molecule.yml | 26 ++++++++ molecule/role-icingadb_redis/prepare.yml | 11 ++++ molecule/role-icingadb_redis/requirements.yml | 2 + molecule/role-icingadb_redis/verify.yml | 9 +++ 10 files changed, 142 insertions(+), 4 deletions(-) create mode 100644 .github/workflows/role-icingadb_redis.yml create mode 100644 molecule/role-icingadb_redis/collections.yml create mode 100644 molecule/role-icingadb_redis/converge.yml create mode 100644 molecule/role-icingadb_redis/dependency.yml create mode 100644 molecule/role-icingadb_redis/host_vars/icinga-default.yaml create mode 100644 molecule/role-icingadb_redis/molecule.yml create mode 100644 molecule/role-icingadb_redis/prepare.yml create mode 100644 molecule/role-icingadb_redis/requirements.yml create mode 100644 molecule/role-icingadb_redis/verify.yml diff --git a/.github/workflows/role-icingadb_redis.yml b/.github/workflows/role-icingadb_redis.yml new file mode 100644 index 00000000..3a01362c --- /dev/null +++ b/.github/workflows/role-icingadb_redis.yml @@ -0,0 +1,66 @@ +--- + # These Jobs should be always be run against the latest version of ansible on the systems + # Feel free to update python and ansible versions + # + # In addition to keep them quick and no additional variables are used. + # +name: role-icingadb_redis +on: + push: + branches: + - main + - 'feature/**' + - 'fix/**' + - '!doc/**' + paths: + - roles/icingadb_redis/** + - molecule/role-icingadb_redis/** + pull_request: + branches: + - 'feature/**' + - 'fix/**' + - '!doc/**' + +jobs: + icingadb_redis-ubuntu2204-latest: + runs-on: ubuntu-latest + + env: + COLLECTION_NAMESPACE: icinga + COLLECTION_NAME: icinga + + strategy: + fail-fast: false + max-parallel: 1 + matrix: + distro: [ubuntu2204] + python: ['3.10'] + ansible: ['2.16.2'] + scenario: [role-icingadb_redis] + + steps: + - name: Check out code + uses: actions/checkout@v4 + + - name: Set up Python + uses: actions/setup-python@v4 + with: + python-version: ${{ matrix.python }} + + - name: Install dependencies ansible + run: | + python3 -m pip install --upgrade pip + python3 -m pip install -r requirements-test-${{ matrix.ansible }}.txt + + - name: Install collection + run: | + mkdir -p ~/.ansible/collections/ansible_collections/$COLLECTION_NAMESPACE + cp -a ../ansible-collection-$COLLECTION_NAME ~/.ansible/collections/ansible_collections/$COLLECTION_NAMESPACE/$COLLECTION_NAME + + - name: Test with molecule + run: | + ansible --version + molecule --version + molecule test -s ${{ matrix.scenario }} + env: + MOLECULE_DISTRO: ${{ matrix.distro }} diff --git a/.github/workflows/role-icingaweb2.yml b/.github/workflows/role-icingaweb2.yml index 67bd2149..6ed31fde 100644 --- a/.github/workflows/role-icingaweb2.yml +++ b/.github/workflows/role-icingaweb2.yml @@ -1,9 +1,12 @@ --- -name: Empty role-icingaweb2 + # These Jobs should be always be run against the latest version of ansible on the systems + # Feel free to update python and ansible versions + # + # In addition to keep them quick and no additional variables are used. + # +name: role-icingaweb2 on: push: - tags: - - '*' branches: - main - 'feature/**' @@ -19,7 +22,7 @@ on: - '!doc/**' jobs: - icingaweb2-ansible-latest: + icingaweb2-ubuntu2204-latest: runs-on: ubuntu-latest env: diff --git a/molecule/role-icingadb_redis/collections.yml b/molecule/role-icingadb_redis/collections.yml new file mode 100644 index 00000000..66cb0953 --- /dev/null +++ b/molecule/role-icingadb_redis/collections.yml @@ -0,0 +1,5 @@ +collections: + - name: community.mysql + - name: community.docker + - name: community.general + version: ">=2,<3" diff --git a/molecule/role-icingadb_redis/converge.yml b/molecule/role-icingadb_redis/converge.yml new file mode 100644 index 00000000..2dea1e2b --- /dev/null +++ b/molecule/role-icingadb_redis/converge.yml @@ -0,0 +1,12 @@ +--- + +- name: Converge + hosts: all + collections: + - icinga.icinga + pre_tasks: + - ansible.builtin.include_role: + name: repos + post_tasks: + - ansible.builtin.include_role: + name: icingadb_redis diff --git a/molecule/role-icingadb_redis/dependency.yml b/molecule/role-icingadb_redis/dependency.yml new file mode 100644 index 00000000..9810d54a --- /dev/null +++ b/molecule/role-icingadb_redis/dependency.yml @@ -0,0 +1,4 @@ +dependency: + name: galaxy + options: + role-file: requirements.yml diff --git a/molecule/role-icingadb_redis/host_vars/icinga-default.yaml b/molecule/role-icingadb_redis/host_vars/icinga-default.yaml new file mode 100644 index 00000000..e69de29b diff --git a/molecule/role-icingadb_redis/molecule.yml b/molecule/role-icingadb_redis/molecule.yml new file mode 100644 index 00000000..c288cf03 --- /dev/null +++ b/molecule/role-icingadb_redis/molecule.yml @@ -0,0 +1,26 @@ +--- +dependency: + name: galaxy +driver: + name: docker +platforms: + - name: icingadb_redis-default + image: "geerlingguy/docker-${MOLECULE_DISTRO:-ubuntu2204}-ansible:latest" + command: ${MOLECULE_DOCKER_COMMAND:-""} + volumes: + - /sys/fs/cgroup:/sys/fs/cgroup:rw + cgroupns_mode: host + privileged: true + pre_build_image: true +provisioner: + name: ansible + inventory: + link: + host_vars: host_vars/ +verifier: + name: testinfra + directory: tests/integration/ +lint: | + set -e + yamllint --no-warnings roles/ + ansible-lint roles/ diff --git a/molecule/role-icingadb_redis/prepare.yml b/molecule/role-icingadb_redis/prepare.yml new file mode 100644 index 00000000..cc5bbc56 --- /dev/null +++ b/molecule/role-icingadb_redis/prepare.yml @@ -0,0 +1,11 @@ +--- +- name: Prepare + hosts: all + tasks: + - name: Install requirements for Debian + apt: + name: + - gpg + - apt-transport-https + update_cache: yes + when: ansible_os_family == "Debian" diff --git a/molecule/role-icingadb_redis/requirements.yml b/molecule/role-icingadb_redis/requirements.yml new file mode 100644 index 00000000..cf94e2e2 --- /dev/null +++ b/molecule/role-icingadb_redis/requirements.yml @@ -0,0 +1,2 @@ +roles: + - geerlingguy.mysql diff --git a/molecule/role-icingadb_redis/verify.yml b/molecule/role-icingadb_redis/verify.yml new file mode 100644 index 00000000..3c4ed28d --- /dev/null +++ b/molecule/role-icingadb_redis/verify.yml @@ -0,0 +1,9 @@ +--- + +- name: Verify + hosts: all + tasks: + - name: Check for running icinga2 + service: + name: icinga2 + state: started From ced49762a6538062f2f238fc433f414f3ec4eab3 Mon Sep 17 00:00:00 2001 From: Thilo W Date: Fri, 5 Jan 2024 15:45:40 +0100 Subject: [PATCH 30/39] adds informations how to run tests on the collection. --- TESTING.md | 63 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 63 insertions(+) create mode 100644 TESTING.md diff --git a/TESTING.md b/TESTING.md new file mode 100644 index 00000000..06fc7d16 --- /dev/null +++ b/TESTING.md @@ -0,0 +1,63 @@ +## Collection Testing Guide + +This guide will quickstart you on our testing environment and eases the way to +contribute on our project. + +### Tests + +Currently we do some molecule tests to check if the roles work in combination +and some variables to check if anything fails. + +Then there are unit tests for the parser which generates the Icinga 2 DSL. + +At last we implemented a way to check if our Icingaweb roles generates valid +ini files. + +### Tools + +Make sure the following tools are available before start testing with the collection. + +``` +pip install ansible-core ansible-lint molecule pytest-testinfra +``` + +To test roles locally without docker/service issues, we created a molecule test +with vagrant. Then you need to install [vagrant](link/to/vagrant) and the molecule plugin. + +`pip install molecule-plugins[vagrant]` + +To use molecule with docker install the docker plugin. + +`pip install molecule-plugins[docker]` + +### Roles Testing + +To test roles over vagrant locally, it is the easiest to run the **local-default** +scenario. The local-default is very big and long running. For shorter tests use +the role- scenarios. + +`molecule test -s local-default` + +The following tests are inplemented based on docker. Per default a **ubuntu2204** +image from geerlingguy's container is used. Thanks [@geerlingguy Dockerhublink](https://hub.docker.com/u/geerlingguy) + +To test other distros use the command with the env **MOLECULE_DISTRO**. + +`MOLECULE_DISTRO=opensuseleap15 molecule test -s role-icingadb_redis` + +### Templating Tests + +The roles are generating configuration for Icingaweb2 and Icinga2 in various files. +To ensure values are written to these files in right syntax we test those too. + +#### Python Unittest + +For testing our Icinga 2 objects syntax we implemented python unittests and try +many combinations which occur in different python versions. + +For more information please have a look at the workflow `Python Unittest`. + +#### Icingaweb2 INI + +To test the INI configuration over Ansible in the Icinga Web 2 role, we implemented +a molecule test to include the template from the role and test it with various values. From d0208a53c8ba07c12d9e736369bd12fa3de06834 Mon Sep 17 00:00:00 2001 From: Johannes Kastl Date: Fri, 12 Jan 2024 08:59:36 +0100 Subject: [PATCH 31/39] Icingaweb2: fix duplicate task name at kickstart tasks (#244) --- roles/icingaweb2/tasks/modules/director.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/roles/icingaweb2/tasks/modules/director.yml b/roles/icingaweb2/tasks/modules/director.yml index d3f0bf18..378ef904 100644 --- a/roles/icingaweb2/tasks/modules/director.yml +++ b/roles/icingaweb2/tasks/modules/director.yml @@ -40,7 +40,7 @@ failed_when: _required.rc|int >= 2 when: vars['icingaweb2_modules']['director']['run_kickstart'] is defined and icingaweb2_modules.director.run_kickstart and vars['icingaweb2_modules']['director']['kickstart'] is defined -- name: Module Director | Check if kickstart is required # noqa: command-instead-of-shell +- name: Module Director | Run kickstart if required # noqa: command-instead-of-shell ansible.builtin.shell: cmd: icingacli director kickstart run when: vars['icingaweb2_modules']['director']['run_kickstart'] is defined and icingaweb2_modules.director.run_kickstart and vars['icingaweb2_modules']['director']['kickstart'] is defined and _required.rc|int == 0 From be451c199308d75d9536e1e8ef81472b3f27b246 Mon Sep 17 00:00:00 2001 From: Thilo W Date: Fri, 12 Jan 2024 09:00:44 +0100 Subject: [PATCH 32/39] added changelog for PR244 --- changelogs/fragments/minor_changes.yml | 1 + 1 file changed, 1 insertion(+) diff --git a/changelogs/fragments/minor_changes.yml b/changelogs/fragments/minor_changes.yml index 06c12f7c..e670073e 100644 --- a/changelogs/fragments/minor_changes.yml +++ b/changelogs/fragments/minor_changes.yml @@ -6,3 +6,4 @@ minor_changes: - added tests for icingaweb2 ini template - added pyinilint as ini validator after templates - ensure backwards compatibility with bool filter (#218) + - "Icingaweb2: fix duplicate task name at kickstart tasks (#244)" From 1cd05d23715d2a8d2f9d0ebfa682034810f6d002 Mon Sep 17 00:00:00 2001 From: Donien <88634789+Donien@users.noreply.github.com> Date: Fri, 12 Jan 2024 11:41:12 +0100 Subject: [PATCH 33/39] Feature/ansible inventory (#177) * Initial commit of working inventory plugin * Allow for negation of in-filter ('group' in host.groups) * Add documentation about the Ansible Inventory Plugin 'icinga.icinga.icinga' * Add test cases for inventory plugin * Add changelog fragment for 'feature/ansible-inventory' --- .github/workflows/python-test.yml | 6 + README.md | 1 + .../fragments/feature_ansible_inventory.yml | 3 + doc/getting-started.md | 6 + .../inventory/icinga-inventory-plugin.md | 631 ++++++++++++++++++ plugins/inventory/icinga.py | 546 +++++++++++++++ tests/unittestpy3/test_inventory.py | 602 +++++++++++++++++ 7 files changed, 1795 insertions(+) create mode 100644 changelogs/fragments/feature_ansible_inventory.yml create mode 100644 doc/plugins/inventory/icinga-inventory-plugin.md create mode 100644 plugins/inventory/icinga.py create mode 100644 tests/unittestpy3/test_inventory.py diff --git a/.github/workflows/python-test.yml b/.github/workflows/python-test.yml index cb2f519e..c5e0588b 100644 --- a/.github/workflows/python-test.yml +++ b/.github/workflows/python-test.yml @@ -59,8 +59,14 @@ jobs: with: python-version: ${{ matrix.python }} + - name: Install dependencies + run: | + python3 -m pip install --upgrade pip + python3 -m pip install "ansible-core>=2.13.10" "requests>=1.0.0" + - name: Test with unittest run: | python -m unittest -v tests.unittestpy3.test_data + python -m unittest -v tests.unittestpy3.test_inventory env: MOLECULE_DISTRO: ${{ matrix.distro }}# diff --git a/README.md b/README.md index 14fdab31..e7cb3e36 100644 --- a/README.md +++ b/README.md @@ -16,6 +16,7 @@ Collection to setup and manage components of the Icinga software stack. * [Role: icinga.icinga.icingaweb2](doc/role-icingaweb2/role-icingaweb2.md) * [Role: icinga.icinga.monitoring_plugins](doc/role-monitoring_plugins/role-monitoring_plugins.md) * [List of Available Check Commands](doc/role-monitoring_plugins/check_command_list.md) +* [Inventory Plugin: icinga.icinga.icinga](doc/plugins/inventory/icinga-inventory-plugin.md) ## Installation diff --git a/changelogs/fragments/feature_ansible_inventory.yml b/changelogs/fragments/feature_ansible_inventory.yml new file mode 100644 index 00000000..31fab3c1 --- /dev/null +++ b/changelogs/fragments/feature_ansible_inventory.yml @@ -0,0 +1,3 @@ +--- +major_changes: + - Add an Ansible Inventory Plugin to fetch host information from Icinga 2's API for use as an Ansible Inventory diff --git a/doc/getting-started.md b/doc/getting-started.md index 8a5b6767..b843a60f 100644 --- a/doc/getting-started.md +++ b/doc/getting-started.md @@ -15,6 +15,12 @@ The collection includes six roles in the current version. * icinga.monitoring_plugins: Role to install and manage Icinga2 compatible monitoring plugins. * [Documentation: doc/role-monitoring_plugins](role-monitoring_plugins/role-monitoring_plugins.md) +--- + +The collection includes a plugin that allows you to use Icinga as an inventory source for Ansible. + +* icinga.icinga.icinga: Ansible Inventory Plugin to fetch hosts from Icinga. + * [Documentation: doc/plugins/inventory/icinga-inventory-plugin.md](plugins/inventory/icinga-inventory-plugin.md) --- **NOTE** diff --git a/doc/plugins/inventory/icinga-inventory-plugin.md b/doc/plugins/inventory/icinga-inventory-plugin.md new file mode 100644 index 00000000..f0e26e16 --- /dev/null +++ b/doc/plugins/inventory/icinga-inventory-plugin.md @@ -0,0 +1,631 @@ +# Icinga as an Inventory Source for Ansible + +- [Requirements](#requirements) +- [Variables](#variables) +- [Using Constructed Inventory and Cache](#using-constructed-inventory-and-cache) +- [Filter Options](#filter-options) + +There is a lot of Ansible Inventory Plugins to pull host information from different sources. This allows for dynamic inventories that adapt to changes made in other applications. +Using this plugin you can use Icinga 2's API to build your Ansible Inventory. + +For this to work you need to create a file ending in either `icinga.yml` or `icinga.yaml` and fill it with all required variables to fetch information from your Icinga 2 API. + +> If you need further information, be sure to pass `-vvv` to your Ansible command to get additional information. This will for example tell you what filter is used by the plugin when querying Icinga's API. + +Example: + +**inventory-icinga.yml:** + +``` +--- +plugin: icinga.icinga.icinga +user: api-user +password: api-user-password +``` + +``` +ansible -i inventory-icinga.yml localhost -m debug -a "msg='{{ groups }}'" +``` + +``` +localhost | SUCCESS => { + "msg": { + "all": [ + "icinga-master", + "dummy_host_1", + "dummy_host_2", + "dummy_host_3", + "dummy_host_4", + "dummy_host_5", + "test" + ], + "icinga_group_testgroup1": [ + "test" + ], + "icinga_group_test_group_2": [ + "test" + ], + "icinga_zone_main": [ + "icinga-master", + "dummy_host_1", + "dummy_host_2", + "dummy_host_3", + "dummy_host_4", + "dummy_host_5" + ], + "icinga_zone_test": [ + "test" + ], + "ungrouped": [] + } +} +``` + +Variables of host `icinga-master`: + +``` +ansible -i inventory-icinga.yml localhost -m debug -a "msg='{{ hostvars[\"icinga-master\"] }}'" +``` + +``` +icinga-master | SUCCESS => { + "msg": { + "ansible_check_mode": false, + "ansible_config_file": "/home/matthias/.ansible.cfg", + "ansible_diff_mode": false, + "ansible_facts": {}, + "ansible_forks": 5, + "ansible_inventory_sources": [ + "/home/matthias/Ansible/inventory-icinga.yml" + ], + "ansible_playbook_python": "/usr/bin/python", + "ansible_verbosity": 0, + "ansible_version": { + "full": "2.15.3", + "major": 2, + "minor": 15, + "revision": 3, + "string": "2.15.3" + }, + "group_names": [ + "icinga_os_linux", + "icinga_zone_main" + ], + "groups": { + "all": [ + "icinga-master", + "dummy_host_1", + "dummy_host_2", + "dummy_host_3", + "dummy_host_4", + "dummy_host_5", + "test" + ], + "icinga_group_testgroup1": [ + "test" + ], + "icinga_group_test_group_2": [ + "test" + ], + "icinga_zone_main": [ + "icinga-master", + "dummy_host_1", + "dummy_host_2", + "dummy_host_3", + "dummy_host_4", + "dummy_host_5" + ], + "icinga_zone_test": [ + "test" + ], + "ungrouped": [] + }, + "icinga___name": "icinga-master", + "icinga_acknowledgement": 0, + "icinga_acknowledgement_expiry": 0, + "icinga_acknowledgement_last_change": 0, + "icinga_action_url": "", + "icinga_active": true, + "icinga_address": "icinga-master", + "icinga_address6": "", + "icinga_check_attempt": 1, + "icinga_check_command": "hostalive", + "icinga_check_interval": 300, + "icinga_check_period": "", + "icinga_check_timeout": null, + "icinga_command_endpoint": "", + "icinga_display_name": "Icinga-Master", + "icinga_downtime_depth": 0, + "icinga_enable_active_checks": true, + "icinga_enable_event_handler": true, + "icinga_enable_flapping": false, + "icinga_enable_notifications": true, + "icinga_enable_passive_checks": true, + "icinga_enable_perfdata": true, + "icinga_event_command": "", + "icinga_executions": null, + "icinga_flapping": false, + "icinga_flapping_current": 0, + "icinga_flapping_ignore_states": null, + "icinga_flapping_last_change": 0, + "icinga_flapping_threshold": 0, + "icinga_flapping_threshold_high": 30, + "icinga_flapping_threshold_low": 25, + "icinga_force_next_check": false, + "icinga_force_next_notification": false, + "icinga_groups": [], + "icinga_ha_mode": 0, + "icinga_handled": false, + "icinga_icon_image": "", + "icinga_icon_image_alt": "", + "icinga_last_check": 1693814617.791436, + "icinga_last_check_result": { + "active": true, + "check_source": "icinga-master", + "command": [ + "/usr/lib/nagios/plugins/check_ping", + "-H", + "icinga-master", + "-c", + "5000,100%", + "-w", + "3000,80%" + ], + "execution_end": 1693814617.791384, + "execution_start": 1693814613.71081, + "exit_status": 0, + "output": "PING OK - Packet loss = 0%, RTA = 0.02 ms", + "performance_data": [ + "rta=0.019000ms;3000.000000;5000.000000;0.000000", + "pl=0%;80;100;0" + ], + "previous_hard_state": 99, + "schedule_end": 1693814617.791436, + "schedule_start": 1693814613.71, + "scheduling_source": "icinga-master", + "state": 0, + "ttl": 0, + "type": "CheckResult", + "vars_after": { + "attempt": 1, + "reachable": true, + "state": 0, + "state_type": 1 + }, + "vars_before": { + "attempt": 1, + "reachable": true, + "state": 0, + "state_type": 1 + } + }, + "icinga_last_hard_state": 0, + "icinga_last_hard_state_change": 1691570988.637088, + "icinga_last_reachable": true, + "icinga_last_state": 0, + "icinga_last_state_change": 1691570988.637088, + "icinga_last_state_down": 0, + "icinga_last_state_type": 1, + "icinga_last_state_unreachable": 0, + "icinga_last_state_up": 1693814617.791384, + "icinga_max_check_attempts": 3, + "icinga_name": "icinga-master", + "icinga_next_check": 1693814916.361451, + "icinga_next_update": 1693815224.5242188, + "icinga_notes": "", + "icinga_notes_url": "", + "icinga_original_attributes": null, + "icinga_package": "director", + "icinga_paused": false, + "icinga_previous_state_change": 1691570988.637088, + "icinga_problem": false, + "icinga_retry_interval": 60, + "icinga_severity": 0, + "icinga_source_location": { + "first_column": 0, + "first_line": 1, + "last_column": 32, + "last_line": 1, + "path": "/var/lib/icinga2/api/packages/director/45f7ec2e-7aee-4110-a414-e32f6228e7ea/zones.d/main/hosts.conf" + }, + "icinga_state": 0, + "icinga_state_type": 1, + "icinga_templates": [ + "icinga-master", + "Default Host" + ], + "icinga_type": "Host", + "icinga_vars": { + "operating_system": "linux" + }, + "icinga_version": 0, + "icinga_volatile": false, + "icinga_zone": "main", + "inventory_dir": "/home/matthias/Ansible", + "inventory_file": "/home/matthias/Ansible/inventory-icinga.yml", + "inventory_hostname": "icinga-master", + "inventory_hostname_short": "icinga-master", + "playbook_dir": "/home/matthias/Ansible" + } +} +``` + +## Requirements + +This inventory plugin needs +- Python `requests` library to make API calls to Icinga 2 + +## Variables + +**plugin** + +This is a token that ensures that the plugin definitions are meant for this inventory plugin. +The form is `namespace.collection_name.plugin_name`. + +This must be `icinga.icinga.icinga` + +Required: `true` +Type: `string` +Default: `None` + +--- + +**url** + +The url to be used for API requests. + +Required: `false` +Type: `string` +Default: `https://localhost` + +--- + +**port** + +The port used by Icinga 2. + +Required: `false` +Type: `int` +Default: `5665` + +--- + +**user** + +The username to be used for API requests. + +Required: `true` +Type: `string` +Default: `None` + +--- + +**password** + +The password to be used for API requests. + +Required: `true` +Type: `string` +Default: `None` + +--- + +**validate_certs** + +Whether the certificates received when requesting the API should be validated to establish trust. + +Required: `false` +Type: `bool` +Default: `true` + +--- + +**ansible_user_var** + +You may decide to define the username for Ansible to connect as within your Icinga 2 host object. This allows the inventory to dynamically adapt the Ansible variable `ansible_user`. + +Required: `false` +Type: `string` +Default: `None` + +--- + +**filters** + +The `filters` variable allows for filtering the Icinga 2 hosts to be used within Ansible. In the background [Icinga 2 API filters](https://icinga.com/docs/icinga-2/latest/doc/12-icinga2-api/#filters) are used. +Options for the `filters` variable are explained in [their own section](#filter-options). + +--- + +**group_prefix** + +The inventory plugin automatically builds Ansible groups based on the Icinga 2 host object attributes `groups` and `zone`. +This prefix is used as a prefix for those groups within Ansible. + +By default groups will be prefixed with `icinga_group_` and `icinga_zone_` respectively. + +Required: `false` +Type: `string` +Default: `icinga_` + +--- + + +**want_ipv4** + +It is common practice to set an Icinga 2 host's name equal to its FQDN. This way DNS is the source of truth. But you may want to use a host's IP address for connections made by Ansible. +If `want_ipv4` is true, the Ansible variable `ansible_host` will be set to the Icinga host's `address` attribute if applicable. +`want_ipv4` takes precedence over `want_ipv6`. + +Required: `false` +Type: `bool` +Default: `false` + +--- + +**want_ipv6** + +Analogous to `want_ipv4` you may want to use a Icinga host's `address6` attribute to establish connections using Ansible. +If `want_ipv6` is true, the Ansible variable `ansible_host` will be set to the Icinga host's `address6` attribute if applicable. +`want_ipv4` takes precedence over `want_ipv6`. + +Required: `false` +Type: `bool` +Default: `false` + +--- + +**vars_prefix** + +All attributes of an Icinga 2 host will be used as host variables within Ansible. Those variables are prefixed with the `vars_prefix`. + +Required: `false` +Type: `string` +Default: `icinga_` + +--- + +## Using Constructed Inventory and Cache + +Other than the variables used for this plugin explicitly, you can also make use of some options offered by Ansible's Inventory Module [**Constructed**](https://docs.ansible.com/ansible/latest/collections/ansible/builtin/constructed_inventory.html). +Specifically `keyed_groups` might be useful. This allows you to build new Ansible groups based on Icinga host attributes, e.g. `vars["operating_system"]`. + +Example: + +**inventory-icinga.yml:** + +``` +--- +plugin: icinga.icinga.icinga +user: api-user +password: api-user-password + +groups: + # simple name matching + dummies: inventory_hostname.startswith('dummy') + +keyed_groups: + - prefix: "icinga_os" + key: vars["operating_system"] +``` + +``` +ansible -i inventory-icinga.yml localhost -m debug -a "msg='{{ groups }}'" +``` + +``` +localhost | SUCCESS => { + "msg": { + "all": [ + "icinga-master", + "dummy_host_1", + "dummy_host_2", + "dummy_host_3", + "dummy_host_4", + "dummy_host_5", + "test" + ], + "dummies": [ + "dummy_host_1", + "dummy_host_2", + "dummy_host_3", + "dummy_host_4", + "dummy_host_5" + ], + "icinga_group_testgroup1": [ + "test" + ], + "icinga_group_test_group_2": [ + "test" + ], + "icinga_os_linux": [ + "icinga-master" + "dummy_host_1", + "dummy_host_2" + ], + "icinga_os_windows": [ + "dummy_host_3", + "dummy_host_4", + "dummy_host_5" + ], + "icinga_zone_main": [ + "icinga-master", + "dummy_host_1", + "dummy_host_2", + "dummy_host_3", + "dummy_host_4", + "dummy_host_5" + ], + "icinga_zone_test": [ + "test" + ], + "ungrouped": [] + } +} +``` + +--- + +In order to minimize API queries made against Icinga you might want to use cache and retrieve host information this way. +The following is an example on how to use the Ansible builtin cache plugin `jsonfile`. + +``` +--- +plugin: icinga.icinga.icinga +user: api-user +password: api-user-password + +cache: true +cache_plugin: "jsonfile" +cache_connection: "/tmp/icinga_cache" +cache_timeout: 1800 +``` + +Here the plugin is told to +- use cache +- use the `jsonfile` cache plugin +- use `/tmp/icinga_cache` as its cache directory +- only use cache that is newer than `1800` seconds / 30 minutes + +If cache is used and the cache is valid, no API calls are made. + +## Filter Options + +You might want to restrict which hosts are part of the API query result. You can for example choose to only fetch host information for hosts within a specific Icinga zone. +If you use custom variables like e.g. *'operating_system'* to distinguish between different operating systems, you can use those variables to narrow down the results. + +Valid subkeys for `filters` are +- name +- group +- zone +- custom +- vars + +The subkeys 'name', 'group' and 'zone' are meant as defaults that allow for quick filtering based on those Icinga host attributes. +Internally Icinga's '**match**' filter is used for 'name' and 'zone'. This way string and boolean comparisons can be made while also allowing for pattern matching. +'group' uses the '**in**' filter to check membership. + +The following example shows filters to restrict the result to hosts +- that are part of **either** zone 'main' or zone 'satellite' +- whose names begin with 'dummy' (matching a pattern) +- that are part of the group 'linux\_hosts' + +**Example** + +Using CURL: + +``` +curl -k -u 'api-user':'api-user-password' -H 'X-HTTP-Method-Override: GET' -X POST https://localhost:5665/v1/objects/hosts -d '{ "filter": "((match(\"main\", host.zone)||match(\"satellite\", host.zone)))&&((match(\"dummy*\", host.name)))&&(((\"linux_hosts\" in host.groups)))" }' +``` + +Using the plugin: + +``` +--- +plugin: icinga.icinga.icinga +user: api-user +password: api-user-password + +filters: + zone: + - main + - satellite + name: + - dummy* + group: + - linux_hosts +``` + +In general, all subkeys of `filters` have to evaluate to true **simultaneously**. In the example above 'zone', 'name' and 'group' **must** all match. +Within one of those filter options **only one** of the list's elements must match. + +It is also possible to negate entries. In that case **either** entry in the list has to match while **neither** of the negated entries is allowed to match. + +To illustrate the logic, we will use 'group' as an example. + +``` +group: + - 'linux_hosts' + - 'windows_hosts' + - '!dns-server' + - '!web-server' +``` + +Logically this results in: ( in group 'linux_hosts' **OR** in group 'windows_hosts' ) **AND** ( **not** in group 'dns-server' **AND** **not** in group 'web-server' ) + +--- + +The 'custom' option lets you simply supply your own filter as it would be passed with CURL (`'{ "filter": "YOUR CUSTOM FILTER" }'`). +Supplying multiple filters will use logic AND to combine them. + +The last subkey you can use is 'vars'. It behaves slightly differently since one cannot anticipate the variables other people might use. +Therefore you have to manually decide on the filter and the custom variable to be used. + +Currently the '**match**', '**in**' and '**is**' filters are supported. As before all their subkeys have to match with only one entry that has to match while none of the negated entries are allowed to match. + +The '**is**' is filter is special in that regard that here you are supposed to only pass one value to it. If multiple values are passed, only the first value in the list will be used. +Also, negation is only allowed for 'set' and 'null' using the '**is**' filter. + +The following values are accepted: +- true +- false +- set +- !set +- null +- !null + +The value 'set' allows to check if the value of variable either evaluates to `true` or is set, meaning the variable **has** a value that is not `false` and not `null`. + +Example: + +``` +--- +plugin: icinga.icinga.icinga +user: api-user +password: api-user-password + +filters: + vars: + match: + linux_distribution: + - '!ubuntu' + - 'debian' + - 'centos' + - 'rhel' + - 'suse' + in: + services: + - 'dns' + - 'database' + is: + ansible_managed: true + ansible_user: set +``` + +This results in the following filter: `((match(\"debian\", host.vars.linux_distribution)||match(\"centos\", host.vars.linux_distribution)||match(\"rhel\", host.vars.linux_distribution)||match(\"suse\", host.vars.linux_distribution))&&(!match(\"ubuntu\", host.vars.linux_distribution))&&((\"dns\" in host.vars.services)||(\"database\" in host.vars.services))&&(host.vars.ansible_managed==true))` + +Prettier version: + +``` +( + ( + match(\"debian\", host.vars.linux_distribution)||match(\"centos\", host.vars.linux_distribution)||match(\"rhel\", host.vars.linux_distribution)||match(\"suse\", host.vars.linux_distribution) + ) + && + ( + !match(\"ubuntu\", host.vars.linux_distribution) + ) + && + ( + (\"dns\" in host.vars.services)||(\"database\" in host.vars.services) + ) + && + ( + host.vars.ansible_managed==true + ) + && + ( + host.vars.ansible_user + ) +) +``` diff --git a/plugins/inventory/icinga.py b/plugins/inventory/icinga.py new file mode 100644 index 00000000..25c1bad5 --- /dev/null +++ b/plugins/inventory/icinga.py @@ -0,0 +1,546 @@ +# -*- coding: utf-8 -*- +# pylint: disable=consider-using-f-string,super-with-arguments,attribute-defined-outside-init,too-many-instance-attributes + + +from urllib.parse import urlparse +from requests.auth import HTTPBasicAuth +from requests.exceptions import SSLError, RequestException +import requests + +from ansible.plugins.inventory import BaseInventoryPlugin, Cacheable, Constructable, to_safe_group_name +from ansible.module_utils._text import to_bytes, to_text +from ansible.errors import AnsibleError +from ansible.parsing.yaml.objects import AnsibleSequence, AnsibleMapping + +DOCUMENTATION = ''' + name: icinga + short_description: Icinga 2 inventory source + requirements: + - requests >= 1.1 + description: + - Get inventory hosts from Icinga 2. + - Uses Icinga 2's API to get information about hosts. + - The use of a custom filter is possible. + - Uses a YAML configuration file that ends with ``icinga.(yml|yaml)``. + extends_documentation_fragment: + - constructed + - inventory_cache + options: + plugin: + description: Token that ensures this is a source file for the C(icinga) plugin. + required: true + choices: ['icinga.icinga.icinga'] + url: + description: + - URL of the Icinga 2 server. + default: 'https://localhost' + port: + description: The port number used for API calls against Icinga 2. + type: int + default: 5665 + user: + description: + - The name of the user who accesses the Icinga 2 API. + required: true + password: + description: + - The password of the user who accesses the Icinga 2 API. + required: true + validate_certs: + description: + - Whether to validate Icinga 2 API certificates. + type: boolean + default: true + group_prefix: + description: + - Prefix to apply to Icinga 2 specific groups. + - By default hosts are also grouped by their zones. + - This prefix applies to both attributes, groups and zone. + - Results in groups named like C(PREFIX(GROUP|ZONE)VALUE). + type: string + default: icinga_ + vars_prefix: + description: + - Prefix to apply to host variables. + - Only affects Icinga 2 host specific attributes. + type: string + default: icinga_ + ansible_user_var: + description: + - The hosts' attribute to set as C(ansible_user). + type: string + want_ipv4: + description: + - Whether C(ansible_host) should be set to the host's C(address) attribute. + - C(want_ipv4) takes precedence over C(want_ipv6). + type: bool + default: false + want_ipv6: + description: + - Whether C(ansible_host) should be set to the host's C(address6) attribute. + - C(want_ipv4) takes precedence over C(want_ipv6). + type: bool + default: false + filters: + suboptions: + name: + description: + - Name(s) or pattern(s) to match specific hosts. + - If any of these match a host, it will be included. + type: list + elements: string + zone: + description: + - Restrict list of hosts to the requested zones. + type: list + elements: string + custom: + description: + - Custom filter(s) that will be passed as is. + type: list + elements: string + vars: + description: + - Restrict list of hosts based on custom variables. + type: list + elements: string +''' + +EXAMPLES = ''' +# inventory-icinga.yml +plugin: icinga.icinga.icinga +url: https://icinga.example.com +user: ansibleinventory +password: changeme + +# icinga.yaml +plugin: icinga.icinga.icinga +url: https://icinga.example.com +user: ansibleinventory +password: changeme +validate_certs: false +filters: + # Only get hosts with 'zone' attribute equal to 'main' or 'sat*' + zone: + - main + - sat* + # Only get hosts which are part of the group 'linux_hosts' + group: + - linux_hosts + vars: + is: + # Only get hosts whose variable 'ansible_managed' is set to true + ansible_managed: true + in: + # Only get hosts who have 'dns' or 'database' in their array variable 'services' + services: + - dns + - database +# Set Ansible's variable 'ansible_user' equal to the host's variable 'ansible_user' +ansible_user_var: vars.ansible_user +# Create groups with name 'icinga_os' + '_{VALUE OF VARIABLE}' and add hosts accordingly +keyed_groups: + - prefix: "icinga_os" + key: vars.operating_system +''' + + + +#class InventoryModule(BaseInventoryPlugin): +class InventoryModule(BaseInventoryPlugin, Cacheable, Constructable): + NAME = 'icinga' + + def verify_file(self, path): + ''' return true/false if this is possibly a valid file for this plugin to consume ''' + valid = False + if super(InventoryModule, self).verify_file(path): + if path.endswith(('icinga.yaml', 'icinga.yml')): + valid = True + else: + self.display.vvv('Skipping due to inventory source not ending in "icinga.yaml" nor "icinga.yml"') + return valid + + + def _get_recursive_sub_element(self, d, key_string): + delimiters = [ '.', '[', ']' ] + post_delimiter = '...' + final_keys = list() + new_d = d + + for delimiter in delimiters: + key_string = key_string.replace(delimiter, post_delimiter) + + keys = key_string.split(post_delimiter) + + # Remove empty entries and cast numbers to integers + for key in keys: + if not key: + continue + if key.isdigit(): + key = int(key) + else: + key = key.strip('\'').strip('"') + final_keys.append(key) + + # Recurse into structure + for index, key in enumerate(final_keys): + try: + new_d = new_d[key] + except IndexError: + self.display.vvvv(f'Structure \'{d}\' has no index \'{index}\' for sub-structure \'{new_d}\'.') + raise + except (KeyError, TypeError): + self.display.vvvv(f'Strucutre \'{d}\' has no key \'{key}\' for sub-structure \'{new_d}\'.') + raise + + return new_d + + + def _get_session(self): + self.session = requests.session() + self.session.auth = HTTPBasicAuth(to_bytes(self.icinga_user), to_bytes(self.icinga_password)) + self.session.headers = { + 'Accept': 'application/json', + 'X-HTTP-Method-Override': 'GET', + } + self.session.verify = self.validate_certs + return self.session + + + def _validate_filter(self): + valid_filter = True + valid_keys = [ 'name', 'group', 'zone', 'custom', 'vars' ] + valid_vars_keys = [ 'match', 'in', 'is' ] + invalid_keys = list() + + if self.filters: + # Check if each key is valid + for key in self.filters.keys(): + if key not in valid_keys: + valid_filter = False + invalid_keys.append(key) + + # Check validity of each 'vars' key subkey + if 'vars' in self.filters: + for sub_key in self.filters['vars']: + if sub_key not in valid_vars_keys: + valid_filter = False + invalid_keys.append(f'vars.{sub_key}') + + else: + valid_filter = False + + if invalid_keys: + raise AnsibleError(f'The following keys are not valid for \'filters\': {invalid_keys}') + + return valid_filter + + + def _create_filter_general(self, filter_string_base, key, values): + # Creates a filter for multiple list entries for a given key + # Handles logical concatenation and negation + # For each key at least on entry in the list must match their hosts according attribute + # The kind of filter ('match', 'in', 'is') is passed to this function + + # Create filters for positive matches logically combined by 'or', and negated matches logically combined by 'and' + # Combine both with a logical 'and' + # E.g.: name must match (A or B or C) AND must match (NOT D and NOT E) + sub_filter = list() + sub_filter_positive = list() + sub_filter_negative = list() + + for value in values: + tmp_string = filter_string_base.format(value, key) + if value.startswith('!'): + tmp_string = '!{}'.format(tmp_string.replace('!', '')) + sub_filter_negative.append(tmp_string) + else: + sub_filter_positive.append(tmp_string) + + if sub_filter_positive: + sub_filter_positive_string = '({})'.format('||'.join(sub_filter_positive)) + sub_filter = sub_filter + [sub_filter_positive_string] + if sub_filter_negative: + sub_filter_negative_string = '({})'.format('&&'.join(sub_filter_negative)) + sub_filter = sub_filter + [sub_filter_negative_string] + + sub_filter_string = '&&'.join(sub_filter) + + return sub_filter_string + + + def _create_is_filter(self, key, values): + true_filter_string = '(host.{}==true)' + false_filter_string = '(host.{}==false)' + null_filter_string = '(host.{}==null)' + set_or_true_filter_string = '(host.{})' + + values = [value.lower() for value in values] + # Only allow one value passed to a key of the 'is' filter + if len(values) > 1: + self.display.vvv(f'Multiple values provided to a key of the \'is\' filter. Only using first value \'{values[0]}\'.') + value = values[0] + + if value == 'true': + filter_string = true_filter_string.format(key) + elif value == 'false': + filter_string = false_filter_string.format(key) + elif value == 'set': + filter_string = set_or_true_filter_string.format(key) + elif value == '!set': + filter_string = '!' + set_or_true_filter_string.format(key) + # 'null' results in 'none' if passed wihtout string enclosure + elif value in [ 'none', 'null' ]: + filter_string = null_filter_string.format(key) + elif value in [ '!none', '!null' ]: + filter_string = '!' + null_filter_string.format(key) + else: + # Only allow 'true', 'false', 'set', '!set', 'null' and '!null' + self.display.vvv('Valid values for the \'is\' filter are: \'true\', \'false\', \'set\', \'!set\', \'null\' and \'!null\'.') + raise ValueError(f'\'{value}\' is not valid value for the \'is\' filter') + + return filter_string + + + def _create_in_filter(self, key, values): + filter_string_base = '(\\"{}\\" in host.{})' + sub_filter_string = self._create_filter_general(filter_string_base, key, values) + return sub_filter_string + + + def _create_match_filter(self, key, values): + filter_string_base = 'match(\\"{}\\", host.{})' + sub_filter_string = self._create_filter_general(filter_string_base, key, values) + return sub_filter_string + + + def _create_filter(self): + filter_string = '' + sub_filters = list() + + # Validate filter. Return empty filter if not valid. + if not self._validate_filter(): + return filter_string + + for key, value in self.filters.items(): + # Skip / ignore if an empty key has been passed + if not value: + self.display.vvv(f'Ignoring empty key \'{key}\'.') + continue + + # Make sure every key within filters is considered a list, cast string to single entry list + if isinstance(value, AnsibleMapping): + value = dict(value) + elif isinstance(value, AnsibleSequence): + value = [to_text(val) for val in value] + else: + value = [to_text(value)] + + # Special treatment for custom filter + if key == 'custom': + for entry in value: + sub_filters.append(entry) + sub_filter_string = '({})'.format(')&&('.join(value)) + + # Special treatment for groups + # Overwrite the filter_string + elif key == 'group': + sub_filter_string = self._create_in_filter('groups', value) + + # Special treatment for custom vars + elif key == 'vars': + local_filter_list = list() + for sub_key, attributes in value.items(): + for attribute_key, attribute_values in attributes.items(): + # Skip / ignore if an empty key has been passed + if not attribute_values and sub_key != 'is': + self.display.vvv(f'Ignoring empty key \'vars.{attribute_key}\'.') + continue + + # Make sure 'attribute_values' is considered a list, cast string to single entry list + if isinstance(attribute_values, AnsibleSequence): + attribute_values = [to_text(val) for val in attribute_values] + else: + attribute_values = [to_text(attribute_values)] + + # Choose correct filter + if sub_key == 'match': + tmp_string = self._create_match_filter(f'vars.{attribute_key}', attribute_values) + elif sub_key == 'in': + tmp_string = self._create_in_filter(f'vars.{attribute_key}', attribute_values) + elif sub_key == 'is': + tmp_string = self._create_is_filter(f'vars.{attribute_key}', attribute_values) + + local_filter_list.append(tmp_string) + + sub_filter_string = '&&'.join(local_filter_list) + + # Anything else will use the 'match' filter + else: + sub_filter_string = self._create_match_filter(key, value) + + sub_filters.append(sub_filter_string) + + if sub_filters: + filter_string = '({})'.format(')&&('.join(sub_filters)) + + return filter_string + + + def _get_hosts(self): + s = self._get_session() + + # Validate connection via API URL + try: + s.get(self.api_url) + except SSLError: + self.display.vvv('SSL Error: You may want to trust the certificate or pass \'validate_certs: false\'') + raise + except RequestException: + self.display.vvv('Error accessing \'{self.api_url}\'.') + raise + + # Create filter + filter_string = self._create_filter() + data = None + if filter_string: + data = '{ "filter": "' + filter_string + '" }' + + self.display.vvv(f'Using filter: \'{data}\'') + + response = s.post(self.api_url + '/objects/hosts', data=data) + + if response.status_code != 200: + raise ValueError(f'Something went wrong. HTTP status code: \'{response.status_code}\'. Icinga 2\'s API most likely did not understand the filter!') + + hosts = response.json()['results'] + return hosts + + + def _populate_inventory(self, hosts): + # Always add keyed groups for attribute 'zone' + zone_wanted = True + for keyed_group in self.keyed_groups: + if 'key' in keyed_group and keyed_group['key'] == 'zone': + zone_wanted = False + break + + if zone_wanted: + zone_key = {'prefix': self.group_prefix + 'zone', 'key': 'zone'} + self.keyed_groups.append(zone_key) + + for host in hosts: + host_name = host['name'] + host_vars = host['attrs'] + + # Add groups and make current host a member based on its 'groups' attribute + for group in host_vars['groups']: + group_name = to_safe_group_name(self.group_prefix + 'group_' + group) + self.inventory.add_group(group_name) + self.inventory.add_host(host_name, group_name) + + # Add host to group 'ungrouped' if it does not belong to a group + if not host_vars['groups']: + self.inventory.add_host(host_name, ) + + # Set attributes as host variables + for key, value in host_vars.items(): + self.inventory.set_variable(host_name, f'{self.vars_prefix}{key}', value) + + # Set 'ansible_host' to IP address if requested + if self.want_ipv4 and host_vars['address']: + self.inventory.set_variable(host_name, 'ansible_host', host_vars['address']) + self.display.vvv(f'Set attribute \'address\' as \'ansible_host\' for host \'{host_name}\'.') + elif self.want_ipv6 and host_vars['address6']: + self.inventory.set_variable(host_name, 'ansible_host', host_vars['address6']) + self.display.vvv(f'Set attribute \'address6\' as \'ansible_host\' for host \'{host_name}\'.') + + # Set 'ansible_user' if requested and defined on the Icinga 2 host + if self.ansible_user: + ansible_user_value = None + try: + ansible_user_value = self._get_recursive_sub_element(host_vars, self.ansible_user) + except (IndexError, KeyError, TypeError): + self.display.vvv(f'Could not set \'{self.ansible_user}\' as \'ansible_user\' for host \'{host_name}\'.') + + # Set 'ansible_user' + if ansible_user_value: + self.inventory.set_variable(host_name, 'ansible_user', ansible_user_value) + + # Add composite vars + self._set_composite_vars(self.compose, host_vars, host_name, strict=self.strict) + + # Add composed groups + self._add_host_to_composed_groups(self.groups, host_vars, host_name, strict=self.strict) + + # Add keyed_groups + self._add_host_to_keyed_groups(self.keyed_groups, host_vars, host_name, strict=self.strict) + + + def _validate_url(self, url): + valid = False + + try: + parsed_url = urlparse(url) + if parsed_url.scheme and parsed_url.netloc and parsed_url.path == '/v1': + valid = True + except ValueError: + pass + + return valid + + + def parse(self, inventory, loader, path, cache=True): + super(InventoryModule, self).parse(inventory, loader, path, cache) + + # Read config options from file for futher use + self._read_config_data(path) + + # Set attributes based on parsed file + self.icinga_url = self.get_option('url').strip('/') + self.icinga_port = self.get_option('port') + self.icinga_user = self.get_option('user') + self.icinga_password = self.get_option('password') + self.validate_certs = self.get_option('validate_certs') + self.filters = self.get_option('filters') + self.group_prefix = self.get_option('group_prefix') + self.vars_prefix = self.get_option('vars_prefix') + self.want_ipv4 = self.get_option('want_ipv4') + self.want_ipv6 = self.get_option('want_ipv6') + self.ansible_user = self.get_option('ansible_user_var') + + # Related to Ansible's Constructable + self.compose = self.get_option('compose') + self.groups = self.get_option('groups') + self.keyed_groups = self.get_option('keyed_groups') + self.strict = self.get_option('strict') + + # Build API URL and validate + self.api_url = f'{self.icinga_url}:{self.icinga_port}/v1' + if not self._validate_url(self.api_url): + raise ValueError(f'\'{self.api_url}\' is not a valid URL.') + + # Check if cache is available and should be used + cache_key = self.get_cache_key(path) + use_cache = self.get_option("cache") and cache + update_cache = self.get_option("cache") and not cache + + hosts = None + + # Get hosts from cache if available and recent + if use_cache: + try: + hosts = self._cache[cache_key] + self.display.vvv('Using existing cache.') + except KeyError: + self.display.vvv('Creating/updating cache.') + update_cache = True + + if not hosts: + # Get hosts from Icinga 2's API + hosts = self._get_hosts() + + if update_cache: + self._cache[cache_key] = hosts + + self._populate_inventory(hosts) diff --git a/tests/unittestpy3/test_inventory.py b/tests/unittestpy3/test_inventory.py new file mode 100644 index 00000000..8584ea80 --- /dev/null +++ b/tests/unittestpy3/test_inventory.py @@ -0,0 +1,602 @@ +#!/usr/bin/python3 + +import sys +sys.path.insert(0,'plugins/inventory') + +import unittest +from unittest.mock import patch, Mock, MagicMock, call + +import requests +from requests.exceptions import SSLError, RequestException +from ansible.parsing.yaml.objects import AnsibleSequence, AnsibleMapping + + +from icinga import InventoryModule + + +class TestInventoryPlugin(unittest.TestCase): + + def test_get_recursive_sub_element(self): + test_module = InventoryModule() + test_dict = { + "vars": { + "base_test": None, + "base test": None, + "index_test": [ { "test": True }, { "test": False } ], + "numeric_key": { "0": "zero" }, + "ansible_vars": { + "ansible_user": "ansible1", + "ansible user": { + "name": "ansible2", + }, + }, + "ansible vars": { + "ansible_user": "ansible3", + "ansible user": { + "name": "ansible4", + }, + }, + }, + } + + # Test cases that must succeed + test_cases = [ + ( 'vars.base_test', None ), + ( 'vars["base_test"]', None ), + ( 'vars[base_test]', None ), + ( 'vars.[base_test]', None ), + ( 'vars...[base_test]', None ), + ( 'vars...base_test', None ), + + ( 'vars.base test', None ), + ( 'vars["base test"]', None ), + ( 'vars[base test]', None ), + ( 'vars.[base test]', None ), + ( 'vars...[base test]', None ), + ( 'vars...base test', None ), + + ( 'vars.index_test[0].test', True ), + ( 'vars.index_test[1].test', False ), + ( 'vars.numeric_key["0"]', 'zero' ), + + ( 'vars.ansible_vars.ansible_user', 'ansible1' ), + ( 'vars["ansible_vars"]["ansible_user"]', 'ansible1' ), + ( 'vars.ansible_vars["ansible_user"]', 'ansible1' ), + ( 'vars.["ansible_vars"].ansible_user', 'ansible1' ), + + ( 'vars.ansible_vars.ansible user.name', 'ansible2' ), + ( 'vars.ansible_vars.["ansible user"].name', 'ansible2' ), + + ( 'vars.ansible vars.ansible_user', 'ansible3' ), + ( 'vars.["ansible vars"].ansible_user', 'ansible3' ), + ( 'vars.[ansible vars].ansible_user', 'ansible3' ), + + ( 'vars.ansible vars.ansible user.name', 'ansible4' ), + ( 'vars.["ansible vars"].ansible user.name', 'ansible4' ), + ( 'vars.[ansible vars].[ansible user].name', 'ansible4' ), + + ( 'vars.ansible vars.ansible user', { "name": "ansible4" } ), + ] + + for search_string, expected in test_cases: + self.assertEqual(test_module._get_recursive_sub_element(test_dict, search_string), expected) + + # Test cases that must throw exceptions + fail_cases = [ + 'vars.bad_key', + 'vars.index_test["0"].test', + 'vars.numeric_key[0]', + 'vars.index_test[100]', + ] + + for search_string in fail_cases: + with self.assertRaises((IndexError, KeyError, TypeError)) as context: + test_module._get_recursive_sub_element(test_dict, search_string) + + + @patch('icinga.InventoryModule') + def test_filter_key_validity(self, mock_init): + # Test with known good filters + mock_init.filters = { + "name": None, + "group": None, + "zone": None, + "custom": None, + "vars": { + "match": None, + "in": None, + }, + } + + expected = True + actual = InventoryModule._validate_filter(mock_init) + self.assertEqual(expected, actual) + + # Test with bad keys + mock_init.filters = { + "bad_key": None, + } + + with self.assertRaises(Exception) as context: + InventoryModule._validate_filter(mock_init) + + # Test with None filters key + mock_init.filters = None + expected = False + actual = InventoryModule._validate_filter(mock_init) + self.assertEqual(expected, actual) + + + @patch('icinga.InventoryModule') + def test_filter_vars_keys_validty(self, mock_init): + # Test with known good keys + mock_init.filters = { + "vars": { + "match": None, + "in": None, + }, + } + + expected = True + actual = InventoryModule._validate_filter(mock_init) + self.assertEqual(expected, actual) + + # Test with bad keys + mock_init.filters = { + "vars": { + "bad_key": None, + } + } + + with self.assertRaises(Exception) as context: + InventoryModule._validate_filter(mock_init) + + + @patch('icinga.InventoryModule') + def test_api_url_validity(self, mock_init): + valid_urls = [ + 'http://localhost:5665/v1', + 'https://localhost:5665/v1', + 'https://localhost/v1', + 'https://127.0.0.1:5665/v1', + 'https://127.0.0.1/v1', + 'https://[::1]:5665/v1', + 'https://[::1]/v1', + ] + invalid_urls = [ + 'https://localhost:5665', + 'localhost:5665/v1', + 'https://[::1]/', + '[::1]/v1', + 'https:///v1', + 'https://', + ] + + expected = True + for url in valid_urls: + actual = InventoryModule._validate_url(mock_init, url) + self.assertEqual(expected, actual) + + expected = False + for url in invalid_urls: + actual = InventoryModule._validate_url(mock_init, url) + self.assertEqual(expected, actual) + + + def test_create_is_filter(self): + items = [ + ( 'test_variable', ['True'], '(host.test_variable==true)' ), + ( 'test_variable', ['true'], '(host.test_variable==true)' ), + + ( 'test_variable', ['False'], '(host.test_variable==false)' ), + ( 'test_variable', ['false'], '(host.test_variable==false)' ), + + ( 'test_variable', ['set'], '(host.test_variable)' ), + ( 'test_variable', ['SET'], '(host.test_variable)' ), + + ( 'test_variable', ['!set'], '!(host.test_variable)' ), + ( 'test_variable', ['!SET'], '!(host.test_variable)' ), + + ( 'test_variable', ['Null'], '(host.test_variable==null)' ), + ( 'test_variable', ['null'], '(host.test_variable==null)' ), + + ( 'test_variable', ['None'], '(host.test_variable==null)' ), + ( 'test_variable', ['none'], '(host.test_variable==null)' ), + + ( 'test_variable', ['!Null'], '!(host.test_variable==null)' ), + ( 'test_variable', ['!null'], '!(host.test_variable==null)' ), + + ( 'test_variable', ['!None'], '!(host.test_variable==null)' ), + ( 'test_variable', ['!none'], '!(host.test_variable==null)' ), + + ( 'test_variable', [ '!set', 'True', 'False', 'set'], '!(host.test_variable)' ), + ] + + test_module = InventoryModule() + + for key, value, expected in items: + actual = test_module._create_is_filter(key, value) + self.assertEqual(expected, actual) + + # Expected Exception + invalid_item = ( 'test_variable', ['anything_else'] ) + with self.assertRaises(ValueError) as context: + test_module._create_is_filter(invalid_item[0], invalid_item[1]) + + + def test_create_in_filter(self): + items = [ + ( 'groups', [ + 'group1', + ], + r'((\"group1\" in host.groups))' + ), + ( 'groups', [ + 'group1', + 'group2', + ], + r'((\"group1\" in host.groups)||(\"group2\" in host.groups))' + ), + ( 'groups', [ + '!group1', + ], + r'(!(\"group1\" in host.groups))' + ), + ( 'groups', [ + '!group1', + 'group2', + ], + r'((\"group2\" in host.groups))&&(!(\"group1\" in host.groups))' + ), + ( 'groups', [ + '!group1', + 'group2', + '!group3', + 'group4', + ], + r'((\"group2\" in host.groups)||(\"group4\" in host.groups))&&(!(\"group1\" in host.groups)&&!(\"group3\" in host.groups))' + ), + ] + + test_module = InventoryModule() + + for key, values, expected in items: + actual = test_module._create_in_filter(key, values) + self.assertEqual(expected, actual) + + + def test_create_match_filter(self): + items = [ + ( 'zone', [ + 'zone1', + ], + r'(match(\"zone1\", host.zone))' + ), + ( 'zone', [ + 'zone1', + 'zone2', + ], + r'(match(\"zone1\", host.zone)||match(\"zone2\", host.zone))' + ), + ( 'zone', [ + '!zone1', + ], + r'(!match(\"zone1\", host.zone))' + ), + ( 'zone', [ + '!zone1', + 'zone2', + ], + r'(match(\"zone2\", host.zone))&&(!match(\"zone1\", host.zone))' + ), + ( 'zone', [ + '!zone1', + 'zone2', + '!zone3', + 'zone4', + ], + r'(match(\"zone2\", host.zone)||match(\"zone4\", host.zone))&&(!match(\"zone1\", host.zone)&&!match(\"zone3\", host.zone))' + ), + ] + + test_module = InventoryModule() + + for key, values, expected in items: + actual = test_module._create_match_filter(key, values) + self.assertEqual(expected, actual) + + + def test_create_filter(self): + self.maxDiff = None + test_module = InventoryModule() + # Cannot pass an AnsibleMapping right now + test_module.filters = { + "name": "satellite", + "group": AnsibleSequence([ "testgroup1", "testgroup2" ]), + "zone": AnsibleSequence([ "zone1", "zone2", "yetanotherzone" ]), + "custom": AnsibleSequence([ 'match(\\"some_custom_filter\\", host.name)' ]), + "vars": AnsibleMapping({ "match": { "operating_system": "win*"}, "in": { "services": None }}), + } + + test_module._validate_filter = MagicMock(return_value=True) + actual = test_module._create_filter() + expected = r'((match(\"satellite\", host.name)))&&(((\"testgroup1\" in host.groups)||(\"testgroup2\" in host.groups)))&&((match(\"zone1\", host.zone)||match(\"zone2\", host.zone)||match(\"yetanotherzone\", host.zone)))&&(match(\"some_custom_filter\", host.name))&&((match(\"some_custom_filter\", host.name)))&&((match(\"win*\", host.vars.operating_system)))' + self.assertEqual(expected, actual) + + # Test with empty return filter if given filter is invalid + test_module._validate_filter = MagicMock(return_value=False) + actual = test_module._create_filter() + expected = '' + self.assertEqual(expected, actual) + + + + + def test_populate_inventory(self): + test_module = InventoryModule() + test_module.inventory = MagicMock() + test_module.inventory.add_host = MagicMock() + test_module.inventory.add_group = MagicMock() + test_module.inventory.set_variable = MagicMock() + test_module._set_composite_vars = MagicMock() + test_module._add_host_to_composed_groups = MagicMock() + test_module._add_host_to_keyed_groups = MagicMock() + test_module._add_host_to_composed_groups = MagicMock() + + # Set object attributes + test_module.vars_prefix = 'icinga_' + test_module.want_ipv4 = True + test_module.want_ipv6 = True + test_module.ansible_user = 'vars.ansible_user' + test_module.keyed_groups = [ { "prefix": "icinga_os", "key": "vars['operating_system']" } ] + test_module.group_prefix = 'icinga_' + test_module.compose = None + test_module.groups = None + test_module.strict = False + + test_hosts = [ + { + "name": "dummy_host1", + "attrs": { + "address": "127.0.0.1", + "address6": "", + "groups": [ "testgroup1", "testgroup2" ], + "zone": "master", + }, + }, + { + "name": "dummy_host2", + "attrs": { + "address": "", + "address6": "::1", + "command_endpoint": "", + "groups": [ "testgroup1", "testgroup3" ], + "templates": [ "dummy_host2", "Default Host" ], + "vars": { + "operating_system": "linux", + "distribution": "debian", + "ansible_user": "myansibleuser", + }, + "zone": "satellite1", + }, + }, + { + "name": "dummy_host3", + "attrs": { + "address": "", + "address6": "", + "groups": [], + "zone": "anotherzone", + }, + }, + ] + + calls_add_group = [ + call('icinga_group_testgroup1'), + call('icinga_group_testgroup2'), + call('icinga_group_testgroup1'), + call('icinga_group_testgroup3'), + ] + calls_add_host = [ + call('dummy_host1', 'icinga_group_testgroup1'), + call('dummy_host1', 'icinga_group_testgroup2'), + call('dummy_host2', 'icinga_group_testgroup1'), + call('dummy_host2', 'icinga_group_testgroup3'), + call('dummy_host3') + ] + calls_set_variable = [ + call('dummy_host1', 'icinga_address', '127.0.0.1'), + call('dummy_host1', 'icinga_address6', ''), + call('dummy_host1', 'icinga_groups', ['testgroup1', 'testgroup2']), + call('dummy_host1', 'icinga_zone', 'master'), + call('dummy_host1', 'ansible_host', '127.0.0.1'), + call('dummy_host2', 'icinga_address', ''), + call('dummy_host2', 'icinga_address6', '::1'), + call('dummy_host2', 'icinga_command_endpoint', ''), + call('dummy_host2', 'icinga_groups', ['testgroup1', 'testgroup3']), + call('dummy_host2', 'icinga_templates', ['dummy_host2', 'Default Host']), + call('dummy_host2', 'icinga_vars', {'operating_system': 'linux', 'distribution': 'debian', 'ansible_user': 'myansibleuser'}), + call('dummy_host2', 'icinga_zone', 'satellite1'), + call('dummy_host2', 'ansible_host', '::1'), + call('dummy_host2', 'ansible_user', 'myansibleuser'), + call('dummy_host3', 'icinga_address', ''), + call('dummy_host3', 'icinga_address6', ''), + call('dummy_host3', 'icinga_groups', []), + call('dummy_host3', 'icinga_zone', 'anotherzone'), + ] + calls_composite = [ + call( None, + { 'address': '127.0.0.1', + 'address6': '', + 'groups': ['testgroup1', 'testgroup2'], + 'zone': 'master' + }, + 'dummy_host1', + strict=False + ), + call( None, + { 'address': '', + 'address6': '::1', + 'command_endpoint': '', + 'groups': ['testgroup1', 'testgroup3'], + 'templates': ['dummy_host2', 'Default Host'], + 'vars': { + 'operating_system': 'linux', + 'distribution': 'debian', + 'ansible_user': 'myansibleuser' + }, + 'zone': 'satellite1' + }, + 'dummy_host2', + strict=False + ), + call( None, + { 'address': '', + 'address6': '', + 'groups': [], + 'zone': 'anotherzone', + }, + 'dummy_host3', + strict=False, + ), + ] + calls_composed = [ + call( None, + { 'address': '127.0.0.1', + 'address6': '', + 'groups': ['testgroup1', 'testgroup2'], + 'zone': 'master' + }, + 'dummy_host1', + strict=False), + call( None, + { 'address': '', + 'address6': '::1', + 'command_endpoint': '', + 'groups': ['testgroup1', 'testgroup3'], + 'templates': ['dummy_host2', 'Default Host'], + 'vars': { + 'operating_system': 'linux', + 'distribution': 'debian', + 'ansible_user': 'myansibleuser' + }, + 'zone': 'satellite1' + }, + 'dummy_host2', + strict=False), + call( None, + { 'address': '', + 'address6': '', + 'groups': [], + 'zone': 'anotherzone' + }, + 'dummy_host3', + strict=False), + ] + calls_keyed = [ + call([ + {'prefix': 'icinga_os', + 'key': "vars['operating_system']"}, + {'prefix': 'icinga_zone', + 'key': 'zone'} + ], + {'address': '127.0.0.1', + 'address6': '', + 'groups': ['testgroup1', 'testgroup2'], + 'zone': 'master' + }, + 'dummy_host1', + strict=False + ), + call([ + {'prefix': 'icinga_os', + 'key': "vars['operating_system']"}, + {'prefix': 'icinga_zone', + 'key': 'zone'} + ], + {'address': '', + 'address6': '::1', + 'command_endpoint': '', + 'groups': ['testgroup1', 'testgroup3'], + 'templates': ['dummy_host2', 'Default Host'], + 'vars': { + 'operating_system': 'linux', + 'distribution': 'debian', + 'ansible_user': 'myansibleuser' + }, + 'zone': 'satellite1'}, + 'dummy_host2', + strict=False), + call([ + {'prefix': 'icinga_os', + 'key': "vars['operating_system']"}, + {'prefix': 'icinga_zone', + 'key': 'zone'} + ], + {'address': '', + 'address6': '', + 'groups': [], + 'zone': 'anotherzone'}, + 'dummy_host3', + strict=False), + ] + + test_module._populate_inventory(test_hosts) + # Assertions + test_module.inventory.add_group.assert_has_calls(calls_add_group, any_order=False) + test_module.inventory.add_host.assert_has_calls(calls_add_host, any_order=False) + test_module.inventory.set_variable.assert_has_calls(calls_set_variable, any_order=False) + test_module._set_composite_vars.assert_has_calls(calls_composite, any_order=False) + test_module._add_host_to_composed_groups.assert_has_calls(calls_composed, any_order=False) + test_module._add_host_to_keyed_groups.assert_has_calls(calls_keyed, any_order=False) + + + @patch('icinga.InventoryModule') + def test_get_hosts(self, mock_init): + + class mocked_get_session_good(): + def get(*args, **kwargs): + return True + def post(*args, **kwargs): + m = Mock() + m.status_code = 200 + m.json.return_value = { "results": [] } + return m + + class mocked_get_session_sslerror(): + def get(*args, **kwargs): + raise SSLError + + class mocked_get_session_requestexception(): + def get(*args, **kwargs): + raise RequestException + + class mocked_get_session_bad_status_code(): + def get(*args, **kwargs): + return True + def post(*args, **kwargs): + m = Mock() + m.status_code = 404 + m.json.return_value = { "error": 404, "status": "No objects found." } + return m + + # Should succeed and return hosts (empty list) + mock_init._get_session.return_value = mocked_get_session_good() + expected = list() + actual = InventoryModule._get_hosts(mock_init) + self.assertEqual(actual, expected) + + # Should raise SSLError + mock_init._get_session.return_value = mocked_get_session_sslerror() + with self.assertRaises(SSLError) as context: + InventoryModule._get_hosts(mock_init) + + # Should raise RequestException + mock_init._get_session.return_value = mocked_get_session_requestexception() + with self.assertRaises(RequestException) as context: + InventoryModule._get_hosts(mock_init) + + # Should raise ValueError + mock_init._get_session.return_value = mocked_get_session_bad_status_code() + with self.assertRaises(ValueError) as context: + InventoryModule._get_hosts(mock_init) From 1b1412ae813beb6de0713da712e41fb61fe02843 Mon Sep 17 00:00:00 2001 From: Thilo W Date: Fri, 12 Jan 2024 16:27:43 +0100 Subject: [PATCH 34/39] removed misleading example. --- doc/role-icinga2/features/feature-api.md | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/doc/role-icinga2/features/feature-api.md b/doc/role-icinga2/features/feature-api.md index 85ebf785..2368247c 100644 --- a/doc/role-icinga2/features/feature-api.md +++ b/doc/role-icinga2/features/feature-api.md @@ -61,13 +61,31 @@ ca_host_port: 5665 ``` > [!INFO] -> In case your agent can't connect to the CA host, you can use the variables `icinga2_delegate_host` +> In case your agent can't connect to the CA host/master, you can change ca_host to your satellite. +> In addition you can use the variables `icinga2_delegate_host` > and `ticket_salt` to delegate ticket creation to one of your satellites instead. +> But is will also work because the delegation task will be initiated by the Ansible controlhost. + +Example if connection and ticket creation should be on the satellite: ```yaml -ca_host: icinga-server.localdomain +icinga2_features: + - name: api + ca_host: icinga-satellite.localdomain + ticket_salt: "{{ icinga2_constants.ticket_salt }}" + [...] icinga2_delegate_host: icinga-satellite.localdomain -ticket_salt: "{{ icinga2_constants.ticket_salt }}" +``` +Example if agent should connect to satellite and the tickets are generated on the +master host. + +```yaml +icinga2_features: + - name: api + ca_host: icinga-satellite.localdomain + ticket_salt: "{{ icinga2_constants.ticket_salt }}" + [...] +icinga2_delegate_host: icinga-master.localdomain ``` By default the FQDN is used as certificate common name, to put a name From ae4e3377e0060f4a48a8b61e3e7131bbc415f584 Mon Sep 17 00:00:00 2001 From: Donien <88634789+Donien@users.noreply.github.com> Date: Fri, 12 Jan 2024 16:51:48 +0100 Subject: [PATCH 35/39] Add variable for dependency repositories within monitoring_plugins role (#209) * Fix #173 This commit allows for a new variable to be used. 'icinga_monitoring_plugins_dependency_repos' defaults to either 'crb' or 'powertools' but can be overriden to temporarily enable one or multiple repositories of choice in case manual intervention is needed. * Add note about use of wildcard for dependency repositories * Add changelog fragment about new variable 'icinga_monitoring_plugins_dependency_repos' --- changelogs/fragments/fix-173-changeable-dependency-repos.yml | 2 ++ doc/role-monitoring_plugins/role-monitoring_plugins.md | 5 +++++ roles/monitoring_plugins/defaults/main.yml | 2 ++ roles/monitoring_plugins/tasks/install_on_RedHat.yml | 2 +- 4 files changed, 10 insertions(+), 1 deletion(-) create mode 100644 changelogs/fragments/fix-173-changeable-dependency-repos.yml diff --git a/changelogs/fragments/fix-173-changeable-dependency-repos.yml b/changelogs/fragments/fix-173-changeable-dependency-repos.yml new file mode 100644 index 00000000..d201297a --- /dev/null +++ b/changelogs/fragments/fix-173-changeable-dependency-repos.yml @@ -0,0 +1,2 @@ +minor_changes: + - Add variable `icinga_monitoring_plugins_dependency_repos` to allow for later modification by the user if specific other repositories need to be activated instead of `powertools` / `crb` diff --git a/doc/role-monitoring_plugins/role-monitoring_plugins.md b/doc/role-monitoring_plugins/role-monitoring_plugins.md index 7fb71f59..6ce4cd5b 100644 --- a/doc/role-monitoring_plugins/role-monitoring_plugins.md +++ b/doc/role-monitoring_plugins/role-monitoring_plugins.md @@ -15,6 +15,11 @@ The list is based on the section *"Plugin Check Commands for Monitoring Plugins" Decides whether to run the equivalent of `dnf --enablerepo crb ...` when installing the necessary packages. Default: `false` Enabling CRB/Powertools may be necessary, depending on the plugins wanted and the repositories already enabled. +- `icinga_monitoring_plugins_dependency_repos: list` + Decides which repositories are temporarily enabled when installing packages. Defaults to either `crb` or `powertools`. + If the need arises, you can manually overwrite this variable to temporarily enable one or multiple repositories of your choice. + You may specify `icinga_monitoring_plugins_dependency_repos: "*"` to temporarily enable every repository present. + - `icinga_monitoring_plugins_remove: boolean` Decides whether to remove packages that have not been asked for by the user. Default `true` The requested check commands are compared against the list of available check commands. Packages not required for installation are removed. diff --git a/roles/monitoring_plugins/defaults/main.yml b/roles/monitoring_plugins/defaults/main.yml index e23ae1e4..9680940b 100644 --- a/roles/monitoring_plugins/defaults/main.yml +++ b/roles/monitoring_plugins/defaults/main.yml @@ -5,3 +5,5 @@ icinga_monitoring_plugins_epel: false icinga_monitoring_plugins_crb: false icinga_monitoring_plugins_remove: true icinga_monitoring_plugins_autoremove: false +icinga_monitoring_plugins_dependency_repos: + - "{{ 'powertools' if ansible_distribution_major_version == '8' and icinga_monitoring_plugins_crb else 'crb' if ansible_distribution_major_version == '9' and icinga_monitoring_plugins_crb }}" diff --git a/roles/monitoring_plugins/tasks/install_on_RedHat.yml b/roles/monitoring_plugins/tasks/install_on_RedHat.yml index 1c95d76f..964bf8a9 100644 --- a/roles/monitoring_plugins/tasks/install_on_RedHat.yml +++ b/roles/monitoring_plugins/tasks/install_on_RedHat.yml @@ -27,7 +27,7 @@ state: present name: "{{ needed_packages }}" update_cache: true - enablerepo: "{{ 'powertools' if ansible_distribution_major_version == '8' and icinga_monitoring_plugins_crb else 'crb' if ansible_distribution_major_version == '9' and icinga_monitoring_plugins_crb }}" + enablerepo: "{{ icinga_monitoring_plugins_dependency_repos }}" when: - ansible_distribution_major_version >= "8" - needed_packages is defined From 00d06081eb4fdb14573e323ea5ea43662dba2c31 Mon Sep 17 00:00:00 2001 From: Thilo W Date: Fri, 12 Jan 2024 16:55:40 +0100 Subject: [PATCH 36/39] Feature/enhance icingadb retention config (#253) * added retention config variables * added local tests, added testinfra test for configs * added test for default build * added documentation for the variables * added changelog fragment --- ...ure_enhance_icingadb_retention_configs.yml | 4 ++ doc/role-icingadb/role-icingadb.md | 15 +++++++ .../default/host_vars/icinga-default.yaml | 11 ++++++ .../integration/test_icingadb_retentions.py | 13 +++++++ .../host_vars/icinga-default.yaml | 11 ++++++ molecule/local-default/molecule.yml | 5 +++ .../integration/test_icingadb_retentions.py | 11 ++++++ roles/icingadb/templates/icingadb.ini.j2 | 39 ++++++++++++------- 8 files changed, 94 insertions(+), 15 deletions(-) create mode 100644 changelogs/fragments/feature_enhance_icingadb_retention_configs.yml create mode 100644 molecule/default/tests/integration/test_icingadb_retentions.py create mode 100644 molecule/local-default/tests/integration/test_icingadb_retentions.py diff --git a/changelogs/fragments/feature_enhance_icingadb_retention_configs.yml b/changelogs/fragments/feature_enhance_icingadb_retention_configs.yml new file mode 100644 index 00000000..a8f430d0 --- /dev/null +++ b/changelogs/fragments/feature_enhance_icingadb_retention_configs.yml @@ -0,0 +1,4 @@ +--- +minor_changes: + - "Enhance IcingaDB retention configs #200" + - "Added tests for retention configs" diff --git a/doc/role-icingadb/role-icingadb.md b/doc/role-icingadb/role-icingadb.md index c8c3604b..dce15d4a 100644 --- a/doc/role-icingadb/role-icingadb.md +++ b/doc/role-icingadb/role-icingadb.md @@ -57,6 +57,21 @@ For logging, currently only the **logging level** can be set. The default is `in |----------|------|-------------|---------| | `icingadb_logging_level` | `fatal\|error\|warn\|info\|debug` | Defines the logging level for IcingaDB. | `info` | +### IcingaDB Retention + +| Variable | Type | Description | Default | +|----------|------|-------------|---------| +|`icingadb_retention_history_days`|`number`|Number of days to retain full historical data.|By default, historical data is retained forever.| +|`icingadb_retention_sla_days`|`number`|Number of days to retain historical data for SLA reporting.|By default, it is retained forever.| +|`icingadb_retention_acknowledgement`|`number`|Number of days to retain acknowledgements|If not limited by icingadb_retention_history_days, forever| +|`icingadb_retention_comment`|`number`|Number of days to retain comments|If not limited by icingadb_retention_history_days, forever| +|`icingadb_retention_downtime`|`number`|Number of days to retain downtimes|If not limited by icingadb_retention_history_days, forever| +|`icingadb_retention_flapping`|`number`|Number of days to retain flapping events|If not limited by icingadb_retention_history_days, forever| +|`icingadb_retention_notification`|`number`|Number of days to retain notifications|If not limited by icingadb_retention_history_days, forever| +|`icingadb_retention_state`|`number`|Number of days to retain states|If not limited by icingadb_retention_history_days, forever| + + + ### Miscellaneous The following variables are used for the IcingaDB setup and are not directly related to the configuration of IcingaDB itself. Normally, you can rely on the defaults to work and should **not** change them unless you know what you are doing. diff --git a/molecule/default/host_vars/icinga-default.yaml b/molecule/default/host_vars/icinga-default.yaml index 9505b147..1370e9a9 100644 --- a/molecule/default/host_vars/icinga-default.yaml +++ b/molecule/default/host_vars/icinga-default.yaml @@ -1,3 +1,14 @@ +# test_icingadb_retentions.py +icingadb_retention_history_days: 10 +icingadb_retention_sla_days: 11 +icingadb_retention_acknowledgement: 20 +icingadb_retention_comment: 30 +icingadb_retention_downtime: 10 +icingadb_retention_state: 60 +icingadb_retention_notification: 4 +icingadb_retention_flapping: 2 + + icinga2_custom_config: - name: icinga2_command path: zones.d/main/commands/custom_commands.conf diff --git a/molecule/default/tests/integration/test_icingadb_retentions.py b/molecule/default/tests/integration/test_icingadb_retentions.py new file mode 100644 index 00000000..c3b91f15 --- /dev/null +++ b/molecule/default/tests/integration/test_icingadb_retentions.py @@ -0,0 +1,13 @@ +# Vars in host_vars + +def test_icingadb_config(host): + i2_file = host.file("/etc/icingadb/config.yml") + assert i2_file.is_file + assert i2_file.contains(' history-days: 10') + assert i2_file.contains(' sla-days: 11') + assert i2_file.contains(' acknowledgement: 20') + assert i2_file.contains(' comment: 30') + assert i2_file.contains(' state: 60') + assert i2_file.contains(' downtime: 10') + assert i2_file.contains(' notification: 4') + assert i2_file.contains(' flapping: 2') diff --git a/molecule/local-default/host_vars/icinga-default.yaml b/molecule/local-default/host_vars/icinga-default.yaml index c1c8f186..6bfebbf6 100644 --- a/molecule/local-default/host_vars/icinga-default.yaml +++ b/molecule/local-default/host_vars/icinga-default.yaml @@ -1,3 +1,14 @@ +# test_icingadb_retentions.py +icingadb_retention_history_days: 10 +icingadb_retention_sla_days: 11 +icingadb_retention_acknowledgement: 20 +icingadb_retention_comment: 30 +icingadb_retention_downtime: 10 +icingadb_retention_state: 60 +icingadb_retention_notification: 4 +icingadb_retention_flapping: 2 + + icinga2_objects: icinga-default: - name: root diff --git a/molecule/local-default/molecule.yml b/molecule/local-default/molecule.yml index 0a29467c..a8355099 100644 --- a/molecule/local-default/molecule.yml +++ b/molecule/local-default/molecule.yml @@ -19,3 +19,8 @@ provisioner: inventory: link: host_vars: host_vars/ +verifier: + name: testinfra + options: + sudo: true + directory: tests/integration/ diff --git a/molecule/local-default/tests/integration/test_icingadb_retentions.py b/molecule/local-default/tests/integration/test_icingadb_retentions.py new file mode 100644 index 00000000..1a356733 --- /dev/null +++ b/molecule/local-default/tests/integration/test_icingadb_retentions.py @@ -0,0 +1,11 @@ +def test_icingadb_config(host): + i2_file = host.file("/etc/icingadb/config.yml") + assert i2_file.is_file + assert i2_file.contains(' history-days: 10') + assert i2_file.contains(' sla-days: 11') + assert i2_file.contains(' acknowledgement: 20') + assert i2_file.contains(' comment: 30') + assert i2_file.contains(' state: 60') + assert i2_file.contains(' downtime: 10') + assert i2_file.contains(' notification: 4') + assert i2_file.contains(' flapping: 2') diff --git a/roles/icingadb/templates/icingadb.ini.j2 b/roles/icingadb/templates/icingadb.ini.j2 index 8b5dd0b6..cf012ac1 100644 --- a/roles/icingadb/templates/icingadb.ini.j2 +++ b/roles/icingadb/templates/icingadb.ini.j2 @@ -84,19 +84,28 @@ logging: # Retention is an optional feature to limit the number of days that historical data is available, # as no historical data is deleted by default. retention: - # Number of days to retain full historical data. By default, historical data is retained forever. -# history-days: - - # Number of days to retain historical data for SLA reporting. By default, it is retained forever. -# sla-days: - - # Map of history category to number of days to retain its data in order to - # enable retention only for specific categories or to - # override the number that has been configured in history-days. +{% if icingadb_retention_history_days is defined and icingadb_retention_history_days is number %} + history-days: {{ icingadb_retention_history_days }} +{% endif %} +{% if icingadb_retention_sla_days is defined and icingadb_retention_sla_days is number %} + sla-days: {{ icingadb_retention_sla_days }} +{% endif %} options: -# acknowledgement: -# comment: -# downtime: -# flapping: -# notification: -# state: +{% if icingadb_retention_acknowledgement is defined and icingadb_retention_acknowledgement is number %} + acknowledgement: {{ icingadb_retention_acknowledgement }} +{% endif %} +{% if icingadb_retention_comment is defined and icingadb_retention_comment is number %} + comment: {{ icingadb_retention_comment }} +{% endif %} +{% if icingadb_retention_downtime is defined and icingadb_retention_downtime is number %} + downtime: {{ icingadb_retention_downtime }} +{% endif %} +{% if icingadb_retention_flapping is defined and icingadb_retention_flapping is number %} + flapping: {{ icingadb_retention_flapping }} +{% endif %} +{% if icingadb_retention_notification is defined and icingadb_retention_notification is number %} + notification: {{ icingadb_retention_notification }} +{% endif %} +{% if icingadb_retention_state is defined and icingadb_retention_state is number %} + state: {{ icingadb_retention_state }} +{% endif %} From de30b3e603b12501b0012cc9f6c39e00b68cf4bc Mon Sep 17 00:00:00 2001 From: Gianmarco Mameli <57061995+gianmarco-mameli@users.noreply.github.com> Date: Tue, 16 Jan 2024 09:34:26 +0100 Subject: [PATCH 37/39] Fix icinga2_objects secondary usage (#255) * fixes icinga2_objects variable usage * adds changelog --- changelogs/fragments/fix_issue_228.yml | 3 +++ roles/icinga2/tasks/objects.yml | 4 ++-- 2 files changed, 5 insertions(+), 2 deletions(-) create mode 100644 changelogs/fragments/fix_issue_228.yml diff --git a/changelogs/fragments/fix_issue_228.yml b/changelogs/fragments/fix_issue_228.yml new file mode 100644 index 00000000..bceb8909 --- /dev/null +++ b/changelogs/fragments/fix_issue_228.yml @@ -0,0 +1,3 @@ +--- +bugfixes: + - Fixed collect of icinga2_objects when icinga2_config_host is not defined (#228) \ No newline at end of file diff --git a/roles/icinga2/tasks/objects.yml b/roles/icinga2/tasks/objects.yml index 6ac27328..c6f36cb2 100644 --- a/roles/icinga2/tasks/objects.yml +++ b/roles/icinga2/tasks/objects.yml @@ -7,8 +7,8 @@ - name: collect all config objects in play vars set_fact: - tmp_objects: "{{ tmp_objects| default([]) + lookup('list', icinga2_objects[icinga2_config_host]) }}" - when: vars['icinga2_objects'][icinga2_config_host] is defined + tmp_objects: "{{ tmp_objects| default([]) + lookup('list', icinga2_objects) }}" + when: vars['icinga2_objects'][icinga2_config_host] is not defined - icinga2_object: args: "{{ item }}" From c4373c52cd076da1d9b9636f27c8ad4a9dd8132f Mon Sep 17 00:00:00 2001 From: Thilo W Date: Tue, 16 Jan 2024 09:36:05 +0100 Subject: [PATCH 38/39] adds documentation for secondary icinga2_objects usage. --- doc/role-icinga2/objects.md | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/doc/role-icinga2/objects.md b/doc/role-icinga2/objects.md index 89eb4355..0be3a5bb 100644 --- a/doc/role-icinga2/objects.md +++ b/doc/role-icinga2/objects.md @@ -32,6 +32,22 @@ icinga2_objects: parent: main ``` +The advantage of the default **icinga2_objects** variable is, that you can run your playbook over many different server without deploying the +monitoring configuration on every host in the playbook. Otherwise the variable should be only placed in `host_vars` files to restrict deployment on every host. + +As a secondary option, you can use the variable without the second level like the following example. + +> **CAUTION!** If not restricted it will be deployed on every host. This should be only defined in `host_vars` unless +you know what you are doing! + +``` +icinga2_objects: + - name: "{{ ansible_fqdn }}" + type: Endpoint + file: "{{ 'conf.d/' + ansible_hostname + '.conf' }}" + order: 20 +``` + More Examples at the end -> [Examples](#examples) ## Managing Config directories From 9fbc7da34399e708e18220d5100f0fca364ccc88 Mon Sep 17 00:00:00 2001 From: Thilo W Date: Tue, 16 Jan 2024 18:03:13 +0100 Subject: [PATCH 39/39] fixes fails if icinga2_objects arent defined which is the default --- roles/icinga2/tasks/objects.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/roles/icinga2/tasks/objects.yml b/roles/icinga2/tasks/objects.yml index c6f36cb2..93d10302 100644 --- a/roles/icinga2/tasks/objects.yml +++ b/roles/icinga2/tasks/objects.yml @@ -8,7 +8,7 @@ - name: collect all config objects in play vars set_fact: tmp_objects: "{{ tmp_objects| default([]) + lookup('list', icinga2_objects) }}" - when: vars['icinga2_objects'][icinga2_config_host] is not defined + when: icinga2_objects is defined and vars['icinga2_objects'][icinga2_config_host] is not defined - icinga2_object: args: "{{ item }}"