Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add containerized Prometheus/Grafana stack deployment #214

Open
wants to merge 3 commits into
base: containers
Choose a base branch
from

Conversation

witekest
Copy link
Contributor

@witekest witekest commented Mar 13, 2024

Install and configure Prometheus, Alertmanager, Grafana and Blackbox exporter containers with Podman.

Implements SUSE/spacewalk#19509

[Container]
Label=app=alertmanager
ContainerName=alertmanager
Image=registry.opensuse.org/devel/bci/sle-15-sp6/containerfile/suse/alertmanager:0.26.0
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you want images from opensuse registry for SUMA as well?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is the point I wanted to discuss. For now I have just hard-coded them here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I checked that monitoring images are not being published to registry.suse.com. Please let me know if we want to have them there for our purposes.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@admd @rjmateus probably for you

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From my point of view we cannot have opensuse images delivered for SUMA. We have a special SKU so we must have SUSE delivered images through registry.suse.com with special access control matching the needed SKU.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct! We must use the images through registry.suse.com

Minor fixes for Prometheus formula
@witekest witekest changed the title Add containerized Prometheus stack deployment Add containerized Prometheus/Grafana stack deployment Mar 15, 2024
@witekest witekest requested a review from agraul March 18, 2024 09:39
Copy link
Member

@agraul agraul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just took a superficial view. I saw that the formula interacts with the service state and execution modules. These won't work when targetting SLE Micro. With SUMA 5.0 itself running on SLE Micro 5.5, I would expect that our formula can deploy onto a SLE Micro system.

I only commented on the first two instances, but there have been more.

Comment on lines +168 to +175
module.run:
- name: service.systemctl_reload
service.running:
- name: grafana
- enable: true
- watch:
- file: /etc/containers/systemd/grafana.*
- file: /etc/grafana/provisioning/datasources/datasources.yml
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The service state/execution module calls won't work on SLE Micro.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What are the limitations here? I read in the documentation that Podman integrates with systemd on SLE Micro.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

state.apply is executed inside a transaction (think: transactional-update run salt-call state.apply ...) and there is no dbus access inside the transaction. At least not as of today, but enabling it also comes with problems...

We have a card to enable SUMA to avoid the transactional-update wrapping, but it's not picked up yet.

Just to be clear, systemd and podman work together. Controlling that with Salt won't work when targeting transactional systems.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. That's a good point.

Comment on lines +196 to +204
service.dead:
- name: grafana
- enable: false
file.absent:
- names:
- /etc/containers/systemd/grafana.container
- /etc/containers/systemd/grafana.volume
module.run:
- name: service.systemctl_reload
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The service state/execution module calls won't work on SLE Micro.

Copy link
Member

@rjmateus rjmateus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similar comments to other reviews

[Container]
Label=app=grafana
ContainerName=grafana
Image=registry.opensuse.org/devel/bci/sle-15-sp6/containerfile/suse/grafana:9.5.8
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The same as the other images, it needs to be configurable and for suse manager it needs to come from suse.registry.com with proper authentication

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ack

Label=app=grafana
ContainerName=grafana
Image=registry.opensuse.org/devel/bci/sle-15-sp6/containerfile/suse/grafana:9.5.8
Volume=/etc/grafana:/etc/grafana:ro
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this means the grafana config files must exist on the Host OS will be mounted as read-only on the container? To adapt the configuration users should change the config files on the host OS?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, the formula takes care of providing the config files.
And yes, users can modify these using the formula or manually.

Label=app=alertmanager
ContainerName=alertmanager
Image=registry.opensuse.org/devel/bci/sle-15-sp6/containerfile/suse/alertmanager:0.26.0
Volume=/etc/prometheus:/etc/prometheus:ro
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same question as before, it will be set on the host os, and mount as read-only on the container, right?

[Container]
Label=app=blackbox_exporter
ContainerName=blackbox_exporter
Image=registry.opensuse.org/devel/bci/sle-15-sp6/containerfile/suse/blackbox_exporter:0.24.0
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same question about the image coming from registry.suse.com

[Container]
Label=app=prometheus
ContainerName=prometheus
Image=registry.opensuse.org/devel/bci/sle-15-sp6/containerfile/suse/prometheus:2.37.6
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

some about the registry

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants