Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow to define custom agents in system test configuration files #1765

Merged
merged 25 commits into from
Apr 18, 2024

Conversation

mrodm
Copy link
Contributor

@mrodm mrodm commented Apr 12, 2024

Relates #787

In this PR, it is added into the system test configuration files first fields in order to run custom agents with different parameters (capabilities, pid mode, user...). This is just taken into account when independent agents are enabled.

A new test package has been added to test this auditd_manager_independent_agent.

@mrodm mrodm self-assigned this Apr 12, 2024
internal/service/boot.go Outdated Show resolved Hide resolved
jsoriano added a commit that referenced this pull request Apr 15, 2024
This will allow to leverage the new custom definitions added in #1765, without
forcing the packages to define a service deployer.
This is useful when the only requirement to test the package is a running agent,
for example when collecting information from the system where it is running.
@mrodm
Copy link
Contributor Author

mrodm commented Apr 15, 2024

Failed CI Steps
:go: Integration test: with-custom-agent

Currently, the error in this step is caused because the previous Elastic Agent created by the previous package tested has not been unenrolled and removed from Fleet.

This could be solved by forcefully removing those agents after each test (without waiting to be acknowledge it by the agent itself). This is done in #1771

Comment on lines +25 to +30
# if it is not a list, rename it back to error.message
- set:
field: error.message
copy_from: auditd.warnings
ignore_empty_value: true
if: ctx?.auditd?.warnings != null && !(ctx.auditd.warnings instanceof List)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added this new step in the pipeline to detect if there is just one warning (string)

warnings as a string

Adding this would make fail this test package when ELASTIC_PACKAGE_TEST_ENABLE_INDEPENDENT_AGENT is false, since this will use the Elastic Agent from the stack that does not have the required capabilities.

Should we skip this test package in the CI scripts ? @jsoriano

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we skip this test package in the CI scripts ? @jsoriano

Yeah, we will need to find a way to test this package only when independent agents are enabled.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added an exception in the test script to not test auditd_manager_independent_agent when using the Elastic Agent from the stack.

Comment on lines 6 to 12
agent:
user: "root"
# shared_pid_address_space: true ?
pid_mode: "host"
capabilities:
- AUDIT_CONTROL
- AUDIT_READ
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently, this file allows additional properties. However, it probably would be better to at least add agent field as an object in this file. WDYT ?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes please.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Created PR for this : elastic/package-spec#739

@mrodm mrodm force-pushed the independent_custom_agents branch from 22d0abd to 97f085d Compare April 16, 2024 16:00
Comment on lines 9 to 10
# shared_pid_address_space: true ?
pid_mode: "host"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This setting looks like that it is more related to docker runtime.

It is needed to specify somehow this setting for auditd_manager package to keep the same behavior. That setting is present in the current configuration:

Should it be added here even if it is more docker related ? or maybe do we need to add a map for docker specific settings ?

agent:
  user: root
  runtime: docker
  capabilities:
    - AUDIT_CONTROL
    - AUDIT_READ
  docker:
    pid_mode: host

Maybe it can be changed the name of this field to use_host_pid_namespace: true

WDYT @jsoriano ?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PID namespacing is not specific of docker, so I would not use an specific docker variable. Not sure about the variable name to use but I wouldn't use a boolean because there can be more than one option.

@mrodm mrodm marked this pull request as ready for review April 17, 2024 13:41
@mrodm mrodm requested a review from jsoriano April 17, 2024 13:41
Copy link
Member

@jsoriano jsoriano left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

Comment on lines 83 to 84
// Capabilities is a list of the capabilities needed to run the Elastic Agent process
Capabilities []string
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit. Maybe we can call this LinuxCapabilities to avoid confusion with other capabilities.

Suggested change
// Capabilities is a list of the capabilities needed to run the Elastic Agent process
Capabilities []string
// LinuxCapabilities is a list of the capabilities needed to run the Elastic Agent process
LinuxCapabilities []string

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, do you think it should be updated that naming in the configuration file for system tests ? @jsoriano

Currently, those settings in the configuration file looks like:

agent:
  runtime: docker
  user: "root"
  pid_mode: "host"
  capabilities:
    - AUDIT_CONTROL
    - AUDIT_READ
   runtime: docker
   user: "root"
   pid_mode: "host"
-  capabilities:
+  linux_capabilities:
     - AUDIT_CONTROL
     - AUDIT_READ

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, call them linux_capabilities in the config.

@elasticmachine
Copy link
Collaborator

💚 Build Succeeded

History

cc @mrodm

@mrodm mrodm merged commit 6eda8fe into elastic:main Apr 18, 2024
3 checks passed
@mrodm mrodm deleted the independent_custom_agents branch April 18, 2024 07:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants