[Meta] Highlight packages that need to be run as root

[jlind23]

We are driving an effort to encourage the declaration packages that need to be run as root.
This option has been added here and will require new package version in order to let developers define this value.

By default all packages are NOT marked as requiring root privileges to properly run which might lead to confusion if users try to deploy a package that need root privileges but that was never updated.

Tasks

Preview Give feedback

1. Mark appropriate linux integrations as requiring root. #8647
   +0
2. Universal Profiling: add privileges section #8651
   Integration:universal_profiling_agent enhancement +2
3. [System tests] Run system tests for integrations with a non root Elastic Agent elastic-package#1586
   Team:Fleet +1
   

[elasticmachine]

Pinging @elastic/fleet (Team:Fleet)

[jlind23]

@epixa @bturquet could you please share with us any related issues you might have on your end?

[nimarezainia]

fyi:

@jamiehynds @andrewkroh for Security external integrations
@SubhrataK @vinaychandrasekhar for the o11y integrations

[epixa]

We're figuring out impacted packages on the security side. @norrietaylor's already linked the linux host integrations that require root, and @pzl's opened a PR to update the endpoint package.

[florianl]

Universal Profiling is affected as well but should be fixed with #8651.

Just out of curiosity:
What is the expected behavior if agent.privileges.root = true from the manifest of the integration package conflicts with runtime.preventions.condition in the specification of EA supervised processes? And why do we need both?

[jlind23]

Both are not linked for now but need to be done in a near future where Agent will need to report through metadata the mode it is running in.
We need both to avoid something like:

• Users install an integration that needs root privileges but the agent is not run as root
• Users will be able to do it as the UI can not prevent this because Fleet does not know in which mode the Agent is running

[andrewkroh]

Should we be doing this for packages (or data streams) that normally require admin access on Windows too? Or should we wait for Elastic Agent to implement this feature for Windows (equivalent to elastic/elastic-agent#3598)?

[nimarezainia]

I'd say do it for all the packages and datastreams. Whether we have the implementation for Windows or not. At the moment, this mainly allows us to notify the user and there are no controls or checks to see what mode the agent is running.

[nimarezainia]

@graphaelli & @simitt wanted to also bring this issue to your attention.

[lucabelluccini]

Added https://github.com/elastic/integration-docs/issues/288

[jlind23]

Thanks @lucabelluccini.
@norrietaylor @epixa were you able to make some progress on your end?

[jlind23]

@bturquet @lalit-satapathy @tommyers-elastic any follow up issue to add here or will all 'your' integrations work out of the box without root/admin privilegeS?

[bturquet]

Last time we discussed it we could not think of any issues for Hosted Services

[nimarezainia]

Last time we discussed it we could not think of any issues for Hosted Services

@bturquet does your team have a suite of smoktests for us to determine how the proposed changes will impact the integrations? We have a linux image that is going through some smoketesting. It would be great to be as comprehensive as possible.

[simitt]

The apm-server itself does not require root privileges, so we do not expect issues with the apm-server. Once this is available for testing, please let us know so we can verify everything still works as expected.

[jlind23]

@simitt this is available for testing on linux host once you provide the --unprivileged flag.
cc @pierrehilbert @cmacknz

[simitt]

@jlind23 I'm a bit confused now, my understanding is as follows: starting in version x (8.13?) the Elastic Agent will by default not run sub-processes with root privileges anymore. Integrations that do require root privileges need to update the package spec to specifically set agent.privileges.root: true. APM Server does not want to be run with root privileges, therefore there are no plans for updating apm package.
My assumption was that once a PR with the Elastic Agent changes is ready (or merged), we can test with this Elastic Agent branch, without any changes on apm side. If everything still works, we are good.

It's not clear to me from reading this and related issues where the --unprivileged flag comes in.

[pierrehilbert]

Hey @simitt
The Agent will continue to run with root privileges by default in 8.13 and the --unprivileged (hidden flag for now as this is only experimental) is the way to run Agent without root privileges.

[lalit-satapathy]

@bturquet @lalit-satapathy @tommyers-elastic any follow up issue to add here or will all 'your' integrations work out of the box without root/admin privilegeS?

Hi @jlind23,

Is there a quick way to find out which integration would require a root access to be run? Are there any static checks today, which will fail, if agent.privileges is not set and the integration requires a root permission?

Out of the long list integrations, How do we find some specific corner case integration requiring root permission, if any ?

[cmacknz]

Out of the long list integrations, How do we find some specific corner case integration requiring root permission, if any ?

More specifically than root, it will be anything that a basic user on the system cannot do without being granted additional privileges. There are generally two categories here:

1. Requiring elevated privileges to use restricted OS or kernel level features. On Linux things like BPF, installing kernel modules, network packet capture, possibly interacting with the audit framework. I will make a bit of an assumption here that most integrations doing this know they are doing it, and you are unlikely to be surprised that you fall into this category.
2. Reading files whose default permissions are restricted to root or specific groups. A good example is the system integration where on Linux it tries to read from /var/log/auth.log and /var/log/syslog by default, on Ubuntu systems you need to be in the adm group to do this. See No System logs are generated for Linux agent, when agent is installed with --unprivileged flag. elastic-agent#4112 (comment). On Windows this would be having additional group memberships like Performance Monitor Users and Event Logs Reader.

That second category will be the one that causes the most problems, because anything that reads a log file is potentially affected depending on where that log file is located.

This should allow us to exclude certain integrations from being affected quickly as well, for example integrations that are only making HTTP requests should not require root.

Is there a quick way to find out which integration would require a root access to be run? Are there any static checks today, which will fail, if agent.privileges is not set and the integration requires a root permission?

One thing that complicates this is that the tests in this repository run in containers, and container file systems do not always match up with the official native file system of that Linux distribution. For example the ubuntu:22.04 container does not contain /var/log/auth.log or /var/log/syslog but an Ubuntu 22.04 VM image does.

[cmacknz]

This should allow us to exclude certain integrations from being affected quickly as well, for example integrations that are only making HTTP requests should not require root.

A quick addition to this, there are ports that require root to bind to them (those <1024) so integrations that implement servers are potentially affected depending on what ports they use.

[lalit-satapathy]

@cmacknz, This is a good list of what to look for checking packages requiring root access. Assuming system & windows packages are being looked into already.

A quick addition to this, there are ports that require root to bind to them (those <1024) so integrations that implement servers are potentially affected depending on what ports they use.

Does stated input fit into this category? This is one of inputs, where we are spawning a server in metricbeat.

[jlind23]

@lalit-satapathy You are probably the most knowledgeable about this input as according to the CODEOWNERS file your team owns it.

The Elastic Agent team will not test each and every integration, this has to be done by the packages owners.
We are working on two things:

• With elastic-package allowing system tests to run with a non root user - [System tests] Run system tests for integrations with a non root Elastic Agent elastic-package#1586
• elastic-agent integration tests were also updated - [Unprivileged] Adjust integration tests to use unprivileged by default elastic-agent#3931

[jlind23]

@kpollich What do you think about closing this as the only remaining task is elastic/elastic-package#1586? Using only elastic/elastic-package#1586 is probably enough wdyt?

[kpollich]

Yes we can close this issue in favor of the Elastic Package work. FYI that issue is blocked on elastic/elastic-package#787 for now.