From f04882ff1bfeb0f1bc490e6178a7083a619ec7d0 Mon Sep 17 00:00:00 2001 From: ripienaar Date: Wed, 18 Dec 2024 21:43:03 +0000 Subject: [PATCH] deploy: 3a432a7e21749b386e4bb4db2a45574d6d62c49e --- 404.html | 2 +- adr/001/index.html | 8 ++++---- adr/index.html | 6 +++--- broker/index.html | 6 +++--- broker/monitoring/index.html | 6 +++--- categories/index.html | 6 +++--- configuration/index.html | 6 +++--- index.html | 10 +++++----- installation/index.html | 6 +++--- packaging/index.html | 6 +++--- previews/index.html | 6 +++--- previews/protov2/index.html | 14 +++++++------- server/index.html | 6 +++--- server/monitoring/index.html | 6 +++--- tags/index.html | 6 +++--- 15 files changed, 50 insertions(+), 50 deletions(-) diff --git a/404.html b/404.html index cbd5daa7..019f9184 100644 --- a/404.html +++ b/404.html @@ -1,2 +1,2 @@ 404 Page not found :: Choria Server and Broker -

44

Not found

Woops. Looks like this page doesn't exist ¯\_(ツ)_/¯.

Go to homepage

\ No newline at end of file +

44

Not found

Woops. Looks like this page doesn't exist ¯\_(ツ)_/¯.

Go to homepage

\ No newline at end of file diff --git a/adr/001/index.html b/adr/001/index.html index 6d0ad2d5..7c1299aa 100644 --- a/adr/001/index.html +++ b/adr/001/index.html @@ -1,8 +1,8 @@ V2 Network Protocol :: Choria Server and Broker -

V2 Network Protocol

Background

Current State

The Choria V1 protocol requires x509, there is no other option.

Experience with mcollective showed that companies had little appetite to understand or use esoteric security protocols, even those based on sound industry standard cryptographic methods. Further most companies already had mature x509 infrastructure either standalone or part of systems like Puppet which made for an easy adoption path.

Choria therefor is built on a protocol that requires x509 semantics and supports nothing else:

  • Every message is signed by a x509 key
  • Every client-initiated message contains the x509 public key
  • Servers listen on TLS using x509 public and private keys
  • User identities are tied to the properties of the certificates
  • User permissions and roles are tied to the identity of the certificate, very limited support for fine-grained roles
  • All JWT files are signed by x509 keys
  • Generally only RSA is supported
  • mTLS is used to create a private network - requiring a purpose specific CA or intermediate CA in practice
  • A local cache of certificates is used like a kind of safety check to catch credentials being re-issued
  • Any NATS client connecting to Choria broker requires a cert/key/ca and do full mTLS

Choria supports a mode combined with the AAA Server where a user does not need her own x509 certificate but holds a JWT instead obtained from an SSO service. In this scenario the AAA Server still holds a x509 cert (called a privileged certificate) and is allowed to sign and encode requests on behalf of others (holders of a JWT) by using its x509 key pair to sign on behalf of the user.

Problem Statement

In practice this works well for most users, but those with very large or complex networks can run into problems either with corporate Certificate Authority policies or corporate CA configuration making it very hard to achieve a secure network with the facilities available. General CA infrastructure is also just getting a bit old and better alternatives exist. JWTs are now used pervasively in modern IT and the user based are more likely to accept their use for our needs.

We would like to explore, and enable, other use cases outside of server management such as IoT, purpose built backplanes, Kubernetes side cars and more, integration into Certificate Authorities in those cases is really problematic.

  • x509 signatures are big and slow
  • x509 certificates are also huge and these go with every request
  • x509 certificate management, especially across multiple client machines is very difficult
  • Dealing with short-lived (minutes) x509 certificates can be really hard
  • Not all CAs have sane enrollment, some requires private keys to be copied around making them useless
  • Newer technologies like ed25519 is attractive as they use small keys and signatures and can also do things like DH Key exchange
  • Using ed25519 opens the possibility of using ssh keys as signing keys, something many users have requested
  • Obtaining private CAs or intermediate CAs is often impossible forcing CA reuse and nullifying the usefulness of the mTLS security
  • Verification of certificates happen during caching rather than a separate check, more importantly the cache is used often as a means of retrieving privileged certs and more by id
  • The local cert cache is deeply embedded in the v1 protocol, but it’s proven to be useless and most people disable its enforcing features - it cannot be disabled entirely
  • As Choria have evolved we need a much more granular role based permissions on each connection - can they use streams, can they admin streams, can they make rpc requests etc
  • More and more servers need to be able to make request either when publishing signed registration payloads or when interacting with Choria Services from within autonomous agents. With v1 protocol this was not possible.

Solution Overview

We will support an additional protocol, while maintaining v1 protocol, that will be based on JWTs signed using ed25519 that embed their public component in the signed JWT.

We have already done the work to design and incorporate the JWTs for clients and servers at the transport level but now need to move that work into the Choria protocol so clients can communicate with Choria without x509 certificates.

Transport Security

Transport security is primarily about how the packets between Choria Broker and Choria Client/Server are secured but touches a bit on how users with access to the NATS Server are limited to just their traffic.

mTLS optional Chain of Trust for transport security

In a mTLS secured network trust is established by way of the CA signing all certificates and all parties verifying this fact. Both sides of the TLS connection will verify the other side to ensure its certificate is signed by a cached local CA chain.

This way there can be no untrusted middlemen or session hijacking in the network path.

In practice, in enterprise networks, there are many challenges:

  • it’s often highly desirable to be able to hijack connections, for example to audit that PII doesn’t leave the legal jurisdiction it belongs in, many companies wish to disable the full mTLS while still maintaining strong identification of entities.
  • it’s also desirable to deploy Choria Brokers using websocket protocol and offload the TLS work onto a load balancer - this would be quite complex with the current model.
  • most enterprise have policies making CA rotation necessary and quite frequent, this makes mTLS completely unworkable at scale.
  • dropping mTLS, or not being able to do mTLS, would impact the strength of identity since we combine the transport and identity in one cert
  • certificate enrolment is done poorly, there are few protocols like ACME in use in Enterprises, often they would design their own enrollment, and mostly it involves moving keys between machines and are very slow

So we will support verified or unverified TLS to the brokers, but clients and servers all must present a signed JWT when not in mTLS mode. This will split the identity from transport security and give people the choice to pick one without compromising the other.

Servers and clients will have:

  • An ed25519 seed file example server.seed kept private and never transmitted over the network
  • A JWT file that holds:
    • the public key of the ed25519 seed
    • an identity
    • a private network ID derived from the identity or seed
    • Standard items like expiry times etc
    • Additional permissions to control access to broker subjects
  • The JWT will be signed by a trusted ed25519 key

The broker will only allow connections that holds a valid, signed JWT where the signature is made using a trusted ed25519 key.

Organization and Chained Issuers

Conceptually Choria Broker can separate connections into something called “accounts” in NATS terms, for Choria we call this an Organization. Today we support just one Organization - choria - but we will look to support more in future.

We will add a concept called an Organization Issuer that is a ed25519 key that must, on a basic level, sign all tokens. To facilitate separation of concerns in our centralized AAA mode the Issuer can delegate JWT issuing in a chain to downstream issuers. The tokens will validate as being signed by the issuer essentially.

We support storing the Organization Issuer offline in something like Vault and at the “AAA Login Server” level that issuing key can also be stored in a Vault like system.

Chain Issuers -Chain Issuers

Thus, we achieve the following chain of trust:

  • We know the JWT was issued by a trusted issuer
  • We verify the connecting server or client has the ed25519 seed that match the JWT because the broker force a NONCE to be signed using the seed, the seed never traverse the network since the public part is embedded in the signed JWT
  • The JWT tokens are therefor not bearer tokens and if they are stolen in-flight (remember no mTLS), one cannot connect with them or make any requests using them
  • We have an identity that isn’t tied to the common name of a cert and so is more fluid and adaptable
  • The broker should support denying all clients without JWTs

In this way we can pick full mTLS when needed or JWT+ed25519 mode when needed and even mix and match the modes.

Reply Security

Reply security is a quite difficult problem to solve since every reply would need to be individually encrypted and decrypted - unlike transport security - this is very CPU intensive, so we have never really done it at scale.

Still, its highly desirable to hide replies destined for a specific user from other users, even those with access to the broker using their own JWT tokens.

Using the callerID as key we calculate a private inbox using the hex encoded md5(callerID), we use this to construct reply subjects for all uses, even access to other subjects like the Streams API.

The broker will set up permissions ensuring that only the callerID can access replies. This way as long as there are unique callerIDs the replies from all systems are private.

We still do not encrypt the traffic in-transit (see point about PII and desired man-in-the-middle in Enterprises), but the replies are private to the user.

% sudo choria jwt client.jwt
+
V2 Network Protocol
Background
Current State
The Choria V1 protocol requires x509, there is no other option.
Experience with mcollective showed that companies had little appetite to understand or use esoteric security protocols, even those based on sound industry standard cryptographic methods. Further most companies already had mature x509 infrastructure either standalone or part of systems like Puppet which made for an easy adoption path.
Choria therefor is built on a protocol that requires x509 semantics and supports nothing else:
  • Every message is signed by a x509 key
  • Every client-initiated message contains the x509 public key
  • Servers listen on TLS using x509 public and private keys
  • User identities are tied to the properties of the certificates
  • User permissions and roles are tied to the identity of the certificate, very limited support for fine-grained roles
  • All JWT files are signed by x509 keys
  • Generally only RSA is supported
  • mTLS is used to create a private network - requiring a purpose specific CA or intermediate CA in practice
  • A local cache of certificates is used like a kind of safety check to catch credentials being re-issued
  • Any NATS client connecting to Choria broker requires a cert/key/ca and do full mTLS
Choria supports a mode combined with the AAA Server where a user does not need her own x509 certificate but holds a JWT instead obtained from an SSO service. In this scenario the AAA Server still holds a x509 cert (called a privileged certificate) and is allowed to sign and encode requests on behalf of others (holders of a JWT) by using its x509 key pair to sign on behalf of the user.
Problem Statement
In practice this works well for most users, but those with very large or complex networks can run into problems either with corporate Certificate Authority policies or corporate CA configuration making it very hard to achieve a secure network with the facilities available. General CA infrastructure is also just getting a bit old and better alternatives exist. JWTs are now used pervasively in modern IT and the user based are more likely to accept their use for our needs.
We would like to explore, and enable, other use cases outside of server management such as IoT, purpose built backplanes, Kubernetes side cars and more, integration into Certificate Authorities in those cases is really problematic.
  • x509 signatures are big and slow
  • x509 certificates are also huge and these go with every request
  • x509 certificate management, especially across multiple client machines is very difficult
  • Dealing with short-lived (minutes) x509 certificates can be really hard
  • Not all CAs have sane enrollment, some requires private keys to be copied around making them useless
  • Newer technologies like ed25519 is attractive as they use small keys and signatures and can also do things like DH Key exchange
  • Using ed25519 opens the possibility of using ssh keys as signing keys, something many users have requested
  • Obtaining private CAs or intermediate CAs is often impossible forcing CA reuse and nullifying the usefulness of the mTLS security
  • Verification of certificates happen during caching rather than a separate check, more importantly the cache is used often as a means of retrieving privileged certs and more by id
  • The local cert cache is deeply embedded in the v1 protocol, but it’s proven to be useless and most people disable its enforcing features - it cannot be disabled entirely
  • As Choria have evolved we need a much more granular role based permissions on each connection - can they use streams, can they admin streams, can they make rpc requests etc
  • More and more servers need to be able to make request either when publishing signed registration payloads or when interacting with Choria Services from within autonomous agents. With v1 protocol this was not possible.
Solution Overview
We will support an additional protocol, while maintaining v1 protocol, that will be based on JWTs signed using ed25519 that embed their public component in the signed JWT.
We have already done the work to design and incorporate the JWTs for clients and servers at the transport level but now need to move that work into the Choria protocol so clients can communicate with Choria without x509 certificates.
Transport Security
Transport security is primarily about how the packets between Choria Broker and Choria Client/Server are secured but touches a bit on how users with access to the NATS Server are limited to just their traffic.
mTLS optional Chain of Trust for transport security
In a mTLS secured network trust is established by way of the CA signing all certificates and all parties verifying this fact. Both sides of the TLS connection will verify the other side to ensure its certificate is signed by a cached local CA chain.
This way there can be no untrusted middlemen or session hijacking in the network path.
In practice, in enterprise networks, there are many challenges:
  • it’s often highly desirable to be able to hijack connections, for example to audit that PII doesn’t leave the legal jurisdiction it belongs in, many companies wish to disable the full mTLS while still maintaining strong identification of entities.
  • it’s also desirable to deploy Choria Brokers using websocket protocol and offload the TLS work onto a load balancer - this would be quite complex with the current model.
  • most enterprise have policies making CA rotation necessary and quite frequent, this makes mTLS completely unworkable at scale.
  • dropping mTLS, or not being able to do mTLS, would impact the strength of identity since we combine the transport and identity in one cert
  • certificate enrolment is done poorly, there are few protocols like ACME in use in Enterprises, often they would design their own enrollment, and mostly it involves moving keys between machines and are very slow
So we will support verified or unverified TLS to the brokers, but clients and servers all must present a signed JWT when not in mTLS mode. This will split the identity from transport security and give people the choice to pick one without compromising the other.
Servers and clients will have:
  • An ed25519 seed file example server.seed kept private and never transmitted over the network
  • A JWT file that holds:
    • the public key of the ed25519 seed
    • an identity
    • a private network ID derived from the identity or seed
    • Standard items like expiry times etc
    • Additional permissions to control access to broker subjects
  • The JWT will be signed by a trusted ed25519 key
The broker will only allow connections that holds a valid, signed JWT where the signature is made using a trusted ed25519 key.
Organization and Chained Issuers
Conceptually Choria Broker can separate connections into something called “accounts” in NATS terms, for Choria we call this an Organization. Today we support just one Organization - choria - but we will look to support more in future.
We will add a concept called an Organization Issuer that is a ed25519 key that must, on a basic level, sign all tokens. To facilitate separation of concerns in our centralized AAA mode the Issuer can delegate JWT issuing in a chain to downstream issuers. The tokens will validate as being signed by the issuer essentially.
We support storing the Organization Issuer offline in something like Vault and at the “AAA Login Server” level that issuing key can also be stored in a Vault like system.
Chain Issuers
+Chain Issuers
Thus, we achieve the following chain of trust:
  • We know the JWT was issued by a trusted issuer
  • We verify the connecting server or client has the ed25519 seed that match the JWT because the broker force a NONCE to be signed using the seed, the seed never traverse the network since the public part is embedded in the signed JWT
  • The JWT tokens are therefor not bearer tokens and if they are stolen in-flight (remember no mTLS), one cannot connect with them or make any requests using them
  • We have an identity that isn’t tied to the common name of a cert and so is more fluid and adaptable
  • The broker should support denying all clients without JWTs
In this way we can pick full mTLS when needed or JWT+ed25519 mode when needed and even mix and match the modes.
Reply Security
Reply security is a quite difficult problem to solve since every reply would need to be individually encrypted and decrypted - unlike transport security - this is very CPU intensive, so we have never really done it at scale.
Still, its highly desirable to hide replies destined for a specific user from other users, even those with access to the broker using their own JWT tokens.
Using the callerID as key we calculate a private inbox using the hex encoded md5(callerID), we use this to construct reply subjects for all uses, even access to other subjects like the Streams API.
The broker will set up permissions ensuring that only the callerID can access replies. This way as long as there are unique callerIDs the replies from all systems are private.
We still do not encrypt the traffic in-transit (see point about PII and desired man-in-the-middle in Enterprises), but the replies are private to the user.
% sudo choria jwt client.jwt
 ...
    Private Network ID: 7419405695a186147a0de38f7e31a509...
 ...
@@ -33,9 +33,9 @@
 }
  • jti is a unique id for this token
  • iss indicates a Chain Issuer with token ID (jti) 0ujsswThIGTUYm2K8FjOOfXtY1K issued this token and his public key is bd2588d3dc309d536461caa11c0d6f639e89d7a09dc43eae052f3fb32e2d8687 (the one from the previous example)
  • issexp indicates when the Chain Issuer expires
  • tcs is made up of first creating sigdata [client token jti].[chain issuer tcs] and then combining that [chain issuer tcs].[sig(sigdata, chainIssuerPrik)]
This way we can, given the signed Client token and the Org Issuer Public key, validate by going backwards over these claims:
  1. Extract the Chain Issuer tcs, public key and id from iss and tcs
  2. Verify the Organization Issuer signed the tcs of the Chain Issuer in this token, which also verifies the public key in the issuer
  3. Verify the tcs signature part of the Client using public key of the Chain Issuer
  4. Verify the expiry of the Chain Issuer
General Improvements
The security plugins handle signing, encoding, extracting of IDs and validating signatures. The current security plugins are all implemented around x509.
We will make some general improvements, rename some functions and add a few bits to the interface, detail to be discovered during implementation.
  • Move the API to []byte based API #1844
  • Remove some string orientated security apis 
  • Make the JWT authoritative for the secure channel name so we can stop using md5
  • Develop a tool that can decode and dump/view network packets #1484
  • The entire concept of the cache to be removed #1842
  • Default collective when v2 is used will be choria #1885
  • Submission can sign messages #1873
  • The protocol code should be instances not a singleton so each can have unique contexts and logging
  • Stronger AAA interactions by signing NONCE like data in login and sign requests
  • Potentially entirely remove the concept of Trusted Signers that was a mid term stop gap till this work is complete, only used by 1 users as far as we are aware

\ No newline at end of file
+ 
\ No newline at end of file diff --git a/adr/index.html b/adr/index.html index 9140b0c4..dbbedb27 100644 --- a/adr/index.html +++ b/adr/index.html @@ -1,11 +1,11 @@ Architecture :: Choria Server and Broker -

Architecture

These are documents describing some Architectural Designs

DocumentDescription
ADR-001Background and design of Protocol version 2 and the Choria Issuer based security layer
\ No newline at end of file diff --git a/broker/index.html b/broker/index.html index b700a54d..64247729 100644 --- a/broker/index.html +++ b/broker/index.html @@ -1,14 +1,14 @@ Choria Broker :: Choria Server and Broker -

Choria Broker

Choria Broker is a set of features that, in a fully decentralized setup, is the only central component Choria needs. It +

Choria Broker

Choria Broker is a set of features that, in a fully decentralized setup, is the only central component Choria needs. It is highly available, clustered and very high performance. The messaging layer is based on NATS with a number of Choria specific additions.

A single Choria Broker can manage 50,000 devices on a low budget compute instance - though a cluster of at least 3 brokers is recommended for availability reasons.

Features

  • Choria Brokers is the core message passing middleware, this is a managed NATS Core instance
  • Choria Streams is the data streaming solution used by various Choria components, this is a managed NATS JetStream instance
  • Choria Federation Broker connects entirely isolated Choria networks into a federated single network
  • Choria Data Adapters to move data from Choria Broker to other technologies
  • A Choria specific authentication layer
\ No newline at end of file + 
\ No newline at end of file diff --git a/broker/monitoring/index.html b/broker/monitoring/index.html index 0dc8036e..7ad43193 100644 --- a/broker/monitoring/index.html +++ b/broker/monitoring/index.html @@ -1,6 +1,6 @@ Monitoring :: Choria Server and Broker -

Monitoring

Choria Broker is just one process running in your server, the command systemctl status choria-broker will show a basic +

Monitoring

Choria Broker is just one process running in your server, the command systemctl status choria-broker will show a basic overview of the process running, you should have basic monitoring for the process in place.

Prometheus Metrics

By default, the broker does not expose any monitoring metrics, if you set the plugin.choria.stats_port configuration to a port number it will listen on that port. You can listen on non localhost by setting plugin.choria.stats_address.

It will then serve a number of URLs, some from Choria and some from the embedded NATS Server - see NATS Monitoring for detail about those.

PathDescription
/choria/Build information, run-time resource use
/choria/metricsPrometheus format metrics

System Account

Monitoring NATS and Choria Streams require a System account enabled:

plugin.choria.network.system.user = system
 plugin.choria.network.system.password = s3cret

This should be set in the Broker configuration and any client who wish to access the broker.

We have a basic Dashboard you can use to view these.

Included Checks

Tip

All the choria broker server commands are from the embedded NATS CLI and so can be an awkward fit within our CLI hierarchy

We include a number of checks in the binary that can be used to monitor various aspects of the service.

CommandDescription
choria broker server check connectionPerforms a basic network connection and round-trip test of the NATS service
choria broker server check streamChecks the health of individual Choria Streams Streams
choria broker server check metaChecks the health of the overall Choria Streams System
choria broker server check jetstreamChecks Choria Streams usage limits
choria broker server check serverChecks the health of the embedded NATS Server
choria broker server check kvChecks the health of Choria Key-Value buckets

All of these Checks require the System Account to be enabled in your broker and the client configuration to have the same settings. @@ -28,9 +28,9 @@ really only useful for developers.

plugin.choria.network.pprof_port = 9090
\ No newline at end of file + 
\ No newline at end of file diff --git a/categories/index.html b/categories/index.html index 2e2eba23..f2133265 100644 --- a/categories/index.html +++ b/categories/index.html @@ -1,10 +1,10 @@ Categories :: Choria Server and Broker -

Categories

\ No newline at end of file diff --git a/configuration/index.html b/configuration/index.html index 891e0472..1bede507 100644 --- a/configuration/index.html +++ b/configuration/index.html @@ -1,7 +1,7 @@ Config Reference :: Choria Server and Broker -

Config Reference

This is a list of all known Configuration settings. This list is based on declared settings within the Choria Go code base and so will not cover 100% of settings - plugins can contribute their own settings which are note known at compile time.

Version Hint

Built on 29 Jul 24 19:20 UTC using version 0.29.3

Run-time configuration

The run-time configuration can be inspected using choria tool config --config /etc/choria/server.cfg, this will show the active configuration.

Search and list directives

In addition to the full list below you can get configuration information for your version using the CLI:

% choria tool config security.provider
+
Config Reference
This is a list of all known Configuration settings. This list is based on declared settings within the Choria Go code base and so will not cover 100% of settings - plugins can contribute their own settings which are note known at compile time.
 Version Hint
Built on 29 Jul 24 19:20 UTC using version 0.29.3
Run-time configuration
The run-time configuration can be inspected using choria tool config --config /etc/choria/server.cfg, this will show the active configuration.
Search and list directives
In addition to the full list below you can get configuration information for your version using the CLI:
% choria tool config security.provider
 ....
 Configuration item: plugin.security.provider
 
@@ -14,9 +14,9 @@
 ╙─
Data Types
A few special types are defined, the rest map to standard Go types
TypeDescription
comma_splitA comma separated list of strings, possibly with spaces between
durationA duration such as 1h, 300ms, -1.5h or 2h45m. Valid time units are ns, ms, s, m, h
path_splitA list of paths split by a OS specific PATH separator
path_stringA path that can include ~ for the users home directory
stringsA space separated list of strings
title_stringA string that will be stored as a Title String
Index
classesfilecollectives
colordefault_discovery_method
default_discovery_optionsdiscovery_timeout
identitylibdir
logfileloglevel
main_collectiveplugin.choria.adapters
plugin.choria.agent_provider.mcorpc.agent_shimplugin.choria.agent_provider.mcorpc.config
plugin.choria.agent_provider.mcorpc.libdirplugin.choria.broker_federation
plugin.choria.broker_networkplugin.choria.discovery.broadcast.windowed_timeout
plugin.choria.discovery.external.commandplugin.choria.discovery.inventory.source
plugin.choria.federation.clusterplugin.choria.federation.collectives
plugin.choria.federation_middleware_hostsplugin.choria.legacy_lifecycle_format
plugin.choria.machine.signing_keyplugin.choria.machine.store
plugin.choria.middleware_hostsplugin.choria.network.client_hosts
plugin.choria.network.client_portplugin.choria.network.client_signer_cert
plugin.choria.network.client_tls_force_requiredplugin.choria.network.deny_server_connections
plugin.choria.network.gateway_nameplugin.choria.network.gateway_port
plugin.choria.network.gateway_remotesplugin.choria.network.leafnode_port
plugin.choria.network.leafnode_remotesplugin.choria.network.listen_address
plugin.choria.network.mapping.namesplugin.choria.network.peer_password
plugin.choria.network.peer_portplugin.choria.network.peer_user
plugin.choria.network.peersplugin.choria.network.pprof_port
plugin.choria.network.provisioning.client_passwordplugin.choria.network.provisioning.provisioner_without_token
plugin.choria.network.provisioning.signer_certplugin.choria.network.public_url
plugin.choria.network.server_signer_certplugin.choria.network.soft_shutdown_timeout
plugin.choria.network.stream.advisory_replicasplugin.choria.network.stream.advisory_retention
plugin.choria.network.stream.event_replicasplugin.choria.network.stream.event_retention
plugin.choria.network.stream.leader_election_replicasplugin.choria.network.stream.leader_election_ttl
plugin.choria.network.stream.machine_replicasplugin.choria.network.stream.machine_retention
plugin.choria.network.stream.manage_streamsplugin.choria.network.stream.store
plugin.choria.network.system.passwordplugin.choria.network.system.user
plugin.choria.network.tls_timeoutplugin.choria.network.websocket_advertise
plugin.choria.network.websocket_portplugin.choria.network.write_deadline
plugin.choria.prometheus_textfile_directoryplugin.choria.puppetca_host
plugin.choria.puppetca_portplugin.choria.puppetdb_host
plugin.choria.puppetdb_portplugin.choria.puppetserver_host
plugin.choria.puppetserver_portplugin.choria.registration.file_content.compression
plugin.choria.registration.file_content.dataplugin.choria.registration.file_content.target
plugin.choria.registration.inventory_content.compressionplugin.choria.registration.inventory_content.target
plugin.choria.require_client_filterplugin.choria.security.certname_whitelist
plugin.choria.security.privileged_usersplugin.choria.security.request_signer.seed_file
plugin.choria.security.request_signer.serviceplugin.choria.security.request_signer.token_file
plugin.choria.security.request_signer.urlplugin.choria.security.server.seed_file
plugin.choria.security.server.token_fileplugin.choria.server.provision
plugin.choria.server.provision.allow_updateplugin.choria.services.registry.cache
plugin.choria.services.registry.storeplugin.choria.srv_domain
plugin.choria.ssldirplugin.choria.stats_address
plugin.choria.stats_portplugin.choria.status_file_path
plugin.choria.status_update_intervalplugin.choria.submission.max_spool_size
plugin.choria.submission.spoolplugin.choria.use_srv
plugin.login.aaasvc.login.urlplugin.machines.bucket
plugin.machines.check_intervalplugin.machines.download
plugin.machines.keyplugin.machines.poll_interval
plugin.machines.purgeplugin.machines.signing_key
plugin.nats.credentialsplugin.nats.pass
plugin.nats.userplugin.rpcaudit.logfile
plugin.rpcaudit.logfile.groupplugin.rpcaudit.logfile.mode
plugin.scout.agent_disabledplugin.scout.goss.denied_local_resources
plugin.scout.goss.denied_remote_resourcesplugin.scout.overrides
plugin.scout.tagsplugin.security.certmanager.alt_names
plugin.security.certmanager.api_versionplugin.security.certmanager.issuer
plugin.security.certmanager.namespaceplugin.security.certmanager.replace
plugin.security.choria.caplugin.security.choria.certificate
plugin.security.choria.keyplugin.security.choria.seed_file
plugin.security.choria.sign_repliesplugin.security.choria.token_file
plugin.security.choria.trusted_signersplugin.security.cipher_suites
plugin.security.client_anon_tlsplugin.security.ecc_curves
plugin.security.file.caplugin.security.file.certificate
plugin.security.file.keyplugin.security.issuer.names
plugin.security.pkcs11.driver_fileplugin.security.pkcs11.slot
plugin.security.providerplugin.security.server_anon_tls
plugin.security.support_legacy_certificatesplugin.yaml
registerintervalregistration
registration_collectiveregistration_splay
rpcauditrpcauthorization
rpcauthproviderrpclimitmethod
soft_shutdown_timeoutttl
classesfile
  • Type: path_string
  • Default Value: /opt/puppetlabs/puppet/cache/state/classes.txt
Path to a file listing configuration classes applied to a node, used in matches using Class filters
collectives
  • Type: comma_split
The list of known Sub Collectives this node will join or communicate with, Servers will subscribe the node and each agent to each sub collective and Clients will publish to a chosen sub collective. Defaults to the build settin build.DefaultCollectives
color
  • Type: boolean
  • Default Value: true
Disables or enable CLI color
default_discovery_method
  • Type: string
  • Validation: enum=mc,broadcast,puppetdb,choria,external,inventory
  • Default Value: mc
The default discovery plugin to use. The default “mc” uses a network broadcast, “choria” uses PuppetDB, external calls external commands
default_discovery_options
  • Type: strings
Default options to pass to the discovery plugin
discovery_timeout
  • Type: integer
  • Default Value: 2
How long to wait for responses while doing broadcast discovery
identity
  • Type: string
The identity this machine is known as, when empty it’s derived based on the operating system hostname or by calling facter fqdn
libdir
  • Type: path_split
The directory where Agents, DDLs and other plugins are found
logfile
  • Type: path_string
  • Default Value: stdout
The file to write logs to, when set to ‘discard’ logging will be disabled. Also supports ‘stdout’ and ‘stderr’ as special log destinations.
loglevel
  • Type: string
  • Validation: enum=debug,info,warn,error,fatal
  • Default Value: info
The lowest level log to add to the logfile
main_collective
  • Type: string
The Sub Collective where a Client will publish to when no specific Sub Collective is configured
plugin.choria.adapters
The list of Data Adapters to activate
plugin.choria.agent_provider.mcorpc.agent_shim
  • Type: string
Path to the helper used to call MCollective Ruby agents
plugin.choria.agent_provider.mcorpc.config
  • Type: string
Path to the MCollective configuration file used when running MCollective Ruby agents
plugin.choria.agent_provider.mcorpc.libdir
  • Type: path_split
Path to the libdir MCollective Ruby agents should have
plugin.choria.broker_federation
Enables the Federation Broker
plugin.choria.broker_network
Enables the Network Broker
plugin.choria.discovery.broadcast.windowed_timeout
  • Type: boolean
Enables the experimental dynamic timeout for choria/mc discovery
plugin.choria.discovery.external.command
  • Type: path_string
The command to use for external discovery
plugin.choria.discovery.inventory.source
  • Type: path_string
The file to read for inventory discovery
plugin.choria.federation.cluster
The cluster name a Federation Broker serves
plugin.choria.federation.collectives
List of known remote collectives accessible via Federation Brokers
plugin.choria.federation_middleware_hosts
Middleware brokers used by the Federation Broker, if unset uses SRV
plugin.choria.legacy_lifecycle_format
  • Type: boolean
  • Default Value: 0
When enabled will publish lifecycle events in the legacy format, else Cloud Events format is used
plugin.choria.machine.signing_key
  • Type: string
Public key used to sign data for watchers like machines watcher. Will override the value compiled in or in the watcher definitions if set here. This is primarily to allow development environments to use different private keys.
plugin.choria.machine.store
Directory where Autonomous Agents are stored
plugin.choria.middleware_hosts
  • Type: comma_split
Set specific middleware hosts in the format host:port, if unset uses SRV
plugin.choria.network.client_hosts
  • Type: comma_split
CIDRs to limit client connections from, appropriate ACLs are added based on this
plugin.choria.network.client_port
Port the Network Broker will accept client connections on
plugin.choria.network.client_signer_cert
  • Type: comma_split
Fully qualified paths to the public certificates used by the AAA Service to sign client JWT tokens. This enables users with signed JWTs to use unverified TLS to connect. Can also be a list of ed25519 public keys.
plugin.choria.network.client_tls_force_required
  • Type: boolean
Force requiring/not requiring TLS for all clients
plugin.choria.network.deny_server_connections
  • Type: boolean
Set ACLs denying server connections to this broker
plugin.choria.network.gateway_name
  • Type: string
  • Default Value: CHORIA
Name for the Super Cluster
plugin.choria.network.gateway_port
  • Type: integer
  • Default Value: 0
Port to listen on for Super Cluster connections
plugin.choria.network.gateway_remotes
  • Type: comma_split
List of remote Super Clusters to connect to
plugin.choria.network.leafnode_port
  • Type: integer
  • Default Value: 0
Port to listen on for Leafnode connections, disabled with 0
plugin.choria.network.leafnode_remotes
  • Type: comma_split
Remote networks to connect to as a Leafnode
plugin.choria.network.listen_address
Address the Network Broker will listen on
plugin.choria.network.mapping.names
  • Type: comma_split
List of subject remappings to apply
plugin.choria.network.peer_password
  • Type: string
Password to use when connecting to cluster peers
plugin.choria.network.peer_port
Port used to communicate with other local cluster peers
plugin.choria.network.peer_user
  • Type: string
Username to use when connecting to cluster peers
plugin.choria.network.peers
List of cluster peers in host:port format
plugin.choria.network.pprof_port
  • Type: integer
  • Default Value: 0
The port the network broker will listen on for pprof requests
plugin.choria.network.provisioning.client_password
  • Type: string
Password the provisioned clients should use to connect
plugin.choria.network.provisioning.provisioner_without_token
  • Type: boolean
Allows a provisioner without a token to connect over TLS using username and password. This facilitates v1 provisioning on an Issuer based network
plugin.choria.network.provisioning.signer_cert
  • Type: path_string
Path to the public cert that signs provisioning tokens, enables accepting provisioning connections into the provisioning account
plugin.choria.network.public_url
  • Type: string
Name:Port to advertise to clients, useful when fronted by a proxy
plugin.choria.network.server_signer_cert
  • Type: comma_split
Fully qualified Paths to the public certificates used by the Provisioner Service to sign server JWT tokens. This enables servers with signed JWTs to use unverified TLS to connect. Can also be a list of ed25519 public keys.
plugin.choria.network.soft_shutdown_timeout
  • Type: integer
  • Default Value: 60
The amount of time to allow the broker to exit, after this memory and thread dumps will be performed and a force exit will be done
plugin.choria.network.stream.advisory_replicas
  • Type: integer
  • Default Value: -1
When configuring Stream advisories storage ensure data is replicated in the cluster over this many servers, -1 means count of peers
plugin.choria.network.stream.advisory_retention
  • Type: duration
  • Default Value: 168h
When not zero enables retaining Stream advisories in the Stream Store
plugin.choria.network.stream.event_replicas
  • Type: integer
  • Default Value: -1
When configuring LifeCycle events ensure data is replicated in the cluster over this many servers, -1 means count of peers
plugin.choria.network.stream.event_retention
  • Type: duration
  • Default Value: 24h
When not zero enables retaining Lifecycle events in the Stream Store
plugin.choria.network.stream.leader_election_replicas
  • Type: integer
  • Default Value: -1
When configuring Stream based Leader Election storage ensure data is replicated in the cluster over this many servers, -1 means count of peers
plugin.choria.network.stream.leader_election_ttl
  • Type: duration
  • Default Value: 1m
The TTL for leader election, leaders must vote at least this frequently to remain leader
plugin.choria.network.stream.machine_replicas
  • Type: integer
  • Default Value: -1
When configuring Autonomous Agent event storage ensure data is replicated in the cluster over this many servers, -1 means count of peers
plugin.choria.network.stream.machine_retention
  • Type: duration
  • Default Value: 24h
When not zero enables retaining Autonomous Agent events in the Stream Store
plugin.choria.network.stream.manage_streams
  • Type: boolean
  • Default Value: 1
When set to zero will disable managing the standard streams on this node
plugin.choria.network.stream.store
  • Type: path_string
Enables Streaming data persistence stored in this path
plugin.choria.network.system.password
  • Type: string
Password used to access the Choria system account
plugin.choria.network.system.user
  • Type: string
Username used to access the Choria system account
plugin.choria.network.tls_timeout
  • Type: integer
  • Default Value: 2
Time to allow for TLS connections to establish, increase on slow or very large networks
plugin.choria.network.websocket_advertise
The URL to advertise for websocket connections
plugin.choria.network.websocket_port
Port to listen on for websocket connections
plugin.choria.network.write_deadline
  • Type: duration
  • Default Value: 10s
How long to allow clients to process traffic before treating them as slow, increase this on large networks or slow networks
plugin.choria.prometheus_textfile_directory
  • Type: path_string
Directory where Prometheus Node Exporter textfile collector reads data
plugin.choria.puppetca_host
  • Type: string
  • Default Value: puppet
The hostname where your Puppet Certificate Authority can be found
plugin.choria.puppetca_port
  • Type: integer
  • Default Value: 8140
The port your Puppet Certificate Authority listens on
plugin.choria.puppetdb_host
  • Type: string
The host hosting your PuppetDB, used by the “choria” discovery plugin
plugin.choria.puppetdb_port
  • Type: integer
  • Default Value: 8081
The port your PuppetDB listens on
plugin.choria.puppetserver_host
  • Type: string
  • Default Value: puppet
The hostname where your Puppet Server can be found
plugin.choria.puppetserver_port
  • Type: integer
  • Default Value: 8140
The port your Puppet Server listens on
plugin.choria.registration.file_content.compression
  • Type: boolean
  • Default Value: true
Enables gzip compression of registration data
plugin.choria.registration.file_content.data
  • Type: string
YAML or JSON file to use as data source for registration
plugin.choria.registration.file_content.target
  • Type: string
NATS Subject to publish registration data to
plugin.choria.registration.inventory_content.compression
  • Type: boolean
  • Default Value: true
Enables gzip compression of registration data
plugin.choria.registration.inventory_content.target
  • Type: string
NATS Subject to publish registration data to
plugin.choria.require_client_filter
  • Type: boolean
  • Default Value: false
If a client filter should always be required, only used in Go clients
plugin.choria.security.certname_whitelist
  • Type: comma_split
  • Default Value: .mcollective$,.choria$
Patterns of certificate names that are allowed to be clients
plugin.choria.security.privileged_users
Patterns of certificate names that would be considered privileged and able to set custom callers
plugin.choria.security.request_signer.seed_file
Path to the seed file used to access a Central Authenticator
plugin.choria.security.request_signer.service
Enables signing requests via Choria RPC requests
plugin.choria.security.request_signer.token_file
Path to the token used to access a Central Authenticator
plugin.choria.security.request_signer.url
URL to the Signing Service
plugin.choria.security.server.seed_file
  • Type: path_string
The server token seed to use for authentication, defaults to server.seed in the same location as server.conf
plugin.choria.security.server.token_file
  • Type: path_string
The server token file to use for authentication, defaults to serer.jwt in the same location as server.conf
plugin.choria.server.provision
Specifically enable or disable provisioning
plugin.choria.server.provision.allow_update
Allows the provisioner to perform in-place version updates
plugin.choria.services.registry.cache
  • Type: path_string
  • Environment Variable: CHORIA_REGISTRY
Directory where the Registry client stores DDLs found in the registry
plugin.choria.services.registry.store
  • Type: path_string
Directory where the Registry service finds DDLs to read
plugin.choria.srv_domain
The domain to use for SRV records, defaults to the domain the server FQDN is in
plugin.choria.ssldir
  • Type: path_string
The SSL directory, auto detected via Puppet, when specifically set Puppet will not be consulted
plugin.choria.stats_address
  • Type: string
  • Default Value: 127.0.0.1
The address to listen on for statistics
plugin.choria.stats_port
  • Type: integer
  • Default Value: 0
The port to listen on for HTTP requests for statistics, setting to 0 disables it
plugin.choria.status_file_path
  • Type: path_string
Path to a JSON file to write server health information to regularly
plugin.choria.status_update_interval
  • Type: integer
  • Default Value: 30
How frequently to write to the status_file_path
plugin.choria.submission.max_spool_size
  • Type: integer
  • Default Value: 500
Maximum amount of messages allowed into each priority
plugin.choria.submission.spool
  • Type: path_string
Path to a directory holding messages to submit to the middleware
plugin.choria.use_srv
If SRV record lookups should be attempted to find Puppet, PuppetDB, Brokers etc
plugin.login.aaasvc.login.url
List of URLs to attempt to login against when the remote signer is enabled
plugin.machines.bucket
  • Type: string
  • Default Value: CHORIA_PLUGINS
The KV bucket to query for plugins to install
plugin.machines.check_interval
  • Type: string
  • Default Value: 30s
How frequently to integrity check deployed autonomous agents
plugin.machines.download
  • Type: boolean
Activate run-time installation of Autonomous Agents
plugin.machines.key
  • Type: string
  • Default Value: machines
The Key to query in KV bucket for plugins to install
plugin.machines.poll_interval
  • Type: string
  • Default Value: 1m
How frequently to poll the KV bucket for updates
plugin.machines.purge
  • Type: boolean
  • Default Value: true
Purge autonomous agents installed using other methods
plugin.machines.signing_key
  • Type: string
The public key to validate the plugins manifest with
plugin.nats.credentials
  • Type: string
  • Environment Variable: MCOLLECTIVE_NATS_CREDENTIALS
The NATS 2.0 credentials to use, required for accessing NGS
plugin.nats.pass
  • Type: string
  • Environment Variable: MCOLLECTIVE_NATS_PASSWORD
The password to use when connecting to the NATS server
plugin.nats.user
  • Type: string
  • Environment Variable: MCOLLECTIVE_NATS_USERNAME
The user to connect to the NATS server as. When unset no username is used.
plugin.rpcaudit.logfile
  • Type: path_string
Path to the RPC audit log
plugin.rpcaudit.logfile.group
  • Type: string
User group to set file ownership to
plugin.rpcaudit.logfile.mode
  • Type: string
  • Default Value: 0600
File mode to apply to the file
plugin.scout.agent_disabled
  • Type: boolean
Disables the scout agent
plugin.scout.goss.denied_local_resources
  • Type: comma_split
List of resource types to deny for Goss manifests loaded from local disk
plugin.scout.goss.denied_remote_resources
  • Type: comma_split
  • Default Value: command
List of resource types to deny when Goss manifests or variables were received over rpc
plugin.scout.overrides
  • Type: path_string
Path to a file holding overrides for Scout checks
plugin.scout.tags
  • Type: path_string
Path to a file holding tags for a Scout entity
plugin.security.certmanager.alt_names
  • Type: comma_split
when using Cert Manager security provider, add these additional names to the CSR
plugin.security.certmanager.api_version
  • Type: string
  • Default Value: v1
the API version to call in cert manager
plugin.security.certmanager.issuer
  • Type: string
When using Cert Manager security provider, the name of the issuer
plugin.security.certmanager.namespace
  • Type: string
  • Default Value: choria
When using Cert Manager security provider, the namespace the issuer is in
plugin.security.certmanager.replace
  • Type: boolean
  • Default Value: true
when using Cert Manager security provider, replace existing CSRs with new ones
plugin.security.choria.ca
  • Type: path_string
When using choria security provider, the path to the optional Certificate Authority public certificate
plugin.security.choria.certificate
  • Type: path_string
When using choria security provider, the path to the optional public certificate
plugin.security.choria.key
  • Type: path_string
When using choria security provider, the path to the optional private key
plugin.security.choria.seed_file
  • Type: path_string
The path to the seed file
plugin.security.choria.sign_replies
  • Type: boolean
  • Default Value: true
Disables signing replies which would significantly trim down the size of replies but would remove the ability to verify signatures or verify message origin
plugin.security.choria.token_file
  • Type: path_string
The path to the JWT token file
plugin.security.choria.trusted_signers
  • Type: comma_split
Ed25119 public keys of entities allowed to sign client and server JWT tokens in hex encoded format
plugin.security.cipher_suites
  • Type: comma_split
List of allowed cipher suites
plugin.security.client_anon_tls
  • Type: boolean
  • Default Value: false
Use anonymous TLS to the Choria brokers from a client, also disables security provider verification - only when a remote signer is set
plugin.security.ecc_curves
  • Type: comma_split
List of allowed ECC curves
plugin.security.file.ca
  • Type: path_string
When using file security provider, the path to the Certificate Authority public certificate
plugin.security.file.certificate
  • Type: path_string
When using file security provider, the path to the public certificate
plugin.security.file.key
  • Type: path_string
When using file security provider, the path to the private key
plugin.security.issuer.names
  • Type: comma_split
List of names of valid issuers this server will accept, set indvidiaul issuer data using plugin.security.issuer..public
plugin.security.pkcs11.driver_file
When using the pkcs11 security provider, the path to the PCS11 driver file
plugin.security.pkcs11.slot
When using the pkcs11 security provider, the slot to use in the device
plugin.security.provider
  • Type: string
  • Validation: enum=puppet,file,pkcs11,certmanager,choria
  • Default Value: puppet
The Security Provider to use
plugin.security.server_anon_tls
  • Type: boolean
  • Default Value: false
Use anonymous TLS to the Choria brokers from a server
plugin.security.support_legacy_certificates
  • Type: boolean
  • Default Value: false
Allow certificates without SANs to be used
plugin.yaml
  • Type: path_string
Where to look for YAML or JSON based facts
registerinterval
  • Type: integer
  • Default Value: 300
How often to publish registration data
registration
  • Type: comma_split
The plugins used when publishing Registration data, when this is unset or empty sending registration data is disabled
registration_collective
  • Type: string
The Sub Collective to publish registration data to
registration_splay
  • Type: boolean
  • Default Value: true
When true delays initial registration publish by a random period up to registerinterval following registration publishes will be at registerinterval without further splay
rpcaudit
When enabled uses rpcauditprovider to audit RPC requests processed by the server
rpcauthorization
When enables authorization is performed on every RPC request based on rpcauthprovider
rpcauthprovider
The Authorization system to use
rpclimitmethod
  • Type: string
  • Validation: enum=first,random
  • Default Value: first
When limiting nodes to a subset of discovered nodes this is the method to use, random is influenced by
soft_shutdown_timeout
  • Type: integer
  • Default Value: 2
The amount of time to allow the server to exit, after this memory and thread dumps will be performed and a force exit will be done
ttl
  • Type: integer
  • Default Value: 60
How long published messages are allowed to linger on the network, lower numbers have a higher reliance on clocks being in sync

\ No newline at end of file
+ 
\ No newline at end of file diff --git a/index.html b/index.html index e4baf301..1af7ef50 100644 --- a/index.html +++ b/index.html @@ -1,16 +1,16 @@ Choria Server and Broker -

Choria Server and Broker

Overview

Choria is a framework that can be used to build Control Planes, Orchestration Systems, Programmable Infrastructure or IoT networks.

This is a Golang Project hosting:

  • A Network Broker used for communications based on NATS
  • A Server that deploys to every node and expose a API over the Choria Network Broker
  • A set of Client libraries and related tools to interact with the exposed APIs
  • Autonomous Agent framework used to create no-code control loops
  • Our JetStream based Choria Streams streaming data technology
  • Various foundational features for monitoring, provisioning, automation and more

General information about the project can be found in the project documentation site, this site -focus on a deep dive on the Server and Broker specifically.

Scalability

The Architecture resembles a client-server architecture with very fast NATS middleware as a transport layer.

Architecture -Architecture

A single Choria Broker deployed on a $40/month cloud VM can be used to manage 50,000 connected devices. The RPC layer is capable +

Choria Server and Broker

Overview

Choria is a framework that can be used to build Control Planes, Orchestration Systems, Programmable Infrastructure or IoT networks.

This is a Golang Project hosting:

  • A Network Broker used for communications based on NATS
  • A Server that deploys to every node and expose a API over the Choria Network Broker
  • A set of Client libraries and related tools to interact with the exposed APIs
  • Autonomous Agent framework used to create no-code control loops
  • Our JetStream based Choria Streams streaming data technology
  • Various foundational features for monitoring, provisioning, automation and more

General information about the project can be found in the project documentation site, this site +focus on a deep dive on the Server and Broker specifically.

Scalability

The Architecture resembles a client-server architecture with very fast NATS middleware as a transport layer.

Architecture +Architecture

A single Choria Broker deployed on a $40/month cloud VM can be used to manage 50,000 connected devices. The RPC layer is capable of communicating with 10s of thousands of nodes in sub-second round trip times.

The system is a cellular, highly scalable and highly available. It can though be run comfortably on a Raspberry PI.

Status

The project is under active development and in use by some of the largest infrastructures in the world.

Agents written 10 years ago for The Marionette Collective are still usable today, we place great value on stability, however the project is under active development and new research and development continues on a daily basis.

\ No newline at end of file + 
\ No newline at end of file diff --git a/installation/index.html b/installation/index.html index bac8286b..ff708219 100644 --- a/installation/index.html +++ b/installation/index.html @@ -2,7 +2,7 @@ Enterprise Linux We publish RPM releases but also nightly builds to our repositories. Users of our Puppet modules will already have these repositories available. Release [choria_release] name=Choria Orchestrator Releases mirrorlist=http://mirrorlists.choria.io/yum/release/el/$releasever/$basearch.txt enabled=True gpgcheck=True repo_gpgcheck=True gpgkey=https://choria.io/RELEASE-GPG-KEY metadata_expire=300 sslcacert=/etc/pki/tls/certs/ca-bundle.crt sslverify=TrueNightly Nightly releases are named and versioned choria-0.99.0.20221109-1.el7.x86_64.rpm where the last part of the version is the date.">Installation :: Choria Server and Broker -

Installation

We distribute an RPM, Debs, MSIs, Exes and Docker containers for the Choria Server and Broker.

Enterprise Linux

We publish RPM releases but also nightly builds to our repositories.

Users of our Puppet modules will already have these repositories available.

Release

[choria_release]
+
Installation
We distribute an RPM, Debs, MSIs, Exes and Docker containers for the Choria Server and Broker.
Enterprise Linux
We publish RPM releases but also nightly builds to our repositories.
Users of our Puppet modules will already have these repositories available.
Release
[choria_release]
 name=Choria Orchestrator Releases
 mirrorlist=http://mirrorlists.choria.io/yum/release/el/$releasever/$basearch.txt
 enabled=True
@@ -23,9 +23,9 @@
 sslverify=True
Debian
We publish release packages for Debian systems on our APT repositories:
deb mirror://mirrorlists.choria.io/apt/release/debian/bullseye/mirrors.txt debian bullseye
Docker
There is a docker container for releases registry.choria.io/choria/choria:latest with nightly builds at registry.choria.io/choria-nightly/choria with tags nightly and nightly-YYYYMMDD.

\ No newline at end of file
+ 
\ No newline at end of file diff --git a/packaging/index.html b/packaging/index.html index 34279394..a29d7303 100644 --- a/packaging/index.html +++ b/packaging/index.html @@ -1,7 +1,7 @@ Custom Packaging :: Choria Server and Broker -

Custom Packaging

Choria binary, being a compiled binary with no external dependencies, needs to be recompiled when adding certain kinds +

Custom Packaging

Choria binary, being a compiled binary with no external dependencies, needs to be recompiled when adding certain kinds of plugin, changing some default locations or adding your own plugins.

The project provides the tooling to perform these builds and has a compile-time configuration that can be adjusted to local needs.

Requirements

The host used to perform the compile need to have Docker on it and be able to fetch the registry.choria.io/choria/packager container. You can build a local version of the Packager using https://github.com/choria-io/packager.

In general you should only do this if you know what you are doing, have special needs, want custom agents etc.

Plugins

A number of plugins types are supported and we build many in at compile time ourselves.

The general process is that all the plugins in packager/plugins.yaml will be included in the build, if you want to add @@ -52,9 +52,9 @@ binary: 64bit_linux

We can now run:

BUILD=acme VERSION=0.26.0acme rake build

When you are done you will have:

  • an rpm called backplane-0.26.0acme.el8.x86_64.rpm
  • the binary will be /opt/acme/bin/backplane
  • config files, log files, services all will be personalized around backplane
  • it will have a custom path to the provisioning.jwt

A number of things are customizable see the section at the top of the buildspec.yaml and comments in the build file.

\ No newline at end of file + 
\ No newline at end of file diff --git a/previews/index.html b/previews/index.html index bfb6ce9d..e7d70359 100644 --- a/previews/index.html +++ b/previews/index.html @@ -1,10 +1,10 @@ Tech Previews :: Choria Server and Broker -

Tech Previews

This section will from time-to-time contain guides on testing upcoming features and architectural changes.

\ No newline at end of file diff --git a/previews/protov2/index.html b/previews/protov2/index.html index eeb148d6..f8422f11 100644 --- a/previews/protov2/index.html +++ b/previews/protov2/index.html @@ -1,9 +1,9 @@ V2 Protocol & Security :: Choria Server and Broker -

V2 Protocol & Security

This is a guide for early adopters who wish to test and study the Version 2 Protocol and Security project.

Warning

This is a Hard Mode guide that does everything manually and with no Configuration Management.

Requirements

Security Credentials

Security is by means of a ed25519 key that signs JWTs, some JWTs form a chain and can sign others. Regardless of the signer verification can be done using the public key associated with the Organization Issuer.

Chain Issuers -Chain Issuers

The Organization Issuer can be kept offline with Provisioning and AAA being delegated authorities capable of signing servers and clients but these are optional components - the Organization Issuer can directly sign Clients and Servers allowing them to operate without the other central components.

Deployment Methods

We demonstrate two deployment methods:

Additionally, we show how Hashicorp Vault can be integrated to manage the Organization Issuer

Tip

We recommend reviewers really dig into the details here, we do have a Docker Compose environment with this all setup.

Decentralized deployment

In this model we will deploy a system that resembles the basic architecture diagram below

Architecture -Architecture

We have only the Brokers as central architecture with no Central AAA or Provisioning.

We will not use mTLS in this case. mTLS is supported but a major advantage of this mode is that it’s not required.

Docker

We will need two Docker networks and 3 instances - broker, server, client and issuer.

$ docker network create choria_v2proto
+
V2 Protocol & Security
This is a guide for early adopters who wish to test and study the Version 2 Protocol and Security project.
 Warning
This is a Hard Mode guide that does everything manually and with no Configuration Management.
Requirements
Security Credentials
Security is by means of a ed25519 key that signs JWTs, some JWTs form a chain and can sign others. Regardless of the signer verification can be done using the public key associated with the Organization Issuer.
Chain Issuers
+Chain Issuers
The Organization Issuer can be kept offline with Provisioning and AAA being delegated authorities capable of signing servers and clients but these are optional components - the Organization Issuer can directly sign Clients and Servers allowing them to operate without the other central components.
Deployment Methods
We demonstrate two deployment methods:
Additionally, we show how Hashicorp Vault can be integrated to manage the Organization Issuer
 Tip
We recommend reviewers really dig into the details here, we do have a Docker Compose environment with this all setup.
Decentralized deployment
In this model we will deploy a system that resembles the basic architecture diagram below
Architecture
+Architecture
We have only the Brokers as central architecture with no Central AAA or Provisioning.
We will not use mTLS in this case. mTLS is supported but a major advantage of this mode is that it’s not required.
Docker
We will need two Docker networks and 3 instances - broker, server, client and issuer.
$ docker network create choria_v2proto
 $ docker network create choria_issuer
 $ docker pull registry.choria.io/choria-nightly/choria:nightly
 $ docker run -ti --rm --entrypoint bash \
@@ -150,8 +150,8 @@
 server.example.net                       time=3 ms
 
 ---- ping statistics ----
-1 replies max: 4ms min: 4ms avg: 4ms overhead: 12ms
Other commands like choria req choria_util info should work demonstrating authorization works and choria broker server list should list the broker indicating Broker System Account access works. After a minute or so choria broker stream ls will show a list of Streams demonstrating Choria Streams authority worked.
Self Provisioned and AAA Integrated Deployment
Thus far we had to manually sign and configure every single server and client, we had to copy files around and more, it’s all a bit tedious.
Lets see how Choria can configure itself and how user management can be centralized for self-service user enrollment.
  • Instead of signing Server JWTs servers will go to Choria Provisioner to obtain credentials and configuration
  • Instead of issuing Client JWTs for every user that are long-lasting we will use a central authorization flow to issue short-lived JWTs and distribute them
Centralized Deployment
-Centralized Deployment
Docker
We will create re-use the networks we made before, and we can keep the same issuer.example.net and broker.example.net.
So stop and recreate your server and client containers, we’ll 2 more container during the guide.
Issuer
The issuer is unchanged from before, so just follow the same steps as before or keep the one you have if you followed the Decentralized section.
Broker
The broker is unchanged from before, so just follow the same steps as before or keep the one you have if you followed the earlier Decentralized section.
You will note we have a setting in the broker.conf:
plugin.choria.network.provisioning.client_password = s3cret
This instructs the broker that we will be connecting servers needing provisioning to it. You should see a log line like:
WARN[0000] Allowing Provisioner connections subject to JWT claims  component=network
Provisioner
The Choria Provisioner is a service that configures new Choria Servers:
  • Enrolls servers into the Issuer
  • Create a per-node configuration
  • Deploys Open Policy Agent policies
  • Configures the server
  • Optionally perform version upgrades
We have CLI tooling allowing you to re-provision servers on demand and more. Review its documentation for full detail.
It needs to connect to the broker, so it needs JWT token, let’s create the container and create the private key:
Since the container does not have the choria command we have to jump some hoops, we’ll make a local storage directory for its configuration and keys and then mount that in.
host$ mkdir provisioner
+1 replies max: 4ms min: 4ms avg: 4ms overhead: 12ms
Other commands like choria req choria_util info should work demonstrating authorization works and choria broker server list should list the broker indicating Broker System Account access works. After a minute or so choria broker stream ls will show a list of Streams demonstrating Choria Streams authority worked.
Self Provisioned and AAA Integrated Deployment
Thus far we had to manually sign and configure every single server and client, we had to copy files around and more, it’s all a bit tedious.
Lets see how Choria can configure itself and how user management can be centralized for self-service user enrollment.
  • Instead of signing Server JWTs servers will go to Choria Provisioner to obtain credentials and configuration
  • Instead of issuing Client JWTs for every user that are long-lasting we will use a central authorization flow to issue short-lived JWTs and distribute them
Centralized Deployment
+Centralized Deployment
Docker
We will create re-use the networks we made before, and we can keep the same issuer.example.net and broker.example.net.
So stop and recreate your server and client containers, we’ll 2 more container during the guide.
Issuer
The issuer is unchanged from before, so just follow the same steps as before or keep the one you have if you followed the Decentralized section.
Broker
The broker is unchanged from before, so just follow the same steps as before or keep the one you have if you followed the earlier Decentralized section.
You will note we have a setting in the broker.conf:
plugin.choria.network.provisioning.client_password = s3cret
This instructs the broker that we will be connecting servers needing provisioning to it. You should see a log line like:
WARN[0000] Allowing Provisioner connections subject to JWT claims  component=network
Provisioner
The Choria Provisioner is a service that configures new Choria Servers:
  • Enrolls servers into the Issuer
  • Create a per-node configuration
  • Deploys Open Policy Agent policies
  • Configures the server
  • Optionally perform version upgrades
We have CLI tooling allowing you to re-provision servers on demand and more. Review its documentation for full detail.
It needs to connect to the broker, so it needs JWT token, let’s create the container and create the private key:
Since the container does not have the choria command we have to jump some hoops, we’ll make a local storage directory for its configuration and keys and then mount that in.
host$ mkdir provisioner
 host$ docker run -v `pwd`/provisioner:/etc/choria-provisioner --user root --entrypoint bash --rm -ti registry.choria.io/choria-nightly/choria:nightly -l
 [root@5d96691fa69f /]# choria jwt keys /etc/choria-provisioner/private.key /etc/choria-provisioner/public.key
 Public Key: a8c15c0a4bbae0646d0c5aa92513f4d58c2c0e51464b4b267bb3a42dbebd1c8a
@@ -385,9 +385,9 @@
     --vault
Here we pass --vault and instead of a path to the Issuer Private Key we give the name choria_issuer that we created in Vault.

\ No newline at end of file
+ 
\ No newline at end of file diff --git a/server/index.html b/server/index.html index cc5ed8ac..6368f054 100644 --- a/server/index.html +++ b/server/index.html @@ -1,13 +1,13 @@ Choria Server :: Choria Server and Broker -

Choria Server

Choria Server is the component that runs on every managed device. It hosts various plugins that can be accessed +

Choria Server

Choria Server is the component that runs on every managed device. It hosts various plugins that can be accessed remotely via a RPC layer.

It’s a stable and robust agent designed to run forever with minimal resource overheads on managed devices.

Features

  • Hosts Choria RPC Agents
  • Hosts Choria Autonomous Agents
  • Hosts foundational technology for Choria Scout
  • Supports optional self-provisioning and enrollment into a Choria network in a IoT device like manner
  • Communicates using a JSON based network protocol with extensive JSON Schemas
  • Deep RBAC integration for Authentication, Authorization and Auditing
  • Supports mTLS or JWT token based security layers with, optional, integration into Enterprise SSO, IAM and systems like Hashicorp Vault
  • Supports Open Policy Agent for Authorization
  • Emits Cloud Events for network management and observability
  • Embeddable in Go applications to provide in-process management for automation backplanes
  • Extensive features related to gathering and streaming of Node Metadata
  • Distributed as RPM, Deb, DMG, MSI across many architectures
\ No newline at end of file + 
\ No newline at end of file diff --git a/server/monitoring/index.html b/server/monitoring/index.html index eaad1f8e..909b7157 100644 --- a/server/monitoring/index.html +++ b/server/monitoring/index.html @@ -5,7 +5,7 @@ Without any opening ports monitoring it is via a state file that it writes regularly when enabled: plugin.choria.status_file_path = /var/log/choria-status.json plugin.choria.status_update_interval = 30The above configuration will cause the status file to update every 30 seconds. This needs to be enabled for any deep introspection. Nagios Check A nagios protocol test is included in the command choria tool status, this can check various aspects of the server operation.">Monitoring :: Choria Server and Broker -

Monitoring

Choria Server is designed to not open any listening ports unless its Apple HomeKit integration is enabled.

Without any opening ports monitoring it is via a state file that it writes regularly when enabled:

plugin.choria.status_file_path = /var/log/choria-status.json
+
Monitoring
Choria Server is designed to not open any listening ports unless its Apple HomeKit integration is enabled.
Without any opening ports monitoring it is via a state file that it writes regularly when enabled:
plugin.choria.status_file_path = /var/log/choria-status.json
 plugin.choria.status_update_interval = 30
The above configuration will cause the status file to update every 30 seconds. This needs to be enabled for any
 deep introspection.
Nagios Check
A nagios protocol test is included in the command choria tool status, this can check various aspects of the server
 operation.
$ choria tool status --status-file /var/log/choria-status.json \
@@ -27,9 +27,9 @@
 observed using choria tool event, this will include start, stop, provisioned etc events from every Choria Server instance.
Some details about these events are in these blog posts:

\ No newline at end of file
+ 
\ No newline at end of file diff --git a/tags/index.html b/tags/index.html index 23433053..3508ca07 100644 --- a/tags/index.html +++ b/tags/index.html @@ -1,10 +1,10 @@ Tags :: Choria Server and Broker -

Tags

\ No newline at end of file