Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Release cert-manager 1.16 #1520

Merged
merged 48 commits into from
Oct 3, 2024
Merged
Show file tree
Hide file tree
Changes from 24 commits
Commits
Show all changes
48 commits
Select commit Hold shift + click to select a range
fb8063b
add documentation to support venafi issuer caBundleSecretRef
sankalp-at-gh May 28, 2024
80a7ee9
Merge pull request #1519 from cert-manager/master
cert-manager-prow[bot] Jul 18, 2024
bdcab60
Add boilerplate for the cert-manager 1.16 release
wallrj Jul 18, 2024
108bdf9
Merge pull request #1521 from wallrj/add-1.16-boilerplate
cert-manager-prow[bot] Jul 18, 2024
7540728
Update the Prometheus Operator documentation to scrape the webhook too
wallrj Jul 18, 2024
cdb90b0
Explain how to set up TLS metrics for the webhook too
wallrj Jul 19, 2024
57109f5
Show how to check the TLS configuration
wallrj Jul 19, 2024
ae3f425
Add a release note snippet
wallrj Jul 19, 2024
93c07ce
Merge pull request #1522 from wallrj/webhook-metrics-server
cert-manager-prow[bot] Jul 23, 2024
9b72ef6
Update the Prometheus Operator documentation to scrape cainjector too
wallrj Jul 23, 2024
3527ee7
Update release note
wallrj Jul 23, 2024
77c910c
Merge pull request #1524 from wallrj/cainjector-webhook-server
cert-manager-prow[bot] Jul 24, 2024
2d80b77
Add release notes for v1.16.0-alpha.0
wallrj Jul 24, 2024
c75a056
Merge pull request #1526 from wallrj/release-notes-1.16.0-alpha.0
cert-manager-prow[bot] Jul 25, 2024
646e8c1
Merge remote-tracking branch 'origin/master' into merge-master
wallrj Aug 6, 2024
7dded24
Merge pull request #1533 from wallrj/merge-master
cert-manager-prow[bot] Aug 7, 2024
b7ebb5e
Document the new renewBeforePercentage field
cbroglie Sep 9, 2024
2bcf5a5
Merge pull request #1551 from cbroglie/renew-before-pct
cert-manager-prow[bot] Sep 25, 2024
0298570
Merge remote-tracking branch 'origin/master' into release-next-merge-…
wallrj Sep 25, 2024
94610bd
Merge pull request #1564 from wallrj/release-next-merge-master
cert-manager-prow[bot] Sep 25, 2024
010a100
Re-run release-notes
wallrj Sep 26, 2024
b2f1081
Update installation version
wallrj Sep 26, 2024
affb38a
Filling in the themes section
wallrj Sep 26, 2024
b6eded5
Merge pull request #1565 from wallrj/release-notest-1.16.0-beta.0
cert-manager-prow[bot] Sep 26, 2024
03cb1ab
More about regional STS endpoints
wallrj Sep 27, 2024
f3e1fad
Update content/docs/releases/release-notes/release-notes-1.16.md
wallrj Sep 27, 2024
65b12a2
Update content/docs/releases/release-notes/release-notes-1.16.md
wallrj Sep 27, 2024
295025e
Merge pull request #1567 from wallrj/more-route53-release-notes
cert-manager-prow[bot] Sep 27, 2024
f49dda2
Apply @maelvls suggestions
wallrj Sep 27, 2024
dba5531
Prioritize and write more about Helm schemas
wallrj Sep 27, 2024
8c722ec
Merge pull request #1568 from wallrj/update-breaking-changes-details-…
cert-manager-prow[bot] Oct 1, 2024
d265fe7
Add some notes about the new UseDomainQualifiedFinalizer feature gate
wallrj Oct 1, 2024
e3d8393
A note about DeletedStateUnknown log noise
wallrj Oct 1, 2024
841ef3c
Add blank lines before the learn more links
wallrj Oct 1, 2024
64f764b
Merge pull request #1571 from wallrj/new-feature-flags
cert-manager-prow[bot] Oct 1, 2024
1a7fc6d
Update release notes for WatchListClient feature support
wallrj Oct 1, 2024
472b69b
Merge pull request #1575 from cert-manager/master
cert-manager-prow[bot] Oct 2, 2024
668d45e
Update the supported versions page for cert-manager 1.16
wallrj Oct 2, 2024
83d1497
Merge pull request #1572 from wallrj/update-watchlistclient-feature-docs
cert-manager-prow[bot] Oct 2, 2024
6de81ed
Merge pull request #1574 from wallrj/update-supported-versions
cert-manager-prow[bot] Oct 2, 2024
6c520ff
Add list of contributors
wallrj Oct 3, 2024
819ce59
Update version variable to 1.16.0
wallrj Oct 3, 2024
19f4f3e
Update API and CLI docs generation script
wallrj Oct 3, 2024
580eea9
Fill in the upgrading documentation
wallrj Oct 3, 2024
28518ae
Fix the post-processing script
wallrj Oct 3, 2024
fd319d3
./scripts/gendocs/generate-new-import-path-docs
wallrj Oct 3, 2024
63cad7d
Add remaining release notes
wallrj Oct 3, 2024
b7d2692
Merge pull request #1576 from wallrj/release-1.16.0
cert-manager-prow[bot] Oct 3, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .spelling
Original file line number Diff line number Diff line change
Expand Up @@ -443,6 +443,7 @@ namespaced
namespaces
ndegory
oauth2
OAuth
onwards
openshift-supported-versions
plaintext
Expand Down
5 changes: 5 additions & 0 deletions content/docs/configuration/venafi.md
Original file line number Diff line number Diff line change
Expand Up @@ -239,6 +239,11 @@ spec:
tpp:
url: https://tpp.venafi.example/vedsdk # Change this to the URL of your TPP instance
caBundle: <base64 encoded string of caBundle PEM file, or empty to use system root CAs>
## Use only caBundle above or the caBundleSecretRef below. Secret can be created from a ca.crt file by running below command
## kubectl create secret generic custom-tpp-ca --from-file=/my/certs/ca.crt -n <cert-manager-namespace>
# caBundleSecretRef:
# name: custom-tpp-ca
# key: ca.crt
credentialsRef:
name: tpp-secret
```
Expand Down
187 changes: 112 additions & 75 deletions content/docs/devops-tips/prometheus-metrics.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,42 +3,26 @@ title: Prometheus Metrics
description: 'cert-manager usage: Prometheus metrics'
---

To help with operations and insights into cert-manager activities, cert-manager exposes metrics in the [Prometheus](https://prometheus.io/) format from the controller component. These are available at the standard `/metrics` path of the controller component's configured HTTP port.
To help with operations and insights into cert-manager activities, cert-manager exposes metrics in the [Prometheus](https://prometheus.io/) format from the controller, webhook and cainjector components. These are available at the standard `/metrics` endpoint on port `9402` of each component Pod.

## Scraping Metrics

How metrics are scraped will depend how you're operating your Prometheus server(s). These examples presume the [Prometheus Operator](https://github.com/prometheus-operator/prometheus-operator) is being used to run Prometheus, and configure Pod or Service Monitor CRDs.

### Helm

If you're deploying cert-manager with helm, a `ServiceMonitor` resource can be configured. This configuration should enable metric scraping, and the configuration can be further tweaked as described in the [Helm configuration documentation](https://github.com/cert-manager/cert-manager/blob/master/deploy/charts/cert-manager/README.template.md#configuration).
If you're deploying cert-manager with helm, a `PodMonitor` resource can be configured. This configuration should enable metric scraping, and the configuration can be further tweaked as described in the [Helm configuration documentation](https://github.com/cert-manager/cert-manager/blob/master/deploy/charts/cert-manager/README.template.md#configuration).

```yaml
prometheus:
enabled: true
servicemonitor:
podmonitor:
enabled: true
```

### Regular Manifests

If you're not using helm to deploy cert-manager and instead using the provided regular YAML manifests, this example `PodMonitor` and deployment patch should be all you need to start ingesting cert-manager metrics.

1. [Apply the following patch](https://kubernetes.io/docs/tasks/manage-kubernetes-objects/update-api-object-kubectl-patch/#use-a-strategic-merge-patch-to-update-a-deployment) to your cert-manager deployment

```yaml
spec:
template:
spec:
containers:
- name: cert-manager-controller
ports:
- containerPort: 9402
name: http
protocol: TCP
```

2. Create the following `PodMonitor`
If you're not using helm to deploy cert-manager and instead using the provided regular YAML manifests, this example `PodMonitor` should be all you need to start ingesting cert-manager metrics.

```yaml
apiVersion: monitoring.coreos.com/v1
Expand All @@ -50,18 +34,28 @@ metadata:
app: cert-manager
app.kubernetes.io/name: cert-manager
app.kubernetes.io/instance: cert-manager
app.kubernetes.io/component: "controller"
spec:
jobLabel: app.kubernetes.io/name
selector:
matchLabels:
app: cert-manager
app.kubernetes.io/name: cert-manager
app.kubernetes.io/instance: cert-manager
app.kubernetes.io/component: "controller"
matchExpressions:
- key: app.kubernetes.io/name
operator: In
values:
- cainjector
- cert-manager
- webhook
- key: app.kubernetes.io/instance
operator: In
values:
- release-name
- key: app.kubernetes.io/component
operator: In
values:
- cainjector
- controller
- webhook
podMetricsEndpoints:
- port: http-metrics
honorLabels: true
```

### TLS
Expand All @@ -70,78 +64,121 @@ TLS can be enabled on the metrics endpoint for end-to-end encryption. This is ac

#### Static certificates

Static certificates can be provided to the cert-manager controller to use when listening on the metric endpoint. If the certificate files are changed then cert-manager will reload the certificates for zero-downtime rotation.
Static certificates can be provided to the cert-manager to use when listening on the metric endpoint. If the certificate files are changed then cert-manager will reload the certificates for zero-downtime rotation.

Static certificates can be specified via the flags `--metrics-tls-cert-file` and `--metrics-tls-private-key-file` or the corresponding config file parameters `metricsTLSConfig.filesystem.certFile` and `metricsTLSConfig.filesystem.keyFile`.

The certificate and private key must be mounted into the controller pod for this to work, if cert-manager is deployed using helm the `.volumes[]` and `.mounts[]` properties can facilitate this.

An example config file would be:
An example Helm values file would be:

```yaml
apiVersion: controller.config.cert-manager.io/v1alpha1
kind: ControllerConfiguration
metricsTLSConfig:
filesystem:
certFile: "/path/to/cert.pem"
keyFile: "/path/to/key.pem"
# values.yaml
prometheus:
enabled: true
config:
metricsTLSConfig:
filesystem:
certFile: "/path/to/cert.pem"
keyFile: "/path/to/key.pem"
webhook:
config:
metricsTLSConfig:
filesystem:
certFile: "/path/to/cert.pem"
keyFile: "/path/to/key.pem"
cainjector:
config:
metricsTLSConfig:
filesystem:
certFile: "/path/to/cert.pem"
keyFile: "/path/to/key.pem"
```

#### Dynamic certificates

In this mode cert-manager will create a CA in a named secret, then use this CA to sign the metrics endpoint certificate. This mode will also take care of rotation, auto rotating the certificate as required.
In this mode cert-manager will create a CA in a named Secret, then use this CA to sign the metrics endpoint certificates. This mode will also take care of rotation, auto rotating the certificate as required.

Dynamic certificates can be specified via the flags `--metrics-dynamic-serving-ca-secret-namespace`, `--metrics-dynamic-serving-ca-secret-name` and `--metrics-dynamic-serving-dns-names` or the corresponding config file parameters `metricsTLSConfig.dynamic.secretNamespace`, `metricsTLSConfig.dynamic.secretName` and `metricsTLSConfig.dynamic.dnsNames`.
Dynamic certificates can be specified via the flags `--metrics-dynamic-serving-ca-secret-namespace`, `--metrics-dynamic-serving-ca-secret-name` and `--metrics-dynamic-serving-dns-names` or the corresponding config file parameters `metricsTLSConfig.dynamic.secretNamespace`, `metricsTLSConfig.dynamic.secretName` and `metricsTLSConfig.dynamic.dnsNames`.

An example config file would be:
An example Helm values file would be:

```yaml
apiVersion: controller.config.cert-manager.io/v1alpha1
kind: ControllerConfiguration
metricsTLSConfig:
dynamic:
secretNamespace: "cert-manager"
secretName: "cert-manager-metrics-ca"
dnsNames:
- cert-manager-metrics
- cert-manager-metrics.cert-manager
- cert-manager-metrics.cert-manager.svc
```

When using Prometheus the CA generated by the generated certificate authority can be trusted as part of the `PodMonitor` or `ServiceMonitor` spec:

```yaml
apiVersion: monitoring.coreos.com/v1
kind: PodMonitor
metadata:
name: cert-manager
namespace: cert-manager
labels:
app: cert-manager
app.kubernetes.io/name: cert-manager
app.kubernetes.io/instance: cert-manager
app.kubernetes.io/component: "controller"
spec:
jobLabel: app.kubernetes.io/name
selector:
matchLabels:
app: cert-manager
app.kubernetes.io/name: cert-manager
app.kubernetes.io/instance: cert-manager
app.kubernetes.io/component: "controller"
podMetricsEndpoints:
- port: http-metrics
# values.yaml
prometheus:
enabled: true
podmonitor:
enabled: true
endpointAdditionalProperties:
scheme: https
honorLabels: true
# TLS config trusting the CA and specifying the server name
tlsConfig:
serverName: cert-manager-metrics
ca:
secret:
name: cert-manager-metrics-ca
key: "tls.crt"
config:
metricsTLSConfig:
dynamic:
secretNamespace: "cert-manager"
secretName: "cert-manager-metrics-ca"
dnsNames:
- cert-manager-metrics
webhook:
config:
metricsTLSConfig:
dynamic:
secretNamespace: "cert-manager"
secretName: "cert-manager-metrics-ca"
dnsNames:
- cert-manager-metrics
cainjector:
config:
metricsTLSConfig:
dynamic:
secretNamespace: "cert-manager"
secretName: "cert-manager-metrics-ca"
dnsNames:
- cert-manager-metrics
```

> ℹ️ This configuration will result in a single new Secret `cert-manager/cert-manager-metrics-ca` containing a CA.
> The first `controller`, `webook`, or `cainjector` Pod will create the CA Secret and the others will then use it.
>
> All the controller, webhook, and cainjector Pods will generate their own unique metrics serving certificates
> and sign them with the CA private key.
>
> The `PodMonitor` is configured to read the public certificate from the CA Secret
> and Prometheus will use that CA when it connects to the metrics servers of each of the matching Pods.
>
> All the serving certificates share the same DNS name.
> That same name must be added to the `PodMonitor`
> and Prometheus will use that hostname when it connects to the metrics servers of each of the matching Pods.

##### Troubleshooting

Check the controller, webhook and cainjector logs to see the CA certificate and serving certificates being created and updated:

```sh
kubectl -n cert-manager logs -l app.kubernetes.io/instance=cert-manager --prefix
```

```console
I0719 15:21:28.113411 1 dynamic_source.go:172] "Detected root CA rotation - regenerating serving certificates" logger="cert-manager"
I0719 15:21:28.115018 1 dynamic_source.go:290] "Updated cert-manager TLS certificate" logger="cert-manager" DNSNames=["cert-manager-metrics"]
```

Check the connection to the metrics endpoint using `kubectl port-forward` and `curl`:

```sh
kubectl port-forward -n cert-manager deployment/cert-manager-webhook 9402
curl --insecure -v https://localhost:9402/metrics
```

Check the health of the cert-manager scrape targets on the Prometheus status page:

![](/docs/devops-tips/prometheus-metrics/prometheus-status-targets.png)

## Monitoring Mixin

Monitoring mixins are a way to bundle common alerts, rules, and dashboards for an application in a configurable and extensible way, using the Jsonnet data templating language. A cert-manager monitoring mixin can be found here https://gitlab.com/uneeq-oss/cert-manager-mixin. Documentation on usage can be found with the `cert-manager-mixin` project.
8 changes: 8 additions & 0 deletions content/docs/manifest.json
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,14 @@
"title": "Supported Releases",
"path": "/docs/releases/README.md"
},
{
"title": "1.16",
"path": "/docs/releases/release-notes/release-notes-1.16.md"
},
{
"title": "Upgrade 1.15 to 1.16",
"path": "/docs/releases/upgrading/upgrading-1.15-1.16.md"
},
{
"title": "1.15",
"path": "/docs/releases/release-notes/release-notes-1.15.md"
Expand Down
Loading