Merge branch 'main' into cassandra-docker-compose-fix-2

ADOPTERS.md

@@ -1,3 +1,5 @@
+* Alauda
+  * Official site: [en](https://www.alauda.io/), [cn](https://www.alauda.cn/)
 * [Base CRM](https://getbase.com/)
 * [Circonus](https://www.circonus.com/)
 * [ContaAzul](https://contaazul.com/)

CHANGELOG.md

@@ -21,6 +21,68 @@ copy from UI changelog
 
 </details>
 
+v1.63.0 / v2.0.0 (2024-11-10)
+-------------------------------
+
+Jaeger v2 is here! 🎉 🎉 🎉 
+
+### Backend Changes
+
+#### ⛔ Breaking Changes
+
+* Remove jaeger-agent from distributions ([@yurishkuro](https://github.com/yurishkuro) in [#6081](https://github.com/jaegertracing/jaeger/pull/6081))
+
+#### 🐞 Bug fixes, Minor Improvements
+
+* Fix possible null pointer deference ([@vaidikcode](https://github.com/vaidikcode) in [#6184](https://github.com/jaegertracing/jaeger/pull/6184))
+* Chore: enable all rules of perfsprint linter ([@mmorel-35](https://github.com/mmorel-35) in [#6164](https://github.com/jaegertracing/jaeger/pull/6164))
+* Chore: enable err-error and errorf rules from perfsprint linter ([@mmorel-35](https://github.com/mmorel-35) in [#6160](https://github.com/jaegertracing/jaeger/pull/6160))
+* [query] move trace handler to server level ([@mahadzaryab1](https://github.com/mahadzaryab1) in [#6147](https://github.com/jaegertracing/jaeger/pull/6147))
+* [fix][query] remove bifurcation for grpc query server ([@mahadzaryab1](https://github.com/mahadzaryab1) in [#6145](https://github.com/jaegertracing/jaeger/pull/6145))
+* [jaeger-v2] add hotrod integration test for jaeger-v2 ([@Saumya40-codes](https://github.com/Saumya40-codes) in [#6138](https://github.com/jaegertracing/jaeger/pull/6138))
+* [query] use otel's helpers for http server ([@mahadzaryab1](https://github.com/mahadzaryab1) in [#6121](https://github.com/jaegertracing/jaeger/pull/6121))
+* Use grpc interceptors instead of explicit context wrappers ([@chahatsagarmain](https://github.com/chahatsagarmain) in [#6133](https://github.com/jaegertracing/jaeger/pull/6133))
+* Fix command in v2 example ([@haoqixu](https://github.com/haoqixu) in [#6134](https://github.com/jaegertracing/jaeger/pull/6134))
+* Fix span deduplication via correct ordering of adjusters ([@cdanis](https://github.com/cdanis) in [#6116](https://github.com/jaegertracing/jaeger/pull/6116))
+* Move all query service http handlers into one function ([@yurishkuro](https://github.com/yurishkuro) in [#6128](https://github.com/jaegertracing/jaeger/pull/6128))
+* [fix][grpc] disable tracing in grpc storage writer clients ([@mahadzaryab1](https://github.com/mahadzaryab1) in [#6125](https://github.com/jaegertracing/jaeger/pull/6125))
+* Feat: automatically publish readme to docker hub ([@inosmeet](https://github.com/inosmeet) in [#6118](https://github.com/jaegertracing/jaeger/pull/6118))
+* Use grpc interceptors for bearer token ([@chahatsagarmain](https://github.com/chahatsagarmain) in [#6063](https://github.com/jaegertracing/jaeger/pull/6063))
+* [fix][query] correct query server legacy condition ([@mahadzaryab1](https://github.com/mahadzaryab1) in [#6120](https://github.com/jaegertracing/jaeger/pull/6120))
+* [query] use otel's helpers for grpc server ([@mahadzaryab1](https://github.com/mahadzaryab1) in [#6055](https://github.com/jaegertracing/jaeger/pull/6055))
+* Enable lint rule: import-shadowing ([@inosmeet](https://github.com/inosmeet) in [#6102](https://github.com/jaegertracing/jaeger/pull/6102))
+* [refractor] switch to enums for es mappings ([@Saumya40-codes](https://github.com/Saumya40-codes) in [#6091](https://github.com/jaegertracing/jaeger/pull/6091))
+* Fix rebuild-ui.sh script ([@andreasgerstmayr](https://github.com/andreasgerstmayr) in [#6098](https://github.com/jaegertracing/jaeger/pull/6098))
+* Use otel component host instead of no op host for prod code ([@chahatsagarmain](https://github.com/chahatsagarmain) in [#6085](https://github.com/jaegertracing/jaeger/pull/6085))
+* [cassandra] prevent fallback to old schema for operation names table in case of db issues ([@arunvelsriram](https://github.com/arunvelsriram) in [#6061](https://github.com/jaegertracing/jaeger/pull/6061))
+
+#### 🚧 Experimental Features
+
+* Add otlp json support for kafka e2e integration tests ([@joeyyy09](https://github.com/joeyyy09) in [#5935](https://github.com/jaegertracing/jaeger/pull/5935))
+* [v2] add es config comments ([@yurishkuro](https://github.com/yurishkuro) in [#6110](https://github.com/jaegertracing/jaeger/pull/6110))
+* [chore][docs] add documentation to elasticsearch configuration ([@mahadzaryab1](https://github.com/mahadzaryab1) in [#6103](https://github.com/jaegertracing/jaeger/pull/6103))
+* [jaeger-v2] refactor elasticsearch/opensearch configurations to have more logical groupings ([@mahadzaryab1](https://github.com/mahadzaryab1) in [#6090](https://github.com/jaegertracing/jaeger/pull/6090))
+* [jaeger-v2] implement utf-8 sanitizer for otlp ([@mahadzaryab1](https://github.com/mahadzaryab1) in [#6078](https://github.com/jaegertracing/jaeger/pull/6078))
+* [jaeger-v2] migrate elasticsearch/opensearch to use otel's tls configuration ([@mahadzaryab1](https://github.com/mahadzaryab1) in [#6079](https://github.com/jaegertracing/jaeger/pull/6079))
+* [jaeger-v2] enable queueing configuration in storage exporter ([@mahadzaryab1](https://github.com/mahadzaryab1) in [#6080](https://github.com/jaegertracing/jaeger/pull/6080))
+* [jaeger-v2] implement empty service name sanitizer for otlp ([@mahadzaryab1](https://github.com/mahadzaryab1) in [#6077](https://github.com/jaegertracing/jaeger/pull/6077))
+* [jaeger-v2] refactor elasticsearch/opensearch storage configurations ([@mahadzaryab1](https://github.com/mahadzaryab1) in [#6060](https://github.com/jaegertracing/jaeger/pull/6060))
+
+#### 👷 CI Improvements
+
+* [v2] use health check in grpc e2e test ([@yurishkuro](https://github.com/yurishkuro) in [#6113](https://github.com/jaegertracing/jaeger/pull/6113))
+* Update node.js github action to use npm lockfile, switch to latest jaeger ui ([@andreasgerstmayr](https://github.com/andreasgerstmayr) in [#6074](https://github.com/jaegertracing/jaeger/pull/6074))
+
+### 📊 UI Changes
+
+#### 🐞 Bug fixes, Minor Improvements
+
+* Migrate from yarn v1 to npm ([@andreasgerstmayr](https://github.com/andreasgerstmayr) in [#2462](https://github.com/jaegertracing/jaeger-ui/pull/2462))
+
+#### 👷 CI Improvements
+
+* Run s390x build on push to main only ([@andreasgerstmayr](https://github.com/andreasgerstmayr) in [#2481](https://github.com/jaegertracing/jaeger-ui/pull/2481))
+
 1.62.0 / 2.0.0-rc2 (2024-10-06)
 -------------------------------
 

RELEASE.md

@@ -1,7 +1,6 @@
 # Jaeger Overall Release Process
 
-1. Determine new version numbers for v1.x.x and v2.x.x-rcN
-    * v2 version is currently in the form v2.0.0-rcN where N is the next number since the last release.
+1. Determine new version numbers for v1.x.x and v2.x.x
 2. Perform UI release according to https://github.com/jaegertracing/jaeger-ui/blob/main/RELEASE.md
 3. Perform Backend release (see below)
 4. [Publish documentation](https://github.com/jaegertracing/documentation/blob/main/RELEASE.md) for the new version on `jaegertracing.io`.
@@ -11,8 +10,8 @@
 
 # Jaeger Backend Release Process
 
-1. Create a PR "Prepare release 1.x.x / 2.x.x-rcN" against main or maintenance branch ([example](https://github.com/jaegertracing/jaeger/pull/543/files)) by updating CHANGELOG.md to include:
-    * A new section with the header `1.x.x / 2.x.x-rcN (YYYY-MM-DD)` (copy the template at the top)
+1. Create a PR "Prepare release 1.x.x / 2.x.x" against main or maintenance branch ([example](https://github.com/jaegertracing/jaeger/pull/543/files)) by updating CHANGELOG.md to include:
+    * A new section with the header `1.x.x / 2.x.x (YYYY-MM-DD)` (copy the template at the top)
     * A curated list of notable changes and links to PRs. Do not simply dump git log, select the changes that affect the users.
       To obtain the list of all changes run `make changelog`.
     * The section can be split into sub-section if necessary, e.g. UI Changes, Backend Changes, Bug Fixes, etc.
@@ -41,7 +40,7 @@
     * Automated:
        * `make draft-release`
     * Manual:
-       * Title "Release 1.x.x / 2.x.x-rcN"
+       * Title "Release 1.x.x / 2.x.x"
        * Tag `v1.x.x` (note the `v` prefix) and choose appropriate branch (usually `main`)
        * Copy the new CHANGELOG.md section into the release notes
        * Extra: GitHub has a button "generate release notes". Those are not formatted as we want,
@@ -79,8 +78,8 @@ Here are the release managers for future versions with the tentative release dat
 
 | Version | Release Manager | Tentative release date |
 |---------|-----------------|------------------------|
-| 1.63.0  | @pavolloffay    | 5 November 2024        |
 | 1.64.0  | @joe-elliott    | 4 December 2024        |
 | 1.65.0  | @jkowall        | 8 January 2025         |
 | 1.66.0  | @yurishkuro     | 3 February 2025        |
 | 1.67.0  | @albertteoh     | 5 March 2025           |
+| 1.68.0  | @pavolloffay    | 5 April 2025           |

cmd/agent/app/builder_test.go

@@ -223,6 +223,7 @@ func TestCreateCollectorProxy(t *testing.T) {
 
 	for _, test := range tests {
 		t.Run("", func(t *testing.T) {
+			t.Parallel()
 			flags := &flag.FlagSet{}
 			grpc.AddFlags(flags)
 			reporter.AddFlags(flags)

cmd/agent/app/reporter/connect_metrics.go

@@ -5,6 +5,7 @@ package reporter
 
 import (
 	"expvar"
+	"sync"
 
 	"github.com/jaegertracing/jaeger/pkg/metrics"
 )
@@ -23,17 +24,21 @@ type ConnectMetrics struct {
 	target  *expvar.String
 }
 
+var (
+	gRPCTarget     *expvar.String
+	gRPCTargetOnce sync.Once
+)
+
 // NewConnectMetrics will be initialize ConnectMetrics
 func NewConnectMetrics(mf metrics.Factory) *ConnectMetrics {
 	cm := &ConnectMetrics{}
 	metrics.MustInit(&cm.metrics, mf.Namespace(metrics.NSOptions{Name: "connection_status"}), nil)
 
-	if r := expvar.Get("gRPCTarget"); r == nil {
-		cm.target = expvar.NewString("gRPCTarget")
-	} else {
-		cm.target = r.(*expvar.String)
-	}
-
+	// expvar.String is not thread-safe, so we need to initialize it only once
+	gRPCTargetOnce.Do(func() {
+		gRPCTarget = expvar.NewString("gRPCTarget")
+	})
+	cm.target = gRPCTarget
 	return cm
 }
 

cmd/agent/app/reporter/grpc/builder_test.go

@@ -300,10 +300,12 @@ func TestProxyClientTLS(t *testing.T) {
 			expectError: false,
 		},
 	}
-	ctx, cancel := context.WithCancel(context.Background())
-	defer cancel()
+
 	for _, test := range tests {
 		t.Run(test.name, func(t *testing.T) {
+			t.Parallel()
+			ctx, cancel := context.WithCancel(context.Background())
+			defer cancel()
 			var opts []grpc.ServerOption
 			if test.serverTLS.Enabled {
 				tlsCfg, err := test.serverTLS.Config(zap.NewNop())

cmd/es-rollover/app/rollover/flags_test.go

@@ -28,5 +28,5 @@ func TestBindFlags(t *testing.T) {
 	require.NoError(t, err)
 
 	c.InitFromViper(v)
-	assert.Equal(t, "{\"max_age\": \"20000d\"}", c.Conditions)
+	assert.JSONEq(t, `{"max_age": "20000d"}`, c.Conditions)
 }

cmd/ingester/app/consumer/offset/concurrent_list_test.go

@@ -19,8 +19,8 @@ func insert(list *ConcurrentList, offsets ...int64) {
 
 func TestInsert(t *testing.T) {
 	for _, testCase := range generatePermutations([]int64{1, 2, 3}) {
-		min, toInsert := extractMin(testCase)
-		s := newConcurrentList(min)
+		m, toInsert := extractMin(testCase)
+		s := newConcurrentList(m)
 		insert(s, toInsert...)
 		assert.ElementsMatch(t, testCase, s.offsets)
 	}
@@ -62,8 +62,8 @@ func TestGetHighestAndReset(t *testing.T) {
 	for _, testCase := range testCases {
 		for _, input := range generatePermutations(testCase.input) {
 			t.Run(fmt.Sprintf("%v", input), func(t *testing.T) {
-				min, input := extractMin(input)
-				s := newConcurrentList(min)
+				m, input := extractMin(input)
+				s := newConcurrentList(m)
 				insert(s, input...)
 				actualOffset := s.setToHighestContiguous()
 				assert.ElementsMatch(t, testCase.expectedList, s.offsets)

cmd/jaeger/README.md

@@ -1,10 +1,8 @@
 # jaeger
 
-This is experimental Jaeger V2 based on OpenTelemetry collector.
+Jaeger V2 based on OpenTelemetry collector.
 Read the [blog post](https://medium.com/jaegertracing/towards-jaeger-v2-moar-opentelemetry-2f8239bee48e).
 
-Tracking issue: https://github.com/jaegertracing/jaeger/issues/4843.
-
 ```mermaid
 flowchart LR
     Receiver1 --> Processor
@@ -36,9 +34,10 @@ flowchart LR
 
 ## Try it out
 
-* Download `docker-compose-v2.yml` from https://github.com/jaegertracing/jaeger/blob/main/examples/hotrod/docker-compose-v2.yml
-* Optional: find the latest Jaeger version (see https://www.jaegertracing.io/download/) and pass it via environment variable `JAEGER_VERSION`. Otherwise `docker compose` will use the `latest` tag, which is fine for the first time you download the images, but once they are in your local registry the `latest` tag is never updated and you may be running stale (and possibly incompatible) verions of Jaeger and the HotROD app.
+* Download `docker-compose-v2.yml` from https://github.com/jaegertracing/jaeger/blob/main/examples/hotrod/docker-compose-v2.yml, e.g.:
+  * `curl -O https://raw.githubusercontent.com/jaegertracing/jaeger/refs/heads/main/examples/hotrod/docker-compose-v2.yml`
+* Optional: find the latest images versions (see https://www.jaegertracing.io/download/) and pass them via environment variables `JAEGER_VERSION` and `HOTROD_VERSION`. Otherwise `docker compose` will use the `latest` tag, which is fine for the first time you download the images, but once they are in your local registry the `latest` tag is never updated and you may be running stale (and possibly incompatible) verions of Jaeger and the HotROD app.
 * Run Jaeger backend and HotROD demo, e.g.:
-  * `JAEGER_VERSION=1.59 docker compose -f path-to-yml-file-v2 up`
+  * `JAEGER_VERSION=2.0.0 HOTROD_VERSION=1.63.0 docker compose -f docker-compose-v2.yml up`
 * Access Jaeger UI at http://localhost:16686 and HotROD app at http://localhost:8080
-* Shutdown / cleanup with `docker compose -f path-to-yml-file down`
+* Shutdown / cleanup with `docker compose -f docker-compose-v2.yml down`

cmd/jaeger/internal/extension/jaegerstorage/extension.go

@@ -12,7 +12,6 @@ import (
 	"go.opentelemetry.io/collector/component"
 	"go.opentelemetry.io/collector/extension"
 
-	"github.com/jaegertracing/jaeger/cmd/jaeger/internal/extension/jaegerstorage/factoryadapter"
 	"github.com/jaegertracing/jaeger/internal/metrics/otelmetrics"
 	"github.com/jaegertracing/jaeger/pkg/metrics"
 	"github.com/jaegertracing/jaeger/plugin/metrics/prometheus"
@@ -22,6 +21,7 @@ import (
 	"github.com/jaegertracing/jaeger/plugin/storage/grpc"
 	"github.com/jaegertracing/jaeger/plugin/storage/memory"
 	"github.com/jaegertracing/jaeger/storage"
+	"github.com/jaegertracing/jaeger/storage_v2/factoryadapter"
 	"github.com/jaegertracing/jaeger/storage_v2/spanstore"
 )
 

docker-compose/elasticsearch/v8/docker-compose.yml

@@ -1,6 +1,6 @@
 services:
   elasticsearch:
-    image: docker.elastic.co/elasticsearch/elasticsearch:8.15.0
+    image: docker.elastic.co/elasticsearch/elasticsearch:8.16.0
     environment:
       - discovery.type=single-node
       - http.host=0.0.0.0

docker-compose/kafka-integration-test/docker-compose.yml

@@ -1,6 +1,6 @@
 services:
   kafka:
-    image: bitnami/kafka:3.8.0
+    image: bitnami/kafka:3.9.0
     ports:
       - "9092:9092"
     environment:

docker-compose/monitor/README.md

@@ -81,7 +81,7 @@ docker compose -f docker-compose-v2.yml up
 **Tips:**
 - Let the application run for a couple of minutes to ensure there is enough time series data to plot in the dashboard.
 - Navigate to Jaeger UI at http://localhost:16686/ and inspect the Monitor tab. Select `redis` service from the dropdown to see more than one endpoint.
-- To visualize the raw metrics stored on the Prometheus server (for debugging and local development use cases), use the built-in Prometheus UI at http://localhost:9090/graph. For example, http://localhost:9090/graph?g0.expr=traces_span_metrics_calls_total&g0.tab=0&g0.range_input=5m
+- To visualize the raw metrics stored on the Prometheus server (for debugging and local development use cases), use the built-in Prometheus UI at http://localhost:9090/query. For example, http://localhost:9090/query?g0.expr=traces_span_metrics_calls_total&g0.tab=0&g0.range_input=5m
 
 **Warning:** The included [docker-compose.yml](./docker-compose.yml) file uses the `latest` version of Jaeger and other components. If your local Docker registry already contains older versions, which may still be tagged as `latest`, you may want to delete those images before running the full set, to ensure consistent behavior:
 

docker-compose/opensearch/v2/docker-compose.yml

@@ -1,6 +1,6 @@
 services:
   opensearch:
-    image: opensearchproject/opensearch:2.17.0
+    image: opensearchproject/opensearch:2.18.0
     environment:
       - discovery.type=single-node
       - plugins.security.disabled=true

docker/debug/Dockerfile

@@ -1,7 +1,7 @@
 # Copyright (c) 2024 The Jaeger Authors.
 # SPDX-License-Identifier: Apache-2.0
 
-FROM golang:1.23.2-alpine AS build
+FROM golang:1.23.3-alpine AS build
 ARG TARGETARCH
 ENV GOPATH /go
 RUN apk add --update --no-cache ca-certificates make git build-base mailcap
@@ -16,7 +16,7 @@ RUN if [[ "$TARGETARCH" == "s390x" ||  "$TARGETARCH" == "ppc64le" ]] ; then \
         cd /go/src/debug-delve && go mod download && go build -o /go/bin/dlv github.com/go-delve/delve/cmd/dlv; \
     fi 
 
-FROM golang:1.23.2-alpine
+FROM golang:1.23.3-alpine
 COPY --from=build /go/bin/dlv /go/bin/dlv
 COPY --from=build /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/ca-certificates.crt
 COPY --from=build /etc/mime.types /etc/mime.types

examples/hotrod/docker-compose-v2.yml

@@ -1,5 +1,5 @@
 # To run a specific version of Jaeger, use environment variable, e.g.:
-#     JAEGER_VERSION=1.52 docker compose up
+#     JAEGER_VERSION=2.0.0 HOTROD_VERSION=1.63.0 docker compose up
 
 services:
   jaeger:
@@ -17,7 +17,7 @@ services:
       - jaeger-example
 
   hotrod:
-    image: ${REGISTRY:-}jaegertracing/example-hotrod:${JAEGER_VERSION:-latest}
+    image: ${REGISTRY:-}jaegertracing/example-hotrod:${HOTROD_VERSION:-latest}
     # To run the latest trunk build, find the tag at Docker Hub and use the line below
     # https://hub.docker.com/r/jaegertracing/example-hotrod-snapshot/tags
     #image: jaegertracing/example-hotrod-snapshot:0ab8f2fcb12ff0d10830c1ee3bb52b745522db6c