nginx inc ingress controller telemetry - 404 Not Found

[serifeturksever]

i am using nginx inc ingress controller in openhift. i configured otlp and telemetry ingress controller ymal as below

otlp:

kind: Ingress
apiVersion: networking.k8s.io/v1
metadata:
name: otlp
namespace: istio-system
annotations:
ingress.kubernetes.io/ssl-redirect: 'false'
kubernetes.io/ingress.class: nginx
nginx.org/basic-auth-realm: Authentication Required - otlp
nginx.org/basic-auth-secret: basic-auth-secret
spec:
tls:

hosts: xxx
secretName: xxx
rules:
host: xxx
http:
paths:
path: /v1/traces
pathType: Prefix
backend:
service:
name: jaeger-collector
port:
number: 4318

zipkin:

kind: Ingress
apiVersion: networking.k8s.io/v1
metadata:
name: zipkin
namespace: istio-system
annotations:
ingress.kubernetes.io/ssl-redirect: 'false'
kubernetes.io/ingress.class: nginx
nginx.org/basic-auth-realm: Authentication Required - zipkin
nginx.org/basic-auth-secret: basic-auth-secret
nginx.org/rewrites: serviceName=jaeger-collector rewrite=/$1
spec:
tls:

hosts: xxx
secretName: xxx
rules:
host: xxx
http:
paths:
path: /(api/.)
pathType: Prefix
backend:
service:
name: jaeger-collector
port:
number: 9411
otlp request worked correctly. But when i request telemetry addresses "https://xxx/api/v2/spans" return 404 Not Found. How can i fix this?

Nginx Ingress Operator version:
2.3.2 provided by NGINX Inc

actually path is in zippin like this:

[github-actions]

Hi @serifeturksever thanks for reporting!

Be sure to check out the docs and the Contributing Guidelines while you wait for a human to take a look at this 🙂

Cheers!

[AlexFenlon]

Hi @serifeturksever,

Can you give us a copy of the relevant location blocks of the nginx.conf?

Can you please format the yaml format so we can have a better idea of the structure of the file?

Please have a look at the Ingress rewrites example as the paths you provided doesn't seem correct.

Have you considered looking at VirtualServer rewrites example as this might have the functionality you are looking for.

[serifeturksever]

hi @AlexFenlon,

First, let me talk about the structure. We will use telemetry in one of our projects. This will be integrated into the code. Normally, we used route, but since there was no authentication in the route, we decided to use nginx ingress. We use nginx ingress operator in Openshift. Since it is our first time using it, we are not fully familiar with the structure, to be honest.

We installed nginx ingress operator in nginx-ingress namespace. We created 2 ingress under istio-system namespace, otlp and zipkin. When we send a request to otlp, it works correctly. However, we get a 404 error in zipkin.

nginx ingress operator created nginxingress-sample-nginx-ingress and nginx-configuration configmaps. I thought it was using nginx-configuration because otlp did not work when I added basic auth. I had previously added configmaps, when I removed it, it started working, but it was not dependent on the environment of the nginx deployment. nginxingress-sample-nginx-ingress deployment is available in the environment. We did not make any changes to these configmaps.

nginxingress-sample-nginx-ingress: Managed by NginxIngress NI nginxingress-sample

nginx-configuration:

When I run the command oc exec -ti -n nginx-ingress /bin/bash on the server

etc/nginx/nginx.conf : (part of it)

etc/nginx/conf.d/ istio-system-otlp.conf:

istio-system-zipkin.conf was also there but it is not there now and I don't know why.

Also a route named zipkin was created automatically, I didn't understand how it was created. I deleted the route because ingress and route might conflict. I am also sending the content of the route I deleted:

I actually used the rewrite annotation as in the link you sent. When I removed it, I get the error
[error] 957#957: *43867 open() "/etc/nginx/html/api/v2/spans" failed (2: No such file or directory).

Zipkin address I sent my request: https://xxx/api/v2/spans
OTLP address I sent my request: https://xxx/v1/traces

[serifeturksever]

By the way, when I give path: /api/v2/spans and send a request from postman, I get "POST /api/v2/spans HTTP/1.1" 202.

[serifeturksever]

In addition, I can see the requests I send in the jaeger logs. When I enter jaeger, the service and operation come but not the traces. Do I need to edit the Logging keys in the configmap for this?

[AlexFenlon]

Hi @serifeturksever,

Do I need to edit the Logging keys in the configmap for this?

You do not need to edit the logging keys in the config map for this.

istio-system-zipkin.conf was also there but it is not there now and I don't know why.

Is the zipkin Ingress still applied here?

By the way, when I give path: /api/v2/spans and send a request from postman, I get "POST /api/v2/spans HTTP/1.1" 202.

Is zipkin now working?
It seems you are getting a 202 response which suggests that it is working.

[vepatel]

@serifeturksever It seems like you're using NIC Ingress annotations in kind: Route, can you try using kind Ingress with proper annotations and paths please.