Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: cassandra container never becomes healthy #6215

Closed
akstron opened this issue Nov 16, 2024 · 2 comments · Fixed by #6214
Closed

[Bug]: cassandra container never becomes healthy #6215

akstron opened this issue Nov 16, 2024 · 2 comments · Fixed by #6214

Comments

@akstron
Copy link
Contributor

akstron commented Nov 16, 2024

What happened?

Post addition of password-based authentication, the Cassandra container spawned by the docker-compose never becomes healthy. This is because the username and password is not provided to the health check cmd.

Image

Upon checking health check stat using: docker inspect --format "{{json .State.Health }}" "v4-cassandra-1" | jq

Image

Steps to reproduce

  1. docker compose -f "docker-compose/cassandra/v4/docker-compose.yaml" up -d

Expected behavior

  1. Getting a healthy status at some point and no authentication error logs.

Relevant log output

 "Status": "unhealthy",
  "FailingStreak": 9,
  "Log": [
    {
      "Start": "2024-11-16T13:37:50.300599482Z",
      "End": "2024-11-16T13:37:50.659393355Z",
      "ExitCode": 1,
      "Output": "Connection error: ('Unable to connect to any servers', {'127.0.0.1:9042': AuthenticationFailed('Remote end requires authentication')})\n"
    },
    {
      "Start": "2024-11-16T13:38:20.658710087Z",
      "End": "2024-11-16T13:38:21.009395221Z",
      "ExitCode": 1,
      "Output": "Connection error: ('Unable to connect to any servers', {'127.0.0.1:9042': AuthenticationFailed('Remote end requires authentication')})\n"
    },

Jaeger backend version

Both v1 and v2

Stogage backend

Cassandra v4

@yurishkuro
Copy link
Member

What about these CI runs on main?

https://github.com/jaegertracing/jaeger/actions/runs/11843397266

@akstron
Copy link
Contributor Author

akstron commented Nov 16, 2024

yurishkuro pushed a commit that referenced this issue Nov 17, 2024
## Which problem is this PR solving?
- Resolves #6215 

## Description of the changes
- Added the username and password values for health check cmd
- Added `healthcheck_cassandra` point to make sure the container is
healthy as per the cmd.

## How was this change tested?
- bash scripts/cassandra-integration-test.sh 4 v004 v1 
- bash scripts/cassandra-integration-test.sh 4 v004 v2

## Checklist
- [x] I have read
https://github.com/jaegertracing/jaeger/blob/master/CONTRIBUTING_GUIDELINES.md
- [x] I have signed all commits
- [ ] I have added unit tests for the new functionality
- [x] I have run lint and test steps successfully
  - for `jaeger`: `make lint test`
  - for `jaeger-ui`: `yarn lint` and `yarn test`

---------

Signed-off-by: Alok Kumar Singh <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants