kombu tries to connect to localhost:6379 for redis backend

[baojd42]

I followed the example for django. When I run the test, I can see the broker and results backend are connected. But the test fail due to kombu looking for connection on localhost:6379. What is weird is in the printed config info, the results backend does not show a port number. Is this normal? Any help is appreciated.

 
 -------------- robin3_regular_worker@821ce159a458 v5.4.0 (opalescent)
--- ***** ----- 
-- ******* ---- Linux-6.5.0-35-generic-x86_64-with-glibc2.29 2024-07-01 14:52:44
- *** --- * --- 
- ** ---------- [config]
- ** ---------- .> app:         robin3_web:0x7c0531f4d730
- ** ---------- .> transport:   redis://7bf1bac6233d:6379/0
- ** ---------- .> results:     redis://35f0c7da356d/0
- *** --- * --- .> concurrency: 16 (prefork)
-- ******* ---- .> task events: OFF (enable -E to monitor tasks in this worker)
--- ***** ----- 
 -------------- [queues]
                .> regular          exchange=regular(direct) key=regular
                

[tasks]
  . django_long_running_task.tasks.task_thru_db
  . robin3.tasks.delete_task_path

[2024-07-01 14:52:45,523: WARNING/MainProcess] /usr/local/lib/python3.8/dist-packages/celery/worker/consumer/consumer.py:508: CPendingDeprecationWarning: The broker_connection_retry configuration setting will no longer determine
whether broker connection retries are made during startup in Celery 6.0 and above.
If you wish to retain the existing behavior for retrying connections on startup,
you should set broker_connection_retry_on_startup to True.
  warnings.warn(

[2024-07-01 14:52:45,530: INFO/MainProcess] Connected to redis://7bf1bac6233d:6379/0
[2024-07-01 14:52:45,530: WARNING/MainProcess] /usr/local/lib/python3.8/dist-packages/celery/worker/consumer/consumer.py:508: CPendingDeprecationWarning: The broker_connection_retry configuration setting will no longer determine
whether broker connection retries are made during startup in Celery 6.0 and above.
If you wish to retain the existing behavior for retrying connections on startup,
you should set broker_connection_retry_on_startup to True.
  warnings.warn(

[2024-07-01 14:52:45,531: INFO/MainProcess] mingle: searching for neighbors
[2024-07-01 14:52:46,538: INFO/MainProcess] mingle: all alone
[2024-07-01 14:52:46,555: INFO/MainProcess] robin3_regular_worker@821ce159a458 ready.

[Nusnus]

Each test node (e.g, broker/backend/worker) has two urls for its container instance.

1. “url” represents the connection string the worker will use and is generated from the hostname of the container. It is seen from the perspective of the worker container so it requires the broker’s container hostname in this case.
2. “host_url” represents the url from the perspective of the tests which are differentiating the containers by different ports on the localhost.

The “url”s are then set for the celery worker container using env vars here.

The host_url’s are used to control the containers from the test code itself.

But the test fail due to kombu looking for connection on localhost:6379

Can you share the test failure logs please?

Take note the django example is passing in the recent CI runs.

See how it is configured in the CI here.

I also recommend reviewing the Django Example doc - it might also give some useful insights.

LMK if any of the above helps @baojd42 🙏