Persistence and Configuration Issues with Langflow Deployment Using Docker Compose

[kozanakyel]

Bug Description

When deploying Langflow using the provided docker-compose.yml and environment configuration from langflow/deploy folder via docker-compose pull and up, the application fails to use the specified PostgreSQL database and instead defaults to SQLite. Even if i use .env.example's db conf credentials. This leads to data loss for flows and API keys between restarts. Additionally, the Celery workers are not running due to a missing celery module, preventing horizontal scaling of worker processes. Streaming functionalities remain in a loading state, and SSL is not being enabled even though a subdomain and related settings are provided.

Reproduction

1-Clone the langflow repository and checkout the v1.1.1 tag.
2-Prepare an .env file with valid PostgreSQL configuration details (e.g., POSTGRES_DB, POSTGRES_USER, POSTGRES_PASSWORD, POSTGRES_HOST, POSTGRES_PORT) within langflow/deploy.
3-Run docker compose pull followed by docker compose up.
4-Observe that a superuser is created, but newly created flows and API keys are not persisted to the PostgreSQL database and will be lost after a restart.
5-Stop the services with docker compose down and bring them up again using docker compose up.
6-Note that previously created flows and API keys are no longer available.
7-Check docker logs deploy-celeryworker-1 to find No module named 'celery' errors.
8-Attempt to use the streaming functionality and observe that the frontend UI remains stuck in a loading state.
9-Configure a domain/subdomain and note that SSL does not activate, leaving the site accessible only over HTTP.

Expected behavior

Database Persistence: The configured PostgreSQL database (set in .env) should be used for storing flows and API keys, ensuring persistence across restarts.
Worker Scaling: Celery workers should start without errors, allowing for horizontal scaling and handling of asynchronous tasks.
Functional Streaming: The frontend UI should properly display streams rather than remaining in a loading state.
SSL Activation: With a valid domain/subdomain configuration, SSL should be automatically enabled, ensuring secure HTTPS access.

Who can help?

@italojohnny @ogabrielluiz @nicoloboschi @zzzming

Operating System

UBUNTU 24.10

Langflow Version

1.1.1

Python Version

3.12

Screenshot

No response

Flow File

No response

[dosubot]

Hey @kozanakyel! I'm here to help you with the Langflow deployment issue you're facing. Let's work together to solve this!

To address the issues you're experiencing with Langflow deployment using Docker Compose, here are some steps you can take:

1. PostgreSQL Database Not Being Used:

   • Ensure that your .env file is correctly configured with the PostgreSQL connection details. The LANGFLOW_DATABASE_URL should be set to use PostgreSQL, and the format should be something like postgresql://<username>:<password>@<host>:<port>/<database>.
   • Verify that the docker-compose.yml file is correctly set up to use the PostgreSQL service and that the langflow service is configured to connect to it. Check the volume mounts to ensure persistence is correctly set up ^[1].

2. Celery Workers Not Running:

   • The error "No module named 'celery'" suggests that the Celery module might not be installed in the backend image. You may need to modify the Dockerfile to include the installation of Celery or ensure that the correct version of the application is being used. Check the requirements.txt or equivalent file to ensure Celery is listed as a dependency ^[2].

3. Streaming Functionalities Stuck in Loading State:

   • This might be due to misconfiguration or connectivity issues between the frontend and backend. Check the logs of the langflow service for any errors and ensure that the frontend is correctly pointing to the backend URL. The backend should be accessible at http://localhost:7860/ ^[2].

4. SSL Not Being Enabled:

   • Ensure that the Traefik configuration in the docker-compose.yml is correctly set up for SSL. The environment variables for the domain and Let's Encrypt should be correctly configured. Check that your domain is pointing to the server's IP address and review Traefik logs for any SSL-related errors ^[1].

These steps should help resolve the issues you're facing. If problems persist, further investigation into the logs and configuration files may be necessary.

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Other}

[kozanakyel]

Hello @dosu i cloned the repository to my server then i go to langflow/deploy folder. i edit the .env file then i commadn the docker-compose pull and docker-compse up. but i got deploy-celeryworker-1 to find No module named 'celery' errors. and also if i create a flow or api key from settings and i try to down and up docker compose. i did not see my created flows or any other conf. you can see my env. file. and also my ip point to my domain that is not pronblem at this phase. celeryworker and concurrency important from that. this is my env file example==>`DOMAIN=xxx.mywebsite.com
STACK_NAME=langflow-stack

TRAEFIK_PUBLIC_NETWORK=traefik-public
TRAEFIK_TAG=langflow-traefik
TRAEFIK_PUBLIC_TAG=traefik-public

Langflow backend configuration

LANGFLOW_AUTO_LOGIN=False
LANGFLOW_SUPERUSER=superuser
LANGFLOW_SUPERUSER_PASSWORD=superuser

Langflow frontend configuration

BACKEND_URL=http://backend:7860

LANGFLOW_WORKER_TIMEOUT=300
LANGFLOW_WORKERS=4

RabbitMQ configuration

RABBITMQ_DEFAULT_USER=langflow
RABBITMQ_DEFAULT_PASS=langflow

Database configuration

DB_USER=langflow
DB_PASSWORD=langflow
DB_HOST=db
DB_PORT=5432
DB_NAME=langflow

DB configuration

POSTGRES_USER=langflow
POSTGRES_PASSWORD=langflow
POSTGRES_DB=langflow
POSTGRES_PORT=5432

Flower configuration

Disable until #2655 gets released

#LANGFLOW_CACHE_TYPE=redis
LANGFLOW_REDIS_HOST=result_backend
LANGFLOW_REDIS_PORT=6379
LANGFLOW_REDIS_DB=0
LANGFLOW_REDIS_EXPIRE=3600
LANGFLOW_REDIS_PASSWORD=
FLOWER_UNAUTHENTICATED_API=True
BROKER_URL=amqp://langflow:langflow@broker:5672
RESULT_BACKEND=redis://result_backend:6379/0
C_FORCE_ROOT="true"

LANGFLOW_DATABASE_URL=postgresql://langflow:langflow@db:5432/langflow

PGAdmin configuration

PGADMIN_DEFAULT_EMAIL=[email protected]
PGADMIN_DEFAULT_PASSWORD=admin
`

[dosubot]

The "No module named 'celery'" error suggests that Celery is not installed in your Docker container. To resolve this, ensure that Celery is included in your Docker setup. Here are some steps to address the issues:

1. Celery Installation:

   • Verify that Celery is listed as a dependency in your pyproject.toml file. It should be included as an optional dependency with extras for Redis ^[1].
   • Check the Dockerfile used for building the backend image to ensure it installs Celery. If it's missing, you may need to modify the Dockerfile to include the installation of Celery.

2. Database Persistence:

   • Ensure that the LANGFLOW_DATABASE_URL is correctly set in your .env file to use PostgreSQL. It should look like postgresql://langflow:langflow@db:5432/langflow.
   • Verify that the docker-compose.yml file correctly maps the langflow service to the PostgreSQL service and that the volumes are set up for data persistence ^[2].

3. Redis Configuration:

   • Ensure that Redis is correctly configured as the result backend. Your .env file should have RESULT_BACKEND=redis://result_backend:6379/0 and related Redis settings properly set.

4. General Configuration:

   • Double-check your .env file for any discrepancies or missing configurations that might affect the services.

By ensuring these configurations are correct, you should be able to resolve the issues with Celery and data persistence. If problems persist, reviewing the logs for more specific error messages can provide further insights.

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Other}