[Question]: Does RAGFlow work without Qwen Models?

[theoalm66]

Describe your problem

Hi,

when I delete the default models ("Tongyi-Qianwen") from the Model providers page, and add an Ollama LLM Chat model and an Embedding model, I get an error "[ERROR]handle_task got exception, please check log" when adding a document to my Knowledge Base. In the logs I can see, that the task says the following:

2024-12-31 13:02:27 2024-12-31 13:02:27,317 INFO     20 set_progress(f40a2190c76e11efab7e0242ac130006), progress: None, progress_msg: 13:02:27 Page(1~13): Start to generate keywords for every chunk ...
2024-12-31 13:02:27 2024-12-31 13:02:27,351 INFO     20 set_progress(f40a2190c76e11efab7e0242ac130006), progress: -1, progress_msg: 13:02:27 [ERROR]handle_task got exception, please check log
2024-12-31 13:02:27 2024-12-31 13:02:27,385 ERROR    20 handle_task got exception for task {"id": "f40a2190c76e11efab7e0242ac130006", "doc_id": "f24a7c06c76e11ef82f90242ac130006", "from_page": 0, "to_page": 12, "retry_count": 0, "kb_id": "3987552ac76b11efaa990242ac130006", "parser_id": "naive", "parser_config": {"auto_keywords": 15, "auto_questions": 0, "raptor": {"use_raptor": true, "prompt": "Bitte fasse die folgenden Abs\u00e4tze zusammen. Sei vorsichtig mit den Zahlen, erfinde keine Dinge. Abs\u00e4tze wie folgt:\n{cluster_content}\nDies ist der Inhalt, den du zusammenfassen musst.", "max_token": 256, "threshold": 0.1, "max_cluster": 64, "random_seed": 0}, "chunk_token_num": 128, "delimiter": "\\n!?;\u3002\uff1b\uff01\uff1f", "layout_recognize": true, "html4excel": false}, "name": "file.pdf", "type": "pdf", "location": "file.pdf", "size": 499137, "tenant_id": "5ce7aba6c6f711ef93200242ac130006", "language": "English", "embd_id": "bge-m3:567m@Ollama", "pagerank": 50, "img2txt_id": "qwen-vl-max@Tongyi-Qianwen", "asr_id": "paraformer-realtime-8k-v1@Tongyi-Qianwen", "llm_id": "qwen-plus@Tongyi-Qianwen", "update_time": 1735646482661}
2024-12-31 13:02:27 Traceback (most recent call last):
2024-12-31 13:02:27   File "/ragflow/rag/svr/task_executor.py", line 513, in handle_task
2024-12-31 13:02:27     do_handle_task(task)
2024-12-31 13:02:27   File "/ragflow/rag/svr/task_executor.py", line 451, in do_handle_task
2024-12-31 13:02:27     chunks = build_chunks(task, progress_callback)
2024-12-31 13:02:27   File "/ragflow/rag/svr/task_executor.py", line 264, in build_chunks
2024-12-31 13:02:27     chat_mdl = LLMBundle(task["tenant_id"], LLMType.CHAT, llm_name=task["llm_id"], lang=task["language"])
2024-12-31 13:02:27   File "/ragflow/api/db/services/llm_service.py", line 228, in __init__
2024-12-31 13:02:27     self.mdl = TenantLLMService.model_instance(
2024-12-31 13:02:27   File "/ragflow/.venv/lib/python3.10/site-packages/peewee.py", line 3128, in inner
2024-12-31 13:02:27     return fn(*args, **kwargs)
2024-12-31 13:02:27   File "/ragflow/api/db/services/llm_service.py", line 126, in model_instance
2024-12-31 13:02:27     raise LookupError("Model({}) not authorized".format(mdlnm))
2024-12-31 13:02:27 LookupError: Model(qwen-plus) not authorized

So it still tries to use the Qwen Models (llm_id and img2txt_id). Why? Are they hardcoded somewhere? I cannot set a different LLM for my Knowledge Base.

Thanks in advance for your time!

[KevinHuSh]

[theoalm66]

Many thanks for your hint.

I changed the System model Settings twice, 1. openAI compatible and 2. Google models. Now I get different errors (see below). The embedding model API keys are working, tested multiple times.

For the first error, it seems like the API key is not picked up properly when creating the embeddings although I configured it in the model provider settings.

With OpenAI compatible API (IONOS) as model provider

embedding model: BAAI/bge-large-en-v1.5:

Docker container log:

2025-01-04 18:31:58 ragflow-server  | openai.AuthenticationError: Error code: 401 - {'httpStatus': 401, 'messages': [{'errorCode': 'paas-auth-1', 'message': 'Unauthorized, wrong or no api key provided to process this request'}]}

In knowledgebase:

Begin at:
Sat, 04 Jan 2025 18:30:36 GMT
Duration:
297.77 s
Progress:
18:30:36 Task has been received.
18:30:37 Page(1~13): OCR started
18:31:27 Page(1~13): OCR finished (50.15s)
18:31:44 Page(1~13): Layout analysis (16.99s)
18:31:46 Page(1~13): Table analysis (2.42s)
18:31:46 Page(1~13): Text merged (0.00s)
18:31:49 Page(1~13): Start to generate keywords for every chunk ...
18:31:58 Page(1~13): Keywords generation completed in 9.06s
18:31:58 Page(1~13): Generate 40 chunks
18:31:58 Page(1~13): [ERROR]Generate embedding error:Error code: 401 - {'httpStatus': 401, 'messages': [{'errorCode': 'paas-auth-1', 'message': 'Unauthorized, wrong or no api key provided to process this request'}]}
18:31:58 [ERROR]handle_task got exception, please check log
18:31:59 Task has been received.
18:31:59 Page(13~17): OCR started
18:35:22 Page(13~17): OCR finished (203.00s)
18:35:27 Page(13~17): Layout analysis (5.17s)
18:35:28 Page(13~17): Table analysis (0.09s)
18:35:28 Page(13~17): Text merged (0.00s)
18:35:30 Page(13~17): Start to generate keywords for every chunk ...
18:35:33 Page(13~17): Keywords generation completed in 2.85s
18:35:33 Page(13~17): Generate 26 chunks
18:35:33 Page(13~17): [ERROR]Generate embedding error:Error code: 401 - {'httpStatus': 401, 'messages': [{'errorCode': 'paas-auth-1', 'message': 'Unauthorized, wrong or no api key provided to process this request'}]}
18:35:33 [ERROR]handle_task got exception, please check log

With Gemini as model provider

Model: text-embedding-004

Docker log:

2025-01-04 18:41:27 ragflow-server  | 2025-01-04 18:41:27,072 INFO     20 set_progress(c706ee2ecac211ef8b250242ac130006), progress: None, progress_msg: 18:41:27 Page(1~13): Generate 40 chunks
2025-01-04 18:41:27 ragflow-server  | 2025-01-04 18:41:27,103 INFO     20 set_progress(c706ee2ecac211ef8b250242ac130006), progress: -1, progress_msg: 18:41:27 Page(1~13): [ERROR]Generate embedding error:list indices must be integers or slices, not tuple
2025-01-04 18:41:27 ragflow-server  | 2025-01-04 18:41:27,131 ERROR    20 Generate embedding error:list indices must be integers or slices, not tuple
2025-01-04 18:41:27 ragflow-server  | Traceback (most recent call last):
2025-01-04 18:41:27 ragflow-server  |   File "/ragflow/rag/svr/task_executor.py", line 463, in do_handle_task
2025-01-04 18:41:27 ragflow-server  |     token_count, vector_size = embedding(chunks, embedding_model, task_parser_config, progress_callback)
2025-01-04 18:41:27 ragflow-server  |   File "/ragflow/rag/svr/task_executor.py", line 321, in embedding
2025-01-04 18:41:27 ragflow-server  |     vts, c = mdl.encode(tts[i: i + batch_size])
2025-01-04 18:41:27 ragflow-server  |   File "<@beartype(api.db.services.llm_service.LLMBundle.encode) at 0x7fa4103db9a0>", line 31, in encode
2025-01-04 18:41:27 ragflow-server  |   File "/ragflow/api/db/services/llm_service.py", line 238, in encode
2025-01-04 18:41:27 ragflow-server  |     embeddings, used_tokens = self.mdl.encode(texts)
2025-01-04 18:41:27 ragflow-server  |   File "<@beartype(rag.llm.embedding_model.GeminiEmbed.encode) at 0x7fa4105d7a30>", line 31, in encode
2025-01-04 18:41:27 ragflow-server  |   File "/ragflow/rag/llm/embedding_model.py", line 508, in encode
2025-01-04 18:41:27 ragflow-server  |     content=texts[i, i + batch_size],
2025-01-04 18:41:27 ragflow-server  | TypeError: list indices must be integers or slices, not tuple
2025-01-04 18:41:27 ragflow-server  | 2025-01-04 18:41:27,145 INFO     20 set_progress(c706ee2ecac211ef8b250242ac130006), progress: -1, progress_msg: 18:41:27 [ERROR]handle_task got exception, please check log
2025-01-04 18:41:27 ragflow-server  | 2025-01-04 18:41:27,178 ERROR    20 handle_task got exception for task {"id": "c706ee2ecac211ef8b250242ac130006", "doc_id": "c4a5fbc0cac211ef90eb0242ac130006", "from_page": 0, "to_page": 12, "retry_count": 0, "kb_id": "a321d348cac211efa6c00242ac130006", "parser_id": "naive", "parser_config": {"auto_keywords": 16, "auto_questions": 0, "raptor": {"use_raptor": false}, "chunk_token_num": 128, "delimiter": "\\n!?;\u3002\uff1b\uff01\uff1f", "layout_recognize": true, "html4excel": false}, "name": "VF-24525_Preisliste_Vodafone_PDFA.pdf", "type": "pdf", "location": "VF-24525_Preisliste_Vodafone_PDFA.pdf", "size": 499137, "tenant_id": "9bf24aaac79111ef83690242ac130006", "language": "English", "embd_id": "text-embedding-004@Gemini", "pagerank": 50, "img2txt_id": "qwen-vl-max@Tongyi-Qianwen", "asr_id": "paraformer-realtime-8k-v1@Tongyi-Qianwen", "llm_id": "gemini-1.5-pro-latest@Gemini", "update_time": 1736012338360}
2025-01-04 18:41:27 ragflow-server  | Traceback (most recent call last):
2025-01-04 18:41:27 ragflow-server  |   File "/ragflow/rag/svr/task_executor.py", line 513, in handle_task
2025-01-04 18:41:27 ragflow-server  |     do_handle_task(task)
2025-01-04 18:41:27 ragflow-server  |   File "/ragflow/rag/svr/task_executor.py", line 463, in do_handle_task
2025-01-04 18:41:27 ragflow-server  |     token_count, vector_size = embedding(chunks, embedding_model, task_parser_config, progress_callback)
2025-01-04 18:41:27 ragflow-server  |   File "/ragflow/rag/svr/task_executor.py", line 321, in embedding
2025-01-04 18:41:27 ragflow-server  |     vts, c = mdl.encode(tts[i: i + batch_size])
2025-01-04 18:41:27 ragflow-server  |   File "<@beartype(api.db.services.llm_service.LLMBundle.encode) at 0x7fa4103db9a0>", line 31, in encode
2025-01-04 18:41:27 ragflow-server  |   File "/ragflow/api/db/services/llm_service.py", line 238, in encode
2025-01-04 18:41:27 ragflow-server  |     embeddings, used_tokens = self.mdl.encode(texts)
2025-01-04 18:41:27 ragflow-server  |   File "<@beartype(rag.llm.embedding_model.GeminiEmbed.encode) at 0x7fa4105d7a30>", line 31, in encode
2025-01-04 18:41:27 ragflow-server  |   File "/ragflow/rag/llm/embedding_model.py", line 508, in encode
2025-01-04 18:41:27 ragflow-server  |     content=texts[i, i + batch_size],
2025-01-04 18:41:27 ragflow-server  | TypeError: list indices must be integers or slices, not tuple

In knowledgebase:

Begin at:
Sat, 04 Jan 2025 18:38:58 GMT
Duration:
181.94 s
Progress:
18:38:58 Task has been received.
18:38:58 Page(1~13): OCR started
18:39:47 Page(1~13): OCR finished (48.91s)
18:40:03 Page(1~13): Layout analysis (15.64s)
18:40:05 Page(1~13): Table analysis (2.20s)
18:40:05 Page(1~13): Text merged (0.00s)
18:40:07 Page(1~13): Start to generate keywords for every chunk ...
18:41:27 Page(1~13): Keywords generation completed in 79.65s
18:41:27 Page(1~13): Generate 40 chunks
18:41:27 Page(1~13): [ERROR]Generate embedding error:list indices must be integers or slices, not tuple
18:41:27 [ERROR]handle_task got exception, please check log
18:41:27 Task has been received.
18:41:27 Page(13~17): OCR started