langchain dependency problem

[peter-axion]

spacy-llm's langchain interface is incompatible with langchain>=0.0.302 because in 302 they stopped exporting modules, so the .llms property that is used in this code no longer exists. This renders spacy-llm unloadable because the registry model initialization calls the Langchain model and the Langchain model's __init__() calls this function.

Stack trace with 0.0.326 looks like this:

Traceback (most recent call last):
  File "/opt/dagster/dagster_home/.venv/lib/python3.10/site-packages/my_spacy_llm_extension/tasks.py", line 6, in <module>
    from spacy_llm.registry import registry
  File "/opt/dagster/dagster_home/.venv/lib/python3.10/site-packages/spacy_llm/__init__.py", line 1, in <module>
    from . import cache  # noqa: F401
  File "/opt/dagster/dagster_home/.venv/lib/python3.10/site-packages/spacy_llm/cache.py", line 11, in <module>
    from .ty import LLMTask, PromptTemplateProvider
  File "/opt/dagster/dagster_home/.venv/lib/python3.10/site-packages/spacy_llm/ty.py", line 14, in <module>
    from .models import langchain
  File "/opt/dagster/dagster_home/.venv/lib/python3.10/site-packages/spacy_llm/models/__init__.py", line 2, in <module>
    from .langchain import query_langchain
  File "/opt/dagster/dagster_home/.venv/lib/python3.10/site-packages/spacy_llm/models/langchain/__init__.py", line 4, in <module>
    LangChain.register_models()
  File "/opt/dagster/dagster_home/.venv/lib/python3.10/site-packages/spacy_llm/models/langchain/model.py", line 106, in register_models
    for class_id, cls in LangChain.get_type_to_cls_dict().items():
  File "/opt/dagster/dagster_home/.venv/lib/python3.10/site-packages/spacy_llm/models/langchain/model.py", line 38, in get_type_to_cls_dict
    return langchain.llms.type_to_cls_dict
  File "/opt/dagster/dagster_home/.venv/lib/python3.10/site-packages/langchain/__init__.py", line 368, in __getattr__
    raise AttributeError(f"Could not find: {name}")
AttributeError: Could not find: llms

I verified that 0.0.331 gives the same error, but somehow that's listed in the dev requirements.txt.

[rmitsch]

Hi @peter-axion, thanks for reporting this. The upcoming v0.6.3 will bump the langchain pin to 0.0.335. In our tests this seems to work without further adjustments though, which is strange. To be sure we fetch type_to_cls_dict with getattr() now, hopefully this should fix things for you too. PR here.

Note: this is the reason we pin the langchain version - it's a very unstable library that quite frequently changes its API (hence also the versioning with 0.0.x). We don't recommend using spacy-llm with a langchain version that's not pinned in our requirements, as we don't guarantee it'll work.

[peter-axion]

Thanks! I'm surprised it didn't replicate but perhaps it has something to do with the order modules are loaded. We're only tangentially using langchain, so I think our solution is going to be to remove it as a dependency.