v1.18.1

• Pin pydantic to 1.x #5244

Full Changelog: v1.18.0...v1.18.1

v1.18.0

⭐️ Highlights

🗄️ Using LLMs Hosted on AWS SageMaker

We are thrilled to share that Haystack now supports the use of open source LLMs deployed to AWS SageMaker! This means you can easily host your models with AWS SageMaker and use them with PromptNode by simply providing the inference endpoint name and your aws_profile_name 👇

from haystack.nodes import PromptNode

prompt_node = PromptNode(
    model_name_or_path="sagemaker-model-endpoint-name", 
    model_kwargs={"aws_profile_name": "my_aws_profile_name", 
	          "aws_region_name": "your-region-k"}
)

For more details on model deployment, check out the documentation.

🗂️ PromptHub

Exciting news! We’re introducing PromptHub: A place for ready-made prompts for the most common NLP tasks. The best part about it is that you can easily import prompts from the hub into Haystack. For example, if you want to use deepset/topic-classification prompt with Haystack, all you need is the prompt name, and that's it! 👇

import os

from haystack.nodes import PromptNode, PromptTemplate

topic_classifier_template = PromptTemplate("deepset/topic-classification")
prompt_node = PromptNode(model_name_or_path="text-davinci-003", api_key=os.environ.get("OPENAI_API_KEY"))
prompt_node.prompt(prompt_template=topic_classifier_template, documents="YOUR_DOCUMENTS", options=["A LIST OF TOPICS"])

Check out the PromptHub and discover other prompt options for various NLP tasks.

🛠️ Adding Tools to ConversationalAgent

The wait is over – you can now greatly enhance your chat application by adding various tools using the tools parameter of ConversationalAgent.

from haystack.agents import Tool
from haystack.agents.conversational import ConversationalAgent

search_tool = Tool(
    name="USA_Presidents_QA",
    pipeline_or_node=presidents_qa,
    description="useful for when you need to answer questions related to the presidents of the USA.",
)

conversational_agent = ConversationalAgent(prompt_node=prompt_node, tools=[search_tool])
conversational_agent.run("YOUR QUERY")

Go to our docs to learn more about various tools you can use.

⚠️ Breaking Changes

🔦 A new optional dependency: inference

To simplify the installation of Haystack, we removed some redundant dependencies (like PyTorch) for users who rely on LLMs through APIs. This means you will get a smaller package and faster Haystack installation if you want to use Haystack with OpenAI models or Transformers models via Hugging Face Inference API. If you prefer to run models locally, for example on your GPU, pip install farm-haystack[inference] will install all required dependencies as before.

🔁 PromptTemplate parameter changes

As of v1.18, PromptTemplate accepts only prompt and output_parser and doesn’t support name and prompt_text parameters anymore. See an example of how you can migrate to the new PromptTemplate 👇🏼

qa_template = PromptTemplate(
    name="question-answering",
    promp_text="""
    Given the context please answer the question. Context: {join(documents)}; 
    Question: {query}; 
    Answer:
    """,
    output_parser=AnswerParser()
)

qa_template = PromptTemplate(
    prompt="""
    Given the context please answer the question. Context: {join(documents)}; 
    Question: {query}; 
    Answer:
    """,
    output_parser=AnswerParser()
)

🚫 Seq2SeqGenerator and RAGenerator deprecation

With Haystack 1.18, we removed the Seq2SeqGenerator and RAGenerator from Haystack. We recommend using the more powerful PromptNode instead. See the announcement for more details.

What's Changed

Breaking Changes

• PromptHub integration in PromptNode by @ZanSara in #4879
• chore: remove deprecated node PDFToTextOCRConverter by @masci in #4982
• refactor: Use globally defined request timeout in ElasticsearchDocumentStore and OpenSearchDocumentStore by @bogdankostic in #5064
• feat!: simplify weaviate auth by @hsm207 in #5115
• feat!: Add extra for inference dependencies such as torch by @julian-risch in #5147
• Remove deprecated param return_table_cell by @masci in #5218

Pipeline

• build: Remove tiktoken alternative by @julian-risch in #4991
• fix: fitz import switcher by @ZanSara in #5012
• refactor: Generate eval result in separate method by @bogdankostic in #5001
• chore: Unpin typing_extensions and remove all its uses by @silvanocerza in #5040
• docs: Fix doc for FARMReader.predict by @pcreux in #5049
• feat: Allow setting custom api_base for OpenAI nodes by @michaelfeil in #5033
• fix: Ensure eval mode for farm and transformer models for predictions by @sjrl in #3791
• chore: remove audio node import by @ZanSara in #5097
• feat: introduce lazy_import by @ZanSara in #5084
• fix: WebRetriever top_k is ignored in a pipeline by @vblagoje in #5106
• build: Move Azure's Form Recognizer dependency to extras by @julian-risch in #5096
• chore: mark some unit tests under test/pipeline by @ZanSara in #5124
• feat: optional transformers by @ZanSara in #5101
• feat: current_datetime shaper function by @ZanSara in #5195
• feat: hard document length limit at max_chars_check by @ZanSara in #5191
• chore: remove safe_import and all usages by @ZanSara in #5139
• fix: Send batches of query-doc pairs to inference_from_objects by @bogdankostic in #5125
• fix: Use add_isolated_node_eval of eval_batch in run_batch by @bogdankostic in #5223

DocumentStores

• docs: updating docstrings to say OpenSearch and backlink to correct docs by @dtaivpp in #5000
• feat: Add batching for querying in ElasticsearchDocumentStore and OpenSearchDocumentStore by @bogdankostic in #5063
• feat: Add batch_size parameter and cast timeout_config value to tuple for WeaviateDocumentStore by @bogdankostic in #5079
• fix: changing document scores by @benheckmann in #5090

Documentation

• fix: Fix CohereInvocationLayer _ensure_token_limit not returning resized prompt by @silvanocerza in #4978
• feat: Add prompt_template to conversational agent init params by @julian-risch in #4994
• feat: Allow setting java options when launching Elasticsearch / OpenSearch by @bogdankostic in #5002
• refactor: Adapt retriever benchmarks script by @bogdankostic in #5004
• refactor: Add reader-retriever benchmark script by @bogdankostic in #5006
• refactor: Adapt running benchmarks by @bogdankostic in #5007
• fix: Move check for default PromptTemplates in PromptTemplate itself by @ZanSara in #5018
• chore: Simplify DefaultPromptHandler logic and add tests by @silvanocerza in #4979
• feat: prompts caching from PromptHub by @ZanSara in #5048
• fix: Fix handling of streaming response in AnthropicClaudeInvocationLayer by @silvanocerza in #4993
• feat: pass model parameters to HFLocalInvocationLayer via model_kwargs, enabling direct model usage by @vblagoje in #4956
• feat: Consider prompt_node's default_prompt_template in agent by @julian-risch in #5095
• fix: rename requests.py into requests_utils.py by @ZanSara in #5099
• docs: update CLI readme by @dfokina in #5129
• fix: small improvement to pipeline v2 tests by @ZanSara in #5153
• feat: Optional Content Moderation for OpenAI PromptNode & OpenAIAnswerGenerator by @benheckmann in #5017
• feat: Update ConversationalAgent by @bilgeyucel in #5065
• fix: Check Agent's prompt template variables and prompt resolver parameters are aligned by @vblagoje in #5163
• feat: add ensure token limit for direct prompting of ChatGPT by @sjrl in https://github...

v1.18.0-rc3

v1.18.0-rc3

v1.18.0-rc2

v1.18.0-rc2

v1.18.0-rc1

v1.18.0-rc1

v1.17.2

This release fixes a bug in telemetry collection originate by the generalimport library. Since we already switched to a different library in the unstable branch, we decided to backport to 1.17.x the introduction of lazy_import, thus fixing the bug inherently.

What's Changed

• feat: introduce lazy_import by @ZanSara in #5084

Full Changelog: v1.17.1...v1.17.2-rc1

v1.17.2-rc1

This release fixes a bug in telemetry collection originate by the generalimport library. Since we already switched to a different library in the unstable branch, we decided to backport to 1.17.x the introduction of lazy_import, thus fixing the bug inherently.

What's Changed

• feat: introduce lazy_import by @ZanSara in #5084

Full Changelog: v1.17.1...v1.17.2-rc1

v1.17.1

• Pin generalimport library to fix #5075

v1.17.0

⭐ Highlights

🗣️ Introducing ConversationalAgent

Great news! We’re introducing the ConversationalAgent – a type of Agent specifically implemented to create chat applications! With its memory integration, the new ConversationalAgent enables human-like conversation with large language models (LLMs). If you’re worried about the token limit of your model, there is an option to condense the chat history with ConversationSummaryMemory before injecting the history into the prompt.

To get started, just initialize ConversationalAgent with a PromptNode and start chatting.

summary_memory = ConversationalSummaryMemory(prompt_node=prompt_node)

conversational_agent = ConversationalAgent(
	prompt_node=prompt_node, 
	memory=summary_memory
)
conversational_agent.run(user_input)

To try it out, check out the new ConversationalAgent Tutorial, see the full example, or visit our documentation!

🎉 Now using transformers 4.29.1

With this release, Haystack depends on the latest version of the transformers library.

🧠 More LLMs

Haystack now supports command from Cohere and claude from Anthropic!

🤖 New error reporting strategy around 3rd-party dependencies

One of the challenges with a multi-purpose NLP framework like Haystack is to find the sweet spot of a turn-key solution implementing multiple NLP use cases without getting into dependency hell. With the new features around generative AI recently shipped, we got several requests about avoiding pulling in too many unrelated dependencies when, say, one just needs a PromptNode.
We heard your feedback and lowered the number of packages a simple pip install farm-haystack pulls in (and we'll keep doing it)! To keep the user experience as smooth as possible, by using the generalimports library, we defer dependency errors from "import time" down to "actual usage time" – so that you don't have to ask yourself "Why do I need this database client to run PromptNode?" anymore.

⚠️ MilvusDocumentStore Deprecated in Haystack

With Haystack 1.17, we have moved the MilvusDocumentStore out of the core haystack project, and we will maintain it in the haystack-extras repo. To continue using Milvus, check out the instructions on how to install the package separately in its readme.

What's Changed

⚠️ Breaking Changes

• refactor: Update schema objects to handle Dataframes in to_{dict,json} and from_{dict,json} by @sjrl in #4747
• chore: remove deprecated MilvusDocumentStore by @masci in #4951
• chore: remove BaseKnowledgeGraph by @masci in #4953
• chore: remove deprecated node PDFToTextOCRConverter by @masci in #4982

Pipeline

• chore: upgrade transformers to 4.28.1 by @vblagoje in #4665
• chore: fixed reader loading test for hf-hub starting 0.14.0 by @mayankjobanputra in #4607
• bug: (rest_api) remove full logging of overwritten env variables by @duffn in #4791
• fix: preserve root_node in JoinNode's output by @ZanSara in #4820
• feat: Send pipeline config hash every 100 runs by @bogdankostic in #4884
• feat: add BLIP support in TransformersImageToText by @anakin87 in #4912
• fix: EvaluationResult serialization changes dataframes by @tstadel in #4906
• fix: shaper exception when retriever return 0 docs. by @yuanwu2017 in #4929
• fix: Use AutoTokenizer instead of DPR specific tokenizer by @bogdankostic in #4898
• fix: Fix necessary extra for MarkdownConverter by @bogdankostic in #4947

DocumentStores

• fix: Add support for _split_overlap meta to Pinecone and dict metadata in general to Weaviate by @bogdankostic in #4805
• fix: str issues in squad_to_dpr by @PhilipMay in #4826
• feat: introduce generalimport by @ZanSara in #4662
• feat: Support authentication using AuthBearerToken and AuthClientCredentials in Weaviate by @hsm207 in #4028

Documentation

• fix: loads local HF Models in PromptNode pipeline by @saitejamalyala in #4670
• fix: README latest and main installation by @dfokina in #4741
• fix: SentenceTransformersRanker's predict_batch returns wrong number of documents by @vblagoje in #4756
• feat: add Google API to search engine providers by @Pouyanpi in #4722
• bug: fix filtering in MemoryDocumentStore (v2) by @ZanSara in #4768
• refactor: Extract ToolsManager, add it to Agent by composition by @vblagoje in #4794
• chore: move custom linter to a separate package by @masci in #4790
• refactor!: Deprecate name param in PromptTemplate and introduce template_name instead by @bogdankostic in #4810
• chore: revert Deprecate name param in PromptTemplate and introduce prompt_nameinstead by @bogdankostic in #4834
• chore: remove optional imports in v2 by @ZanSara in #4855
• test: Update unit tests for schema by @sjrl in #4835
• feat: allow filtering documents on all fields (v2) by @ZanSara in #4773
• feat: Add Anthropic invocation layer by @silvanocerza in #4818
• fix: improve Document comparison (v2) by @ZanSara in #4860
• feat: Add Cohere PromptNode invocation layer by @vblagoje in #4827
• fix: Support for gpt-4-32k by @dnetguru in #4825
• fix: Document v2 JSON serialization by @ZanSara in #4863
• fix: Dynamic max_answers for SquadProcessor (fixes IndexError when max_answers is less than the number of answers in the dataset) by @benheckmann in #4817
• feat: Add agent memory by @vblagoje in #4829
• fix: Make sure summary memory is cumulative by @vblagoje in #4932
• feat: Add conversational agent by @vblagoje in #4931
• docs: Small fix to PromptTemplate API docs by @sjrl in #4870
• build: Remove mmh3 dependency by @julian-risch in #4896
• docstrings update in web.py by @dfokina in #4921
• feat: Add max_tokens to BaseGenerator params by @vblagoje in #4168
• fix: change parameter name to request_with_retry by @ZanSara in #4950
• fix: Adjust tool pattern to support multi-line inputs by @vblagoje in #4801
• feat: enable passing generation_kwargs to the PromptNode in pipeline.run() by @faaany in #4832
• fix: Remove streaming LLM tracking; they are all streaming now by @vblagoje in #4944
• feat: HFInferenceEndpointInvocationLayer streaming support by @vblagoje in #4819
• fix: Fix request_with_retry kwargs by @silvanocerza in #4980

Other Changes

• fix: Allow to set num_beams in HFInvocationLayer by @sywangyi in #4731
• ci: Execute pipelines and utils unit tests in CI by @bogdankostic in #4749
• refactor: Make agent test more robust by @vblagoje in #4767
• ci: Add coverage tracking with Coveralls by @silvanocerza in #4772
• test: move several modeling tests in e2e/ by @ZanSara in #4308
• chore: Added deprecation tests for seq2seq generator and RAG Generator by @mayankjobanputra in #4782
• feat: Add HF local runtime token streaming support by @vblagoje in #4652
• fix: load the local finetuning model from pipeline yaml (#4729) by @yuanwu2017 in #4760
• test: Add others folder to unit test job by @silvanocerza in https://github.com/deepset-ai/haysta...

v1.17.0-rc2

v1.17.0-rc2