Astra Assistants Built-in Tools

[phact]

We had an ask from a user on discord who wanted full control over chunking and embeddings but still wanted to leverage the rest of astra-assistants. I decided to build this functionality on the client leveraging Assistant's tool (formerly function calling) capabilities.

We're thinking of these as astra-assistants built-in tools. They will be a collection of curated tools can be used with astra-assistants, with openai assistants directly, or even with the completions API with some additional boiler plate. There are similar concepts in langchain and cohere's cohere-toolkit seems to be working on something similar but I was surprised to find that this concept doesn't exist for Assistants API (or maybe I just haven't been able to find it?).

Our implementation is modular and re-usable based on an abstract base class: tool_interface and a custom AssistantEventHandler for streaming called AstraEventHandler. We hope implementing this class will be relatively straight forward and encourage community contributions.

We just shipped a couple of built-in tools as part of the astra-assistants client (astradb data-api and support for code interpreter using https://e2b.dev/docs/sandbox/api/metadata).

Built-in tools can be used with assistants by:

1. creating an instance of the tool:

from astra_assistants.tools.astra_data_api import AstraDataAPITool

data_api_tool = AstraDataAPITool(
    db_url=db_url,
    collection_name=collection_name,
    namespace=namespace,
    vectorize=False,
    openai_client=client,
    embedding_model="text-embedding-ada-002",
)

2. Providing the tool to your assistant:

assistant = client.beta.assistants.create(
    name="Smart bot",
    instructions="You are a bot. Use the provided functions to answer questions about movies.",
    model="gpt-3.5-turbo",
    tools=[data_api_tool.to_function()],
)

3. Instantiating an AstraEventHandler and registering the tool with it

from astra_assistants.astra_assistants_event_handler import AstraEventHandler

event_handler = AstraEventHandler(client)
event_handler.register_tool(data_api_tool)

4. and finally passing the handler to your run (and optionally specifying the tool_choice to force usage):

with client.beta.threads.runs.create_and_stream(
        thread_id=thread.id,
        assistant_id=assistant.id,
        event_handler=event_handler,
        tool_choice=data_api_tool.tool_choice_object(),
) as stream:
    for text in stream.text_deltas:
        print(text, end="", flush=True)

Here's an end to end example of using the astra data-api tool.

Finally, Built-In tools might not be the right name, open to ideas.

[chymian]

since the discord link is expired…
can you pls. give an working example/how-to how to use assistants & DB locally?
i.e. which local (vector)DB can one use instead of the cloud astradb if everything has to be local?
a bit more beef to it than a piece of code would help enormous –for all non devs.
a docker-compose would be perfect.

TIA

[phact]

Hi, thanks for the comment! I fixed the discord link. TL;DR, astradb is managed cassandra on the cloud. You should be able to run cassandra locally but it would require a couple of code changes in astra-assistants where the db connection gets established since the code is currently built to talk to astra.

[chymian]

thx for the fast answer, @phact ,

a bit more beef to it than a piece of code would help enormous –for all non devs.

You should be able to run cassandra locally but it would require a couple of code changes in astra-assistants where the db connection gets established since the code is currently built to talk to astra.

ok, and how can that look like - for non-devs?

context:
I want to setup vrsen.ai agents as service but it has to run locally, due to the laws in europe and privacy concerns.
a solution for a sysadmin - but no dev - would be really helpful here.
can you please give some help?

[phact]

Right now pointing astra-assistants at a local cassandra requires code changes (so it requires a developer). I can update this discussion when this changes.

[chymian]

TIA, looking forward to it.
any ETA?

[phact]

https://discord.com/invite/MEFVXUvsuy for convenience