Merge pull request #155 from acon96/release/v0.3

Release v0.3

README.md

@@ -1,19 +1,23 @@
 # Home LLM
-![Banner Logo](/docs/banner.svg)  
-
-This project provides the required "glue" components to control your Home Assistant installation with a completely local Large Language Model acting as a personal assistant. The goal is to provide a drop in solution to be used as a "conversation agent" component by Home Assistant.  The 2 main pieces of this solution are Home LLM and Llama Conversation.
+This project provides the required "glue" components to control your Home Assistant installation with a **completely local** Large Language Model acting as a personal assistant. The goal is to provide a drop in solution to be used as a "conversation agent" component by Home Assistant.  The 2 main pieces of this solution are the Home LLM model and Local LLM Conversation integration.
 
 ## Quick Start
 Please see the [Setup Guide](./docs/Setup.md) for more information on installation.
 
-## LLama Conversation Integration
+## Local LLM Conversation Integration
 In order to integrate with Home Assistant, we provide a custom component that exposes the locally running LLM as a "conversation agent".
 
 This component can be interacted with in a few ways:  
 - using a chat interface so you can chat with it.
 - integrating with Speech-to-Text and Text-to-Speech addons so you can just speak to it.
 
-The component can either run the model directly as part of the Home Assistant software using llama-cpp-python, or you can run [Ollama](https://ollama.com/) (simple) or the [oobabooga/text-generation-webui](https://github.com/oobabooga/text-generation-webui) project (advanced) to provide access to the LLM via an API interface.
+The integration can either run the model in 2 different ways:
+1. Directly as part of the Home Assistant software using llama-cpp-python
+2. On a separate machine using one of the following backends:
+    - [Ollama](https://ollama.com/) (easier)
+    - [LocalAI](https://localai.io/) via the Generic OpenAI backend (easier)
+    - [oobabooga/text-generation-webui](https://github.com/oobabooga/text-generation-webui) project (advanced)
+    - [llama.cpp example server](https://github.com/ggerganov/llama.cpp/blob/master/examples/server/README.md) (advanced)
 
 ## Home LLM Model
 The "Home" models are a fine tuning of various Large Languages Models that are under 5B parameters.  The models are able to control devices in the user's house as well as perform basic question and answering.  The fine tuning dataset is a [custom synthetic dataset](./data) designed to teach the model function calling based on the device information in the context.
@@ -27,10 +31,12 @@ The latest models can be found on HuggingFace:
 <summary>Old Models</summary>  
 
 3B v2 (Based on Phi-2): https://huggingface.co/acon96/Home-3B-v2-GGUF  (ChatML prompt format)  
-3B v1 (Based on Phi-2): https://huggingface.co/acon96/Home-3B-v1-GGUF  (ChatML prompt format)  
 1B v2 (Based on Phi-1.5): https://huggingface.co/acon96/Home-1B-v2-GGUF  (ChatML prompt format)  
 1B v1 (Based on Phi-1.5): https://huggingface.co/acon96/Home-1B-v1-GGUF  (ChatML prompt format)  
 
+NOTE: The models below are only compatible with version 0.2.17 and older!
+3B v1 (Based on Phi-2): https://huggingface.co/acon96/Home-3B-v1-GGUF  (ChatML prompt format)  
+
 </details>
 
 The model is quantized using Llama.cpp in order to enable running the model in super low resource environments that are common with Home Assistant installations such as Raspberry Pis.
@@ -126,6 +132,7 @@ In order to facilitate running the project entirely on the system where Home Ass
 ## Version History
 | Version | Description                                                                                                                                                                                                          |
 |---------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| v0.3    | Adds support for Home Assistant LLM APIs, improved model prompting and tool formatting options, and automatic detection of GGUF quantization levels on HuggingFace                                                   |
 | v0.2.17 | Disable native llama.cpp wheel optimizations, add Command R prompt format                                                                                                                                            |
 | v0.2.16 | Fix for missing huggingface_hub package preventing startup                                                                                                                                                           |
 | v0.2.15 | Fix startup error when using llama.cpp backend and add flash attention to llama.cpp backend                                                                                                                          |

TODO.md

@@ -1,10 +1,16 @@
 # TODO
-- [x] detection/mitigation of too many entities being exposed & blowing out the context length
+- [ ] support new LLM APIs  
+    - rewrite how services are called  
+    - handle no API selected  
+    - rewrite prompts + service block formats  
+    - implement new LLM API that has `HassCallService` so old models can still work  
+    - update dataset so new models will work with the API  
+- [ ] make ICL examples into conversation turns  
+- [ ] translate ICL examples + make better ones  
 - [ ] areas/room support  
-- [ ] figure out DPO to improve response quality
-- [ ] train the model to respond to house events  
-    - present the model with an event + a "prompt" from the user of what you want it to do (i.e. turn on the lights when I get home = the model turns on lights when your entity presence triggers as being home)  
-    - basically lets you write automations in plain english  
+- [ ] convert requests to aiohttp  
+- [x] detection/mitigation of too many entities being exposed & blowing out the context length  
+- [ ] figure out DPO to improve response quality  
 - [x] setup github actions to build wheels that  are optimized for RPIs
 - [x] mixtral + prompting (no fine tuning)  
     - add in context learning variables to sys prompt template
@@ -45,3 +51,6 @@
     - set up vectordb  
     - ingest home assistant docs  
     - "context request" from above to initiate a RAG search  
+- [ ] train the model to respond to house events  
+    - present the model with an event + a "prompt" from the user of what you want it to do (i.e. turn on the lights when I get home = the model turns on lights when your entity presence triggers as being home)  
+    - basically lets you write automations in plain english  

custom_components/llama_conversation/__init__.py

@@ -1,16 +1,21 @@
-"""The Local LLaMA Conversation integration."""
+"""The Local LLM Conversation integration."""
 from __future__ import annotations
 
 import logging
 
 import homeassistant.components.conversation as ha_conversation
 from homeassistant.config_entries import ConfigEntry
+from homeassistant.const import ATTR_ENTITY_ID
 from homeassistant.core import HomeAssistant
-from homeassistant.helpers import config_validation as cv
+from homeassistant.exceptions import HomeAssistantError
+from homeassistant.helpers import config_validation as cv, llm
+from homeassistant.util.json import JsonObjectType
+
+import voluptuous as vol
 
 from .agent import (
-    LLaMAAgent,
-    LocalLLaMAAgent,
+    LocalLLMAgent,
+    LlamaCppAgent,
     GenericOpenAIAPIAgent,
     TextGenerationWebuiAgent,
     LlamaCppPythonAPIAgent,
@@ -26,7 +31,10 @@
     BACKEND_TYPE_GENERIC_OPENAI,
     BACKEND_TYPE_LLAMA_CPP_PYTHON_SERVER,
     BACKEND_TYPE_OLLAMA,
+    ALLOWED_LEGACY_SERVICE_CALL_ARGUMENTS,
     DOMAIN,
+    HOME_LLM_API_ID,
+    SERVICE_TOOL_NAME,
 )
 
 _LOGGER = logging.getLogger(__name__)
@@ -38,19 +46,19 @@ async def update_listener(hass: HomeAssistant, entry: ConfigEntry):
     hass.data[DOMAIN][entry.entry_id] = entry
 
     # call update handler
-    agent: LLaMAAgent = await ha_conversation._get_agent_manager(hass).async_get_agent(entry.entry_id)
-    agent._update_options()
+    agent: LocalLLMAgent = ha_conversation.get_agent_manager(hass).async_get_agent(entry.entry_id)
+    await hass.async_add_executor_job(agent._update_options)
 
     return True
 
 async def async_setup_entry(hass: HomeAssistant, entry: ConfigEntry) -> bool:
-    """Set up Local LLaMA Conversation from a config entry."""
+    """Set up Local LLM Conversation from a config entry."""
 
     def create_agent(backend_type):
         agent_cls = None
 
         if backend_type in [ BACKEND_TYPE_LLAMA_HF, BACKEND_TYPE_LLAMA_EXISTING ]:
-            agent_cls = LocalLLaMAAgent
+            agent_cls = LlamaCppAgent
         elif backend_type == BACKEND_TYPE_GENERIC_OPENAI:
             agent_cls = GenericOpenAIAPIAgent
         elif backend_type == BACKEND_TYPE_TEXT_GEN_WEBUI:
@@ -78,7 +86,7 @@ def create_agent(backend_type):
 
 
 async def async_unload_entry(hass: HomeAssistant, entry: ConfigEntry) -> bool:
-    """Unload Local LLaMA."""
+    """Unload Local LLM."""
     hass.data[DOMAIN].pop(entry.entry_id)
     ha_conversation.async_unset_agent(hass, entry)
     return True
@@ -87,18 +95,79 @@ async def async_migrate_entry(hass, config_entry: ConfigEntry):
     """Migrate old entry."""
     _LOGGER.debug("Migrating from version %s", config_entry.version)
 
-    if config_entry.version > 1:
-      # This means the user has downgraded from a future version
-      return False
-
-    # if config_entry.version < 2:
-    #     # just ensure that the defaults are set
-    #     new_options = dict(DEFAULT_OPTIONS)
-    #     new_options.update(config_entry.options)
-
-    #     config_entry.version = 2
-    #     hass.config_entries.async_update_entry(config_entry, options=new_options)
+    # 1 -> 2: This was a breaking change so force users to re-create entries
+    if config_entry.version == 1:
+        _LOGGER.error("Cannot upgrade models that were created prior to v0.3. Please delete and re-create them.")
+        return False
 
     _LOGGER.debug("Migration to version %s successful", config_entry.version)
 
     return True
+
+class HassServiceTool(llm.Tool):
+    """Tool to get the current time."""
+
+    name = SERVICE_TOOL_NAME
+    description = "Executes a Home Assistant service"
+
+    # Optional. A voluptuous schema of the input parameters.
+    parameters = vol.Schema({
+        vol.Required('service'): str,
+        vol.Required('target_device'): str,
+        vol.Optional('rgb_color'): str,
+        vol.Optional('brightness'): float,
+        vol.Optional('temperature'): float,
+        vol.Optional('humidity'): float,
+        vol.Optional('fan_mode'): str,
+        vol.Optional('hvac_mode'): str,
+        vol.Optional('preset_mode'): str,
+        vol.Optional('duration'): str,
+        vol.Optional('item'): str,
+    })
+
+    async def async_call(
+        self, hass: HomeAssistant, tool_input: llm.ToolInput, llm_context: llm.LLMContext
+    ) -> JsonObjectType:
+        """Call the tool."""
+        domain, service = tuple(tool_input.tool_args["service"].split("."))
+        target_device = tool_input.tool_args["target_device"]
+
+        service_data = {ATTR_ENTITY_ID: target_device}
+        for attr in ALLOWED_LEGACY_SERVICE_CALL_ARGUMENTS:
+            if attr in tool_input.tool_args.keys():
+                service_data[attr] = tool_input.tool_args[attr]
+        try:
+            await hass.services.async_call(
+                domain,
+                service,
+                service_data=service_data,
+                blocking=True,
+            )
+        except Exception:
+            _LOGGER.exception("Failed to execute service for model")
+            return { "result": "failed" }
+
+        return { "result": "success" }
+
+class HomeLLMAPI(llm.API):
+    """
+    An API that allows calling Home Assistant services to maintain compatibility 
+    with the older (v3 and older) Home LLM models
+    """
+
+    def __init__(self, hass: HomeAssistant) -> None:
+        """Init the class."""
+        super().__init__(
+            hass=hass,
+            id=HOME_LLM_API_ID,
+            name="Home-LLM (v1-v3)",
+        )
+
+    async def async_get_api_instance(self, llm_context: llm.LLMContext) -> llm.APIInstance:
+        """Return the instance of the API."""
+        return llm.APIInstance(
+            api=self,
+            api_prompt="Call services in Home Assistant by passing the service name and the device to control.",
+            llm_context=llm_context,
+            tools=[HassServiceTool()],
+        )