llms for auto-complete?

[lurebat]

We all know that llms are good at completion, that's basically all they do.

Right now some keyboards integrate llms but usually for heavy operations - rewriting, summary, etc.

I think we can do better, and use them for constant additional completion, like co-pilot.

Basically send a request every time there is a delay in typing.

Send the sentence context and ask for completions, and display them as they're available.
It doesn't replace normal typing, but enhances it.

Since we are sending very little data plus we don't need a very fancy llms, it can be quite cheap.

Or if you're worried about privacy spin out your own Ollama server.
Obviously it should be optional and not on by default, but I do wonder how powerful it can be