Streaming text responses

[sixlive]

Description

Enabling streaming of generated text, which will be particularly useful for real-time applications and improved user experiences.

[petervandijck]

I would like to second that this is a must-have for even starting to use the package 👍

[tognee]

Hi, is there a timeframe for this feature?
I could help to develop it if there is a clear definition on how it should be implemented.

The first thing that needs to be discussed is if the streaming option should be an extension of the Prism::text static function or a new Prism::streamingText function.

After that the implementation seems simple, there should be a generate function that calls the API endpoint with the streaming parameter and returns a generator. This generator should collect the data from the request and parses the SSE events coming from the provider, map each chunk to a new ChunkResponse object and yield the object one at a time.

This ChunkResponse should be following the OpenAI Chunk type or should it be something custom?

If we can get to a specification together I could start working on this feature

[tomtev]

Really need this also :)

[petervandijck]

Same here, waiting for it to start implementing prism

…

On Sat, Nov 23, 2024 at 9:30 AM Tommy Vedvik ***@***.***> wrote: Really need this also :) — Reply to this email directly, view it on GitHub <#27 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAD6PVG2R2A3VPQOPTWZ5C32CCGSFAVCNFSM6AAAAABQBQRNEGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIOJVGQ4TONZUGY> . You are receiving this because you commented.Message ID: ***@***.***>

-- -- My new company: we do AI: https://www.getsimplyput.ai/ https://www.linkedin.com/in/petervandijck/ http://petervandijck.com/ Cell: +57 (300) 836-7690