Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

LLamaSharp integration #44

Merged
merged 4 commits into from
Nov 3, 2023
Merged

LLamaSharp integration #44

merged 4 commits into from
Nov 3, 2023

Conversation

TesAnti
Copy link
Collaborator

@TesAnti TesAnti commented Nov 3, 2023

implemented IChatModel for LLamaSharp. See LangChain.Providers.LLamaSharp.IntegrationTests for usage examples.

  • tests are excluded from CI.
  • model is expected to be downloaded manually. working models can be found on LLamaSharp's HugginFace.
  • for tests i used model ggml-model-f32-q4_0.bin because it's the smallest.
  • i'm using Cpu as a backend in tests so anyone could test it without graphics card and cuda.

to use model in own project - the corresponding backed should be added to the project. This can be done using nuget. For example, LLamaSharp.Backend.Cpu.

closes #32

@HavenDV HavenDV enabled auto-merge (squash) November 3, 2023 16:21
@HavenDV HavenDV merged commit c776a41 into tryAGI:main Nov 3, 2023
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Integrate LLamaSharp project
2 participants