Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: fixed propagation of exceptions from upstream endpoints #13

Merged
merged 38 commits into from
Dec 2, 2024

Conversation

adubovik
Copy link
Collaborator

@adubovik adubovik commented Nov 21, 2024

  • fix: typo in README
  • fix: debug logs for response/request weren't logged (it happened because _debug constant was evaluated before the log level was set)
  • fix: merging of chat completion chunks in non-streaming wasn't working properly, e.g. object fields were concatenated which resulted in "object": "chat.completionchat.completionchat.completion". Added a test witnessing the bugfix.
  • fix[examples]: en_core_web_sm model is (a) downloaded as part of the package build and (b) pre-loaded on the server start-up instead of spending time of the first request into PII interceptor
  • fix[examples]: pinned versions of en_core_web_sm model (3.7.1) and spacy (3.7.5) to avoid model downloading issue on the server start up
  • deps: migrated FileStorage from aiohttp to httpx, thus aiohttp is removed from the package dependencies
  • deps: aiostream package dependency moved to the examples extras
  • deps: migrated from aidial-sdk 0.13.0 to 0.16.0
  • tests: built infrastructure for testing chains of interceptors
  • tests: introduced X-UPSTREAMS header and refactored DIAL Client to ease interceptor testing
  • tests: added tests that validate response headers propagation from the upstream in streaming and non-streaming scenarios
  • feat: DIALException are materialised instead of being treated as normal chat completion chunks (that's what handle_streaming_errors method used to do). This led to ChatCompletionInterceptor.call_upstreams changing type to accommodate the new kind of stream values which hold DIAL Exception.
  • chore: migrated from the custom _UnstructuredChunk to ArbitraryChunk from DIAL SDK

pyproject.toml Outdated Show resolved Hide resolved
@adubovik adubovik requested review from igorlagutin and roman-romanov-o and removed request for igorlagutin November 26, 2024 15:43
@adubovik adubovik self-assigned this Nov 26, 2024
@adubovik adubovik changed the title fix: various fixes fix: fixed propagation of exceptions from upstream endpoints Nov 26, 2024
aidial_interceptors_sdk/chat_completion/adapter.py Outdated Show resolved Hide resolved
aidial_interceptors_sdk/chat_completion/adapter.py Outdated Show resolved Hide resolved
aidial_interceptors_sdk/utils/_debug.py Outdated Show resolved Hide resolved
tests/utils/interceptors.py Show resolved Hide resolved
tests/utils/interceptors.py Show resolved Hide resolved
tests/examples/test_trace.py Show resolved Hide resolved
@adubovik adubovik merged commit 48ad4d9 into development Dec 2, 2024
8 checks passed
@adubovik adubovik deleted the fix/various-fixes branch December 2, 2024 11:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants