Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(backend): proxy s3 attachments by default #1030

Merged
merged 3 commits into from
Aug 14, 2024
Merged

chore(backend): proxy s3 attachments by default #1030

merged 3 commits into from
Aug 14, 2024

Conversation

detj
Copy link
Contributor

@detj detj commented Aug 13, 2024

Summary

In order to reduce onboarding friction for self host users, we'll default to using minio for uploading and fetching attachment blobs. This way, users will not require to acquire a DNS domain/subdomain for handling of attachments. Thereby, making the self hosting experience a tad bit smoother.

Tasks

  • Modify all call sites where presigned S3 URLs are generated for attachments
  • Create a new /attachments?payload=[partial-presigned-url] API for reverse proxying to S3 backend.
  • Update self-host/config.sh script to reflect updated config state
  • Reorganized all backend route definitions to make sure 404 Not Found cases does not get eaten up by other routes.
  • Fix a couple of typos

Flow

Here's the flow showing both proxy & no proxy mechanism.

No Proxy

sequenceDiagram
    Frontend->>+Backend: want session replay
    Backend-->>-Frontend: here you go
    Frontend->>+Minio: fetch me this attachment
    Minio-->>-Frontend: here you go
Loading

Proxy

sequenceDiagram
    Frontend->>+Backend: want session replay
    Backend-->>-Frontend: here you go
    Frontend->>+Backend: fetch me this attachment
    Backend->>+Minio: fetch me this attachment
    Minio-->>-Backend: here you go
    Backend-->>-Frontend: here you go
Loading

@detj detj self-assigned this Aug 13, 2024
Copy link

vercel bot commented Aug 13, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Skipped Deployment
Name Status Preview Comments Updated (UTC)
measure-web-app ⬜️ Ignored (Inspect) Visit Preview Aug 13, 2024 6:15pm

detj added 2 commits August 13, 2024 20:01
- by default all attachments are proxied via s3 presigned urls
- previous no proxy behavior can also be configured
- reorganize routes so that not found route can function normally
- add new api endpoint to reverse proxy presigned s3 attachments
- update attachment origin settins in `self-host/config.sh`
@detj detj requested a review from anupcowkur August 13, 2024 18:15
@detj detj merged commit 421ecd4 into main Aug 14, 2024
8 checks passed
@detj detj deleted the obj-store-proxy branch August 14, 2024 06:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

2 participants