Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add background blur feature for supported devices #2812

Open
wants to merge 30 commits into
base: livekit
Choose a base branch
from

Conversation

toger5
Copy link
Contributor

@toger5 toger5 commented Nov 20, 2024

This adds background blur via @livekit/track-processors for supported browsers.
The option can be found in the video tab of the settings.

The fallback will be a greyed out checkbox (on firefox for instance)
This way firefox users get informed why it does not work on their side if other users tell them where to set it up.

Sadly this is not exactly done the way we would want to set it up see: livekit/track-processors-js#50

  • bundle local assets not CDN
  • stringify related hang on startup

@toger5
Copy link
Contributor Author

toger5 commented Nov 20, 2024

how do we deal with lk modules that dont fullfill the ts lints?

@toger5
Copy link
Contributor Author

toger5 commented Nov 27, 2024

how do we deal with lk modules that dont fullfill the ts lints?

I found what we needed:
@types/dom-mediacapture-transform

Copy link
Member

@hughns hughns left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've suggested some cleanup. Furthermore I think you could deduplicate some of the code into something like a useBlur() or canBlur(). This might make adding tests easier too.

locales/en-GB/app.json Outdated Show resolved Hide resolved
src/livekit/useLiveKit.ts Outdated Show resolved Hide resolved
src/livekit/useLiveKit.ts Outdated Show resolved Hide resolved
src/livekit/useLiveKit.ts Outdated Show resolved Hide resolved
src/livekit/useLiveKit.ts Outdated Show resolved Hide resolved
src/settings/SettingsModal.tsx Outdated Show resolved Hide resolved
src/settings/SettingsModal.tsx Outdated Show resolved Hide resolved
src/settings/SettingsModal.tsx Outdated Show resolved Hide resolved
src/room/LobbyView.tsx Outdated Show resolved Hide resolved
locales/en-GB/app.json Outdated Show resolved Hide resolved
@toger5
Copy link
Contributor Author

toger5 commented Dec 2, 2024

Furthermore I think you could deduplicate some of the code into something like a useBlur() or canBlur(). This might make adding tests easier too.

I think @robintown has an idea how we might be able to reuse the track from the lobby to the in call view. Than this might come for free.

But in this pr we should at least get rid of the duplicated let blur = useMemo sections

locales/en-GB/app.json Outdated Show resolved Hide resolved
@hughns hughns changed the title Add background Blur Add background blur feature for support devices Dec 4, 2024
@hughns hughns mentioned this pull request Dec 4, 2024
@hughns hughns force-pushed the toger5/track-processor-blur branch from dcb8a21 to 1860646 Compare December 9, 2024 22:09
Comment on lines +25 to +26
wasmLoaderPath: new URL(
"../../node_modules/@mediapipe/tasks-vision/wasm/vision_wasm_internal.js",
Copy link
Member

@sandhose sandhose Dec 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would be better to use

import wasmLoaderPath from "@mediapipe/tasks-vision/wasm/vision_wasm_internal.js?url";

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did try that, but I ended up with a vite error and so did this approach instead. 🤷

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants