Skip to content

Latest commit

 

History

History
61 lines (51 loc) · 4.39 KB

README.md

File metadata and controls

61 lines (51 loc) · 4.39 KB
Raw

OpenVINO™ AI Plugins for Audacity* 🤘

License: GPL v3

A set of AI-enabled effects, generators, and analyzers for Audacity®. These AI features run 100% locally on your PC 💻 -- no internet connection necessary! OpenVINO™ is used to run AI models on supported accelerators found on the user's system such as CPU, GPU, and NPU.

  • Music Separation🎵 -- Separate a mono or stereo track into individual stems -- Drums, Bass, Vocals, & Other Instruments.
  • Music Style Remix💿 -- Uses Stable Diffusion to alter a mono or stereo track using a text prompt.
  • Noise Suppression🧹 -- Removes background noise from an audio sample.
  • Music Generation🎶 -- Uses Stable Diffusion to generate snippets of music from a text prompt.
  • Whisper Transcription🎤 -- Uses whisper.cpp to generate a label track containing the transcription or translation for a given selection of spoken audio or vocals.

Installation 💾

Go here to find installation packages & instructions for the latest Windows release.

Build Instructions 🔨

Help, Feedback, & Bug Reports 🙋‍♂️

We welcome you to submit an issue here for

  • Questions
  • Bug Reports
  • Feature Requests
  • Feedback of any kind -- how can we improve this project?

Contribution 🤝

Your contributions are welcome and valued, no matter how big or small. Feel free to submit a pull-request!

Acknowledgements & Citations 🙏

  • Audacity® development team & Muse Group-- Thank you for your support!
  • Audacity® GitHub -- https://github.com/audacity/audacity
  • Whisper transcription & translation analyzer uses whisper.cpp (with OpenVINO™ backend): https://github.com/ggerganov/whisper.cpp
  • Music Generation & Music Style Remix use Riffusion's UNet model, Riffusion pipelines that were ported to C++ from this project: https://github.com/riffusion/riffusion
  • Music Separation effect uses Meta's Demucs v4 model (https://github.com/facebookresearch/demucs), which has been ported to work with OpenVINO™
  • Noise Suppression:
    • noise-suppression-denseunet-ll: from OpenVINO™'s Open Model Zoo: https://github.com/openvinotoolkit/open_model_zoo
    • DeepFilterNet2 & DeepFilterNet3:
      • Ported the models & pipeline from here: https://github.com/Rikorose/DeepFilterNet
      • We also made use of @grazder's fork / branch (https://github.com/grazder/DeepFilterNet/tree/torchDF-changes) to better understand the Rust implementation, and so we also based some of our C++ implementation on torch_df_offline.py found here.
      • Citations: 
        @inproceedings{schroeter2022deepfilternet2,
title = {{DeepFilterNet2}: Towards Real-Time Speech Enhancement on Embedded Devices for Full-Band Audio},
author = {Schröter, Hendrik and Escalante-B., Alberto N. and Rosenkranz, Tobias and Maier, Andreas},
booktitle={17th International Workshop on Acoustic Signal Enhancement (IWAENC 2022)},
year = {2022},
}
  
@inproceedings{schroeter2023deepfilternet3,
title = {{DeepFilterNet}: Perceptually Motivated Real-Time Speech Enhancement},
author = {Schröter, Hendrik and Rosenkranz, Tobias and Escalante-B., Alberto N. and Maier, Andreas},
booktitle={INTERSPEECH},
year = {2023},
}

Disclaimer ⚠️

Stable Diffusion & Riffusion's data model is governed by the Creative ML Open Rail M license, which is not an open source license. https://github.com/CompVis/stable-diffusion. Users are responsible for their own assessment whether their proposed use of the project code and model would be governed by and permissible under this license.