Serving WhisperX with BentoML

WhisperX provides fast automatic speech recognition with word-level timestamps and speaker diarization.

This is a BentoML example project, demonstrating how to build a speech recognition inference API server, using the WhisperX project. See here for a full list of BentoML example projects.

Prerequisites

If you want to test the project locally, install FFmpeg on your system.

Install dependencies

git clone https://github.com/bentoml/BentoWhisperX.git
cd BentoWhisperX

# Recommend Python 3.11
pip install -r requirements.txt

Run the BentoML Service

We have defined a BentoML Service in service.py. Run bentoml serve in your project directory to start the Service. Please note that you may need to request access to pyannote/segmentation-3.0 and pyannote/speaker-diarization-3.1, then provide your Hugging Face token when running the Service.

$ HF_TOKEN=<your hf access token> bentoml serve .

2024-01-18T09:01:15+0800 [INFO] [cli] Starting production HTTP BentoServer from "service:BentoWhisperX" listening on http://localhost:3000 (Press CTRL+C to quit)

The server is now active at http://localhost:3000. You can interact with it using the Swagger UI or in other different ways.

CURL

curl -s \
     -X POST \
     -F '[email protected]' \
     http://localhost:3000/transcribe

Python client

from pathlib import Path
import bentoml

with bentoml.SyncHTTPClient('http://localhost:3000') as client:
    audio_url = 'https://example.org/female.wav'
    response = client.transcribe(audio_file=audio_url)
    print(response)

For detailed explanations of the Service code, see WhisperX: Speech recognition.

Deploy to BentoCloud

After the Service is ready, you can deploy the application to BentoCloud for better management and scalability. Sign up if you haven't got a BentoCloud account.

Make sure you have logged in to BentoCloud and set your Hugging Face access token in bentofile.yaml, then run the following command to deploy it.

bentoml deploy .

Once the application is up and running on BentoCloud, you can access it via the exposed URL.

Note: For custom deployment in your own infrastructure, use BentoML to generate an OCI-compliant image.