This repository hosts the backend services for the Auto Twitch Compiler web-app. It manages the entire video processing pipeline, from curating Twitch clips based on user preferences to the final assembly of the video ready for YouTube upload.
The backend is built using AWS Serverless Application Model (SAM) and consists of multiple Lambda functions, each responsible for a specific part of the video processing workflow:
- Curate: Fetches Twitch clips based on provided game IDs, quantity, and languages.
- Download: Downloads the selected Twitch clips.
- PreProcess: Processes each clip, preparing them for assembly.
- Concatenate: Merges all processed clips into a single video file ready for upload.
These Lambda functions are containerized and deployed as Docker images, allowing for more consistent and portable executions.
This serverless architecture ensures scalability, resilience, and efficient use of resources.
Previously, the backend was hosted elsewhere, but in the spirit of evolution and leveraging best-of-breed cloud services, we've transitioned to AWS. This move provides improved scalability, flexibility, and a range of AWS services to further optimize the workflow.
For details about the frontend transition to Next.js and the shift from Firebase to MongoDB Atlas, please refer to the frontend repository.
The lambdas are developed using Node.js, specifically:
- Node version: v19.4.0
The backend is orchestrated using AWS SAM. To deploy:
-
Ensure you have AWS SAM CLI installed.
-
Navigate to the project root and run:
sam build -
Followed by:
sam deploy --guided -
Follow the prompts to deploy the Lambdas and associated resources.
Sensitive data, like API keys, are stored securely using AWS Systems Manager Parameter Store. Ensure that the necessary parameters are set up in the AWS Management Console.
Pull requests are welcome. For significant changes, please open an issue first to discuss the proposed changes.