swiss-wowbagger

Let yourself be insulted in swiss german.

Inspired by kthxbye's figgdi. Thanks to Franz Hohler's Totemügerli, Stiller Has' grusig.

Huere Siech, säg mou öppis!

Developer's Guide

Prerequisites:

1. JDK 17
2. Optional: MBROLA (see [Install MBROLA] for instructions)
3. Optional: GraalVM CE 21.3+ based on Java 17 (If you need to re-record native-image config files)

The apps and the docker containers are hosted at Google Cloud. Schaltstelle Members can add themselves as Owner of the project to gain access.

• Cloud Run Overview (instances setup, logs, etc.)
• Secret Manager (credentials for Twitter/Telegram)
• Cloud Scheduler (Trigger for daily random Tweet)

Everything is built and deployed automatically on push to master by Github Actions. See Workflow config file for details.

Overview

The project can be built and tested completely using mvn clean verify. The applications get containerized and integration tests are run against these containers. Wiremock is used to mock Twitter and Telegram APIs using recorded responsed.

For production the applications are compiled to native applications using GraalVM. This greatly reduces container startup time which is beneficial with Google Cloud Run as there are no instances running all the time.

Switching between native and non-native builds is made by enabling the corresponding profile in Maven: non-native or native-shade. To build locally using native builds run mvn -P native-shade clean verify. There's no need to have GraalVM available locally as the build happens within a Docker container.

If you make significant code changes you might need to re-record native-image config files. See [Record native-image config files] for instructions. If you are in panic you can remove -Pnative-shadefrom the Github Actions Workflow config file and non-native applications are built and deployed.

How to install MBROLA

The speech synthesis part of Swiss Wowbagger needs MBROLA to be available on your machine. If MBROLA is not installed locally, it tries to use the MBROLA docker image.

To install MBROLA on your machine:

• Debian/Ubuntu Linux: you can install everything using apt. See MBROLA Dockerfile for instructions.
• Fedora Linux: MBROLA is not available in common repositories so just copy it from the base docker file using setup-mbrola-on-fedora.sh
• Windows or MacOS: copy the binaries from mbrola-jvm and make them available in PATH

Auzure Speech synthesis

• https://de.wikipedia.org/wiki/Liste_der_IPA-Zeichen
• https://en.wikipedia.org/wiki/Pronunciation_Lexicon_Specification
• http://web.mit.edu/6.mitx/www/24.900%20IPA/IPAapp.html
• https://speech.microsoft.com/portal/af9f421b07ad40c4ac493125e0e1dfdc/audiocontentcreation

Commonly used phonemes: æ ɘ ɪ œ øː ʏ

χ ʃ ɡ ŋ

How to register Telegram Bot Webhook

See https://core.telegram.org/bots for instructions how to setup a developer account and get the needed credentials.

1. Set the 'telegram-*' secrets in Secret Manager
2. Register webhook: curl -F "url=https://swiss-wowbagger-telegram-ultgi7by3q-oa.a.run.app/${WOWBAGGER_BOT_TOKEN}/webhook" https://api.telegram.org/bot${WOWBAGGER_BOT_TOKEN}/setWebhook

For local testing:

1. create a public tunnel to localhost using ngrok http 8080
2. Switch the Telegram Bot registration to your public ngrok HTTPS URL using the curl command above

Twitter Bot development

You need to have a Developer account with free Premium Account Activity API subscription.

Caveats:

• After creating the app in the Developer Portal make sure that "App permission" are set to "Read and write and Direct message" before creating the access tokens. You have to recreate them after changing this setting.

Wiremock

To re-record Wiremock mappings files:

1. Download Wiremock Standalone
2. `java -jar wiremock-jre8-standalone-2.31.0.jar --port 9999
3. Open http://localhost:9999/__admin/recorder and set "Target URL" to the Twitter or Telegram API. More info
4. Start the application using environment variables to redirect API calls to Wiremock. e. g. TWITTER_BASE_URL=http://localhost:9999/1.1/

Record native-image config files

native-image builds uses static code analysis and Tree Shaking to only include methods, properties, resources, etc. which are actually used. This cannot work automatically where Reflection is used. To detect these cases GraalVM provides a JRE agent which records every call at runtime and generates config files. See Assisted Configuration with Tracing Agent for more information.

For this to work make sure that the integration tests (classes ending in *IT) are realistic and cover 100% of use cases where reflection is involved. This is the case in the Twitter and Telegram API libraries.

Instructions for re-recording:

1. Set JAVA_HOME: export JAVA_HOME=/<path-to>/graalvm-ce-java17-21.3.0
2. Cd into the app's directory.
3. mvn -Pnative -Dagent=true -DskipTests=true -DskipNativeBuild=true package exec:exec@java-agent
4. Run the integration tests.
5. Stop the mvn process.
6. Copy output from target/native/agent-output to src/main/resources/META-INF/native-image/guru.nidi/swiss-wowbagger-xxx

Setup Github Actions

To allow our Github Actions workflow to push docker images and make deployments we added a service account in GCP with just the right permissions. An authentication key has to be added as Github Secret for this to work.

1. Go to https://console.cloud.google.com/iam-admin/serviceaccounts?project=swiss-wowbagger
2. For the github-actions service account follow these instructions to download the key as JSON: https://cloud.google.com/iam/docs/creating-managing-service-account-keys#creating
3. In Github Repository settings add a Secret named GCP_SA_KEY and past the JSON file's contents as value

Setup Google Cloud SDK access

Only needed if you want to deploy from localhost.

1. Install Google Cloud SDK https://cloud.google.com/sdk/docs/install
2. Setup Google Cloud SDK https://cloud.google.com/sdk/docs/initializing (Project ID: swiss-wowbagger)
3. Configure authentication for the docker repo: gcloud auth configure-docker europe-west6-docker.pkg.dev