How to create a virtual env to run the Python code

• git clone via SSH or HTTPS
• make sure virtualenv is installed pip/pip3 install virtualenv
• create a new venv virtualenv venv
• Activate the venv source venv/bin/activate
• Install the requirements.txt pip install -r requirements.txt
• Usually pytorch doesn't work, install that manually with pip/pip3 install torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/cu116

If you however use conda, like the IAT server does sigh, then run the following commands.

• conda create -n venv python=3.10
• conda activate venv
• pip3 install -r requirements.txt
• Usually pytorch doesn't work, install that manually with pip/pip3 install torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/cu116

SAC implementation for dynamic systems

This repository is the implementation of a Soft-Actor-Critic Agent that is used to control a dynamic system via deep learning.

Settings/Configs

There are several experiments that can be run. Within the original_config.yaml settings file you will find a line under base_settings that states the experiment. Enter a value between 1-6 here to choose the experiment that shall be run when training new models. Additonally the max velocity of the wind can be set under the wind settings within this file.\ The following settings can be used:

• 1: No wind, just drive left to right and stay in the middle
• 2: Changing starting y value, no wind
• 3: Constant wind velocity, wind direction bottom to top
• 4: Changing wind velocity, wind direction bottom to top
• 5: Constant wind velocity, swap between b to t or t to b
• 6: Changing wind velocity, all wind directions random

Running the Python code

The Python code can be run in several ways. You can start the training, training with rendering or just render a previously trained experiment.\ Use the following commandos:

• python3 main.py -t, this trains a new model with default config and saves all it's data in the dedicated experiment directory, parse a config file to train a specific model
• python3 main.py -tr, this trains and renders a new experiment/model with default config
• python3 main.py -r [path], this renders a previously trained experiment. Note, the experiment path has to look like experiments/foo/experiment_12-19... if you are already in the directory where main.py is located.
• python3 main.py -p [n_models], this trains n models with random hp defined in the hp tuner. Set it to -pr to also render the experiments. Or just checkout the overview.csv in the corresponding experiment directory do see what the best model is and render just that as described in point 3.

Usage: main.py [-h] [-t [TRAIN]] [-r [RENDER]] [-p [PARAMSTUNE]]:

options:\

• h, --help show this help message and exit
• t [TRAIN], --train [TRAIN] train new model with default or parsed config file
• r [RENDER], --render [RENDER] render trained model or parsed previous experiment dir
• p [PARAMSTUNE], --paramstune [PARAMSTUNE] use hp tuner for model_generation