qa-se-webapp

Software Engineering & Agile Assignment - Web Application Development. Application written in Python. Any information stored in database is dummy information.

Introduction

As part of the Software Engineering & Agile module of my degree I was required to design, develop and test a web database application written in Python. The Python Web Framework can be any (Django, FastAPI, Flask), I have used Flask. The work for this module is within commits up to the end of September 2022.

The application has been reused for the Software Engineering & DevOps module of my degree. For this module I have increased security of the app in regards to the OWASP Top Ten 2021. The work for this module is within commits from May 2023 onwards.

Requirements of the Degree (Software Engineering and Agile)

The web app has restrictions / requirements as per the brief, these are:

• It must feature a relational database (e.g. SQL) backend. The web app currently uses a different SQLlite backend for the production (live) version of the app and the development (test) version. SQLAlchemy handles the connectivity and the database settings can be set via environment variables or within the config.py file.
• The database can have a minimum of 2 tables and a maximum of 4 tables. The database has 2 tables. One for Users and one for Assets. Both tables have a Primary Key set on their ID fields. A relationship exists between Assets.AssignedBy and Users.ID to record who has assigned / updated the Asset record.
• The web application should have 2 types of users: regular and admin. The web application has regular users and admin users. The application looks for an environment variable or a variable within the config.py file for the admin user email address. Upon signing up with the appropriate email address the admin user gains admin privileages.
• Regular users should be able to carry out CREATE, READ and UPDATE actions on the database via the web app. Regular users can create assets, read asset details and update asset details.
• Admin users should be able to carry out CREATE, READ, UPDATE and DELETE actions on the database via the web app. The admin user can do everything a regular user can, and also has an option to delete an asset when on the "Edit Asset" screen.
• Admin users should be able to make changes to the underlaying database. The underlaying database can be accessed from the device running the application using the following command from within the "flaskwebapp" directory: flask shell Database commands such as User.query.all() to list all users can be run. To exit the Flask Shell using the command: exit()

Modularisation

The web app has been split into modules to assist with maintaining it and to (hopefully) make it easier to understand. The web app resides within the "flaskwebapp" directory. The README.md and requirements.txt, alongside non-essentials (.github, .vscode, .gitignore, venv), are outside this directory to keep the flaskwebapp directory free from clutter and files that are generated on first run.

Within the flaskwebapp directory is assets.py which is the application and a config.py file containing the configuration that the app requires. Configuration can also be set via environment variables if preferred.

The app directory contains templates for "auth" and "main". Each of these contain views.py which tell Flask what should be shown where (e.g. which URL endpoints are available, who they are available to, where they are available and what functions they carry out). forms.py tells Flask what should be on a form and what should be expected from the form, also carrying out entry validation. tables.py tells Flask what details should be on a table. errors.py is to enable custome error pages.

The static/css directory contains the CSS style sheet which can be used to overide CSS settings in the flask-bootstrap library.

The templates directory contains a base.html written in HTML with Jinja2 formatting that acts as a template for the other pages. The templates tell Flask how to display (or not display) information on web pages. The subdirectories are are named auth for authorisation pages (e.g. account registration, account login), errors for error pages and views for regular pages.

models.py contains the classes that are used to create and allow interaction with the database. Each table (e.g., Users, Assets) is it's own class.

The tests directory contains the Unit Test files (each starting with the name test_) that are used to test the web application.

Naming Conventions

Functions, variables and routes within the web app have been named appropriatly. For example the /add and /delete routes of /asset/ do as they are named (add and delete). The AddAsset, SearchAsset, EditAsset classes from within forms.py also do as labelled (add, search and edit).

Usability (Notifications)

The flash module of Flask is used to display notifications to the end user upon certain actions such as creating an account, logging out, adding an asset and editing an asset. A warning message is displayed to admin users upon trying to delete an asset asking them if they are sure they want to delete the asset.

ENVIRONMENT VARIABLES

The app can detect environment variables to change features (e.g. turn off CRSF during in test and turn it on in production).

FLASK_APP - expects the the location of where assets.py is running from FLASK_CONFIG - requires a value of 'production', 'test' or 'azurewebapp' each alters settings defined in config.py ADMIN_USER - the email address of the default admin user SECRET_KEY - a long and secure secret key for the app APP_DEBUG - TRUE or FALSE, not currently implemented DATABASE_URL - the URL of the database to use for Production TEST_DATABASE_URL - the URL of the database to use for Test

Running The Application Locally (DEV)

Running In DEV Mode - NOTE: Commands are case-sensitive:

• Clone the repository
• Open the repository (change to the repository directory)
• Create a Virtual Environment using the command "python3 -m venv ./venv"
• Activate the Virtual Environment using the command "source ./venv/bin/activate"
• Install the required Python modules using the command "pip install -r requirements.txt"
• Set the FLASK_APP environment variable using the command "export FLASK_APP=assets.py"
• Set the
• Change to the flaskwebapp directory
• Initialise database using command "flask db init"
• Migrate any database changges using command "flask db migrate"
• Upgrade database using command "flask db upgrade"
• Run the Flask app using the command "flask run" OR
• Run the unit tests using the command "flask test" OR
• Run the unit tests via Coverage, using the command "coverage run -m flask test"

UPDATES - Summer 2023

For the Software Engineering and DevOps module of my assignment I have made several updates to the original web app.

Running The Application Locally (Container)

The app can now be run locally via a Docker Container: "docker pull geektechstuff/assets-app:latest". By default the app runs on port 8000 within the container.

-- Deployment Pipeline The app has three GitHub Actions (pipelines) attached to it. 'Flask App Tests' runs when any branch (except MAIN) is pushed. This action runs all the unit tests automatically and checks the Python code against a linter. 'Deploy via CI/CD' runs only when the MAIN branch is pushed. This action runs all the unit tests automatically and checks the Python code against a linter, and if there are no failures it automatically deploys the app to https://assets.geektechstuff.com. 'Publisher Docker Image' runs only when a RELEASE is created. This action is similar to the 'Deploy via CI/CD' action however it publishes the app as a container on Docker Hub (https://hub.docker.com/r/geektechstuff/assets-app) instead of deploying it live as a website.

-- Badges The README.md displays badges for the GitHub Actions to indicate if the actions are passing or failing.

-- Test Coverage The Python library Coverage is now included in the requirements.txt and can be run against tests to check the coverage of testing. As of 6th June 2023, overall coverage is at 92%. Coverage can be run using the command 'coverage run -m flask test', and results can be check using the command 'coverage report -m'. Check out https://geektechstuff.com/2023/05/08/code-coverage-python-devops/ for more information.

-- Increased Testing More tests have been added to flaskwebapp/tests/ to increase what is tested. This includes testing for 'OWASP A01:2021 Broken Access Control' to test views (test_views.py) and make sure logged out users do not see details where login is required.

Testing for 'OWASP A02:2021 Cryptographic Failures' has test_db_users.py test password setting and hashing/salting. test_user.py now checks that duplicate username or duplicate email address cannot be used.

Testing for 'Improper Error Handling' has test_error_pages.py testing that the correct error pages are used when an error occurs and that default error messages are not displayed.

-- Logging Logging has been added to the app, with settings (e.g. log file name, log level) viewable in flaskwebapp/app/init.py. By default the events are logged to LogFiles/assets.log. Logging has been added to help with 'OWASP A09:2021 Security Logging and Monitoring Failures'.

The logging includes:

• User attempted login (auth/views.py)
• User successful login (auth/views.py)
• User failed login (auth/views.py)
• User logout (auth/views.py)
• Attempting to register (auth/views.py)
• Registering succesfully (auth/views.py)
• Successfully changing password (auth/views.py)
• Failing to change password (auth/views.py)
• adding an asset (main/views.py)
• editing an asset (main/views.py)
• deleting an asset (main/views.py)

-- Response Headers To help limit 'A03:2021 Injection', previous vulnerabilities from A7:2017 Cross-Site Scripting (XSS) have had response headers set in assets.py. For example these headers set a Content Security Policy (CSP) to only allow content from the listed resources.