File system layout

This page contains information about how build and projects data are laid out in the file system.

There are (2) directories that are key to mistry: the "data" and the "projects" directory.

Data directory

This directory contains:

• the build artifacts
• build logs
• build result metadata (exit code, status etc.)

Given a project named "foo" and the data_path set to /var/lib/mistry/data, an example data directory look like this:

/var/lib/mistry/data/
`-- foo # project name
    |-- pending # currently running jobs
        |-- 48f96a905fa3de9dc150d28973233bd6d83cb3349812946b4013e2d82fce9504
        |   |-- data
        |   |   |-- artifacts
        |   |   |-- cache
        |   |   `-- params
        |   |-- out.log
    `-- ready # completed jobs
        |-- 589e9563457ed1cec9b1d33549fe1565eab51b0c839bb0c2d53ad476afa6503c # job ID
        |   |-- data
        |   |   |-- artifacts # results of the build
        |   |   |-- cache     # data persisted across builds for incremental building
        |   |   `-- params    # job parameters (as files)
        |   |-- out.log     # the log of the build
        |   `-- result.json # metadata regarding the result of the build (exit code etc.)
        |-- 645d23ee67b17f091eeba36536898d3519dd89676077bfdb1f90b9be075ef665
        |   |-- data
        |   |   |-- artifacts
        |   |   |-- cache
        |   |   `-- params
        |   |-- out.log
        |   `-- result.json
`-- projectB
  # ...

The data directories in the host are mounted at /data inside job execution containers. For example, the container of the job 48f96a9... would have /var/lib/mistry/data/foo/pending/48f96a905fa3de9dc150d28973233bd6d83cb3349812946b4013e2d82fce9504/data mounted at /data.

This path is where all the magic happens and it contains (3) sub-directories:

1. /data/artifacts/: contains build results (aka. artifacts). These are the contents that are served when someone uses mistry-cli
2. /data/cache/: contains contents that are used for caching purposes, written from previous builds. Files that should be retained between builds and are not part of the artifacts should be placed here.
3. /data/params/: contains the job parameters

Projects directory

This is where the job execution environments exist, in the form of Docker images. For an example, take a look at the project directory used in the tests.

It is set via the projects_path setting.