Skip to content

Latest commit

 

History

History
327 lines (204 loc) · 14.4 KB

DEVELOPMENT.md

File metadata and controls

327 lines (204 loc) · 14.4 KB
Raw

Development Guide for Athens

The proxy is written in idiomatic Go and uses standard tools. If you know Go, you'll be able to read the code and run the server.

Athens uses Go Modules for dependency management. You will need Go v1.11 or later to get started on Athens.

See our Contributing Guide for tips on how to submit a pull request when you are ready.

All the instructions in this document assume that you have checked out the code to your local machine.

If you haven't done that, please do with the below command before you proceed:

$ git clone https://github.com/gomods/athens.git

Go version

Athens is developed on Go 1.11+.

To point Athens to a different version of Go set the following environment variable:

GO_BINARY_PATH=go1.11.X
# or whichever binary you want to use with athens

Run the Proxy

We provide four ways to run the proxy on your local machine:

  1. Using Docker and Docker Compose (we suggest this one if you're getting started)
  2. Natively on your host as a service (only SystemD is currently supported)
  3. Natively on your host as a simple binary
  4. Using Sail

See below for instructions for each option!

Run Using Docker

As we said above, we suggest that you use this approach because it simulates a more realistic Athens deployment. This technique does the following, completely inside containers:

  1. Builds Athens from scratch
  2. Starts up MongoDB and Jaeger
  3. Configures Athens to use MongoDB for its storage and Jaeger for its distributed tracing
  4. Runs Athens

You'll need Docker and Docker Compose installed. Once you do, run the below command to set everything up:

$ make run-docker

You should see some output that look like this:

docker-compose -p athensdockerdev up -d dev
Creating network "athensdockerdev_default" with the default driver
Creating athensdockerdev_jaeger_1 ... done
Creating athensdockerdev_minio_1  ... done
Creating athensdockerdev_mongo_1  ... done
Creating athensdockerdev_dev_1    ... done

After that runs, Athens should be up and running and serving on port 3000. To test it out, run this command:

$ curl localhost:3000

... and you should see the standard Athens response:

"Welcome to The Athens Proxy"

When you're ready to stop Athens and all its dependencies, run this command:

$ make run-docker-teardown

Run Natively on Your Host as a Service

There are many service execution environments. On Linux, two important ones are SystemD and SysV, but at the moment only SystemD is supported; others may follow soon. For other systems, please see the next section.

SystemD on Linux

If you're inside GOPATH, make sure GO111MODULE=on, if you're outside GOPATH, then Go Modules are on by default.

The Makefile builds the necessary athens binary. Then a script sets up the service for you.

$ make athens
$ sudo ./scripts/systemd.sh install

After the server starts, you can manage it as usual via systemctl, e.g.:

sudo systemctl status athens

which is the same as

$ sudo ./scripts/systemd.sh status

The systemd.sh script also has a remove option to uninstall the service.

SystemD allows logs to be collected and inspected; more information is in this tutorial by Digital Ocean, amongst others. So tailing the logs can be done like this:

$ sudo journalctl -u athens --since today --follow

Run Natively on Your Host

If you're inside GOPATH, make sure GO111MODULE=on, if you're outside GOPATH, then Go Modules are on by default.

The main package is inside cmd/proxy and is run like any go project as follows:

$ cd cmd/proxy
$ go build
$ ./proxy

After the server starts, you'll see some console output like:

Starting application at 127.0.0.1:3000

Run Using Sail

Follow instructions at sail.dev to setup the sail CLI. Then simply run:

sail run gomods/athens

The command will automatically clone the athens repo and give you a local URL that you can use to open an editor and development environment directly in your browser.

Dependencies

Services that Athens Needs

Depending on its configuration, Athens may rely on several external services (i.e. databases, etc...) to function properly. We use Docker images to configure and run those services. However, Athens does not require any of these external services by default. For example, the default storage driver is memory, but you can opt-in to using the fs driver. Neither would require any external service dependencies.

But if you'd like to test out Athens against a different storage backend like MongoDB, Minio, or a cloud blob storage system, this section is for you.

If you're not familiar with Docker, that's ok. We've tried to make it easy to get up and running with the below steps.

  1. Download and install docker-compose (docker-compose is a tool for easily starting and stopping lots of services at once)
  2. Run make dev from the root of this repository

That's it! After the make dev command is done, everything will be up and running and you can move on to the next step.

If you want to stop everything at any time, run make down.

Note: make dev only runs the minimum dependencies needed for things to work. If you'd like to run all the possible dependencies, run make alldeps. Keep in mind, though, that make alldeps does not start up Athens, but only its dependencies. All the services that get started by make alldeps are also available in the docker-compose.yml file, so if you're familiar with Docker Compose, you can also start up services as you need.

Run unit tests

There are two methods for running unit tests:

Completely In Containers

This method uses Docker Compose to set up and run all the unit tests completely inside Docker containers.

We highly recommend you use this approach to run unit tests on your local machine.

It's nice because:

  • You don't have to set up anything in advance or clean anything up
  • It's completely isolated
  • All you need is to have Docker Compose installed

To run unit tests in this manner, use this command:

make test-unit-docker

On the Host

This method uses Docker Compose to set up all the dependencies of the unit tests (databases, etc...), but runs the unit tests directly on your host, not in a Docker container. This is a nice approach because you can keep all the dependency services running at all times, and you can run the actual unit tests very quickly.

To run unit tests in this manner, first run this command to set up all the dependencies:

make alldeps

Then run this to execute the unit tests themselves:

make test-unit

And when you're done with unit tests and want to clean up all the dependencies, run this command:

make dev-teardown

Run End to End Tests

End to end tests ensure that the Athens server behaves as expected from the go CLI tool. These tests run exclusively inside Docker containers using Docker Compose, so you'll have to have those dependencies installed. To run the tests, execute this command:

make test-e2e-docker

This will create the e2e test containers, run the tests themselves, and then shut everything down.

Build the Docs

To get started with developing the docs we provide a docker image, which runs Hugo to render the docs. Using the docker image, we mount the /docs directory into the container. To get it up and running, from the project root run:

make docs
docker run -it --rm \
        --name hugo-server \
        -p 1313:1313 \
        -v ${PWD}/docs:/src:cached \
        gomods/hugo

Then open http://localhost:1313.

Linting

In our CI/CD pass, we use golint, so feel free to install and run it locally beforehand:

go get golang.org/x/lint/golint

For People Doing a Release

This section is written primarily for maintainers, so we don't forget how to initiate and complete a release. If you're not a maintainer and you're curious how we release new versions of Athens, read on!

Take a look at our releases page. This is how we release official new builds of Athens. Almost all of the release process is automatic.

Notes about the release number

Before we continue, please be sure you're familiar with our release numbering scheme.

We follow the semver convention for our release numbers. At the moment, we're not at v1, so release versions are v0.x.y. x is the MINOR number, and y is the PATCH number. You'll need to decide which one to update.

If there are significant new features in this release, choose to update x and set y to 0. If there are just bugfixes, just increment the y version number.

Code freeze

The first step to a release is a code freeze. This is 1-2 weeks (depending on the features and bugfixes we intend to release) during which we don't merge anything but critical bugfixes to the master branch. The code in master is essentially a release candidate (we don't cut a new branch for RC's at the moment) to test.

If you are doing a patch release, you do not need to do a code freeze.

Release branch

You'll be creating a new branch that represents the code that will be released. It always looks like release-v0.x.y. The v0.x.y represents the semver version number.

Reminder, the x and y make up the semver number. x is the MINOR and y is the PATCH

You'll need to have permissions to create a new branch in origin, whether through the GitHub site or running push origin release-v0.x.x.

Minor releases

If you're doing a minor release, you'll be incrementing x and setting y to 0 in the branch name. For example, if the previous release was v0.1.2, the previous release branch will be release-v0.1.2. Your new version will be v0.2.0 and new release branch will be release-v0.2.0.

Cut minor release branches from the master branch.

Patch releases

If you're doing a patch release, you'll be incrementing only the y version number. In this case, the new version will be v0.2.1 and new branch will be release-v0.2.1.

Cut patch release branches from the most recent release branch. Since these branches will only fix bugs, you'll need to find the commits from master that have the fixes in them and cherry pick them into the new patch release branch. For example:

$ git checkout -b release-v0.2.1 upstream/release-v0.2.0
$ git cherry-pick <commit from master>
....

Updating the helm chart

Regardless of which branch you created, you'll need to update the helm chart number. After you've cut the branch, make sure to change the versions in the Chart.yaml file:

  • If this is a new release of Athens, make sure to update the Docker image version value
  • Increment the patch number in the version field
  • Set the appVersion field to the semver of the new branch. Do not include the v prefix

Creating the new release in GitHub

Go to the create new release page and draft a new release. See below for what data to put into the fields you see:

  • Tag version - This should be the same v0.x.y number you put into the release branch. Make sure this tag starts with v and that the tag target is the proper release branch.
  • Release Title - Make sure the title is prefixed by the release number including the v. If you want to write something creative in the rest of the title, go for it!
  • Describe this release - Make sure to write what features this release includes, and any notable bugfixes. Also, thank all the folks who contributed to the release. You can find that information in a link that looks like this: https://github.com/gomods/athens/compare/$PREVIOUS_TAG...release-$CURRENT_TAG. Substitute $PREVIOUS_TAG for the last semver and $CURRENT_TAG to the version in the new release branch

When you're done, press the "Publish Release" button. After you do, our Drone job will do almost everything.

Make sure the Drone CI/CD job finished, and check in Docker Hub to make sure the new release showed up in the tags section.

Finishing up

The Drone job will do everything except:

  • Tweet out about the new release
  • Update the helm chart in the master branch

If you are a core maintainer and don't have access to the @gomods account, ask one of the maintainers to give you access. Here is an example showing the general format of these tweets. Obviously you should use your creativity here though!

Finally, you'll need to update the helm version number in the master branch. Create a new branch called update-helm-$CURRENT_TAG and update the following files:

  • charts/athens-proxy/values.yaml - update the image.tag field to the latest version number you created, including the v. This field should be near the top of the file
  • charts/athens-proxy/Chart.yaml - update the version field and the appVersion field
    • Increment the patch number in the version field
    • Change the appVersion field to the tag name of the GitHub version you created, including the v

Here is an example of how to do this.

Finally, create a pull request from your new branch into the master branch. It will be reviewed and merged as soon as possible.