Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: add docs about network mode #1873

Open
wants to merge 6 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/content/configuration/download.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ download "github.com/pkg/*" {

The first two lines describe the _default_ behavior for all modules. This behavior is overridden for select module groups below. In this case, the default behavior is:

- Immediatley redirect all requests to `https://proxy.golang.org`
- Immediately redirect all requests to `https://proxy.golang.org`
- In the background, download the module from the version control system (VCS) and store it

The rest of the file contains `download` blocks. These override the default behavior for specific groups of modules.
Expand Down
2 changes: 1 addition & 1 deletion docs/content/configuration/filter.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: Filtering modules (deprecated)
description: Configuring modules that are stored on the proxy
weight: 6
weight: 7
---

>Note: the filter file that this page documents is deprecated. Please instead see ["Filtering with the download mode file"](/configuration/download) for updated instructions on how to filter modules in Athens.
Expand Down
41 changes: 41 additions & 0 deletions docs/content/configuration/network.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
---
title: Network mode
description: Configures how Athens will return the results of the /list endpoint
weight: 6
---

The NetworkMode configuration in Athens determines how results are returned by the `/list` endpoint. This endpoint can retrieve information from both Athens' own [storage](/configuration/storage) and the upstream version control system (VCS).

> Note: The NetworkMode configuration can also affect the behavior of other endpoints by improving error messaging

## Network mode keywords

There are 3 modes available for the NetworkMode configuration. To configure the `NetworkMode` settings at `config.dev.toml`, set the `NetworkMode` to 1 of the 3 available modes:

1. `strict`: In this mode, Athens will merge versions from the VCS and storage but will fail if either of them fails. This mode provides the most consistent results.

2. `offline`: This mode only retrieves versions from Athens' storage and never reaches out to the VCS.

3. `fallback`: This mode retrieves versions from Athens' storage only if the VCS fails. Note that using this mode may result in inconsistent results since fallback mode does its best to provide the available versions at the time of the request.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This mode retrieves versions from Athens' storage only if the VCS fails.

I don't believe that's true. We always retrieve versions from the Athens storage. We just fallback to only providing this list in case of an upstream outage

Copy link
Contributor Author

@ngshiheng ngshiheng May 30, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We always retrieve versions from the Athens storage

Hmm sorry, I don't exactly follow

  1. This was taken from this snippet.
  2. That aside, I can attest this behavior is correct based on existing live deployments
  3. Also referring to /pkg/download/protocol.go

maybe I could have worded it better? 🤔

edit: AFAIK go get will reach out to VCS to retrieve versions. But commands like go mod download and go build will always return whatever that is from Athens storage

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@marwan-at-work WDYT? can you recommend what can I change here to move this PR forward?


## Use cases

### Ensuring consistency in module version retrieval

When working in environments that prioritize consistency and reliability, configuring Athens with the `strict` mode guarantees a dependable and predictable module version resolution mechanism.

Using the `strict` mode, Athens merges module versions from its storage and the VCS, while ensuring that any failure in either source results in a failure response.

Choose Athens' `strict` network mode for a reliable and consistent approach to module version retrieval.

### Fetching modules for offline environments

When working in offline environments with private networks lacking direct internet access, Athens' `offline` mode can be useful.

For example, you can [pre-download](/configuration/prefill-disk-cache/) modules using Athens from a machine with internet access. Subsequently, the pre-downloaded modules are accessible within the offline network through Athens, facilitating development and builds without requiring an active internet connection.

### Ensuring availability of modules versions

In certain situations, the stability or availability of the VCS may vary. To mitigate potential disruptions, Athens offers the `fallback` mode. If the VCS encounters issues or fails to respond, Athens falls back to serving modules from its storage to ensure continued availability.

However, using the `fallback` mode may result in inconsistent results. Athens will provide the available versions at the time of the request, which may differ from the latest versions in the VCS.
75 changes: 28 additions & 47 deletions docs/content/install/install-on-kubernetes.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,45 +6,26 @@ weight: 1

When you follow the instructions in the [Walkthrough](/walkthrough), you end up with an Athens Proxy that uses in-memory storage. This is only suitable for trying out the Athens proxy for a short period of time, as you will quickly run out of memory and Athens won't persist modules between restarts. In order to run a more production-like proxy, you may want to run Athens on a [Kubernetes](https://kubernetes.io/) cluster. To aid in deployment of the Athens proxy on Kubernetes, a [Helm](https://www.helm.sh/) chart has been provided. This guide will walk you through installing Athens on a Kubernetes cluster using Helm.

* [Prerequisites](#prerequisites)
* [Configure Helm](#configure-helm)
* [Deploy Athens](#deploy-athens)
- [Prerequisites](#prerequisites)
- [Configure Helm](#configure-helm)
- [Deploy Athens](#deploy-athens)

---

## Prerequisites

In order to install Athens on your Kubernetes cluster, there are a few prerequisites that you must satisfy. If you already have completed the following steps, please continue to [configuring helm](#configure-helm). This guide assumes you have already created a Kubernetes cluster.

* Install the [Kubernetes CLI](#install-the-kubernetes-cli).
* Install the [Helm CLI](#install-the-helm-cli).
- Install the [Kubernetes CLI](#install-the-kubernetes-cli).
- Install the [Helm CLI](#install-the-helm-cli).

### Install the Kubernetes CLI

In order to interact with your Kubernetes Cluster, you will need to [install kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/).
To work with your Kubernetes Cluster, it's necessary to have the Kubernetes CLI, also known as `kubectl`, installed on your computer. You can download and install it by following the instructions provided in the [official Kubernetes documentation](https://kubernetes.io/docs/tasks/tools/install-kubectl/).

### Install The Helm CLI
### Install the Helm CLI

[Helm](https://github.com/kubernetes/helm) is a tool for installing pre-configured applications on Kubernetes.
Install `helm` by running the following command:

#### MacOS

```console
brew install kubernetes-helm
```

#### Windows

1. Download the latest [Helm release](https://storage.googleapis.com/kubernetes-helm/helm-v2.7.2-windows-amd64.tar.gz).
1. Decompress the tar file.
1. Copy **helm.exe** to a directory on your PATH.

#### Linux

```console
curl https://raw.githubusercontent.com/kubernetes/helm/master/scripts/get | bash
```
[Helm](https://github.com/helm/helm) is a tool for installing pre-configured applications on Kubernetes. You can download and install `helm` by following the instructions available on the [official Helm documentation](https://helm.sh/docs/intro/install/).

## Configure Helm

Expand Down Expand Up @@ -88,7 +69,7 @@ $ helm repo update

Next, install the chart with default values to `athens` namespace:

```
```console
$ helm install gomods/athens-proxy -n athens --namespace athens
```

Expand Down Expand Up @@ -133,10 +114,10 @@ When using the `disk` storage provider, you can configure a number of options re

```yaml
persistence:
enabled: false
accessMode: ReadWriteOnce
size: 4Gi
storageClass:
enabled: false
accessMode: ReadWriteOnce
size: 4Gi
storageClass:
```

Add it to `override-values.yaml` file and run:
Expand All @@ -151,7 +132,7 @@ helm install gomods/athens-proxy -n athens --namespace athens -f override-values

To use the Mongo DB storage provider, you will first need a MongoDB instance. Once you have deployed MongoDB, you can configure Athens using the connection string via `storage.mongo.url`. You will also need to set `storage.type` to "mongo".

```
```console
helm install gomods/athens-proxy -n athens --namespace athens --set storage.type=mongo --set storage.mongo.url=<some-mongodb-connection-string>
```

Expand All @@ -162,7 +143,7 @@ S3 bucket name, respectively. By default, Athens will attempt to load AWS creden
variables, shared credentials files, and EC2 instance credentials. To manually specify AWS credentials, set `storage.s3.access_key_id`,
`storage.s3.secret_access_key`, and change `storage.s3.useDefaultConfiguration` to `false`.

```
```console
helm install gomods/athens-proxy -n athens --namespace athens --set storage.type=s3 --set storage.s3.region=<your-aws-region> --set storage.s3.bucket=<your-bucket>
```

Expand All @@ -173,21 +154,21 @@ This URL can also be an kubernetes-internal one (e.g. something like `minio-serv
You need to create a bucket inside your minio-installation or use an existing one. The bucket needs to be referenced in `storage.minio.bucket`.
Last athens need authentication credentials for your minio in `storage.minio.accessKey` and `storage.minio.secretKey`.

```
```console
helm install gomods/athens-proxy -n athens --namespace athens --set storage.type=minio --set storage.minio.endpoint=<your-minio-endpoint> --set storage.minio.bucket=<your-bucket> --set storage.minio.accessKey=<your-minio-access-key> --set storage.minio.secretKey=<your-minio-secret-key>
```

#### Google Cloud Storage

To use Google Cloud Storage storage with Athens, set `storage.type` to `gcp`. You need to set `storage.gcp.projectID` and `storage.gcp.bucket` to the
desired GCP project and bucket name, respectively.
To use Google Cloud Storage storage with Athens, set `storage.type` to `gcp`. You need to set `storage.gcp.projectID` and `storage.gcp.bucket` to the
desired GCP project and bucket name, respectively.

Depending on your deployment environment you will also need to set `storage.gcp.serviceAccount` to a key which has read/write access to
the GCS bucket. If you are running Athens inside GCP, you will most likely not need this as GCP figures out internal authentication between products for you.
Depending on your deployment environment you will also need to set `storage.gcp.serviceAccount` to a key which has read/write access to
the GCS bucket. If you are running Athens inside GCP, you will most likely not need this as GCP figures out internal authentication between products for you.

```
helm install gomods/athens-proxy -n athens --namespace athens --set storage.type=gcp --set storage.gcp.projectID=<your-gcp-project> --set storage.gcp.bucket=<your-bucket>
```
```console
helm install gomods/athens-proxy -n athens --namespace athens --set storage.type=gcp --set storage.gcp.projectID=<your-gcp-project> --set storage.gcp.bucket=<your-bucket>
```

### Kubernetes Service

Expand Down Expand Up @@ -237,16 +218,16 @@ You can set the `URL` for the [upstream module repository](https://docs.gomods.i

You have a few good options for what you can set as an upstream:

- `https://gocenter.io` to use JFrog's GoCenter
- `https://proxy.golang.org` to use the Go Module Mirror
- The URL to any other Athens server
- `https://gocenter.io` to use JFrog's GoCenter
- `https://proxy.golang.org` to use the Go Module Mirror
- The URL to any other Athens server

The example below shows you how to set GoCenter up as upstream module repository:

```yaml
upstreamProxy:
enabled: true
url: "https://gocenter.io"
enabled: true
url: "https://gocenter.io"
```

Add it to `override-values.yaml` file and run:
Expand Down
2 changes: 1 addition & 1 deletion docs/content/install/using-docker.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ description: Information about Athens' Docker images
weight: 1
---

Whether setting Athens up using [Kubernetes](install/install-on-kubernetes/) or using the [Walkthrough](/Walkthrough), you'll most likely be using one of the images that the Athens project produces. This document details what images are available, and has a recap from the Walkthrough of how to use them on their own.
Whether setting Athens up using [Kubernetes](/install/install-on-kubernetes/) or using the [Walkthrough](/Walkthrough), you'll most likely be using one of the images that the Athens project produces. This document details what images are available, and has a recap from the Walkthrough of how to use them on their own.

---

Expand Down
11 changes: 10 additions & 1 deletion docs/layouts/partials/custom-footer.html
Original file line number Diff line number Diff line change
Expand Up @@ -4,5 +4,14 @@
</script>
-->
<a href="https://github.com/gomods/athens">
<img style="position: absolute; top: 0; right: 0; border: 0;" src="https://s3.amazonaws.com/github/ribbons/forkme_right_green_007200.png" alt="Fork me on GitHub">
<img
decoding="async"
loading="lazy"
width="149"
height="149"
style="position: absolute; top: 0; right: 0; border: 0"
src="https://github.blog/wp-content/uploads/2008/12/forkme_right_red_aa0000.png?resize=149%2C149"
alt="Fork me on GitHub"
data-recalc-dims="1"
/>
</a>