Skip to content

Release Process

Luke Karrys edited this page Feb 15, 2023 · 230 revisions

Overview

  • The release cadence is optimized for consistency – new releases happen weekly, on the same day of the week (Wednesday)
  • Out-of-band releases can & do happen to address critical security bugs or patch a broken release & we sometimes elect to skip weeks where there is competing priorities or vacation/OOO
  • There is only one officially supported release branch: npm@latest (ie. the latest/stable version of npm)
    • Our team does not support any LTS releases - you can read more about the versions of npm we support here.

Note: We're currently shipping weekly releases of npm@latest every Wednesday (& occasionally even more than once a week).


Roles

Each release has a release manager, who is responsible for landing that release's pull requests and dependency upgrades, ensuring that the tests pass for the release branch, writing the release notes, and running through the actual release process (see below). The current release managers are the npm CLI team:

Setting up to do releases

If you're a new release manager, you need a GPG key to sign releases with. We've been using https://keybase.io/ to store/verify our public keys.

In order to sign the release tags, you will need to set this also: npm config set sign-git-tag true


Release Please

Much of the release process is automated using a customized release-please script in @npmcli/template-oss.

The configuration for release-please is located in a root repo config file release-please-config.json.

Major Releases and Prereleases

When creating a new major release, first set the prerelease: true flag in the release-please config, and then continue to use conventional commits to indicate changes and cut a branch for the latest version of the previous major.

  1. Cut a release/v<CURRENT_MAJOR> branch

    git checkout latest
    git checkout -b release/v<CURRENT_MAJOR>
    git push -u origin release/v<CURRENT_MAJOR>
    git checkout latest
  2. Update releases.json in npm/documentation with the new branches

  3. Set prerelease: true for all packages in release-please-config.json

    // ./release-please-config.json
    {
      ...
      "packages": {
        ".": {
          ...
          "prerelease": true
        }
      }
      ...
    }
  4. Commit prerelease: true as feat:

    git checkout -b prerelease
    git commit -am "feat: set prerelease flag"
    gh pr create -f
  5. Continue to use conventional commits and feature PRs

    Everything is now identical to normal release development. The only difference is that release-please will version and tag all releases with a -pre.<PRE_ID> suffix.

    The usual release process should be followed, except for the sections marked NOT FOR PRERELEASE.

    git checkout latest
    git checkout -b me/my-breaking-feature
    git commit -m 'feat: break stuff
    
    BREAKING CHANGE: this message gets added to the changelog'
    gh pr create -f
  6. One month later…, remove prerelease flag and commit

    Warning: All breaking changes must be merged AND released before this step

    git checkout latest
    # Remove `prerelease` flags from manifest
    git commit -am "feat: remove prerelease flags"
    # `release-please` uses the convention of an empty commit
    # to force all packages to be released
    git commit -m "feat: trigger release process" --allow-empty
    # this cannot be made into a PR because rebasing on GitHub
    # will remove the empty commit
    git push origin latest

    This will trigger a new release without a -pre.<PRE_ID> suffix that can be merged and published normally.

  7. Combine all prerelease changelogs into the final GitHub Release notes

    release-please only generates changelogs based on the diff from the previous prerelease. So when coming out of prerelease mode, we combine all the prerelease changelogs and manually update the notes of the GitHub Release after it has been created.

    The GitHub Release notes are the source of truth for the Node.js pull request as well.

Setting a New Major Version as Latest

The following steps need to be followed when a new major version is tagged latest for the first time. This will not always occur at the same time as a major version coming out of prerelease mode.

  1. Update releases.json in npm/documentation so the branch and spec of the new and previous major releases point to the correct place.

  2. Set the latest dist-tag of the new major version:

    node . dist-tag add npm@<NEW-MAJOR.Y.Z> latest
  3. TODO: There may be more of these?

    Warning: There are probably more steps here!

Syncing Chore and Template-OSS Commits

Any commit to a branch with the pattern release/v<MAJOR> will create a backport release. Since our releases are run via local workflows, the chore commits updating these workflows should be ported to all active release/v* branches.

This is not a perfect process, since some commits may contain code that is not runnable in CI on older supported versions of Node. Most of the non-test workflows are only run on the latest supported version of Node (eg if v8 supports ^12.13.0 || ^14.15.0 || >=16.0.0 then they will run on 16) which should overlap with later npm Node engines.

With the above in mind, here are some commands we can run to apply the differences between a release/v* branch and latest:

  1. Switch to release/v<MAJOR> and branch from there:

    git checkout release/v<MAJOR>
    git checkout -b sync-release-v<MAJOR>-chores
  2. Apply the non-branch-specific diff in scripts/

    Any @npmcli/template-oss config also lives in this directory, so later running template-oss-apply will apply anything synced in scripts/.

    git diff release/v<MAJOR> latest -- scripts/ ':(exclude)scripts/template-oss/branch-specific-config.js' | git apply

    These should be inspected manually and non-compatible changes should be reverted. Then the result can be committed.

    git commit -am 'chore: sync `scripts/` from `latest`'
  3. Get the non-content diff in docs/:

    The docs workspace can also contain tooling that needs to be ported over so that building the docs via the workflows will work across branches.

    git diff release/v<MAJOR> latest -- docs/ ':(exclude)docs/lib/content/' | git apply

    These should be inspected manually and non-compatible changes should be reverted. Then the result can be committed:

    git commit -am 'chore: sync `docs/` from `latest`'
  4. Update @npmcli/template-oss

    Now that any config has been synce, we can install the latest @npmcli/template-oss and apply the changes.

    node . i @npmcli/template-oss@latest -ws -iwr --save-exact
    node . run template-oss-apply
    git commit -am 'chore(deps): @npmcli/template-oss@<version>'
  5. Open a PR to release/v<MAJOR>

    gh pr create --base release/v<MAJOR>

Weekly Tasks

Merging Pull Requests

  1. Search for PRs to merge

    Some helpful gh searches:

    gh pr list --draft=false --search "status:pending"
    gh pr list --draft=false --search "status:success"
    gh pr list --draft=false --search "review:required"
    gh pr list --draft=false --search "review:approved"
  2. Make sure the PR is targeting latest and not something else

    gh pr view --json baseRefName -q '.baseRefName'
  3. Merge the pull request using squash or rebase

    Note: Prefer using rebase if the PR has multiple conventional commits.

    Squash

    gh pr merge -s <pr-num>

    Rebase

    gh pr merge -r <pr-num>

Updating Dependencies

Note: Workspace do not need to be updated, since the automated release process will sync all workspace versions as part of the release pull request.

Dependency updates should all go through pull requests using the deps: conventional commit prefix. This section shows how to check for any outdated deps and make pull requests for them.

  1. Check all top level deps

    node . outdated
  2. Check all levels of production deps

    node . outdated -a --omit=dev
  3. Install and commit each dep to a PR

    git checkout -b deps/updates
    node . install <pkg>@<version>
    node . run dependencies
    # Use prefix `chore:` for dev deps
    git commit -am 'deps: <pkg>@<version>'
    git push origin deps/updates
    gh pr create -f
  4. Merge the PR if things look ok

Updating @npmcli/template-oss

Note: @npmcli/template-oss is a dependency of all workspaces and must be installed with --save-exact

  1. Update @npmcli/template-oss in all workspaces and root

    node . i @npmcli/template-oss@latest -ws -iwr --save-exact
  2. Commit and open PR

    git checkout -b deps/template-oss
    git commit -am 'chore(deps): @npmcli/template-oss@<version>'
    git push origin deps/template-oss
    gh pr create -fg

Release Day Tasks

Update any last minute deps

See Updating Dependencies for the process.

Release the CLI and workspaces

Note: A single release pull request is created for the CLI and all dependent workspaces by release-please.

The release pull request is created automatically by release-please whenever a commit is pushed to the default branch. This PR should be left open until Release Day, when it should be checked to make sure all workflows are passing. You should checkout the branch locally and publish from there. Once everything is published the PR can be merged into latest.

  1. Checkout the release branch

    Ensure git status is not dirty on this branch after resetting deps. If it is, then something is probably wrong with the automated release process.

    gh pr checkout <PR-NUMBER> --force
    node . run resetdeps
    node scripts/git-dirty.js
  2. Check CI status

    gh pr checks --watch
  3. Publish the CLI and workspaces

    Warning: This will publish all updated workspaces to latest, prerelease or backport depending on their version, and will publish the CLI with the dist-tag set to next-<MAJOR>.

    Note: The --test argument can optionally be omitted to run the publish process without running any tests locally.

    node scripts/publish.js --test
  4. Optionally install and test npm@<X.Y.Z> locally

    npm i -g npm@<X.Y.Z>
    npm --version
    npm whoami
    npm help install
    # etc
  5. Set latest dist-tag to newly published version

    Warning: NOT FOR PRERELEASE: Do not run this step for prereleases or if <MAJOR> is not being set to latest.

    node . dist-tag add npm@<X.Y.Z> latest
  6. Trigger docs.npmjs.com update

    gh workflow run update-cli.yml --repo npm/documentation
  7. Merge release PR

    gh pr merge --rebase
    git checkout <BASE-BRANCH>
    git fetch
    git reset --hard origin/<BASE-BRANCH>
    node . run resetdeps
  8. Wait For Release Tags

    Warning: The remaining steps all require the GitHub tags and releases to be created first. These are done once this PR has been labelled with autorelease: tagged.

    Release Please will run on the just merged release commit and create GitHub releases and tags for each package. The release bot will will comment on this PR when the releases and tags are created.

    Note: The release workflow also includes the Node integration tests which do not need to finish before continuing.

    You can watch the release workflow in your terminal with the following command:

    gh run watch `gh run list -R npm/cli -w release -b <BASE-BRANCH> -L 1 --json databaseId -q ".[0].databaseId"`
    
  9. Open nodejs/node PR to update npm to latest

    Warning: You must wait for CI to create the release tags before running this step. These are done once this PR has been labelled with autorelease: tagged.

    Trigger the Create Node PR action. This will open a PR on nodejs/node to the main branch.

    Note: The resulting PR may need to be labelled if it is not intended to land on old Node versions.

    First, sync our fork of node with the upstream source:

    gh repo sync npm/node --source nodejs/node --force

    Then, if we are opening a PR against the latest version of node:

    gh workflow run create-node-pr.yml -f spec=next-<MAJOR>

    If the PR should be opened on a different branch (such as when doing backports) then run it with -f branch=<BRANCH_NAME>. There is also a shortcut to target a specific Node version by specifying a major version number with -f branch=18 (or similar).

    For example, this will create a PR on nodejs/node to the v16.x-staging branch:

    gh workflow run create-node-pr.yml -f spec=next-<MAJOR> -f branch=16
  10. Label and fast-track nodejs/node PR

    Note: This requires being a nodejs collaborator. Currently @lukekarrys is so ping them to do these steps!

    • Thumbs-up reaction on the Fast-track comment
    • Add an LGTM / Approval
    • Add request-ci label to get it running CI
    • Add commit-queue label once everything is green