Skip to content

Tips for Developers (1)

Jump to bottom Edit New page
Jiwoo Lee edited this page Jun 26, 2023 · 1 revision

Developers' corner

Introduction

If you are interested in contributing to the the development of the PMP please contact the core team ([email protected]) so that you can be registered as a contributor. Since this package has many users, it is critical that the master code repository remains as clean as possible. For this reason, a few rules are being adhered to:

  1. "master" contains the latest end-user-ready code version
  2. Official releases will be tagged
  3. A "devel" branch is also associated with this repo. This branch contains fixes and improvements that any developer sees "fit-for-master" but has not been extensively tested by others. This can be considered a "pre-release" branch.

Patch procedure

In order to ensure that "master" stays clean, only a few "advanced" developers have repo write access. The procedure to fix/enhance the code is as follows:

  1. Fork "master" to your favorite git server (we recommend github)
  2. Clone your forked repo to your local machine
  3. Create a new branch from the cloned master (Branch names should be descriptive, we recommend the following format: issueNumber_author_shortDescription)
  4. Edit/fix/enhance at will
  5. Send a pull request (PR) to the PCMDI metrics repo administrator so changes are pulled back into the PCMDI metrics repo as a separate branch (issueNumber_author_shortDescription) and your code can be tested by others
  6. Once your changes have been tested and confirmed to work, the PR/branch is approved and your changes are merged back into master

Practical command-line example (user: durack1)

  1. Go to github.com/PCMDI/pcmdi_metrics and "fork" the repo to your github account

  2. Clone your forked repo to your local machine (do this once only)

    $ git clone [email protected]:durack1/pcmdi_metrics.git
$ git remote add PCMDI git://github.com/PCMDI/pcmdi_metrics.git
$ git fetch --all
$ git checkout -b pcmdi_master PCMDI/master ; # create local branch name (pcmdi_master) following source branch

  3. Bring PCMDI changes back to your repo - make sure you have the most up-to-date copy of the remote repo

    $ git checkout pcmdi_master ; # Pull changes for master repo
$ git pull
$ git checkout master
$ git merge pcmdi_master

  4. Create a new branch (using issueNumber_author_shortDescription format)

    $ git checkout master
$ git checkout -b issueNumber_author_shortDescription ; # Create a new branch based on master

  5. Make the metrics package cooler - edit away..

  6. Commit your changes to your local repo, with an appropriate comment describing your changes

    $ git commit -am 'A much cooler metrics package'

  7. Push your local changes back to your remote branch (which you named following issueNumber_author_shortDescription format in step 3 above) first time:

    $ git push -u origin HEAD:issueNumber_author_shortDescription

    Subsequent pushes:

    $ git push

  8. From github (user account webpage) request changes to be pulled back into PCMDI/pcmdi_metrics or if you're not using github send the PCMDI repo administrator the fork and branch information you would like to merge.

IF THERE ARE ANY CONFLICTS DURING THE MERGE, YOUR PULL REQUEST WILL BE REJECTED - IT IS YOUR RESPONSIBILITY TO MERGE CONFLICTS PRIOR TO ISSUING A PULL REQUEST - see step 7 above

Notes

  1. Branch names should be descriptive; we recommend the following format: issueNumber_author_shortDescription

  2. Multiple bug fixes/enhancements should be split into multiple branches

  3. Once your merge back into the master repo is complete, we suggest purging your local branch

    $ git branch -D issueNumber_author_shortDescription [deletes local branch]
$ git push origin :issueNumber_author_shortDescription [deletes remote branch]

  4. And then synchronize your repos with the remotes

    $ git remote show [show remotes]
$ git remote show PCMDI [show specific remote detail]
$ git checkout pcmdi_master
$ git pull
$ git checkout master
$ git pull
$ git merge --no-ff pcmdi_master [merge pcmdi_master into master branch]
$ git checkout -b issueNumber_author_shortDescription [create new branch]
< Make edits >
$ git commit -am 'issueNumber_author_shortDescription'
< test code for bugs and fix >
$ git push -u origin HEAD:issueNumber_author_shortDescription [-u ensures changes are tracked across repos]
< further testing before merging into pcmdi_master >
< EXTRAS >
$ git push remote -u HEAD:issueNumber_author_shortDescription [push changes to remote branch and track [-u]]

  5. If you haven't been at it for a while, here's how you blast your repo and update it to ensure that you're working with the latest source

    $ git clone https://github.com/PCMDI/pcmdi_metrics.git [creates a pcmdi_metrics directory in CWD]
$ cd pcmdi_metrics
$ git remote add durack1 https://github.com/durack1/pcmdi_metrics
$ git fetch --all -p [--all is equivalent to --force and -p purges/deletes branches that no longer exist remotely]
$ git checkout master [switch to master branch - there is only one the UV-CDAT master]
$ git branch -vv [validate that there is only one master branch]
$ git status [check to see if everything is up to date]
$ git checkout -b issue601_durack1_python278 [create new branch before making changes - update to python 2.7.8]
# Do some other stuff, download new *.tgz and get an md5 checksum
$ git diff [check to see that only what you wanted to change has changed]
$ git status [confirm which branch you're on before committing]
$ git commit -am 'fix #601 - update python to latest 2p7p8' [commit changes to new branch issue601_durack1_python278 as created above]
$ git push -u durack1 HEAD:issue601_durack1_python278 [push changes into remote branch]
$ git push -u origin HEAD:issue601_durack1_python278 [push changes into local branch]

Test development branch before merging its pull request (PR) to main

  • Create a conda development env for use in local development:

    conda env create -f conda-env/dev.yml or

    conda env create -f conda-env/dev.yml -n my_pmp_conda_env

  • Clone the repo to the local

  • Switch to the working branch

  • Activate the local development conda env and install from the source code:

    conda activate pcmdi_metrics_dev (if env name was not given above) or conda activate my_pmp_conda_env

    python setup.py install (at pcmdi_metrics directory)

Note for CI process

Adds the conda development env for use in local development and in the GH Actions CI/CD workflow(s).

  • conda env create -f conda-env/dev.yml or conda env create -f conda-env/dev.yml -n my_pmp_conda_env
  • PMP uses Github Action for its CI process.
  • pre-commit checks were included for the CI process
    • isort for sorting import statements, and
    • flake8 and black for python code styling.
  • To run pre-commit in your local branch: pre-commit run --all-files

Conda-forge releases

Basic instructions are provided here for maintaining the conda-forge feedstock. For more help, please consult the official documentation.

Set-up (this only needs to be done once):

  1. Fork the pcmdi_metrics feedstock at https://github.com/conda-forge/pcmdi_metrics-feedstock.
  2. Clone your fork locally to work (optional. You can also use the Github UI).

Release

When a new, tagged version of the PMP has been released, follow these steps. Note that it is important to work in a fork and not in the pcmdi_metrics-feedstock directly. It is also important to work in a new branch, and not in the "master" branch.

  1. Generate sha256 hash for the new version
    a. Download the tar file.
    b. Open a terminal and run openssl sha256 tar_file_name.

  2. In your fork of the pcmdi_metrics-feedstock, check out a new branch. Use a descriptive name, for example "v2.2.1".

  3. Open the file recipe/meta.yaml in your preferred text editor.

  4. In recipe/meta.yaml, update two lines:
    L2: update the version number in quotes (e.g. {% set version = "2.2.1" %}, but with your version in place of 2.2.1)
    L10: Update the sha256 field with the hash generated in step 1.
    Save these changes.

  5. Add and commit recipe/meta.yaml in your new branch

  6. Push the new branch to "origin" if working locally.

  7. Go to your pcmdi_metrics-feedstock fork on Github.com. Create a PR to the PMP feedstock from your new branch. You can use the default PR template. Once it is created, follow the check boxes in the comment.

  8. Wait for the feedstock tests to finish. Once you are satisfied with the PR, it can be merged.

  9. After a couple of hours, the new version should be available on conda-forge at https://anaconda.org/conda-forge/pcmdi_metrics.

Alpha/Beta/Dev versions

The release process for a dev version is the same as the "Release" process outlined above, with two additional steps. We want to use a different label for the dev version so that users who want the latest stable release from the pcmdi_metrics channel do not download this version by default.

This example uses the label name 'pcmdi_metrics_dev'. Consult PCMDI metrics anaconda page to see what labels are in use.

These are the additional steps (following steps 4 and 5 in the "Release" process):

4a. Create recipe/conda_build_config.yaml. This file will contain two lines of text:

channel_targets:
  - conda-forge pcmdi_metrics_dev

5a. Add and commit recipe/conda_build_config.yaml.

The file "recipe/conda_build_config.yaml" should be deleted the next time a non-dev release is done.

To create an environment with the dev version (use your label if different from pcmdi_metrics_dev):
conda create -y -n pcmdi_metrics_test -c conda-forge/label/pcmdi_metrics_dev python=3.9 pcmdi_metrics

Add a custom footer
Add a custom sidebar
Clone this wiki locally