This is the notebooks and tools repository of Digital Earth Australia (DEA).
Contribute: We'd love your contribution! DEA Notebooks is an open-source project and welcomes contributions from everyone.
License: The code in this repository is licensed under the Apache License, Version 2.0. DEA data is licensed under the Creative Commons by Attribution 4.0 license.
Documentation: See the DEA Notebooks Wiki.
Contact: For assistance with any of these notebooks and tools, please ask a question on our Open Data Cube Slack channel or on the GIS Stack Exchange using the open-data-cube tag. You can also report an issue on this repository.
Citation: If you use this repository in your work, please reference it with the following citation.
Krause, C., Dunn, B., Bishop-Taylor, R., Adams, C., Burton, C., Alger, M., Chua, S., Phillips, C., Newey, V., Kouzoubov, K., Leith, A., Ayers, D., Hicks, A., DEA Notebooks contributors 2021. Digital Earth Australia notebooks and tools repository. Geoscience Australia, Canberra. https://doi.org/10.26186/145234
We would also appreciate it if you add a citation of your work to our USAGE page.
This repository hosts Jupyter Notebooks, Python scripts and workflows for analysing Digital Earth Australia (DEA) satellite data and derived products. This documentation is designed to provide a guide to getting started with DEA, and to showcase the wide range of geospatial analyses that can be achieved using DEA data and open-source software including Open Data Cube and xarray.
The repository is based around the following directory structure (from simple to increasingly complex applications):
- Beginners_guide: Introductory notebooks aimed at introducing Jupyter Notebooks and how to load, plot and interact with DEA data
- DEA_products: Notebooks introducing DEA's satellite datasets and derived products, including how to load each dataset and any special features of the data
- Interactive_apps: Interactive apps and widgets that require little or no coding to run
- How_to_guides: A recipe book of simple code examples demonstrating how to perform common geospatial analysis tasks using DEA and open-source software
- Real_world_examples: More complex case study workflows demonstrating how DEA can be used to address real-world problems
Supporting functions and data for the notebooks are kept in the following directories:
- Tools: Python module dea-tools, containing functions and algorithms developed to assist in analysing DEA data (e.g. loading data, plotting, spatial analysis, machine learning)
- Supplementary_data: Supplementary files required for the analyses above (e.g. images, rasters, shapefiles, training data)
All notebooks in the dea-notebooks repository contain tags describing their functionality. If you are searching for a specific functionality, use the Tags Index to search for a suitable example. If there is a functionality that has not been documented that you think should be, please create an 'Issue' in the dea-notebooks repository.
We encourage you to check out the other usages of our notebooks, code and tools at our USAGE page.
To get started, see either of these articles.
The dea-notebooks repository uses 'branches' to manage individuals' notebooks, and to allow easy publishing of notebooks ready to be shared. There are two main types of branches:
- develop branch: The
developbranch is the default branch where notebooks are put as they are being prepared to be shared publicly. Notebooks added to this branch will be periodically merged into thestablebranch after testing and evaluation. Thedevelopbranch is protected and requires changes to be approved via a 'pull request' and review checklist before they appear on the branch. - stable branch: The
stablebranch contains DEA's collection of publicly available notebooks. Notebooks added to this branch will become part of the official DEA documentation and are published on the DEA Knowledge Hub. Thestablebranch is protected, and is periodically updated with new content from thedevelopbranch via a 'pull request' (fordevelop>stablepull requests, merge using the 'Create a merge commit' option). - Working branches: All other branches in the repository are working spaces for users of
dea-notebooks. They have a unique name (typically named after the user, e.g.ClaireK,BexDunn). The notebooks on these branches can be works-in-progress and do not need to be pretty or complete. By using a working branch, it is easy to use scripts and algorithms fromdea-notebooksin your own work or share and collaborate on a working version of a notebook or code.
Once you have a notebook that is ready to be published from the develop branch to the DEA Knowledge Hub, you can submit a 'pull request' in the Pull requests tab at the top of the repository. The default pull request template contains a check-list to ensure that all stable branch Jupyter notebooks are consistent and well-documented so they can be understood by future users, and rendered correctly in the DEA Knowledge Hub. Please ensure that as many of these checklist items are complete as possible or leave a comment in the pull request asking for help with any remaining checklist items.
For pull requests you would like help with or that are a work in progress, consider using Github's draft pull request feature. This indicates that your work is still a draft, allowing you to get feedback from other DEA users before it is published on the ``develop` branch.
A template notebook has been developed to make it easier to create new notebooks that meet all the pull request checklist requirements. The template notebook contains a simple structure and useful general advice on writing and formatting Jupyter notebooks. The template can be found here: DEA_notebooks_template.ipynb
Using the template is not required for working branch notebooks but is highly recommended as it will make it much easier to publish any notebooks on develop in the future.
Anyone with admin access to the dea-notebooks repository can approve 'pull requests'. You can see a list of the 'pull requests' ready for review in the Pull requests tab at the top of the repository. Click this tab, then click on the open pull request. You will need to review the code before you can approve the request. Ensure that all items in the pull request checklist have been ticked off and incorporated into the notebook. To make changes to someone else's pull request directly, first check out the branch you want to edit (e.g. pull_request_branch):
git pull
git checkout --track origin/pull_request_branchCommit and push any changes you make, which will become part of the open pull request.
If the notebook meets all the checklist requirements, click the green 'Review' button and click 'Approve' (with an optional comment). You can also 'Request changes' here if any of the checklist items are not complete.
Once the pull request has been approved, you can merge it into the develop branch. Select the 'Squash and merge' option from the drop-down menu to the right of the green 'merge' button. Once you have merged the new branch in, you need to delete the branch. There is a button on the page that asks you if you would like to delete the now merged branch. Select 'Yes' to delete it.