Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

create test and pylint CI workflows; use devops_tests as a submodule; add show_anim(); bump checkout & setup-python actions in pylint CI job; move more setup entries into pyproject.toml; add example notebook with animation #30

Merged
merged 36 commits into from
Nov 28, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
9642001
add show_anim() function with usage example notebook
Sfonxu Nov 27, 2024
6aa3682
add show_anim() implementation
Sfonxu Nov 27, 2024
be99053
fixed formatting errors
Sfonxu Nov 27, 2024
bb385f1
bumped python version and added imageio to requirements
Sfonxu Nov 27, 2024
6e165d4
reversed python version to 3.9
Sfonxu Nov 27, 2024
f23c97d
fixes to pyproject.toml to help with compiling
Sfonxu Nov 27, 2024
177e375
fixes to pyproject.toml to help with compiling
Sfonxu Nov 27, 2024
69dd187
Bump python version of pylint to 3.12
Sfonxu Nov 27, 2024
88e0b60
Temporary edits to setup.py to test workflows interactions
Sfonxu Nov 27, 2024
077bee3
change action tools versions
Sfonxu Nov 27, 2024
215cc18
change setuptools version
Sfonxu Nov 27, 2024
6c3c251
changes to pylint.yml
Sfonxu Nov 27, 2024
02191bc
more temp changes, to be cleaned up
Sfonxu Nov 27, 2024
4dbf906
emptied setup.py
Sfonxu Nov 27, 2024
a480122
force installation of setuptools
Sfonxu Nov 27, 2024
ff41ab5
reverse temp changes to setup.py
Sfonxu Nov 27, 2024
3ee435c
fix setup.py formatting
Sfonxu Nov 27, 2024
3fe6104
add show_anim() to README.md with a short description
Sfonxu Nov 28, 2024
2118d90
add CI wokrflow and fix notebook output
Sfonxu Nov 28, 2024
778f03b
fixes to notebook output
Sfonxu Nov 28, 2024
83e9b77
added line so pylint ignores a warning
Sfonxu Nov 28, 2024
74f68cd
fixed a typo in pylint disable
Sfonxu Nov 28, 2024
175b677
fix yml formatting
Sfonxu Nov 28, 2024
5d18a9c
fixed typo in tests.yml
Sfonxu Nov 28, 2024
4697a35
changed submodule name
Sfonxu Nov 28, 2024
9d0c042
add ls to tests.yml for debuging purposes
Sfonxu Nov 28, 2024
aab1de0
add ls to tests.yml for debuging purposes
Sfonxu Nov 28, 2024
c7f4314
add ls to tests.yml for debuging purposes
Sfonxu Nov 28, 2024
8006c96
add submodules to checkout in tests.yml
Sfonxu Nov 28, 2024
54e999f
fixed typo in tests.yml test section
Sfonxu Nov 28, 2024
5d2ea5c
added requirements.txt to jupyter-utils for workflows usage
Sfonxu Nov 28, 2024
3b35486
actually upload the file this time
Sfonxu Nov 28, 2024
0fae733
added open_atmos_jupyter_utils to modules in tests.yml
Sfonxu Nov 28, 2024
db4bdc5
cleanup README text
slayoo Nov 28, 2024
d9a564f
delete requirements.txt since we have this information in the package…
slayoo Nov 28, 2024
7f470b7
do not do pip install -r requirements.txt (pip install should do it)
slayoo Nov 28, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions .github/workflows/pylint.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,14 +12,14 @@ jobs:
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v2
- name: Set up Python 3.9
uses: actions/setup-python@v2
- uses: actions/checkout@v4
- name: Set up Python 3.12
uses: actions/setup-python@v5
with:
python-version: 3.9
python-version: 3.12
- name: Install dependencies
run: |
python -m pip install --upgrade pip
python -m pip install --upgrade pip setuptools
pip install pylint
python setup.py egg_info
pip install -r *.egg-info/requires.txt
Expand Down
26 changes: 26 additions & 0 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
name: tests

on:
push:
branches: [ main ]
pull_request:
branches: [ main ]

jobs:
test:
runs-on: ubuntu-latest

steps:
- name: Checkout
uses: actions/checkout@v4
with:
submodules: true
- name: Set up Python 3.12
uses: actions/setup-python@v5
with:
python-version: 3.12
- name: Run tests
run: |
pip install -e .
pip install -r tests/devops_tests/requirements.txt
pytest -vv -rP -We tests/devops_tests/test_notebooks.py::test_run_notebooks
3 changes: 3 additions & 0 deletions .gitmodules
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
[submodule "tests/devops_tests.git"]
path = tests/devops_tests
url = [email protected]:open-atmos/devops_tests.git
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@

Utility routines used in Jupyter notebooks in [PySDM](https://github.com/open-atmos/PySDM), [PyMPDATA](https://github.com/open-atmos/PyMPDATA) and [PyPartMC](https://github.com/open-atmos/PyPartMC) projects:
- [``show_plot()``](https://open-atmos.github.io/jupyter-utils/open_atmos_jupyter_utils/show_plot.html) - a drop-in replacement for matplotlib's show() displaying the figure inline using vector graphics (svg) by default and offering a download-as-pdf-or-svg widget just below (on Colab the widget triggers Google Drive download)
- [``show_anim()``](https://open-atmos.github.io/jupyter-utils/open_atmos_jupyter_utils/show_anim.html) - a replacement for matplotlib's FuncAnimate() that inline-displays animations in gif format (thus github renderer compatible) and offers a way to download the .gif file (on Colab the widget triggers Google Drive download)
- [``TemporaryFile``](https://open-atmos.github.io/jupyter-utils/open_atmos_jupyter_utils/temporary_file.html) - a class equipped with ``make_link_widget()`` method returning a click-to-download Colab-compatible widget to be display()-ed in a Jupyter notebook
- [``pip_install_on_colab('package_a', 'package_b', ...)``](https://open-atmos.github.io/jupyter-utils/open_atmos_jupyter_utils/pip_install_on_colab.html) - a function handling execution of ``pip`` (and ``ldconfig``) on Colab

Expand Down
83 changes: 83 additions & 0 deletions examples/show_anim_example.ipynb

Large diffs are not rendered by default.

38 changes: 38 additions & 0 deletions open_atmos_jupyter_utils/show_anim.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
"""inline animation function that allows rendering on github
preview and showing click to download gif button"""

#pylint: disable=consider-using-with
import os
import tempfile
import base64
import imageio
import matplotlib.pyplot as plt
from IPython.display import HTML, display
from open_atmos_jupyter_utils.temporary_file import TemporaryFile


def show_anim(plot_func, frame_range, duration=0.01, loop=0):
"""plot_func is called with one argument - the frame number from frame_range
and is expected to return a matplotlib figure instance (on which savefig()
and close() are subsequently called)"""
gif_file = TemporaryFile(suffix=".gif")
with tempfile.TemporaryDirectory() as tmpdirname:
for frame in frame_range:
fig = plot_func(frame)
fig.savefig(f"{tmpdirname}/{frame:05d}.png")
plt.close(fig)
__merge_images_into_gif(tmpdirname, gif_file, duration, loop)

b64 = base64.b64encode(open(gif_file.basename,'rb').read()).decode("ascii")
display(HTML(f'<img src="data:image/gif;base64,{b64}" />'))
display(gif_file.make_link_widget())


def __merge_images_into_gif(image_folder, gif_name, duration, loop):
"""creates a GIF file from a series of animation frames"""
with imageio.get_writer(
gif_name.basename, duration=duration, loop=loop, mode="I"
) as writer:
for filename in sorted(os.listdir(image_folder)):
image = imageio.v3.imread(os.path.join(image_folder, filename))
writer.append_data(image)
17 changes: 16 additions & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,9 +1,24 @@
[tool.setuptools]

[tool.setuptools_scm]

# TODO
[build-system]
requires = ['setuptools==56.0.0', 'setuptools-scm==6.0.1']
requires = ['setuptools==75.3.0', 'setuptools-scm==8.1.0']
build-backend = "setuptools.build_meta"

[project]
name = 'open-atmos-jupyter-utils'
description = "utility routines used in PySDM, PyMPDATA and PyPartMC examples and tests"
license = {text = "GPL-3.0"}
readme = "README.md"
dynamic = ["version"]
dependencies = [
"ipywidgets",
"IPython",
"matplotlib",
"imageio",
]
[project.urls]
"Homepage" = "https://github.com/open-atmos/jupyter-utils"
"Source" = "https://github.com/open-atmos/jupyter-utils"
Expand Down
21 changes: 8 additions & 13 deletions setup.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
"""
the magick behind ``pip install ...``
"""

from setuptools import setup, find_packages


Expand All @@ -12,19 +13,13 @@ def get_long_description():


setup(
name='open-atmos-jupyter-utils',
description='utility routines used in PySDM, PyMPDATA and PyPartMC examples and tests',
use_scm_version={
"local_scheme": lambda _: "",
"version_scheme": "post-release"
},
setup_requires=['setuptools_scm'],
install_requires=['ipywidgets', 'IPython', 'matplotlib'],
author='https://github.com/open-atmos/jupyter-utils/graphs/contributors',
author_email='[email protected]',
license="GPL-3.0",
packages=find_packages(include=['open_atmos_jupyter_utils', 'open_atmos_jupyter_utils.*']),
use_scm_version={"local_scheme": lambda _: "", "version_scheme": "post-release"},
setup_requires=["setuptools_scm", "setuptools"],
install_requires=["ipywidgets", "IPython", "matplotlib", "imageio"],
packages=find_packages(
include=["open_atmos_jupyter_utils", "open_atmos_jupyter_utils.*"]
),
long_description=get_long_description(),
long_description_content_type="text/markdown",
url='https://github.com/open-atmos/jupyter-utils',
url="https://github.com/open-atmos/jupyter-utils",
)
1 change: 1 addition & 0 deletions tests/devops_tests
Submodule devops_tests added at 8eb4d9