diff --git a/03_building_and_packaging/pypi_exercise.md b/03_building_and_packaging/pypi_exercise.md index 41415ef0..6303d935 100644 --- a/03_building_and_packaging/pypi_exercise.md +++ b/03_building_and_packaging/pypi_exercise.md @@ -2,8 +2,8 @@ ## Starting remarks -- [Exercise repository link](https://github.com/Simulation-Software-Engineering/diffusion2d) -- Deadline for submitting this exercise is **Thursday 3rd November 09:00**. +- [Exercise repository link](https://github.com/Simulation-Software-Engineering/diffusion2D) +- Deadline for submitting this exercise is **Wednesday 20th November 09:00**. - The code in this exercise produces plots and in order to view them you need to use a GUI-based operating system or environment. ## Brief idea of the exercise diff --git a/03_building_and_packaging/pypi_slides.md b/03_building_and_packaging/pypi_slides.md index ad576b7d..caaa16f9 100644 --- a/03_building_and_packaging/pypi_slides.md +++ b/03_building_and_packaging/pypi_slides.md @@ -31,17 +31,9 @@ slideOptions: --- -## What is pip? - -- pip is a package installer to install packages from the [Python Package Index PyPI]((https://pypi.org/)). -- pip is itself a [package](https://pypi.org/project/pip/), which is available on PyPI. -- pip is open-source and is developed on [GitHub](https://github.com/pypa/pip). - ---- - -## Python packaging is itself evolving +## Python packaging standard is continuously evolving -Files which are commonly seen: +Commonly seen files: `setup.py`, `setup.cfg`, `pyproject.toml`, `requirements.txt`. @@ -56,30 +48,31 @@ All are files which packaging-related tools consume. What do these files do? - Packaging workflows are also standardized through PEPs. Examples are - [PEP 427](https://www.python.org/dev/peps/pep-0427/) which introduces the built-package format "wheel". - [PEP 518](https://peps.python.org/pep-0518/) which introduces a configuration file for packaging. -- Tip: read the *Rationale* section (or introduction) of the **PEP** convention of a particular feature. +- Tip: read the *Rationale* section of the **PEP** convention of a particular feature. --- ## Python libraries used to install packages -- `disutils`: old and deprecated, to be phased out in Python 3.12. +- `disutils`: old and deprecated ([PEP 632](https://peps.python.org/pep-0632/)). - `setuptools`: actively maintained packaging tool which is shipped with Python (built on top of `disutils`). +- Many more options, but `setuptools` is lowest common denominator. --- ## setup.py - setup.cfg - pyproject.toml - Names of all these files are standardized. -- `setup.py` is the interface to the command line. Needs to be at the root of the repository. -- `setup.cfg` has metadata of all the options that can also be specified in `setup.py`. -- `pyproject.toml` is similar to `setup.cfg` but has additionally the `build-system` table. +- `setup.py` is a script which uses `setuptools`. Needs to be at the root of the repository. +- `setup.cfg` is a config file which has metadata of all the options that can also be specified in `setup.py`. +- `pyproject.toml` has logic and metadata necessary to build and package. --- ## Comparison of various approaches - `setup.py` has been widely popular but main limitation is that it cannot be executed without knowing its dependencies. *Chicken and egg* problem regarding dependencies. -- Does `setup.cfg` solve the dependencies problem? No, because packaging tools can directly read dependencies from it. +- Does `setup.cfg` solve the dependencies problem? No, because no packaging tool can directly read dependencies from it. - Solution is to use an additional `pyproject.toml` with the `[build-system]` table specified. - [PyPA sample project](https://github.com/pypa/sampleproject) shows an example using all three files. @@ -95,28 +88,30 @@ import setuptools setup( name="package-name", - version="0.0.1", - author="", + version="", + author="Your Name", description="A small description", url="package-website-url", package_dir={"": ""}, packages=setuptools.find_packages(where=""), - python_requires=">=3.6", install_requires=[""] + entry_points={ + 'console_scripts': ['package-import-name = '] + } ) ``` --- -## Using setup.cfg +## Using setup.cfg and setup.py Entries moved to `setup.cfg` would look like: ```python [metadata] name="package-name" -version="0.0.1" -author="" +version="" +author="Your Name" url="package-website-url" description="A small description" @@ -124,7 +119,19 @@ description="A small description" packages = find: install_requires = "" - python_version>"3.6" + +[options.entry_points] +console_scripts = + executable-name = +``` + +A nominal `setup.py` is still required + +```python +from setuptools import setup + +if __name__ == "__main__": + setup() ``` --- @@ -139,31 +146,21 @@ Example `pyproject.toml` can look like ```python [build-system] -requires = ["setuptools"] -build-backend = "setuptools.build_meta" +requires = ["setuptools", "wheel"] [project] name = "package-name" description = "A small description" readme = "README.md" -version = "0.0.1" -requires-python = ">=3.6" keywords = ["keyword1", "keyword2"] -license = {text = "BSD License"} -dependencies = [] -``` - ---- - -## Minimal setup.py is still required - -A minimal `setup.py` is required along with `setup.cfg` or `pyproject.toml` - -```python -from setuptools import setup - -if __name__ == "__main__": - setup() +classifiers = [ + "Programming Language :: Python :: 3" +] +dependencies = [ + "requests", + 'importlib-metadata; python_version<"3.8"', +] +dynamic = [""] ``` --- @@ -187,11 +184,86 @@ Is there a better way? Yes! Use pip! --- +## What is pip? + +- pip is a package installer to install packages from the [Python Package Index PyPI]((https://pypi.org/)). +- pip is itself a [package](https://pypi.org/project/pip/) which is available on PyPI. +- pip is open-source and is developed on [GitHub](https://github.com/pypa/pip). + +--- + +## Using pip 1/2 + +Executing: + +```bash +pip install package-name +``` + +leads to pip choosing a distribution file for the package and installing it in the environment. + +```bash +python -m pip install package-name +``` + +is basically the same as: + +```bash +pip install package-name +``` + +--- + +## Using pip 2/2 + +- pip tracks metadata to allow for easy updating and uninstalling of packages. +- pip is bundled together with Python 3.x, making it even easier to use. +- pip can install a package from a source distribution (`.tar.gz`) or a wheel distribution (`.whl`). + +**Important**: Do not use + +```bash +sudo pip install +``` + +Various security issues with doing so! Go for a venv or + +```bash +pip install --user +``` + +--- + +## pip vs. pipx + +- pip installs packages in the global namespace. +- pipx installs packages in individual virtual environments. +- pipx is meant to be used for applications run directly via the command line. +- For libraries, the recommended way is using pip in a virtual environment. + +--- + +## Using pip + +Uninstall a package + +```bash +pip uninstall +``` + +Update a package + +```bash +pip install --upgrade +``` + +--- + ## What is PyPI? - [PyPI](https://pypi.org/) = **Python Package Index** is a repository of software developed in the Python community. - PyPI itself is developed on GitHub through another software called [Warehouse](https://github.com/pypa/warehouse). -- PyPI has a informative [public dashboard](https://p.datadoghq.com/sb/7dc8b3250-85dcf667bd) to show its activity. +- PyPI has an informative [public dashboard](https://p.datadoghq.com/sb/7dc8b3250-85dcf667bd) to show its activity. - A major advantage is the active maintenance of PyPI and the packages indexed in it. - Not to be confused with **PyPA** which is Python Packaging Authority, a working group which maintains projects in Python packaging. @@ -203,49 +275,41 @@ Is there a better way? Yes! Use pip! --- -## Folder structure for packaging 1/3 +## File structure for code packaging 1/2 Four places where naming is relevant: - Name of the repository (on GitHub or GitLab). - Name of the folder which has the source code. -- Name of the package as seen my PyPI. +- Name of the package as seen by PyPI. - Name of the package to be used in the `import` statement. - -Tip: Keep the last three names the same. - - -**All three names are independent of each other.** - ---- - -## Folder structure for packaging 2/3 +**All names are independent of each other.** Example folder structure: ```bash -repository-name/ -└── package-name/ +generic_folder_name/ +└── src/ ├── __init__.py └── source-code.py ``` -- The file `__init__.py` is required to import the `package_name/` package as a package. This file is mostly empty +- The file `__init__.py` is required to import the `package_name/` as a package. This file is mostly empty - `source-code.py` contains the code. It can be multiple files. --- -## Folder structure for packaging 3/3 +## File structure for code packaging 2/2 - Once the file structuring is complete, the repository will look like: ```bash -repository-name/ +generic_folder_name/ ├── LICENSE ├── setup.py ├── README.md -├── package-name/ +├── src/ │ ├── __init__.py │ └── source-code.py └── tests/ @@ -302,6 +366,9 @@ The option `entry_points` provide metadata which are exposed after installation. ```python from setuptools import setup +with open("README.md", "r", encoding="utf-8") as fh: + long_description = fh.read() + setup( ... entry_points={ @@ -358,7 +425,7 @@ Both the commands must be run in the same directory as `setup.py`. - Secure authentication of the user to PyPI over HTTPS using a verified connection. - Its predecessor `python setup.py upload` required careful configuration. - Encourages users to create distribution files to promote testing before releasing. -- The archive files are uploaded to a package index from where pip can get them +- The archive files are uploaded to a package index from where pip can get them. --- @@ -410,25 +477,6 @@ pip install --user --index-url https://test.pypi.org/simple/ - Conda is configured to work with [Anaconda installers and packages](https://repo.anaconda.com/). - Conda is often the preferred way to run Python packages on Windows and MacOS. See example in [FEniCS on Anaconda section](https://fenicsproject.org/download/). ---- - -## Difference between Conda and pip - - - -| Feature | Conda | pip | -| ------- | ----- | --- | -| Multi-language dependency | Yes | No | -| Package installation | Anaconda installer / download binaries | build / wheels | -| Package availability | ~8000 | ~336,000 | -| Virtual environment functionality | In-built environment management | No, but support for `virtualenv` | - ---- - ## Be careful while using Python environments