Python package policies: Difference between revisions

Revision as of 15:24, 15 August 2022

Python packages in Alpine Linux should follow a general set of standards for their APKBUILDs.

This material is work-in-progress ...

Pending consensus on this approach, some discussion is taking place on the talk page
(Last edited by Psykose on 15 Aug 2022.)

Guidelines

Prefix Python 3 libraries with py3-. Do not prefix programs (distinct from libraries) at all.

General Template

Be sure to make the following changes:

Update maintainer to yourself
Set the pkgname to the Alpine package name (prefixed with "py3-")
Set _pyname (see talk page) to the name of the package on PyPI
Update the version number, pkgdesc, url, and license
Build it and address any issues that come up
Read the build output and be vigilant for issues listed in the following sections - the build may complete successfully even though these issues are present

Package template

Note that if you are removing python2 support from a package which previously had it, you should add

replaces="py2-example"

as well. If the old package was a split package, also add

replaces="py-example"
provides="py-example=$pkgver-r$pkgrel"

.

# Maintainer: Joe Bloe <joe@example.org>
pkgname=py3-alpine-name
_pyname=pypi-name
pkgver=1.2.3
pkgrel=0
pkgdesc="Example Python package"
url="https://example.org"
arch="noarch"
license="MIT"
depends="python3"
makedepends="py3-setuptools"
_pypiprefix="${_pyname%${_pyname#?}}"
source="https://files.pythonhosted.org/packages/source/$_pypiprefix/$_pyname/$_pyname-$pkgver.tar.gz"
builddir="$srcdir/$_pyname-$pkgver"

build() {
	python3 setup.py build
}

check() {
	python3 setup.py test
}

package() {
	python3 setup.py install --root="$pkgdir" --skip-build
}

Common issues

No source package available (only the wheel is on PyPI)

Seek out the upstream source (e.g. GitHub) and swap out the URL.

No tests in PyPI package "(0 tests run)"

Seek out the upstream source (e.g. GitHub) and swap out the URL.

setup.py test downloads a lot of dependencies

Watch out for this and be sure to add any of these packages to checkdepends so that setuptools isn't downloading and testing against packages/versions which aren't in aports.

You need to use the 'tox' utility to run tests

tox (knows as py3-tox on Alpine Linux) downloads all dependencies into a virtual environment by default, which bypasses the system tooling, making testing somewhat less useful.

Remember to use *--sitepackages* on your invocation of tox in the check() phase so it uses the system packages instead of downloading ones into a virtual environment.

setup.py does not exist

In case a package uses a pyproject.toml instead of setup.py, you can use py3-build+py3-installer. See community/py3-rich for an example. Things usually need either py3-poetry-core/py3-flit-core or just py3-wheel/py3-setuptools in makedepends alongside py3-build to build successfully (read the pyproject.toml `build-backend` to figure out which). Also note how to run tests against the built package in that example.

Alpine+Python projects

aports normalization project

Many Python packages in aports (if not most) do not follow these guidelines.

TODO: obtain consensus and organize this work.

Python 2 deprecation

This material is work-in-progress ...

Pending consensus on this approach
(Last edited by Psykose on 15 Aug 2022.)

The general approach to Python 2 deprecation and removal requires three broad steps:

Do not add new python 2 packages to aports, effective immediately.
In the course of the aports normalization project, drop Python 2 support if it's easy or triage it and make a note for later if not.
Making judgement calls for difficult packages on a case-by-case basis, based on the upstream's progress/amicability towards a Python 3 port. Upstreams which are unwilling to port to Python 3 should be removed from aports.

TODO: Prepare some discussion place, git repos, scripts, etc, to organize this work.

@@ Line 1: / Line 1: @@
 Python packages in Alpine Linux should follow a general set of standards for their APKBUILDs.
-{{Draft|Pending consensus on this approach}}
+{{Draft|Pending consensus on this approach, some discussion is taking place on the [[Talk:Python_package_policies|talk page]]}}
+== Guidelines ==
+* Prefix Python 3 libraries with py3-. Do not prefix programs (distinct from libraries) at all.
 == General Template ==
-This APKBUILD template includes a split py2-/py3- package and a py- metapackage, and downloads the source package from PyPI. Be sure to make the following changes:
+Be sure to make the following changes:
 * Update maintainer to yourself
-* Set the pkgname to the Alpine package name (prefixed with "py-")
+* Set the pkgname to the Alpine package name (prefixed with "py3-")
-* Set _pkgname to the name of the package on PyPI
+* Set _pyname (see [[Talk:Python_package_policies|talk page]]) to the name of the package on PyPI
 * Update the version number, pkgdesc, url, and license
 * Build it and address any issues that come up
 * Read the build output and be vigilant for issues listed in the following sections - the build may complete successfully even though these issues are present
+=== Package template ===
+Note that if you are removing python2 support from a package which previously had it, you should add <pre>replaces="py2-example"</pre> as well. If the old package was a split package, also add <pre>replaces="py-example"
+provides="py-example=$pkgver-r$pkgrel"</pre>.
 <pre># Maintainer: Joe Bloe <joe@example.org>
-pkgname=py-alpine-name
+pkgname=py3-alpine-name
-_pkgname=pypi-name
+_pyname=pypi-name
 pkgver=1.2.3
 pkgrel=0
 pkgdesc="Example Python package"
-url=http://example.org/
+url="https://example.org"
-arch=noarch
+arch="noarch"
-license=MIT
+license="MIT"
-subpackages="py2-${pkgname#py-}:_py2 py3-${pkgname#py-}:_py3"
+depends="python3"
-makedepends="py-setuptools"
+makedepends="py3-setuptools"
-source="https://files.pythonhosted.org/packages/source/${_pkgname:0:1}/$_pkgname/$_pkgname-$pkgver.tar.gz"
+_pypiprefix="${_pyname%${_pyname#?}}"
-builddir=$srcdir/$_pkgname-$pkgver
+source="https://files.pythonhosted.org/packages/source/$_pypiprefix/$_pyname/$_pyname-$pkgver.tar.gz"
+builddir="$srcdir/$_pyname-$pkgver"
-prepare() {
-	cp -r "$builddir" "$builddir"-py3
-}
 build() {
-	cd "$builddir"
-	python2 setup.py build
-	cd "$builddir"-py3
 	python3 setup.py build
 }
 check() {
-	cd "$builddir"
-	python2 setup.py test
-	cd "$builddir"-py3
 	python3 setup.py test
 }
 package() {
-	mkdir -p "$pkgdir"
+	python3 setup.py install --root="$pkgdir" --skip-build
-}
-_py2() {
-	cd "$builddir"
-	_py python2
-}
-_py3() {
-	cd "$builddir"-py3
-	_py python3
-}
-_py() {
-	python="$1"
-	pkgdesc="$pkgdesc (for $python)"
-	depends="$depends $python"
-	install_if="$pkgname=$pkgver-r$pkgrel $python"
-	$python setup.py install --prefix=/usr --root="$subpkgdir"
 }</pre>
@@ Line 82: / Line 64: @@
 Watch out for this and be sure to add any of these packages to checkdepends so that setuptools isn't downloading and testing against packages/versions which aren't in aports.
-=== Different dependencies between py2 and py3 versions ===
+=== You need to use the 'tox' utility to run tests ===
+tox (knows as py3-tox on Alpine Linux) downloads all dependencies into a virtual environment by default, which bypasses the system tooling, making
+testing somewhat less useful.
+Remember to use *--sitepackages* on your invocation of tox in the check() phase so it uses the system packages instead of downloading ones into
+a virtual environment.
-Add depends="..." to the respective _py2 or _py3 subpackage functions.
+=== setup.py does not exist ===
-=== No python2 support ===
+In case a package uses a pyproject.toml instead of setup.py, you can use py3-build+py3-installer. See community/py3-rich for an example. Things usually need either py3-poetry-core/py3-flit-core or just py3-wheel/py3-setuptools in makedepends alongside py3-build to build successfully (read the pyproject.toml `build-backend` to figure out which). Also note how to run tests against the built package in that example.
-In this case, rename the package to "py-example" and remove the -py2 subpackage. Merge the _py(), _py3,() and package() functions. Remove the prepare() function and drop the -py3 suffix from the builddir. An updated py3-only template is provided here:
+== Alpine+Python projects ==
-<pre># Maintainer: Joe Bloe <joe@example.org>
+=== aports normalization project ===
-pkgname=py-alpine-name
-_pkgname=pypi-name
-pkgver=1.2.3
-pkgrel=0
-pkgdesc="Example Python package"
-url=http://example.org/
-arch=noarch
-license=MIT
-depends="python3"
-makedepends="py-setuptools"
-source="https://files.pythonhosted.org/packages/source/${_pkgname:0:1}/$_pkgname/$_pkgname-$pkgver.tar.gz"
-builddir=$srcdir/$_pkgname-$pkgver
-build() {
+Many Python packages in aports (if not most) do not follow these guidelines.
-	cd "$builddir"
-	python3 setup.py build
-}
-check() {
+TODO: obtain consensus and organize this work.
-	cd "$builddir"
-	python3 setup.py test
-}
-package() {
+=== Python 2 deprecation ===
-	python3 setup.py install --prefix=/usr --root="$pkgdir"
-}</pre>
-Note that if you are removing python2 support from a package which previously had it, you should add <pre>replaces="py3-example"</pre> as well.
+{{Draft|Pending consensus on this approach}}
-== Future work ==
+The general approach to Python 2 deprecation and removal requires three broad steps:
-=== aports normalization project ===
+# Do not add new python 2 packages to aports, effective immediately.
+# In the course of the aports normalization project, drop Python 2 support if it's easy or triage it and make a note for later if not.
+# Making judgement calls for difficult packages on a case-by-case basis, based on the upstream's progress/amicability towards a Python 3 port. Upstreams which are unwilling to port to Python 3 should be removed from aports.
-Many Python packages in aports (if not most) do not follow these guidelines. TODO: figure out an approach and obtain consensus.
+TODO: Prepare some discussion place, git repos, scripts, etc, to organize this work.
-=== Python 2 deprecation ===
-Python 2 is EoL in less than a year. Let's get rid of it. TODO: figure out an approach and obtain consensus.
+[[category: python]]