Python package policies: Difference between revisions

From Alpine Linux
mNo edit summary
(→‎Package template: Remove reference to python2 packages)
 
(25 intermediate revisions by 9 users not shown)
Line 1: Line 1:
Python packages in Alpine Linux should follow a general set of standards for their APKBUILDs.
Python packages in Alpine Linux should follow a general set of standards for their APKBUILDs.


{{Draft|Pending consensus on this approach}}
{{Draft|Pending consensus on this approach, some discussion is taking place on the [[Talk:Python_package_policies|talk page]]}}


== Guidelines ==
== Guidelines ==


* Prefix Python 3 libraries with py3-, and Python 2 libraries with py2-. Do not prefix programs (distinct from libraries) at all.
* Prefix Python 3 libraries with <code>py3-</code>. Do not prefix programs (distinct from libraries) at all.


== General Template ==
== General Template ==
Line 12: Line 12:


* Update maintainer to yourself
* Update maintainer to yourself
* Set the pkgname to the Alpine package name (prefixed with "py-")
* Set the pkgname to the Alpine package name (prefixed with "py3-")
* Set _pyname to the name of the package on PyPI
* Set _pyname (see [[Talk:Python_package_policies|talk page]]) to the name of the package on PyPI  
* Update the version number, pkgdesc, url, and license
* Update the version number, pkgdesc, url, and license
* Build it and address any issues that come up
* Build it and address any issues that come up
* Read the build output and be vigilant for issues listed in the following sections - the build may complete successfully even though these issues are present
* Read the build output and be vigilant for issues listed in the following sections - the build may complete successfully even though these issues are present


=== Python 3 only ===
=== Package template ===
 
Note that if you are removing python2 support from a package which previously had it, you should add <pre>replaces="py2-example"</pre> as well. If the old package was a split package, also add <pre>replaces="py-example"</pre>.


<pre># Maintainer: Joe Bloe <joe@example.org>
<pre># Maintainer: Joe Bloe <joe@example.org>
Line 28: Line 26:
pkgrel=0
pkgrel=0
pkgdesc="Example Python package"
pkgdesc="Example Python package"
url=https://example.org
url="https://example.org"
arch=noarch
arch="noarch"
license=MIT
license="MIT"
depends="python3"
depends="python3"
makedepends="py3-setuptools"
makedepends="py3-setuptools"
_pypiprefix="${_pyname%${_pyname#?}}"
_pypiprefix="${_pyname%${_pyname#?}}"
source="https://files.pythonhosted.org/packages/source/$_pypiprefix/$_pyname/$_pyname-$pkgver.tar.gz"
source="https://files.pythonhosted.org/packages/source/$_pypiprefix/$_pyname/$_pyname-$pkgver.tar.gz"
builddir=$srcdir/$_pyname-$pkgver
builddir="$srcdir/$_pyname-$pkgver"
 
build() {
cd "$builddir"
python3 setup.py build
}
 
check() {
cd "$builddir"
python3 setup.py test
}
 
package() {
python3 setup.py install --prefix=/usr --root="$pkgdir"
}</pre>
 
=== Python 2 & 3 ===
 
Specify all of the runtime dependencies for both Python 2 & 3 in the makedepends.
 
<pre># Maintainer: Joe Bloe <joe@example.org>
pkgname=py-alpine-name
_pyname=pypi-name
pkgver=1.2.3
pkgrel=0
pkgdesc="Example Python package"
url=https://example.org
arch=noarch
license=MIT
subpackages="py2-${pkgname#py-}:_py2 py3-${pkgname#py-}:_py3"
makedepends="python2 python3 py-setuptools"
_pypiprefix="${_pyname%${_pyname#?}}"
source="https://files.pythonhosted.org/packages/source/$_pypiprefix/$_pyname/$_pyname-$pkgver.tar.gz"
builddir=$srcdir/$_pyname-$pkgver
 
prepare() {
cp -r "$builddir" "$builddir"-py2
}


build() {
build() {
cd "$builddir"
python3 setup.py build
python3 setup.py build
cd "$builddir"-py2
python2 setup.py build
}
}


check() {
check() {
cd "$builddir"
python3 setup.py test
python3 setup.py test
cd "$builddir"-py2
python2 setup.py test
}
}


package() {
package() {
mkdir -p "$pkgdir"
python3 setup.py install --root="$pkgdir" --skip-build
}
 
_py2() {
cd "$builddir"-py2
_py python2
}
 
_py3() {
cd "$builddir"
_py python3
}
 
_py() {
_python="$1"
pkgdesc="$pkgdesc (for $_python)"
depends="$depends $_python"
install_if="$pkgname=$pkgver-r$pkgrel $_python"
$_python setup.py install --prefix=/usr --root="$subpkgdir"
}</pre>
}</pre>


Line 124: Line 61:
Watch out for this and be sure to add any of these packages to checkdepends so that setuptools isn't downloading and testing against packages/versions which aren't in aports.
Watch out for this and be sure to add any of these packages to checkdepends so that setuptools isn't downloading and testing against packages/versions which aren't in aports.


=== Different dependencies between py2 and py3 versions ===
=== Upstream uses 'tox' to run tests ===


Add depends="..." to the respective _py2 or _py3 subpackage functions.
tox (packaged as {{Pkg|py3-tox}}) downloads all dependencies into a virtual environment. This bypasses the system packages negating a lot of the value in these tests.
 
You can use <code>--sitepackages</code> on your invocation of tox in the check() phase so it uses the system packages instead of downloading ones into a virtual environment.
 
=== setup.py does not exist ===
 
In case a package uses a <code>pyproject.toml</code> instead of <code>setup.py</code>, you can use <code>py3-gpep517</code> and <code>py3-installer</code>. See {{Pkg|py3-rich}} for an example. Also note how to run tests against the built package in that example.
 
Many projects require either <code>py3-poetry-core</code>/<code>py3-flit-core</code> or just <code>py3-wheel</code>/<code>py3-setuptools</code> in makedepends alongside <code>py3-gpep517</code> to build successfully (check the <code>build-backend</code> in <code>pyproject.toml</code> to confirm the exact requirements).


== Alpine+Python projects ==
== Alpine+Python projects ==
Line 136: Line 81:
TODO: obtain consensus and organize this work.
TODO: obtain consensus and organize this work.


=== Python 2 deprecation ===
[[category: python]]
 
{{Draft|Pending consensus on this approach}}
 
The general approach to Python 2 deprecation and removal requires three broad steps:
 
# Do not add new python 2 packages to aports, effective immediately.
# In the course of the aports normalization project, drop Python 2 support if it's easy or triage it and make a note for later if not.
# Making judgement calls for difficult packages on a case-by-case basis, based on the upstream's progress/amicability towards a Python 3 port. Upstreams which are unwilling to port to Python 3 should be removed from aports.
 
TODO: Prepare some discussion place, git repos, scripts, etc, to organize this work.

Latest revision as of 13:18, 26 February 2024

Python packages in Alpine Linux should follow a general set of standards for their APKBUILDs.

This material is work-in-progress ...

Pending consensus on this approach, some discussion is taking place on the talk page
(Last edited by WhyNotHugo on 26 Feb 2024.)

Guidelines

  • Prefix Python 3 libraries with py3-. Do not prefix programs (distinct from libraries) at all.

General Template

Be sure to make the following changes:

  • Update maintainer to yourself
  • Set the pkgname to the Alpine package name (prefixed with "py3-")
  • Set _pyname (see talk page) to the name of the package on PyPI
  • Update the version number, pkgdesc, url, and license
  • Build it and address any issues that come up
  • Read the build output and be vigilant for issues listed in the following sections - the build may complete successfully even though these issues are present

Package template

# Maintainer: Joe Bloe <joe@example.org>
pkgname=py3-alpine-name
_pyname=pypi-name
pkgver=1.2.3
pkgrel=0
pkgdesc="Example Python package"
url="https://example.org"
arch="noarch"
license="MIT"
depends="python3"
makedepends="py3-setuptools"
_pypiprefix="${_pyname%${_pyname#?}}"
source="https://files.pythonhosted.org/packages/source/$_pypiprefix/$_pyname/$_pyname-$pkgver.tar.gz"
builddir="$srcdir/$_pyname-$pkgver"

build() {
	python3 setup.py build
}

check() {
	python3 setup.py test
}

package() {
	python3 setup.py install --root="$pkgdir" --skip-build
}

Common issues

No source package available (only the wheel is on PyPI)

Seek out the upstream source (e.g. GitHub) and swap out the URL.

No tests in PyPI package "(0 tests run)"

Seek out the upstream source (e.g. GitHub) and swap out the URL.

setup.py test downloads a lot of dependencies

Watch out for this and be sure to add any of these packages to checkdepends so that setuptools isn't downloading and testing against packages/versions which aren't in aports.

Upstream uses 'tox' to run tests

tox (packaged as py3-tox) downloads all dependencies into a virtual environment. This bypasses the system packages negating a lot of the value in these tests.

You can use --sitepackages on your invocation of tox in the check() phase so it uses the system packages instead of downloading ones into a virtual environment.

setup.py does not exist

In case a package uses a pyproject.toml instead of setup.py, you can use py3-gpep517 and py3-installer. See py3-rich for an example. Also note how to run tests against the built package in that example.

Many projects require either py3-poetry-core/py3-flit-core or just py3-wheel/py3-setuptools in makedepends alongside py3-gpep517 to build successfully (check the build-backend in pyproject.toml to confirm the exact requirements).

Alpine+Python projects

aports normalization project

Many Python packages in aports (if not most) do not follow these guidelines.

TODO: obtain consensus and organize this work.

Retrieved from "https://wiki.alpinelinux.org/w/index.php?title=Python_package_policies&oldid=26482"