Apk spec - Revision history

Zcrayfish: Fix a wikilink... Use path template in a few places, as appropriate.

2024-10-14T22:34:37Z

Fix a wikilink... Use path template in a few places, as appropriate.

← Older revision		Revision as of 22:34, 14 October 2024
Line 1:		Line 1:
	{{Draft}}		{{Draft}}

	For end-user facing documentation about apk, check out the [[~~Alpine_Package_Keeper\|~~Alpine Package Keeper]] page.		For end-user facing documentation about apk, check out the [[Alpine Package Keeper]] page.

	This page is an attempt to document the internal data structures of the apk package manager. The canonical implementation of the apk format is [https://gitlab.alpinelinux.org/alpine/apk-tools apk-tools] and much of this information is gleaned from reading the source code.		This page is an attempt to document the internal data structures of the apk package manager. The canonical implementation of the apk format is [https://gitlab.alpinelinux.org/alpine/apk-tools apk-tools] and much of this information is gleaned from reading the source code.
Line 19:		Line 19:
	The signature file is a DER encoded PKCS1v15 RSA signature of the SHA1 hash of the control tar segment gzip stream. The filename has the format <tt>.SIGN.RSA.<key_name>.rsa.pub</tt> (for example <tt>.SIGN.RSA.alpine-devel@lists.alpinelinux.org-5261cecb.rsa.pub</tt>). This file is placed inside of a tar record with permissions 0644, uid 0, and gid 0. This tar record (lacking end-of-tar records) is gzip compressed, forming a signature tar segment, and is concatenated onto the front of the combined control and data segments. [https://gitlab.alpinelinux.org/alpine/abuild/-/blob/master/abuild-sign.in abuild-sign] is responsible for generating these signature segments.		The signature file is a DER encoded PKCS1v15 RSA signature of the SHA1 hash of the control tar segment gzip stream. The filename has the format <tt>.SIGN.RSA.<key_name>.rsa.pub</tt> (for example <tt>.SIGN.RSA.alpine-devel@lists.alpinelinux.org-5261cecb.rsa.pub</tt>). This file is placed inside of a tar record with permissions 0644, uid 0, and gid 0. This tar record (lacking end-of-tar records) is gzip compressed, forming a signature tar segment, and is concatenated onto the front of the combined control and data segments. [https://gitlab.alpinelinux.org/alpine/abuild/-/blob/master/abuild-sign.in abuild-sign] is responsible for generating these signature segments.

	Apk will verify a signature by looking in ~~<tt>~~/etc/apk/keys~~</tt>~~ for a file called ''exactly'' ~~<tt>~~<key_name>.rsa.pub~~</tt>~~ in ~~<tt>~~/etc/apk/keys~~</tt>~~. If no such file is found, verification fails.		Apk will verify a signature by looking in {{path\|/etc/apk/keys}} for a file called ''exactly'' {{path\|<key_name>.rsa.pub}} in {{path\|/etc/apk/keys}}. If no such file is found, verification fails.

	The control segment contains the package metadata in a ~~<tt>~~.PKGINFO~~</tt>~~ file as well as all of the scripts (if any) that are used by apk during installation and removal of the package. For historical reasons all files in the control tar segment are prefixed with a dot (<tt>.</tt>). The control segment is constructed by placing each file for the package into a tar record, concatenating those tar records, gzipping the tar records, and concatenating them onto the front of the data tarball. The SHA1 hash of this gzip stream is used as the checksum <tt>C:</tt> field in the APKINDEX file.		The control segment contains the package metadata in a {{path\|.PKGINFO}} file as well as all of the scripts (if any) that are used by apk during installation and removal of the package. For historical reasons all files in the control tar segment are prefixed with a dot (<tt>.</tt>). The control segment is constructed by placing each file for the package into a tar record, concatenating those tar records, gzipping the tar records, and concatenating them onto the front of the data tarball. The SHA1 hash of this gzip stream is used as the checksum <tt>C:</tt> field in the APKINDEX file.

	The data tarball is a standard gzipped tarball with extra PAX headers that contain the SHA1 hash of each file in the tar header for that file. The hash is contained in a header called ~~<tt>~~APK-TOOLS.checksum.SHA1~~</tt>~~. Unlike the other tar streams this tarball does contain the two end-of-tar null records. It is always the final segment of an APK package. Hashes are added with the [https://gitlab.alpinelinux.org/alpine/abuild/-/blob/master/abuild-tar.c abuild-tar] tool.		The data tarball is a standard gzipped tarball with extra PAX headers that contain the SHA1 hash of each file in the tar header for that file. The hash is contained in a header called {{path\|APK-TOOLS.checksum.SHA1}}. Unlike the other tar streams this tarball does contain the two end-of-tar null records. It is always the final segment of an APK package. Hashes are added with the [https://gitlab.alpinelinux.org/alpine/abuild/-/blob/master/abuild-tar.c abuild-tar] tool.

	== PKGINFO Format ==		== PKGINFO Format ==
Line 142:		Line 142:

	= Installed Database V2 =		= Installed Database V2 =
	The installed database is used by apk to track which packages are installed and what modifications those packages have made to the system. This file is located at ~~<tt>~~/lib/apk/db/installed~~</tt>~~. The installed file is a plaintext file of the same format as APKINDEX (contained in APKINDEX.tar.gz). It is neither compressed nor signed. Each record in the installed file starts with a package index record with the same fields as the APKINDEX file. The installed file adds some additional fields that are defined in [https://gitlab.alpinelinux.org/alpine/apk-tools/-/blob/ff7c8f6ee9dfa2add57b88dc271f6711030e72a0/src/database.c#L937 database.c]. As of July 2022 these additional fields are:		The installed database is used by apk to track which packages are installed and what modifications those packages have made to the system. This file is located at {{path\|/lib/apk/db/installed}}. The installed file is a plaintext file of the same format as APKINDEX (contained in APKINDEX.tar.gz). It is neither compressed nor signed. Each record in the installed file starts with a package index record with the same fields as the APKINDEX file. The installed file adds some additional fields that are defined in [https://gitlab.alpinelinux.org/alpine/apk-tools/-/blob/ff7c8f6ee9dfa2add57b88dc271f6711030e72a0/src/database.c#L937 database.c]. As of July 2022 these additional fields are:

	* <tt>r:</tt> - packages which this package replaces, space separated list		* <tt>r:</tt> - packages which this package replaces, space separated list

EvertHeylen: Add note about the importance of filenames

2024-10-14T10:17:56Z

Add note about the importance of filenames

← Older revision		Revision as of 10:17, 14 October 2024
Line 18:		Line 18:

	The signature file is a DER encoded PKCS1v15 RSA signature of the SHA1 hash of the control tar segment gzip stream. The filename has the format <tt>.SIGN.RSA.<key_name>.rsa.pub</tt> (for example <tt>.SIGN.RSA.alpine-devel@lists.alpinelinux.org-5261cecb.rsa.pub</tt>). This file is placed inside of a tar record with permissions 0644, uid 0, and gid 0. This tar record (lacking end-of-tar records) is gzip compressed, forming a signature tar segment, and is concatenated onto the front of the combined control and data segments. [https://gitlab.alpinelinux.org/alpine/abuild/-/blob/master/abuild-sign.in abuild-sign] is responsible for generating these signature segments.		The signature file is a DER encoded PKCS1v15 RSA signature of the SHA1 hash of the control tar segment gzip stream. The filename has the format <tt>.SIGN.RSA.<key_name>.rsa.pub</tt> (for example <tt>.SIGN.RSA.alpine-devel@lists.alpinelinux.org-5261cecb.rsa.pub</tt>). This file is placed inside of a tar record with permissions 0644, uid 0, and gid 0. This tar record (lacking end-of-tar records) is gzip compressed, forming a signature tar segment, and is concatenated onto the front of the combined control and data segments. [https://gitlab.alpinelinux.org/alpine/abuild/-/blob/master/abuild-sign.in abuild-sign] is responsible for generating these signature segments.

			Apk will verify a signature by looking in <tt>/etc/apk/keys</tt> for a file called ''exactly'' <tt><key_name>.rsa.pub</tt> in <tt>/etc/apk/keys</tt>. If no such file is found, verification fails.

	The control segment contains the package metadata in a <tt>.PKGINFO</tt> file as well as all of the scripts (if any) that are used by apk during installation and removal of the package. For historical reasons all files in the control tar segment are prefixed with a dot (<tt>.</tt>). The control segment is constructed by placing each file for the package into a tar record, concatenating those tar records, gzipping the tar records, and concatenating them onto the front of the data tarball. The SHA1 hash of this gzip stream is used as the checksum <tt>C:</tt> field in the APKINDEX file.		The control segment contains the package metadata in a <tt>.PKGINFO</tt> file as well as all of the scripts (if any) that are used by apk during installation and removal of the package. For historical reasons all files in the control tar segment are prefixed with a dot (<tt>.</tt>). The control segment is constructed by placing each file for the package into a tar record, concatenating those tar records, gzipping the tar records, and concatenating them onto the front of the data tarball. The SHA1 hash of this gzip stream is used as the checksum <tt>C:</tt> field in the APKINDEX file.

Sertonix: use https link

2023-08-25T02:58:19Z

use https link

← Older revision		Revision as of 02:58, 25 August 2023
Line 88:		Line 88:
	= Index Format V2 =		= Index Format V2 =
	== Binary Format ==		== Binary Format ==
	The index is served as [~~http~~://dl-cdn.alpinelinux.org/alpine/edge/main/x86_64/APKINDEX.tar.gz APKINDEX.tar.gz] and is downloaded by apk to power the package database. The index is signed similarly to packages. The main difference between the index and packages is that the index file contains only two segments.		The index is served as [https://dl-cdn.alpinelinux.org/alpine/edge/main/x86_64/APKINDEX.tar.gz APKINDEX.tar.gz] and is downloaded by apk to power the package database. The index is signed similarly to packages. The main difference between the index and packages is that the index file contains only two segments.

	The signature segment is identical to a package segment and is concatenated, in its own gzip stream, to the beginning of the APKINDEX tarball.		The signature segment is identical to a package segment and is concatenated, in its own gzip stream, to the beginning of the APKINDEX tarball.

Ilanschnell: Fixed mistake about what the SHA1 hash input of the RSA signature is.

2023-05-06T04:55:36Z

Fixed mistake about what the SHA1 hash input of the RSA signature is.

← Older revision		Revision as of 04:55, 6 May 2023
Line 17:		Line 17:
	APK v2 packages contain two tar segments followed by a tarball each in their own gzip stream (3 streams total). These streams contain the package signature, control data, and package data. The package data is a tarball of the files contained in a package laid out in a way that allows it to be unpacked at the filesystem root such that all files are placed in the correct location on the system. The control tar segment contains the package metadata along with any install scripts. The signature tar segment contains a single file that is a binary signature over the control segment.		APK v2 packages contain two tar segments followed by a tarball each in their own gzip stream (3 streams total). These streams contain the package signature, control data, and package data. The package data is a tarball of the files contained in a package laid out in a way that allows it to be unpacked at the filesystem root such that all files are placed in the correct location on the system. The control tar segment contains the package metadata along with any install scripts. The signature tar segment contains a single file that is a binary signature over the control segment.

	The signature file is a DER encoded PKCS1v15 RSA signature of the SHA1 hash of the ~~concatenated~~ control ~~and data~~ gzip ~~streams~~. The filename has the format <tt>.SIGN.RSA.<key_name>.rsa.pub</tt> (for example <tt>.SIGN.RSA.alpine-devel@lists.alpinelinux.org-5261cecb.rsa.pub</tt>). This file is placed inside of a tar record with permissions 0644, uid 0, and gid 0. This tar record (lacking end-of-tar records) is gzip compressed, forming a signature tar segment, and is concatenated onto the front of the combined control and data segments. [https://gitlab.alpinelinux.org/alpine/abuild/-/blob/master/abuild-sign.in abuild-sign] is responsible for generating these signature segments.		The signature file is a DER encoded PKCS1v15 RSA signature of the SHA1 hash of the control tar segment gzip stream. The filename has the format <tt>.SIGN.RSA.<key_name>.rsa.pub</tt> (for example <tt>.SIGN.RSA.alpine-devel@lists.alpinelinux.org-5261cecb.rsa.pub</tt>). This file is placed inside of a tar record with permissions 0644, uid 0, and gid 0. This tar record (lacking end-of-tar records) is gzip compressed, forming a signature tar segment, and is concatenated onto the front of the combined control and data segments. [https://gitlab.alpinelinux.org/alpine/abuild/-/blob/master/abuild-sign.in abuild-sign] is responsible for generating these signature segments.

	The control segment contains the package metadata in a <tt>.PKGINFO</tt> file as well as all of the scripts (if any) that are used by apk during installation and removal of the package. For historical reasons all files in the control tar segment are prefixed with a dot (<tt>.</tt>). The control segment is constructed by placing each file for the package into a tar record, concatenating those tar records, gzipping the tar records, and concatenating them onto the front of the data tarball. The SHA1 hash of this gzip stream is used as the checksum <tt>C:</tt> field in the APKINDEX file.		The control segment contains the package metadata in a <tt>.PKGINFO</tt> file as well as all of the scripts (if any) that are used by apk during installation and removal of the package. For historical reasons all files in the control tar segment are prefixed with a dot (<tt>.</tt>). The control segment is constructed by placing each file for the package into a tar record, concatenating those tar records, gzipping the tar records, and concatenating them onto the front of the data tarball. The SHA1 hash of this gzip stream is used as the checksum <tt>C:</tt> field in the APKINDEX file.

Sfzhi: Corrected the signature calculation description to match the actual implementation.

2023-04-28T10:37:38Z

Corrected the signature calculation description to match the actual implementation.

← Older revision		Revision as of 10:37, 28 April 2023
Line 15:		Line 15:
	= Package Format V2 =		= Package Format V2 =
	== Binary Format ==		== Binary Format ==
	APK v2 packages contain two tar segments followed by a tarball each in their own gzip stream (3 streams total). These streams contain the package signature, control data, and package data. The package data is a tarball of the files contained in a package laid out in a way that allows it to be unpacked at the filesystem root such that all files are placed in the correct location on the system. The control tar segment contains the package metadata along with any install scripts. The signature tar segment contains a single file that is a binary signature over the ~~concatenated~~ control segment ~~and data tarball~~.		APK v2 packages contain two tar segments followed by a tarball each in their own gzip stream (3 streams total). These streams contain the package signature, control data, and package data. The package data is a tarball of the files contained in a package laid out in a way that allows it to be unpacked at the filesystem root such that all files are placed in the correct location on the system. The control tar segment contains the package metadata along with any install scripts. The signature tar segment contains a single file that is a binary signature over the control segment.

	The signature file is a DER encoded PKCS1v15 RSA signature of the SHA1 hash of the concatenated control and data gzip streams. The filename has the format <tt>.SIGN.RSA.<key_name>.rsa.pub</tt> (for example <tt>.SIGN.RSA.alpine-devel@lists.alpinelinux.org-5261cecb.rsa.pub</tt>). This file is placed inside of a tar record with permissions 0644, uid 0, and gid 0. This tar record (lacking end-of-tar records) is gzip compressed, forming a signature tar segment, and is concatenated onto the front of the combined control and data segments. [https://gitlab.alpinelinux.org/alpine/abuild/-/blob/master/abuild-sign.in abuild-sign] is responsible for generating these signature segments.		The signature file is a DER encoded PKCS1v15 RSA signature of the SHA1 hash of the concatenated control and data gzip streams. The filename has the format <tt>.SIGN.RSA.<key_name>.rsa.pub</tt> (for example <tt>.SIGN.RSA.alpine-devel@lists.alpinelinux.org-5261cecb.rsa.pub</tt>). This file is placed inside of a tar record with permissions 0644, uid 0, and gid 0. This tar record (lacking end-of-tar records) is gzip compressed, forming a signature tar segment, and is concatenated onto the front of the combined control and data segments. [https://gitlab.alpinelinux.org/alpine/abuild/-/blob/master/abuild-sign.in abuild-sign] is responsible for generating these signature segments.

User4708: Changed Package_managment to Alpine Package Keeper.

2022-12-12T21:55:11Z

Changed Package_managment to Alpine Package Keeper.

← Older revision		Revision as of 21:55, 12 December 2022
Line 1:		Line 1:
	{{Draft}}		{{Draft}}

	For end-user facing documentation about apk, check out the [[~~Package_management~~]] page.		For end-user facing documentation about apk, check out the [[Alpine_Package_Keeper\|Alpine Package Keeper]] page.

	This page is an attempt to document the internal data structures of the apk package manager. The canonical implementation of the apk format is [https://gitlab.alpinelinux.org/alpine/apk-tools apk-tools] and much of this information is gleaned from reading the source code.		This page is an attempt to document the internal data structures of the apk package manager. The canonical implementation of the apk format is [https://gitlab.alpinelinux.org/alpine/apk-tools apk-tools] and much of this information is gleaned from reading the source code.

Mcrute at 03:32, 18 July 2022

2022-07-18T03:32:32Z

@@ Line 138: / Line 138: @@
 p:cmd:jool=4.1.5-r0 cmd:jool_siit=4.1.5-r0 cmd:joold=4.1.5-r0
 </pre>
 [[Category:Package Manager]] [[Category:Development]]

Mcrute at 01:41, 18 July 2022

2022-07-18T01:41:09Z

← Older revision		Revision as of 01:41, 18 July 2022
Line 7:		Line 7:
	There are three generations of the APK data formats. Version 1 is deprecated and no longer used, version 2 is currently the main version in use by apk-tools, and version 3 is under development. This page mostly describes the data formats used in version 2.		There are three generations of the APK data formats. Version 1 is deprecated and no longer used, version 2 is currently the main version in use by apk-tools, and version 3 is under development. This page mostly describes the data formats used in version 2.

	= ~~Package Format V2~~ =		= Background =
	== Tar Segments ==		== Tar Segments ==
	Tar segments are a set of tar records. Normal tar files contain two null records at the end of the tar file to signal the end of the tarball. Tar segments are lacking these two records and can thus be concatenated before other tar files and will behave as one continuous tar file. The APK v2 package format makes use of both tar segments and tarballs.		Tar segments are a set of tar records. Normal tar files contain two null records at the end of the tar file to signal the end of the tarball. Tar segments are lacking these two records and can thus be concatenated before other tar files and will behave as one continuous tar file. The APK v2 package format makes use of both tar segments and tarballs.
Line 13:		Line 13:
	Tar segments can be compressed using gzip compression. Gzip is a stream-based file format and multiple streams can be concatenated together. Most tooling will treat multiple gzip streams within a file as if it were a single stream. APK v2 files are aware of gzip streams and use them for file segmentation.		Tar segments can be compressed using gzip compression. Gzip is a stream-based file format and multiple streams can be concatenated together. Most tooling will treat multiple gzip streams within a file as if it were a single stream. APK v2 files are aware of gzip streams and use them for file segmentation.

			= Package Format V2 =
	== Binary Format ==		== Binary Format ==
	APK v2 packages contain two tar segments followed by a tarball each in their own gzip stream (3 streams total). These streams contain the package signature, control data, and package data. The package data is a tarball of the files contained in a package laid out in a way that allows it to be unpacked at the filesystem root such that all files are placed in the correct location on the system. The control tar segment contains the package metadata along with any install scripts. The signature tar segment contains a single file that is a binary signature over the concatenated control segment and data tarball.		APK v2 packages contain two tar segments followed by a tarball each in their own gzip stream (3 streams total). These streams contain the package signature, control data, and package data. The package data is a tarball of the files contained in a package laid out in a way that allows it to be unpacked at the filesystem root such that all files are placed in the correct location on the system. The control tar segment contains the package metadata along with any install scripts. The signature tar segment contains a single file that is a binary signature over the concatenated control segment and data tarball.

Mcrute at 01:39, 18 July 2022

2022-07-18T01:39:26Z

← Older revision		Revision as of 01:39, 18 July 2022
Line 98:		Line 98:
	The <tt>apk_pkg_write_index_entry</tt> function of [https://gitlab.alpinelinux.org/alpine/apk-tools/-/blob/ff7c8f6ee9dfa2add57b88dc271f6711030e72a0/src/package.c#L905 package.c] defines the currently accepted fields. As of July 2022, these are:		The <tt>apk_pkg_write_index_entry</tt> function of [https://gitlab.alpinelinux.org/alpine/apk-tools/-/blob/ff7c8f6ee9dfa2add57b88dc271f6711030e72a0/src/package.c#L905 package.c] defines the currently accepted fields. As of July 2022, these are:

	* <tt>C:</tt> - file checksum, ~~the SHA1 hash of the second gzip stream of the package, base64-encoded and prefixed with <tt>Q1</tt> to indicate SHA1~~		* <tt>C:</tt> - file checksum, see below
	* <tt>P:</tt> - package name (corresponds to <tt>pkgname</tt> in PKGINFO)		* <tt>P:</tt> - package name (corresponds to <tt>pkgname</tt> in PKGINFO)
	* <tt>V:</tt> - package version (corresponds to <tt>pkgver</tt> in PKGINFO)		* <tt>V:</tt> - package version (corresponds to <tt>pkgver</tt> in PKGINFO)

Mcrute at 01:37, 18 July 2022

2022-07-18T01:37:23Z

Show changes