Alpine Package Testing Suite

From Alpine Linux
Revision as of 06:52, 5 November 2009 by Amanison (talk | contribs) (Using kernel modules)
Jump to: navigation, search

Alpine Package Testing Suite (APTS) is a set of GNU make scripts to test that the packages are working. The idea is to catch errors in upgrades and make sure that the packages shipped on the release will work.

Current Package Status

APTS Package Status Page lists the current packages and APTS status.

How to get APTS

Currently the only way to get it is via subversion. Anonymous read access to svn is available from To install subversion and fetch the APTS scripts run:

apk_add subversion
svn co svn:// apts

How to run the tests

Use the run-tests shell script to run the tests.

cd apts

It is also possible to run one or more individual tests by specifying the .mk files on the command line. To test the packages that starts on a:

./run-tests a*.mk

Installed packages will be skipped

If a package is already installed, it will be assumed that the package is working and the test will be skipped. This also prevents the test suite from damaging a running system. There is no way to override this and force an already installed package to be tested.

Analyzing failed packages

The failed tests will have their output in .failed/<package>/stdout and .failed/<package>/stderr. Note that the failed packages will not be uninstalled so you can check the config files and run the commands manually to find out why the test is failing.

Run tests again

Packages that have passed the test will no be run again. If you want to force run a specific package again you can remove the .passed/<package> directory. All tests that have failed will be re-run, uninstalling the package first. To avoid that you can define the SKIP_FAILED variable.

SKIP_FAILED=yes ./run-tests

How to create new tests

To create a new test, create a new file named <package>.mk where <package> is the name of the package to test. This makefile must have a target with the name of the package.

The rules for this target should install the package, verify that it is working and uninstall it afterwards.

stdout and stderr will automatically be redirected to out/<package>/std{out,err}.

A test script for nmap might look like:

       apk_add $@
       nmap localhost
       apk_delete $@

The special variable $@ will expand to the target itself, in this case nmap.

A slightly more advanced example might look like this:

       apk_add $@
       echo "visible_hostname test.domain" >> /etc/squid/squid.conf
       /etc/init.d/$@ start
       pidof $@
       /etc/init.d/$@ stop
       ! pidof $@
       apk_delete $@

There is a helper script called new to genereate new templates.

./new <package>

Using kernel modules

Some packages need kernel modules, for example a the snd-dummy for a dummy soundcard. Instead of loading the modules in your .mk file, you can specify the variable NEED_MODULES. All the modules needed to run tests will be loaded before running the tests. That way the same kernel module will be loaded only once for several test scripts.

NEED_MODULES=snd-dummy snd-pcm-oss

How to submit your new test

Once you have a test script that works you can submit it for inclusion in the APTS svn tree. Follow these steps if you do not have direct svn write access to

  • Add your file to svn. We use squid as example here:
svn add
  • When you check svn status you should see that your file is added
# svn st
?      out
M      Makefile
  • Subversion here correctly shows that the out directory is not in subversion control.
  • You can also run a diff to verify it looks ok:
svn diff
  • You can now send a patch. Use sendbug for this. (remeber to run setup-sendbug unless you already have configure it)
svn diff | sendbug

Set the following fields:

>Synopsis:      [PATCH] APTS new test for squid
>Severity:      non-critical
>Priority:      medium
>Category:      misc                                             
>Class:         change-request

If you have any comments, please add those in the >Description field.

Save and quit. Send your bug report. Bugs marked with '[PATCH]' will most likely be handled faster.