Porting GHC to Alpine

This material is work-in-progress ... Do not follow instructions here until this notice is removed. (Last edited by Dubiousjim on 21 May 2013.)

I wanted to port the Glasgow Haskell Compiler to Alpine, and to do that, you need to start with some alreay-compiled GHC binary. So that means cross-compiling from some system where the binaries were already available.

After this is done once for each of Alpine's target architectures, other users can use those existing Alpine binaries to compile newer versions of GHC directly on Alpine, much more easily than is detailed below. (It does still take a long time.) But I am recording the steps necessary to port to Alpine in the first place, so that others can verify or reproduce my work. This may also help others porting GHC to other systems.

I assume anyone following these directions has already setup a cross-compiler targeting Alpine inside a chroot holding an existing Linux system where GHC binaries are available. These pages explain how to do that:

• ArchLinux install inside an Alpine chroot
• Cross-Compiler targeting Alpine

So here's how we use Arch's GHC to cross-compile a version of GHC we can run on Alpine:

1. In your outside Alpine system, do this. I assume that $ARCH_ROOT contains the absolute path on your Alpine system of the Arch chroot's /. I assume also that $ARCH_HOME contains the absolute path on your Arch system of the home directory of the non-root Arch user you'll be using to build GHC. (On the other hand, the plain $ARCH variable we set below indicates your machine architecture---nothing to do with ArchLinux.)

   sudo apk add paxctl libedit-dev libiconv-dev gmp-dev mkdir -p $HOME/alien-scripts ln -s $ARCH_ROOT/$ARCH_HOME/ghc-7.6.3 $HOME/ghc-7.6.3 mkdir /tmp/ghc-bootstrap $ARCH_ROOT/tmp/ghc-bootstrap sudo mount --bind /tmp/ghc-bootstrap $ARCH_ROOT/tmp/ghc-bootstrap case `uname -m` in x86_64) ARCH=x86_64; T=$ARCH;; i?86) ARCH=x86 T=i486;; *) echo Unknown architecture;; esac mkdir -p $HOME/buildroot-$ARCH/host/usr/bin/ cat << "__EOF__" > $HOME/buildroot-$ARCH/host/usr/bin/$T-buildroot-linux-uclibc-gcc #!/bin/sh /usr/bin/gcc -nopie "$@" __EOF__ chmod +x $HOME/buildroot-$ARCH/host/usr/bin/$T-buildroot-linux-uclibc-gcc

2. Inside the Arch chroot, as the non-root user, do this:

   sudo pacman -S base-devel python2 wget ghc openssh ssh-keygen

   You can give the ssh key a passphrase, but it's not necessary.

3. Back in the outside Alpine system, do this:

   cat $ARCH_ROOT/$ARCH_HOME/.ssh/id_rsa.pub >> $HOME/.ssh/authorized_keys sudo paxctl -cm $ARCH_ROOT/usr/lib/ghc-7.6.3/ghc

4. Returning again to the Arch chroot, do:

   eval `ssh-agent` ssh-add

   and unlock your ssh key if you gave it a passphrase. Now from this Arch session, we can do a password-less ssh or scp to the outside Alpine system.

5. Still inside the Arch chroot, do:

   mkdir -p sources && cd sources wget -N http://www.haskell.org/ghc/dist/7.6.3/ghc-7.6.3-src.tar.bz2 wget -N http://www.haskell.org/ghc/dist/7.6.3/ghc-7.6.3-testsuite.tar.bz2 mkdir -p patches/ghc/7.6.3 && cd patches/ghc/7.6.3 GIST_BASE=https://gist.github.com/dubiousjim/5607734/raw wget -N $GIST_BASE/configure.patch wget -N $GIST_BASE/libraries-configure.patch wget -N $GIST_BASE/h_wcwidth.patch cd $HOME/sources cat << "__EOF__" > alien #!/bin/sh PORT=22 # change if you use different ssh port on your Alpine system CMD=$2; shift 2 set -x scp -P$PORT "$CMD" $USER@localhost:alien-scripts/ || exit 1 ssh -p$PORT $USER@localhost alien-scripts/"${CMD##*/}" "$@" __EOF__ cat << "__EOF__" > alien-ghc #!/bin/sh PORT=22 # change if you use different ssh port on your Alpine system set -x ssh -p$PORT $USER@localhost "sh -c 'cd $PWD; $0.exe $*'" __EOF__ chmod +x alien alien-ghc

   The two alien scripts assume that you're working with the same username on both the Alpine and Arch systems; if that's incorrect, then replace $USER with the relevant Alpine username. The alien-ghc script also assumes that you won't be working underneath any directories whose names need escaping for the shell.

6. Still inside the Arch chroot, do this:

   cd case `uname -m` in x86_64) ARCH=x86_64; T=$ARCH;; i?86) ARCH=x86 T=i486;; *) echo Unknown architecture;; esac tar -xjf sources/ghc-7.6.3-src.tar.bz2 # next is optional, if you want to ... # tar -xjf sources/ghc-7.6.3-testsuite.tar.bz2 BUILDROOT=$HOME/buildroot-$ARCH export PATH=$HOME/python2-path:$PATH:$BUILDROOT/host/usr/bin cd ghc-7.6.3 sed -e 's/^#BuildFlavour = unreg/BuildFlavour = unreg/' mk/build.mk.sample > mk/build.mk patch -p1 -i $HOME/sources/patches/ghc/7.6.3/configure.patch patch -p1 -i $HOME/sources/patches/ghc/7.6.3/libraries-configure.patch patch -p1 -i $HOME/sources/patches/ghc/7.6.3/h_wcwidth.patch ./configure --prefix=/tmp/ghc-bootstrap \ --host=$T-buildroot-linux-uclibc \ --target=$T-buildroot-linux-uclibc \ --with-alien=$HOME/sources/alien 2>&1 | tee build.log

   The GHC wiki page on cross-compiling seems to suggest that you should omit the --host=..., but if you compare the 7.6 and (at this point, pre-release) 7.7 GHC build system sources, you'll see that this is true (if at all) only post-7.6.

   Continue with:

   make 2>&1 | tee -a build.log

7. The build will eventually fail with this error message:

   ".../buildroot-x86_64/host/usr/bin/x86_64-buildroot-linux-uclibc-gcc"  -fno-stack-protector   libraries/integer-gmp/cbits/mkGmpDerivedConstants.c -o libraries/integer-gmp/cbits/mkGmpDerivedConstants
   libraries/integer-gmp/cbits/mkGmpDerivedConstants > libraries/integer-gmp/cbits/GmpDerivedConstants.h
   /bin/sh: libraries/integer-gmp/cbits/mkGmpDerivedConstants: No such file or directory
   make[1]: *** [libraries/integer-gmp/cbits/GmpDerivedConstants.h] Error 127
   make[1]: *** Deleting file `libraries/integer-gmp/cbits/GmpDerivedConstants.h'
   make: *** [all] Error 2
   

   Type this to continue:

   $HOME/sources/alien run libraries/integer-gmp/cbits/mkGmpDerivedConstants \ > libraries/integer-gmp/cbits/GmpDerivedConstants.h make 2>&1 | tee -a build.log

   
   

8. A bit later, the build will again fail with this error message:

     HC [stage 1] libraries/base/dist-install/build/Foreign/C/Types.o
   
   libraries/base/Foreign/C/Types.hs:162:25:
       Not in scope: type constructor or class `HTYPE_FLOAT'
   
   libraries/base/Foreign/C/Types.hs:162:215:
       Not in scope: type constructor or class `HTYPE_FLOAT'
   
   libraries/base/Foreign/C/Types.hs:162:290:
       Not in scope: type constructor or class `HTYPE_FLOAT'
   
   libraries/base/Foreign/C/Types.hs:162:398:
       Not in scope: type constructor or class `HTYPE_FLOAT'
   
   libraries/base/Foreign/C/Types.hs:162:470:
       Not in scope: type constructor or class `HTYPE_FLOAT'
   
   libraries/base/Foreign/C/Types.hs:162:548:
       Not in scope: type constructor or class `HTYPE_FLOAT'
   
   libraries/base/Foreign/C/Types.hs:164:27:
       Not in scope: type constructor or class `HTYPE_DOUBLE'
   
   libraries/base/Foreign/C/Types.hs:164:219:
       Not in scope: type constructor or class `HTYPE_DOUBLE'
   
   libraries/base/Foreign/C/Types.hs:164:295:
       Not in scope: type constructor or class `HTYPE_DOUBLE'
   
   libraries/base/Foreign/C/Types.hs:164:405:
       Not in scope: type constructor or class `HTYPE_DOUBLE'
   
   libraries/base/Foreign/C/Types.hs:164:478:
       Not in scope: type constructor or class `HTYPE_DOUBLE'
   
   libraries/base/Foreign/C/Types.hs:164:557:
       Not in scope: type constructor or class `HTYPE_DOUBLE'
   make[1]: *** [libraries/base/dist-install/build/Foreign/C/Types.o] Error 1
   make: *** [all] Error 2
   

   Type this to continue:

   cat << __EOF__ >> libraries/base/include/HsBaseConfig.h #define HTYPE_DOUBLE Double #define HTYPE_FLOAT Float __EOF__ make 2>&1 | tee -a build.log

   The HsBaseConfig.h file is only generated during the build, so you can't make this change ahead of time; you have to wait for the build to fail, make the change, and then reissue make.

9. Finally, the build will fail a third time with this error:

     HC [stage 2] utils/ghctags/dist-install/build/Main.o
   inplace/bin/ghc-stage2: line 7: .../ghc-7.6.3/inplace/lib/ghc-stage2: No such file or directory
   make[1]: *** [utils/ghctags/dist-install/build/Main.o] Error 127
   make: *** [all] Error 2
   

   Type this to continue:

   mv inplace/lib/ghc-stage2 inplace/lib/ghc-stage2.exe cp $HOME/sources/alien-ghc inplace/lib/ghc-stage2 make 2>&1 | tee -a build.log

10. After this, the build should complete. We still have to do a bit of work to get it to install properly, though. Do this:

    mkdir -p /tmp/ghc-bootstrap/lib/ghc-7.6.3 mv ghc/stage2/build/tmp/ghc-stage2 /tmp/ghc-bootstrap/lib/ghc-7.6.3/ghc.exe mv utils/ghc-pkg/dist-install/build/tmp/ghc-pkg /tmp/ghc-bootstrap/lib/ghc-7.6.3/ghc-pkg.exe cp $HOME/sources/alien-ghc ghc/stage2/build/tmp/ghc-stage2 cp $HOME/sources/alien-ghc utils/ghc-pkg/dist-install/build/tmp/ghc-pkg make install 2>&1 | tee -a build.log

    Once the installation finishes, clean up with:

    cd /tmp/ghc-bootstrap mv lib/ghc-7.6.3/ghc.exe lib/ghc-7.6.3/ghc mv lib/ghc-7.6.3/ghc-pkg.exe lib/ghc-7.6.3/ghc-pkg for f in bin/*-buildroot-linux-uclibc-*; do mv $f bin/${f#*-uclibc-}; done ln -sf ghc-7.6.3 bin/ghc ln -sf ghc-pkg-7.6.3 bin/ghc-pkg cd -

11. Now switch back to your outside Alpine system and do this:

    paxctl -cm /tmp/ghc-bootstrap/lib/ghc-7.6.3/ghc sudo umount $ARCH_ROOT/tmp/ghc-bootstrap sed -i -e 's|"C compiler command",.*|"C compiler command", "/usr/bin/gcc"),|' \ -e 's|"C compiler flags", "|"C compiler flags", " -nopie |' /tmp/ghc-bootstrap/lib/ghc-7.6.3/settings rm -r $HOME/alien-scripts $HOME/buildroot-$ARCH rm $HOME/ghc-7.6.3

    Now everything in your Alpine system is cleaned up, and you've got a working GHC installation in /tmp/ghc-bootstrap, which the Alpine APKBUILD for ghc will recognize and use. You can delete the $ARCH_HOME/ghc-7.6.3 folder inside your Arch chroot now, and close that session.

(I will prepare an Alpine APKBUILD for ghc, and arrange with the dev team how to make the initial binaries available, soon.)