Add support for multiple archs in autopkgtest CI

This took **a lot** of time, as autopkgtest-build-qemu has been able to
build non-x86 for less than a release (version 5.17, unreleased),
and it isn't meant to be (easly) used on Ubuntu (and unfortunately GH
Actions only support Ubuntu).

This is extremely fragile; adding a CPU core breaks something, adding a
MB of RAM breaks something, installing recommended packages breaks
something... you get it. But it works. Beautiful, isn't it?

This currently tests amd64, arm64. armhf, i386 and ppc64el. s390x seems
unsupported by autopkgtest-build-qemu.

If you're curious to see how many trials it took, have a look at this
test branch:
github.com/Tachi107/pistache/tree/actions-autopkgtest-multiarch-test
parent 05629072
...@@ -5,8 +5,13 @@ ...@@ -5,8 +5,13 @@
name: autopkgtest name: autopkgtest
on: on:
[push, pull_request] push:
paths-ignore: pistache.io/**
pull_request:
paths-ignore: pistache.io/**
# The default Debian shell (dash) is faster than bash at running scripts, and
# using bash when it is not needed doesn't make sense.
defaults: defaults:
run: run:
shell: sh shell: sh
...@@ -15,22 +20,89 @@ jobs: ...@@ -15,22 +20,89 @@ jobs:
test: test:
runs-on: ubuntu-latest runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
arch: [ amd64, arm64, armhf, i386, ppc64el ] # , s390x
steps: steps:
- uses: actions/checkout@v2 - uses: actions/checkout@v2
# The Debian repository is needed by autopkgtest-build-qemu when building
# for foreign architectures, since the tool tries to install
# linux-image-${{ matrix.arch }}, that is unavailable on Ubuntu; adding
# the Debian Testing repo and lowering its priority solves the issue
# while not risking to break the system mixing Debian and Ubuntu pkgs.
# The pgp key required to use the repository is downloaded through the
# debain-archive-keyring package, and instead of using the deprecated
# apt-key utility to add the keyring to the trusted ones I use the
# Signed-By option (deb822 source format), as recommended in the Debian
# wiki (https://wiki.debian.org/DebianRepository/UseThirdParty).
# Node.js stuff is purged since it is not needed and for whatever reason
# causes a warning to be generated when building autopkgtest.
# Also qemu-system-gui is holded to prevent installation since it is not
# needed, and both vmdb2 and qemu-user-static are installed from testing
# because the version available in Ubuntu does not work; the old vmdb2
# causes image generation failures for all architectures, qemu-user-static
# only for arm64, but causes autopkgtest to fail for armhf and ppc64el.
# The debian-archive.trafficmanager.net mirror is used since it is the
# Azure mirror, and GitHub Actions run on that network.
# The --no-install-recommends apt option is important, as installing
# recommended packages causes autopkgtest to fail for arm64 and armhf;
# I don't know why, I don't know how, I only know that it tooks hours to
# figure it out.
- name: Install dependencies - name: Install dependencies
run: | run: |
sudo apt -qq install debian-archive-keyring
printf 'Enabled: yes\nTypes: deb\nURIs: http://debian-archive.trafficmanager.net/debian/\nSuites: testing\nComponents: main\nSigned-By: /usr/share/keyrings/debian-archive-keyring.gpg\n' | sudo tee /etc/apt/sources.list.d/debian.sources
printf 'Package: *\nPin: release o=Debian,a=testing\nPin-Priority: 400\n' | sudo tee /etc/apt/preferences.d/99debian-testing
sudo apt -qq update sudo apt -qq update
sudo apt -qq install autopkgtest genisoimage qemu-system vmdb2 meson cppcheck debhelper-compat dh-exec libc6-dev libcurl4-openssl-dev libgtest-dev libssl-dev libstdc++6 lintian pkg-config rapidjson-dev valgrind devscripts --no-install-recommends sudo apt -qq autopurge '*yarn*' '*npm*' '*node*'
sudo rm -rf /usr/local/lib/node_modules
sudo apt --assume-yes install genisoimage qemu-system vmdb2/testing devscripts --no-install-recommends qemu-user-static/testing qemu-efi-aarch64 qemu-efi-arm ovmf ovmf-ia32 && sudo apt-mark auto qemu-user-static qemu-efi-aarch64 qemu-efi-arm ovmf ovmf-ia32
sudo apt -qq build-dep .
- name: Build image # Build and install autopkgtest from the latest master (5.17), since
# previous versions can't build non-x86 images. It is done in a nested /tmp
# directory because otherwise debuild would try to write in /, resulting in
# an error.
- name: Install autopkgtest from source
run: | run: |
sudo autopkgtest-build-qemu --size 12G testing $HOME/autopkgtest-testing.img mkdir --parent /tmp/autopkgtest-build
curl --silent --output /tmp/autopkgtest-build/autopkgtest-master.tar --proto https --tlsv1.3 --http2-prior-knowledge https://salsa.debian.org/ci-team/autopkgtest/-/archive/master/autopkgtest-master.tar
tar --extract --file=/tmp/autopkgtest-build/autopkgtest-master.tar --directory=/tmp/autopkgtest-build
cd /tmp/autopkgtest-build/autopkgtest-master
sudo apt -qq build-dep . --default-release=testing
DEB_BUILD_OPTIONS=nocheck debuild --unsigned-source --unsigned-changes
sudo apt -qq install ../autopkgtest*.deb
cd -
rm -rf /tmp/autopkgtest-build
- name: Build Debian package - name: Build image
run: sudo autopkgtest-build-qemu --architecture ${{ matrix.arch }} --size 12G testing $HOME/autopkgtest-testing.img
- name: Build Debian package source
run: | run: |
./debian/rules get-orig-source ./debian/rules get-orig-source
debuild -S -sa -uc -us dpkg-source --build .
# The increased timeout is needed because the CI can be too slow to boot
# the image. --dpkg-architecture is needed because autopkgtest-virt-qemu
# defaults to the host arch instead of reading it from the image.
# The ram size is set to 3072 MB because the default 1 GiB is not enough
# to build Pistache, the amount must be less than 4 GiB otherwise the i386
# image won't boot, ppc64el requires it to be aligned to 256 MiB and an
# amout not convertible to a GiB integer causes arm64 to fail, while extra
# CPU cores can't be used as they can cause arm runs to fail with the error
# below. Instead of explicitly calling autopkgtest-virt-qemu it should be
# enough to just write qemu.
# Currently this sometimes fails on i386 with
# <VirtSubproc>: failure: timed out waiting for 'login prompt on serial
# console'
# and on arm64 and armhf with
# host login: <VirtSubproc>: failure: The VM does not start a root shell
# on ttyS1 or hvc1 already. The only other supported login mechanism is
# through --user and -*** the guest ttyS0
- name: Run autopkgtest - name: Run autopkgtest
run: autopkgtest ../pistache_*.dsc -- qemu --timeout-reboot=120 $HOME/autopkgtest-testing.img run: autopkgtest ../pistache_*.dsc -- autopkgtest-virt-qemu --dpkg-architecture=${{ matrix.arch }} --timeout-reboot=120 --ram-size=3072 $HOME/autopkgtest-testing.img
...@@ -8,7 +8,7 @@ SPDX-License-Identifier: Apache-2.0 ...@@ -8,7 +8,7 @@ SPDX-License-Identifier: Apache-2.0
[![N|Solid](pistache.io/static/img/logo.png)](https://www.github.com/pistacheio/pistache) [![N|Solid](pistache.io/static/img/logo.png)](https://www.github.com/pistacheio/pistache)
[![Build Status](https://travis-ci.org/pistacheio/pistache.svg?branch=master)](https://travis-ci.org/pistacheio/pistache) [![autopkgtest](https://github.com/pistacheio/pistache/actions/workflows/autopkgtest.yaml/badge.svg)](https://github.com/pistacheio/pistache/actions/workflows/autopkgtest.yaml)
[![REUSE status](https://api.reuse.software/badge/github.com/pistacheio/pistache)](https://api.reuse.software/info/github.com/pistacheio/pistache) [![REUSE status](https://api.reuse.software/badge/github.com/pistacheio/pistache)](https://api.reuse.software/info/github.com/pistacheio/pistache)
Pistache is a modern and elegant HTTP and REST framework for C++. It is entirely written in pure-C++17 and provides a clear and pleasant API. Pistache is a modern and elegant HTTP and REST framework for C++. It is entirely written in pure-C++17 and provides a clear and pleasant API.
......
...@@ -53,7 +53,7 @@ foreach test_name : pistache_test_files ...@@ -53,7 +53,7 @@ foreach test_name : pistache_test_files
date_dep date_dep
] ]
), ),
timeout: 60, timeout: 600,
workdir: meson.current_build_dir(), workdir: meson.current_build_dir(),
is_parallel: false is_parallel: false
) )
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment