vm_based_simulator_env.md 4.14 KB
OAI CI Virtual-Machine-based Simulator Test Environment

Table of Contents

  1. Introduction
  2. Prerequisites
    1. uvtool installation
    2. apt-cacher-server installation

1. Introduction

This document explains how the master pipeline works and how any developer could contribute to add testing.

It is an extension to the wiki page.

The purpose of this master pipeline is to:

  • Validate that a Merge Request is mergeable
  • Validate that a Merge Request is following coding guidelines
  • Validate that a Merge Request is not breaking any typical build variant
  • Validate that a Merge Request is not breaking any legacy simulator-based test

We will mainly focused on the 2 last items.

Last point, this documentation is valid for all CI-supported branches:

  • master
  • develop
  • develop-nr

But the feature set may not be aligned. The principles still apply.

2. Prerequisites

Some details are available on this wiki section.

Currently we have a host server (bellatrix) with the current parameters:

  • 40 x Intel(R) Xeon(R) CPU E5-2640 v4 @ 2.40GHz
  • 64 Gbytes RAM
  • Ubuntu 16.04.3 LTS (xenial)

For you to replicate this environment, you need a strong server:

  • At least 16 cores
  • Ubuntu 16 (xenial) or higher (such as bionic, not tested)

Also we are using a Virtual Machine (VM for the rest of document) based strategy for the moment.

So you need to install 2 tools:

  • uvt-kvm
  • apt-cacher

We are planning to also add:

  • A Red Hat Linux Entreprise 7.6 host environment
  • A Container-based strategy (such as Docket and Kubernetes)

2.1. uvtool installation

$ sudo apt-get install uvtool
# if you don't have already, create an ssh key
$ ssh-keygen -b 2048
# retrieve an image
$ sudo uvt-simplestreams-libvirt sync arch=amd64 release=xenial
# we might soon switch to an Ubuntu 18.04 version
$ sudo uvt-simplestreams-libvirt sync arch=amd64 release=bionic
$ uvt-simplestreams-libvirt query
release=bionic arch=amd64 label=release (20190402)
release=xenial arch=amd64 label=release (20190406)

On our server, I don't update (sync) that often (every 2-4 months).

For more details:

  • uvtool syntax is here
  • more readable tutorial is here

2.2. apt-cacher-server installation

I recommend to follow to the letter this tutorial.

The reason: we are creating/using/destroying a lot of VM instances and we are always installing the same packages. This service allows to cache on the host and, doing so, decreases the pressure on your internet bandwith usage. It also optimizes time at build stage.

$ sudo apt-get install apt-cacher apache2
$ sudo vi /etc/default/apt-cache
$ sudo service apache2 restart

# Server configuration
$ sudo vi /etc/apt-cacher/apt-cacher.conf 
 --> allowed_hosts = *
 --> fix the installer_files_regexp
$ sudo vi /etc/apt/apt.conf.d/01proxy
--> add `Acquire::http::Proxy "http://<IP address or hostname of the apt-cacher server>:3142";`
$ sudo service apt-cacher restart

This last file (/etc/apt/apt.conf.d/01proxy) is very important since it is tested in any CI script.


We can now switch to the next step: how to deal with oai sources

You can also go back to the CI dev main page