issues focused on Virtual machine ecosystem and developing core set of SW
Go to file
Disassembler ce325cf3d0
Squashed commit of the following:
- Bump basic OS to Alpine 3.9
 - Restructure repo and add layer versioning
 - Use JSON for all metadata
 - Merge abuild branch (but without abuild)
2019-02-26 20:24:02 +01:00
_build Squashed commit of the following: 2019-02-26 20:24:02 +01:00
_extra Squashed commit of the following: 2019-02-26 20:24:02 +01:00
_vm Squashed commit of the following: 2019-02-26 20:24:02 +01:00
acme-sh Squashed commit of the following: 2019-02-26 20:24:02 +01:00
activemq Squashed commit of the following: 2019-02-26 20:24:02 +01:00
alpine3.8 Squashed commit of the following: 2019-02-26 20:24:02 +01:00
alpine3.8-nodejs8 Squashed commit of the following: 2019-02-26 20:24:02 +01:00
alpine3.8-php5.6 Squashed commit of the following: 2019-02-26 20:24:02 +01:00
alpine3.8-ruby2.4 Squashed commit of the following: 2019-02-26 20:24:02 +01:00
alpine3.9 Squashed commit of the following: 2019-02-26 20:24:02 +01:00
alpine3.9-java8 Squashed commit of the following: 2019-02-26 20:24:02 +01:00
alpine3.9-nodejs10 Squashed commit of the following: 2019-02-26 20:24:02 +01:00
alpine3.9-php7.2 Squashed commit of the following: 2019-02-26 20:24:02 +01:00
alpine3.9-python2.7 Squashed commit of the following: 2019-02-26 20:24:02 +01:00
alpine3.9-python3.6 Squashed commit of the following: 2019-02-26 20:24:02 +01:00
alpine3.9-ruby2.4 Squashed commit of the following: 2019-02-26 20:24:02 +01:00
alpine3.9-tomcat8 Squashed commit of the following: 2019-02-26 20:24:02 +01:00
ckan Squashed commit of the following: 2019-02-26 20:24:02 +01:00
ckan-datapusher Squashed commit of the following: 2019-02-26 20:24:02 +01:00
crisiscleanup Squashed commit of the following: 2019-02-26 20:24:02 +01:00
cts Squashed commit of the following: 2019-02-26 20:24:02 +01:00
ecogis Squashed commit of the following: 2019-02-26 20:24:02 +01:00
frontlinesms Squashed commit of the following: 2019-02-26 20:24:02 +01:00
gnuhealth Squashed commit of the following: 2019-02-26 20:24:02 +01:00
kanboard Squashed commit of the following: 2019-02-26 20:24:02 +01:00
mariadb Squashed commit of the following: 2019-02-26 20:24:02 +01:00
mifosx Squashed commit of the following: 2019-02-26 20:24:02 +01:00
motech Squashed commit of the following: 2019-02-26 20:24:02 +01:00
odoo Squashed commit of the following: 2019-02-26 20:24:02 +01:00
opendatakit Squashed commit of the following: 2019-02-26 20:24:02 +01:00
opendatakit-build Squashed commit of the following: 2019-02-26 20:24:02 +01:00
openmapkit Squashed commit of the following: 2019-02-26 20:24:02 +01:00
pandora Squashed commit of the following: 2019-02-26 20:24:02 +01:00
postgres Squashed commit of the following: 2019-02-26 20:24:02 +01:00
rabbitmq Squashed commit of the following: 2019-02-26 20:24:02 +01:00
redis Squashed commit of the following: 2019-02-26 20:24:02 +01:00
sahana Squashed commit of the following: 2019-02-26 20:24:02 +01:00
sahana-demo Squashed commit of the following: 2019-02-26 20:24:02 +01:00
sahana-shared Squashed commit of the following: 2019-02-26 20:24:02 +01:00
sambro Squashed commit of the following: 2019-02-26 20:24:02 +01:00
seeddms Squashed commit of the following: 2019-02-26 20:24:02 +01:00
sigmah Squashed commit of the following: 2019-02-26 20:24:02 +01:00
solr Squashed commit of the following: 2019-02-26 20:24:02 +01:00
ushahidi Squashed commit of the following: 2019-02-26 20:24:02 +01:00
vmmgr@d64e57ba69 Squashed commit of the following: 2019-02-26 20:24:02 +01:00
_vm.sh Squashed commit of the following: 2019-02-26 20:24:02 +01:00
.gitmodules Squashed commit of the following: 2019-02-26 20:24:02 +01:00
README.md Squashed commit of the following: 2019-02-26 20:24:02 +01:00

This is the main repository containing all build and installation scripts, configuration and customization of operating system and all applications present in SpotterVM virtual machine.

Virtual machine specifications

  • Memory: 4 GB
  • CPU: 1 processor, 2 cores
  • Hard Disk: SCSI, 300 MB
  • CD/DVD: IDE
  • Network Adapter: Bridged

In case you're setting up a VMWare virtual machine, select OS type Other Linux 3.x kernel 64-bit and after the VM is created, manually edit the *.vmx file using a text editor and add mem.hotadd = "FALSE". Failing to do so will result in system unable to boot. Other hypervisors don't need this adjustment.

Virtual Machine creation

Download Alpine Virtual 3.9.0 x86_64 from https://alpinelinux.org/downloads/ and boot from it. At the login prompt, use the root user without password to log in.

# Set up interfaces (leave the default choices)
setup-interfaces
ifup eth0

# Download and launch the setup script  
wget dl.dasm.cz/_vm.sh
sh _vm.sh

The script will perform installation and configuration of Alpine linux, LXC and the whole VMMgr platform. Virtual machine is protected by LUKS-on-LVM disk encryption. The encryption password, which is simultaneously also a password for VMMgr web administration interface, will be asked for at the beginning of the script execution. Root access is disabled.

After the script finishes and shuts down the virtual machine, remove CD/DVD drive from the virtual machine settings and extend the HDD to 80 GB. This is to minimize the overall size of the OVA (Open Virtual Appliance) file to which the VM will be exported.

Application Build environment creation

First time setup

Follow the VM creation steps as above, but comment the line disabling the root login in _vm.sh file before its execution. After the script finishes and HDD size is adjusted, add serial port in the virtual machine settings and set it to use Unix socket or named pipe (eg. \\.\pipe\alpine). Then start the VM and connect to the serial port using screen, PuTTY or any other terminal client.

# Install git and OpenSSH client
apk --no-cache add git openssh-client

# Create SSH key
ssh-keygen -t ecdsa

Assign the newly generated key to your GitLab account

# Clone the repository
git clone --recurse-submodules ssh://git@gitlab.dasm.cz:2222/Spotter-Cluster/Spotter-Cluster.git

# Install the build toolchain
Spotter-Cluster/_build/install-toolchain.sh

After the script finishes, it is possible to connect via SSH (provided you have personal key in authorized_keys) as the toolchain sets up also openssh-server. The serial connection will work only until disconnected. If it should work indefinitely, run

sed -i 's/ttyS0::once/ttyS0::respawn/' /etc/inittab
kill -1 1

Building the packages

There are 3 distinct packaging systems.

  1. Just a plain tar for basic OS setup used by _vm.sh installation script.
  2. Abuild for the native Alpine linux packages (APK) used for LXC, ACME client and VMMgr packaging.
  3. lxc-build/lxc-pack for LXC images building and packaging.

Full build of all components is automated via build-all.sh script. Before the script can be run, signing keys need to be put in place

# Supply Abuild signing key
vi /srv/build/repokey.rsa

# Or generate a new pair
echo '/srv/build/repokey.rsa' | abuild-keygen

# Build all packages
Spotter-Cluster/_build/build-all.sh

The full build process takes considerable amount of time, so it is advised to have a persistent build VM.

Host assignment

Application Container UID/GID Internal Port Host
ActiveMQ activemq 61616 61616 (ActiveMQ) -
CKAN ckan 8003 8080 (HTTP) ckan
CKAN Datapusher ckan-datapusher 8004 8080 (HTTP) -
Crisis Cleanup crisiscleanup 8005 8080 (HTTP) cc
CTS cts 8006 8080 (HTTP) cts
EcoGIS ecogis 8020 8080 (HTTP) ecogis
FrontlineSMS frontlinesms 8018 8080 (HTTP) sms
GNU Health gnuhealth 8008 8080 (HTTP) gh
KanBoard kanboard 8009 8080 (HTTP) kb
MariaDB mariadb 3306 3306 (MySQL) -
Mifos X mifosx 8012 8080 (HTTP) mifosx
Motech motech 8013 8080 (HTTP) motech
ODK Aggregate opendatakit 8015 8080 (HTTP) odk
ODK Build opendatakit-build 8017 8080 (HTTP) odkbuild
Odoo odoo 8019 8080 (HTPP) odoo
OpenMapKit openmapkit 8007 8080 (HTTP) omk
Pan.do/ra pandora 8002 8080 (HTTP) pandora
Postgres postgres 5432 5432 (Postgres) -
RabbitMQ rabbitmq 5672 5672 (AMQP) -
Redis redis 6379 6379 (Redis) -
Sahana sahana 8001 8080 (HTTP) sahana
SAMBRO sambro 8001 8080 (HTTP) sambro
SeedDMS seeddms 8010 8080 (HTTP) dms
Sigmah sigmah 8011 8080 (HTTP) sigmah
Solr solr 8983 8983 (HTTP) -
Ushahidi ushahidi 8014 8080 (HTTP) ush