Installation

Note

PAVICS is built to work within POSIX and POSIX-like systems (e.g. Unix/Linux). For Windows users who want to administer a PAVICS data server, you may need to either install PuTTY, a Unux/Linux virtual machine or use a POSIX API environment like the Windows Subsytem for Linux or Cygwin to interface with the PAVICS server system.

Deployment methods

PAVICS can be installed from source on GitHub or by using Docker. Git/Github is a versioning software and platform that can fetch development branches of pavics while Docker is a virtualization tool for running isolated service images built with specific library environments. Depending on your needs and interests (e.g. debugging vs deploying) either platform may suit your needs.

Proposing changes to PAVICS or Contributing requires free a GitHub account but anyone can download the source code for PAVICS withour registering. Git is normally in most standard Linux software repositories and can be installed (using Debian/Ubuntu-based systems) with the following:

# Enable root access
sudo su

# Updating software catalogues and and installed libraries
apt-get update; sudo apt-get upgrade

# Installing git will also install dependent libraries
apt-get install -y git

# Configuring git for code commits using your user credentials
exit # Leave superuser
git config --global user.name "Your Name"
git config --global user.email "youremail@domain.com"

For Docker installations, begin by following the guide for installing Docker Community Edition on your home machine from the Docker Installation Page. Docker Community Edition is a set of command line tools for creating and launching container-based applications.

Depending on your Linux distribution, you can either download and launch the installer directly or you may need to add a new ppa/apt/yum/dnf/etc. software repository to your system to install the most recent version. For Debian/Ubuntu-based systems:

# Enable root access
sudo su

# Install base library requirements
apt-get install -y apt-transport-https ca-certificates wget software-properties-common

# Install the Docker suite of tools
apt-get install -y docker docker.io docker-engine

For users not using Linux, refer to the following installation guides for Mac OS and for Windows.

PAVICS and Dockerhub images

All code produced for the PAVICS project is Open Source and hosted publicly through GitHub repositories. Among our members/users, the preferred method for packaging and deployment relies on Docker images. As such, we use the Docker Image building and hosting features of DockerHub. This feature automatically builds a new image for all major releases to the GitHub repository, for all repositories that include a Dockerfile. The resulting images are all publicly available for distribution and deployment. Any PAVICS Docker image can be obtained using the following command:

docker pull pavics/[image_name]

Most of the code base for PAVICS is forked from the Birdhouse Project which already has multiple DockerHub build processes. Using our own DockerHub build gives us the advantage of having our own upstream build process for the code being modified by CRIM / Ouranos. Many of the modifications to these birdhouse components are merged upstream, but some are specific to the PAVICS project and we felt it was worthwhile to maintain an independent build process.

PAVICS installation with docker-compose

Todo

Base PAVICS installation is incomplete. The following lines refer to Phoenix instance. Need to specify which birds are needed for a bare installation of PAVICS: Phoenix, FlyingPigeon, Malleefowl, Emu, etc.

First mount or create a symlink for the datasets storage at /data. Mount or make a symlink for the geoserver data storage so that /geoserver_data could be used (read/write) by geoserver.

To install the suite of docker images you must use docker-compose. Docker-compose is a docker helper for coordinating multiple docker images at once. Docker-compose exists in most Linux software repositories but can also be installed using pip or conda. Depending on your Linux distribution and whether you have Anaconda/miniconda installed, run any of the following:

# For standard libraries (Ubuntu/Debian)
sudo apt-get install docker-compose

# For system-installed Python2 or Python3 (requires sudo)
sudo apt-get install -y python-dev python3-dev
sudo -H pip install docker-compose

# For Anaconda/miniconda (Python2 or Python3)
conda install docker-compose

Note

Presently, the main PAVICS repo is closed to the public while it is under heavy development

After installing docker-compose, clone the PAVICS repository and navigate to the docker build recipe within PAVICS:

# Cloning the PAVICS repository
git clone https://github.com/Ouranosinc/PAVICS.git
cd pavics-sdi/birdhouse
# Open the docker-compose.yml using a text editor
nano docker-compose.yml

Note

The docker-compose.yml contains many of the setup configurations needed to successfully launch Birdhouse. Be sure to read the Birdhouse service configuration and the Load balancing suggestions before continuing the installation.

After modifying the necessary variables in the docker-compose.yml file, simply run the following command, taking care to select an appropriate host name:

HOSTNAME='<public-ip>' bash -c 'docker-compose up -d'

This installation will run on a single server instance, but there are instructions for Load balancing.

Updating containers to the latest version

To synchronize a deployment with the latest container available on dockerhub:

# Set working directory where the docker-compose.yml is located. Usually :
cd ~/PAVICS/birdhouse

# Docker requires root privileges
sudo su

# Pull the latest containers (container_name is optional but can limit the operation to only one container rather than applying to all containers)
HOSTNAME='<public-ip>' bash -c 'docker-compose pull [container_name]'

# Stopping containers
HOSTNAME='<public-ip>' bash -c 'docker-compose stop [container_name]'

# Start again containers (-d is for detached, avoid it to get all output to the command line)
HOSTNAME='<public-ip>' bash -c 'docker-compose up -d [container_name]'

Note

container_name is the name chosen in the docker-compose.yml, not the name of the docker image.

Resetting Bird services

If a Bird Service becomes unusable, the docker containers can easily be reset to default settings:

# Docker requires root privileges
sudo su

# Stopping and removing containers (this will flush their states)
HOSTNAME='<public-ip>' bash -c 'docker-compose down'

# Start again containers (-d is for detached, avoid it to get all output to the command line)
HOSTNAME='<public-ip>' bash -c 'docker-compose up -d'

Note

Some manual tasks are required after resetting the birdhouse environment : See the manual steps under the Phoenix configuration

Restarting Bird services

If the dockers containers need to be stopped (including the docker service, if required) these steps will preserve the docker state and all configurations set via Bird Service web portals:

# Docker requires root privileges
sudo su

# Stopping running containers
HOSTNAME='<public-ip>' bash -c 'docker-compose stop'

# Stopping the docker service
service docker stop

Restarting Bird Services (required after restarting the host vm):

# Docker requires root privileges
sudo su

# Starting the docker service
service docker start

# Starting the birdhouse containers
HOSTNAME='<public-ip>' bash -c 'docker-compose start'

Port specification

The docker-compose.yml config file exposes ports for each docker container (left part is the public one, right part the container internal one). We try to respect the following convention:

8xxx : port usually responding for the service (The HTTP port)
28xxx: https port
38xxx: the output port (To be documented)
48xxx: the supervisor port of the container

Container xxx value:

Phoenix : 443 (With 8081 as http and 8443 as https)
Malleefowl : 091
Flyingpigeon : 093
Emu : 094
Solr : 983 (No https or output ports)
ncWMS2 : 080 (No https or output ports)
thredds : 083 (No https or output ports)
pavics-catalog : 086 (No https or output ports)
geoserver : 087 (No https or output ports)

The exception is the Pavics-frontend, which has port 3000.

To find which shell process uses a given port, use:

netstat -nlp | grep :<port #>

Todo

Update the installation and config with security changes