Summer 2019 OS Upgrades

Owned by Anna Battenhouse
Last updated: Sep 18, 2019
3 min read

Overview

Before summer of 2019, BRCF POD compute and storage servers were running Ubuntu Linux version 14.04 LTS (Long Term Support), which was released in 2014 with 5 years of support/updates. As of April 2019, updates for Ubuntu 14.04 were frozen and no longer available in the standard repositories. As a result, all BRCF POD servers need to be migrated to a newer Ubuntu LTS release. We selected Ubuntu 18.04 LTS, as it will be supported until April 2023.

All storage servers were migrated to Ubuntu 18.04 by July 2019. Migration of compute servers to 18.04 began in July 2019, scheduled as follows:

  • July 24 - Chen/Wallingford and Iyer/Kim PODs
  • July 31 - Dickinson/Cambronne, Ochman, Wilke and EDU PODs
  • Aug 7 - Georgiou/WCAAR, GSAF, Lambowitz/CCBB and Marcotte PODs

How the compute server O/S upgrade may affect you

There are two categories of programs installed on compute servers:

  1. Programs we install globally, accessible by all users
  2. Programs you install yourself, generally accessible only by you

These can be binaries built from source, or in the case of Python/Pyton3 and R/R Studio server, add-on packages.

Missing globally-installed programs

Compute servers under the previous Ubuntu 14.04 release included a layer of bioinformatics software called BioLinux 7. That layer is not available for Ubuntu 18.04. We have attempted to globally install the most important BioLinux 7 tools under Ubuntu 18.04; however some may be missing.

Additionally, there are some 3rd party programs that we cannot yet build successfully under Ubuntu 18.04.

If a program (or R package or Python package) you regularly used under Ubuntu 14.04 is missing in the new Ubuntu 18.04 environment, please Contact Us.

We are working to update our POD Software Information to reflect globally-installed software available under Ubuntu 18.04, but it currently reflects only the 14.04 environment.

Incompatible user-installed programs

Programs you installed under Ubuntu 14.04 may no longer work.

The Ubuntu O/S is installed on local hard disks on each POD compute server, so that's where system programs, system libraries, most add-on programs and add-on libraries are installed. (The exception is for large 3rd party programs or tool suites, which are built in /stor/system/opt, a directory on the shared storage server.)

If you installed 3rd party programs, however, the installation binaries are usually written to your home directory, /stor/home/<user_name>, which is on the shared storage server. These programs may refer to system libraries (or R or Python packages) that are incompatible with the Ubuntu 18.04 environment.

The steps needed to address such incompatibilities depends on the type of program and installation:

  • For programs downloaded as pre-built binaries, check the maintainer's website for a version compatible with Ubuntu 18.04.
  • For programs downloaded as source and built by you, move the previous binaries (often in ~/bin or ~/.local/bin), then perform a clean build from existing (or updated) source code.
  • For R packages, your local installations are in your ~/R directory. Rename that directory (e.g. to ~/R.previous) so you can refer to it to see which packages you installed. You may then need to re-install any packages you need that are not already installed globally (you can check this by doing library(<module_name>) in R).
  • For Python packages, your local installations are in your ~/.local/lib/python2.7 or ~/.local/lib/python3 directories. Rename those directories (e.g. to ~/.local/lib/python2.7.previous) so you can refer to it to see which packages you installed, then re-install those packages.

Notes on R Studio Server and JupyterHub Server

In our new Ubuntu 18.04 environment, all compute servers have both web-based R Studio Server and JupyterHub Server installed.

The default version of R under Ubuntu 18.04 is 3.4.4, and this is the version used by the R Studio Server web application. We are aware that many new R packages are only available for R versions 3.5 and later, and will be installing a 3.5.x version of R soon. It will be available from the command line via a command like R-3.5.x, but will not be used by R Studio Server.

The JupyterHub Server's Python environment is maintained in a Python3-based virtual environment separate from the system (command-line) Python.






Confluence Documentation | Web Privacy Policy | Web Accessibility