Winter 22-23 OS Upgrades

Overview

Before December 2022, BRCF POD compute and storage servers were running Ubuntu Linux version 18.04 LTS (Long Term Support), which was released in 2018 with 5 years of support/updates. As of April 2023, updates for Ubuntu 18.04 are frozen and no longer available in the standard repositories. As a result, all BRCF POD servers need to be migrated to a newer Ubuntu LTS release. We selected Ubuntu 20.04 LTS, which will be supported until April 2025.

All storage servers were migrated to Ubuntu 20.04 in November 2022. This has little effect on users since most storage servers have little user-relevant software installed (but see below for outstanding issues).

Migration of POD compute servers to 20.04 began in with the December 20 maintenance window and is now complete, dates shown below. Numbers indicate how many compute servers each pod has.

• December 20 - 23, 2022 - CBRS (2), Chen/Wallingford(1) and EDU (3)
• January 3 - 4, 2023 - Dickinson/Cambronne (1), Georgiou/WCAAR (3), Lambowitz/CCBB (3)
• January 17 - 18, 2023 - Wilke (2), Iyer (2), Ochman (2), Kirkpatrick (1)
• January 31 - Feb 1, 2023 - Hopefog/Ellington (3), Livestrong (4)
• February 28 - March 1, 2023 - Marcotte (3), Rental (2)
• March 14 - 15, 2023 - GSAF (3)

Find your OS version

To determine which OS version a server is running, do the following in a Terminl (including RStudio Server or JupyterHub Server Terminal): lsb_release -a. On the Release: line you should see either 20.04 or 18.04.

Known outstanding issues

• JupyterHub Server
  
  • Clicking "Logout" in JupyterHub server does not work – it displays a "404 page not found" error
  • The Python3 menu item is missing from JupyterHub's New menu on some servers. On other servers where the Python3 menu item is present, an error occurs when selecting it to open a new notebook.
• Command-line programs
  • On the EDU pod, the 3rd party programs used by Josh Beckham's FRI stream (e.g. Gold) are not yet working
  • The singularity program is not the latest version needed for many open source bioinformatics containers
  • The FASTX-Toolkit has not been deployed
• Storage server access
  • The rssh program that allows direct access to the shared storage server via SSH (e.g. scp, rsync, sftp) is not available in Ubuntu 20.04. We are actively working to implement a suitable replacement. Meanwhile, files can be transferred to/from the storage server through any of a POD's compute server via scp, rsync, sftp

How R Studio Server and JupyterHub Server changes may affect you

All compute servers have both web-based RStudio Server and JupyterHub Server installed. A menu allowing you to select from the installed web-based applications is available by specifying any POD compute server name in a web browser; for example: https://gsafcomp01.ccbb.utexas.edu/

Also see R and Python versions for a summary of changes to versions of R and Python.

RStudio differences

The version of R in RStudio under Ubuntu 18.04 was either R 3.6.1 or R 4.0.3, as documented at Web-based RStudio and JupyterHub software. Servers with RStudio using R 4.0.3 will still use that version, although we plan to upgrade the R version to R 4.2 going forward. However the servers using R 3.6.1 will now run R 3.6.3. We do not anticipate that this will affect user-installed packages, but if you run into issues, move your ~/R directory out of the way as described at Global package installation conflicts, then re-install your package(s). If the problem persists please Contact Us.

The default version of R under Ubuntu 20.04 is 3.6.3, and this is the version used by the R Studio Server web application. We are aware that many new R packages are only available for later R versions, and will be installing R 4.2 later in 2023.

As part of the upgrade to the base R program, some R packages have also been upgraded.  While we don't anticipate any problems arising from these updates, if you do notice a problem or have any questions, please Contact Us.

JupyterHub differences

The default system Python under Ubuntu 18.04 was Python 3.6.9, and the web-based JupyterHub Server used that version. The default system Python under 20.04 is Python 3.8.10. The differences between the Python 3.6 and 3.8 environments may affect your Python Notebook programs.

As for the set of available JupyterHub Server packages, the JupyterHub Server's Python environment is maintained in a Python3-based virtual environment separate from the system (command-line) Python, so all those packages are centrally managed by us. As part of the upgrade to the base Python program, some python packages have also been upgraded.  While we don't anticipate any problems arising from these updates, if you do notice a problem or have any questions, please Contact Us.

How the compute server O/S upgrade may affect you

There are two categories of programs installed on compute servers:

1. Programs we install globally, accessible by all users
2. Programs you install yourself, generally accessible only by you

These can be binaries built from source, or in the case of Python and R/RStudio Server, add-on packages.

Globally-installed programs

If a program or R package or Python package you regularly used under Ubuntu 18.04 is missing or doesn't work in the new Ubuntu 20.04 environment, please Contact Us after trying the troubleshooting tips described below.

We are working to update our POD Software Information to reflect globally-installed software available under Ubuntu 20.04, but it currently reflects only the 18.04 environment.

Incompatible user-installed programs

Programs you installed under Ubuntu 18.04 may no longer work.

The Ubuntu OS is installed on local hard disks on each POD compute server, so that's where system programs, system libraries, most add-on programs and add-on libraries are installed. (The exception is for large 3rd party programs or tool suites, which are built in /stor/system/opt, a directory on the shared storage server.)

If you installed 3rd party programs, however, the installation binaries are usually written somewhere under your home directory, /stor/home/<user_name>, which is on the shared storage server. These programs may refer to system libraries (or R or Python packages) that are incompatible with the Ubuntu 20.04 environment.

The steps needed to address such incompatibilities depends on the type of program and installation:

• For programs downloaded as prebuilt binaries, check the maintainer's website for a version compatible with Ubuntu 20.04.
• For programs downloaded as source and built by you, move the previous binaries (often in ~/bin or ~/.local/bin), then perform a clean build from existing (or updated) source code.

R and Python versions

After the upgrade, R and Python versions are as follows:

• Python
  • Default comand-line python is still 2.7
  • Default python3 3.8.10, is the Python in JupyterHub Server
  • Python 3.6 is no longer available
• R
  • Default command-line R is 3.6.3
  • R 4.0.3 is accessible from the command line as R-4.0.3
  • RStudio Server R is either 3.6.3 (if 3.6.1 previously) or still 4.0.3
  • R 3.4 and 3.6.1 are no longer available

If you run into problems running Python or R programs that used to work before:

• For R packages, your local installations are in your ~/R directory. Rename that directory (e.g. to ~/R.bak) so you can refer to it to see which packages you installed. You may then need to re-install any packages you need that are not already installed globally (you can check this by doing library(<module_name>) in R).
  • See above for the different R versions available in RStudio Server.
• For Python packages, your local installations are in your ~/.local/lib/python2.7 or ~/.local/lib/python3 directories. Rename those directories (e.g. to ~/.local/lib/python2.7.previous) so you can refer to it to see which packages you installed, then re-install those packages.

Also see How RStudio Server and JupyterHub Server changes may affect you.