This lesson is being piloted (Beta version)

Containers on Blue Waters

This is an introduction to container solutions for High-Performance Computing (HPC) systems. Although we will go over a lot of material, our end goal is to teach you basic concepts related to software containers in HPC.

Attention

  1. This is an early (“alpha”) version of the lesson. As such, typos, minor mistakes, and incomplete instructions are to be expected. Please let us know of any issues by emailing us at bw-eot@ncsa.illinois.edu or by submitting an issue or a pull request on GitHub. Any and all contributions are welcome!
  2. At the very bottom of this (and all other) pages you can see that this lesson uses the template of The Carpentries organization. This lesson, however, is not part of The Carpentries official lesson portfolio. Therefore, please direct all questions and comments related to this lesson to us.
  3. Docker, Singularity, Shifter, and HPC Container Maker are complex software with plethora of subcommands, options, and possible use cases. The main goal of this lesson is to help you understand how to use these tools for computational research. If you use any of these tools in your research in a way not described in this lesson, please let us know!
  4. This lesson assumes that you’re comfortable with using command shell on the machine you’re using for the workshop, whether this is a Linux, macOS, or Windows computer.
  5. This lesson assumes that you can install software on the machine you’re using for the workshop. Usually, this means that your account on the machine has administrative privileges.

Prerequisites

  1. Familiarity with the command shell provided by your operating system. On Windows, you can either use Command Prompt, Powershell, or any third-party terminal emulator, such as MobaXterm or Git Bash.
  2. Docker and Singularity.
  3. Python.

Consult with our Setup page for more information on required software.

Schedule

Setup Download files required for the lesson
00:00 1. Software Containers in HPC: What and Why What are software containers?
How are containers different from Virtual Machines?
What is the container runtime landscape?
00:20 2. Docker How can one use Docker?
What is the difference between Docker image and container?
02:20 3. Singularity How can one build and run HPC applications in a container?
03:30 4. HPCCM: HPC Container Maker How can one build and run HPC applications in a container?
04:30 5. Using shifter to run TensorFlow on Blue Waters How to use Shifter as a way to build complex software stacks for Blue Waters?
05:00 Finish

The actual schedule may vary slightly depending on the topics and exercises chosen by the instructor.