This lesson is being piloted (Beta version)

Containers on Blue Waters: Glossary

Key Points

Software Containers in HPC: What and Why
  • Containers are “ligh-weight virtual machines.”

  • Not all container solutions are suited for HPC.

Docker
  • To minimize the container image size, do not include development tools and other build artifacts

  • Docker images are ‘layered’

  • ‘Layered’ images have build time advantages

  • Multi-stage Docker builds are a powerful technique to minimize image size.

Singularity
  • Singularity images are ‘flat’

  • ‘Flat’ images have runtime advantages

  • Docker images can be easily converted to Singularity images

HPCCM: HPC Container Maker
  • HPC Container Maker (HPCCM) is an open source tool that makes it easier to build container images for HPC applications

  • HPCCM building blocks uplevel container specification

  • HPCCM recipes are Python

  • HPCCM can generate both Dockerfiles and Singularity definition files

Using shifter to run TensorFlow on Blue Waters
  • TensorFlow is a popular machine learning framework

  • available binary packages no longer support Blue Waters

  • TensorFlow’s build system is quite complex

Glossary

FIXME