Skip to main content

Container Setup

Containers offer a consistent software environment, simplifying deployment across systems. This page explains how to configure a ROCm container to use AMD GPUs and Mango BoostX™ RoCE AI cards within a containerized environment. A CUDA container with NVIDIA GPUs can be configured in a similar way.

Before proceeding, confirm that the host-side Mango BoostX™ RoCE AI setup is complete:

(host) ~$ rdma link
link mb_0/1 state ACTIVE physical_state LINK_UP netdev ens102np0
link mb_1/1 state ACTIVE physical_state LINK_UP netdev ens104np0

Option 1 — Building a Docker Image from a Dockerfile​

Download a base ROCm Docker image​

ROCm Docker images are available at hub.docker.com/r/rocm/pytorch.

(host) ~$ docker pull rocm/pytorch:rocm6.3.2_ubuntu22.04_py3.10_pytorch_release_2.4.0

Build the Docker image​

Use the following Dockerfile snippet to install the Mango BoostX™ RoCE AI software stack inside the image. This can be applied to the downloaded image or a Dockerfile built from scratch.

Dockerfile
FROM rocm/pytorch:rocm6.3.2_ubuntu22.04_py3.10_pytorch_release_2.4.0

ADD mango-repo-rdma_26XXXX_amd64.deb .
RUN dpkg -i mango-repo-rdma_26XXXX_amd64.deb && \
    apt update && \
    apt install -y --no-install-recommends libc6-dev libglib2.0-dev libstdc++-10-dev \
        ibverbs-providers mango-cli perftest rdma-core
(host) ~$ docker build -t <image-name> .
info

If the base image contains packages that conflict with ours (e.g., Mellanox OFED), uninstall them. See Troubleshooting → Uninstall Mellanox OFED.

Option 2 — Building a Docker Image from a Running Container​

Install the Mango BoostX™ RoCE AI package​

Install the user-space packages inside the running container:

(container) ~$ dpkg -i mango-repo-rdma_26XXXX_amd64.deb
(container) ~$ apt update
(container) ~$ apt install -y --no-install-recommends libc6-dev libglib2.0-dev libstdc++-10-dev \
    ibverbs-providers mango-cli perftest rdma-core
info

The kernel-space modules (mango-drivers and mango-drivers-dkms) must be installed on the host, not inside the container.

Commit the container​

Save the container as a new image:

(host) ~$ docker commit <container-name>
(host) ~$ docker tag <image-name-in-hex> <image-name>

Docker Execution​

Run the container using the following docker run command:

(host) ~$ docker run \
    --group-add=video --group-add=render --cap-add=SYS_PTRACE \
    --security-opt seccomp=unconfined --shm-size 8G \
    --network=host --privileged -v /dev:/dev \
    ... \
    <image-name>
  • The first two lines (--group-add=video through --shm-size 8G) are recommended for AMD GPU usage.
  • The third line (--network=host --privileged -v /dev:/dev) is required for using the Mango BoostX™ RoCE AI interfaces.
  • Additional options may be appended as needed.