Skip to main content

Custom Runner Images

If you would like to include extra dependencies which aren't packaged with your module repository objects, you can create a custom runner image. This is useful in cases like the following:

  • You need to install some pip packages
  • You use Bazel and your code's dependencies are not fully hermetic

There are two important types of images:

  • Runner image: This is the image used by the remote GPU workers ("runners") that the ExaDeploy scheduler orchestrates.
  • Local E2E image: This is an image that can be used to stand up a full ExaDeploy cluster within a single Docker container locally for end-to-end testing. This lets you avoid setting up or depending on cloud infrastructure to iterate on code.

Both types can be created in similar ways. The below documentation focuses on the runner image, but the instructions should also apply to the local E2E image.

Creating images with Bazel​

If your code requires dependencies that are not hermetically included in their Bazel rules, it's necessary to build a custom image to run and test code that runs with ExaDeploy. Here's an example:

WORKSPACE
load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_file")

# (Exafunction setup omitted)

http_file(
    name = "com_exafunction_layer_runner",
    downloaded_file_path = "runner_layer.tar.gz",
    url = "https://storage.googleapis.com/exafunction-dist/exafunction_runner_layer-0000000-000000000000.tar.gz",
    sha256 = "0075097b94782c5b19c0f48e5211b30061587dd13a0ac3071ccc32b865955878",
)

BUILD
load("@com_exafunction_dist//bazel:container_image.bzl", "runner_image")

runner_image(
    name = "runner_image",
    layer_archive = "@com_exafunction_layer_runner//file:runner_layer.tar.gz",
    base = "//foo:base_image",
)

Creating images with the Python CLI​

Consult the reference documentation for python3 -m exa docker here.