We have a lot of historical tests that violate this line, but our goal is to only maintain tests in this repository which apply to more than one image - for example, we have tests for Python that apply to both python and pypy.

If you're using our test suite to test your image in your own CI, see the "Alternate config files" section below for information about how to run both our tests and your own from our (admittedly naïve) test wrapper.

$ ./run.sh usage: run.sh [-t test ...] image:tag [...]  ie: run.sh debian:trixie  run.sh -t utc python:3  run.sh -t utc python:3 -t python-imports This script processes the specified Docker images to test their running environments.

Run all the tests that are applicable to the debian:trixie image:

$ ./run.sh debian:trixie testing debian:trixie 'utc' [1/4]...passed 'no-hard-coded-passwords' [2/4]...passed 'override-cmd' [3/4]...passed 'debian-apt-get' [4/4]...passed

Try to run just the python-imports test against the debian:trixie image: (which doesn't contain Python)

$ ./run.sh -t python-imports debian:trixie testing debian:trixie image has no tests...skipping

Run the utc and python-imports tests against python:3, pypy:3, and debian:trixie:

$ ./run.sh -t utc -t python-imports python:3 pypy:3 debian:trixie testing python:3 'utc' [1/2]...passed 'python-imports' [2/2]...passed testing pypy:3 'utc' [1/2]...passed 'python-imports' [2/2]...passed testing debian:trixie 'utc' [1/1]...passed

Each test lives in a separate directory inside tests/, and config.sh determines which images each test applies to.

This is the actual "test script". In the script, $1 is the name of the image we're testing. An exit code of zero indicates success/passing, and a non-zero exit code indicates failure.

For example, tests/utc/run.sh consists of:

#!/bin/bash set -e docker run --rm --entrypoint date "$1" +%Z

Which runs date inside the given image, and outputs the current timezone according to date +%Z, which should be UTC. See tests/<test name>/expected-std-out.txt below for the other half of this test.

There are several pre-written run.sh scripts to help with common tasks. See the tests/run-*-in-container.sh section below for more details about these.

If this file exists, then this test's run.sh is expected to output exactly the contents of this file on standard output, and differences will be assumed to denote test failure.

To continue with our utc test example, its contents in this file are simply:

UTC 

If a test fails due to having incorrect output, the diff between the generated output and the expected output will be displayed:

$ ./run.sh -t utc fedora:21 testing fedora:21 'utc' [1/1]...failed; unexpected output: --- tests/utc/expected-std-out.txt	2015-02-05 16:52:05.013273118 -0700 +++ -	2015-03-13 15:11:49.725116569 -0600 @@ -1 +1 @@ -UTC +EDT

(this is an example of fedora:21 failing the utc test)

These scripts are intended to be used as symlink targets for test run.sh scripts.

For example, if tests/python-sometest/run.sh is a symlink to ../run-python-in-container.sh, then tests/run-python-in-container.sh will execute python tests/python-sometest/container.py inside the image.

To illustrate further:

$ ls -l tests/hylang-sh/ total 8 -rw-rw-r-- 1 tianon tianon 167 Mar 11 13:24 container.hy -rw-rw-r-- 1 tianon tianon 5 Mar 11 13:24 expected-std-out.txt lrwxrwxrwx 1 tianon tianon 25 Mar 11 13:25 run.sh -> ../run-hy-in-container.sh $ ls -l tests/python-imports/ total 4 -rw-r--r-- 1 tianon tianon 1096 Aug 29 16:14 container.py lrwxrwxrwx 1 tianon tianon 29 Dec 15 2016 run.sh -> ../run-python-in-container.sh $ head -n3 tests/python-imports/container.py import platform, sys isWindows = platform.system() == 'Windows'

This file controls which tests apply (or don't apply) to each image.

When testing an image, the list of tests to apply are calculated by doing globalTests + imageTests[testAlias[image]] + imageTests[image]. Any tests for which globalExcludeTests[image_test] is set are removed. If -t is specified one or more times, any tests not specified explicitly via -t are also removed.

This list of tests applies to every image minus combinations listed in globalExcludeTests (such as hello-world not getting the utc test, since it has no date binary in order for the test to work).

globalTests+=(	utc	no-hard-coded-passwords )

This array defines image aliases -- for example, the pypy image should inherit all the standard python image tests, since it's a functionally equivalent implementation of Python. As noted in globalTests, any image+test combinations that shouldn't or can't be tested as a result of this aliasing can be excluded via globalExcludeTests.

testAlias+=(	[pypy]='python'	[jruby]='ruby'	[mariadb]='mysql'	[percona]='mysql' )

This array defines the list of explicit tests for each image. For example, the mysql image (and aliased derivatives via testAlias) has a test which verifies that basic functionality works, such as starting up the image and connecting to it from a separate container.

imageTests+=(	[python]=' python-imports python-pip-requests-ssl '	[mysql]=' mysql-basics ' )

Defines image+test combinations which shouldn't ever run (usually because they won't work, like trying to run date in the hello-world image for the utc test, which has only one binary).

globalExcludeTests+=( # single-binary images	[hello-world_utc]=1 )

If you would like to run the Official Image tests against your own images, you can use the -c/--config flag to specify one or more alternate config files. These config files should configure the same environment variables used by the default config.sh (see above).

imageTests+=(	[myorg/myimage]=' my-custom-test ' )

Note: If you do use your own config file, the config.sh included here will no longer be loaded by default. If you want to load it in addition to your own config file (for example, to run the globalTests against your own image), use an additional --config flag.

$ /path/to/official-images/test/run.sh --config /path/to/official-images/test/config.sh --config ./my-config.sh myimage