Skip to content

Run independent steps at the same time inside a job

In the long-pole job, start independent steps such as image pulls, dependency installs, per-target builds and packaging together, wait for them before the step that needs their results, and keep every failure visible.

Saves wait timelow risk

Signals in your baseline

  • The long pole's rows in "Steps inside the long-pole jobs" show two or more slow steps in a row that don't use each other's output, such as an image pull followed by a dependency install.
  • A step builds, compresses or uploads several targets one at a time in a loop.
  • The long pole starts a container, database or emulator only after an unrelated install step has finished.

Why it works

Steps in a job run one after another unless you say otherwise. An image pull waits on the network, a dependency install waits on the network and the disk, and a loop that compiles one target at a time leaves the other cores idle. When two steps don't use each other's output, starting them together makes the pair take as long as the slower step instead of the sum of both.

This shortens the run only in a job on the critical path, and only when the steps don't compete for the same resource. Two CPU-bound compiles on a 2-core runner won't finish sooner side by side.

Apply it

Parallel and background steps

GitHub Actions can run steps of one job concurrently. A parallel group starts its steps together and waits for all of them. A step with background: true keeps running while later steps start, until a wait step names it.

jobs:
  e2e:
    runs-on: ubuntu-24.04
    steps:
      - uses: actions/checkout@v7
      - uses: actions/setup-node@v7
        with:
          node-version: 24
          cache: npm
      - parallel:
          - name: Pull service images
            run: docker compose -f compose.ci.yml pull --quiet
          - name: Install dependencies
            run: npm ci
      - name: Build
        id: build
        run: npm run build
        background: true
      - name: Start services
        run: docker compose -f compose.ci.yml up --detach --wait
      - wait: build
      - run: npm run test:e2e
  • The parallel group ends with an implicit wait, so the build starts only after the pull and the install have both finished.
  • The build runs in the background while the services start. wait: build blocks until it's done, and fails the job if the build failed.
  • Outputs and environment changes from a background step, including GITHUB_PATH changes made by setup actions, become visible only after the wait.
  • A job runs at most 10 background steps at once. Composite actions can't declare background steps.

These keywords arrived in June 2026. If you use self-hosted runners, check that their runner version supports them before you rely on them.

Shell background jobs

Inside one run script, start a command with & and wait for its process ID:

docker compose -f compose.ci.yml pull --quiet &
pull_pid=$!
npm ci
wait "$pull_pid"
  • wait "$pull_pid" returns the background command's exit status, so a failed pull fails the step. A bare wait with no arguments returns 0 even when a background command failed.

Loops over targets

Replace a loop that handles one target at a time with a pool of workers:

find dist -name '*.tar' -print0 | xargs -0 -n 1 -P "$(nproc)" zstd -q -19 --rm
  • -P "$(nproc)" runs one worker per core, and xargs exits with status 123 if any invocation failed, so the step still fails.
  • Build tools have their own switches: make -j"$(nproc)", gradle --parallel and Turborepo, which runs the tasks in one turbo run lint typecheck call concurrently. Cargo already compiles crates in parallel.

Keep it safe

  • Check every exit code. A background command nobody waits for can fail silently while the job goes green. Wait on native background steps explicitly where the result matters, and wait on each process ID in shell scripts.
  • Don't overlap steps that write the same files. Two installs into one node_modules, two Gradle builds in one project directory, or a step that reads a directory another step is still writing can give corrupt or flaky results.
  • Keep parallelism at or below the runner's cores (nproc) and watch memory. A runner that runs out of memory loses contact with GitHub, and the job fails with an infrastructure error instead of a clear message.
  • Don't move credential setup, such as a registry login, into a background step that later steps depend on. Its environment changes appear only after the wait, so those steps would run unauthenticated.
  • Job names don't change, so required checks keep matching.

Verify

  • In the branch baseline's Steps inside the long-pole jobs, the overlapped steps should take about as long as the slowest of them, and the job's p50 duration should fall by the rest.
  • Runner minutes fall by the same amount, because the job finishes sooner.
  • Break one overlapped command on a branch, for example with a wrong image tag, and confirm the job fails.

Sources