No space left on device on a CI runner
Fix "No space left on device" on a CI runner: remove unused toolchains, run docker system prune, and trim caches. Covers self-hosted runners and larger runners.
ENOSPC: no space left on device, write
Incident Overview
Fix "No space left on device" on a CI runner: remove unused toolchains, run docker system prune, and trim caches. Covers self-hosted runners and larger runners.
medium fix·5 min read
By Vishvesh Patel·DevOps Engineer·Updated 21 September 2026
ENOSPC: no space left on device, write failed to register layer: write /usr/lib/...: no space left on device Error: No space left on device ##[error]Process completed with exit code 1 fatal: write error: No space left on device
Do this first3 steps
Run these in order. Each one tells you what its output means before you change anything.
• Measure before deleting anything
Add this as the first step of the failing job. Knowing whether the disk is consumed by the preinstalled toolchain, Docker layers or your own build output decides which of the fixes below is worth applying.
• Reclaim Docker space, which is usually the largest consumer
On an ephemeral hosted runner this is safe, because nothing is shared with another job. On a self-hosted runner it destroys caches other jobs rely on, so use a filter such as --filter until=24h instead.
• On hosted runners, remove preinstalled toolchains you do not use
A GitHub-hosted Ubuntu runner ships with Android SDKs, .NET, GHC and several language runtimes, which together occupy tens of gigabytes. Removing the ones your job does not need reclaims more than anything else.
The runner's disk filled. Where the space went depends heavily on whether the runner is hosted or self-hosted, so measure before changing anything.
Add this as the first step of the failing job:
df -h / && docker system df 2>/dev/null
Hosted runners: the toolchain is the problem
A GitHub-hosted ubuntu-latest runner provides roughly 14 GB free on a 75 GB disk, because the image ships with a very large set of preinstalled software. Most jobs use almost none of it.
That reclaims 30 GB or more in about twenty seconds, and it is the single most effective fix for a Docker build that runs out of space on a hosted runner.
The jlumbroso/free-disk-space action packages the same idea with flags, if you prefer not to maintain the list.
Only delete what you do not need. Removing $AGENT_TOOLSDIRECTORY breaks actions/setup-node and its siblings, so do it only in jobs that do not use them.
- name: Free disk space
run: |
sudo rm -rf /usr/share/dotnet # ~20 GB
sudo rm -rf /usr/local/lib/android # ~12 GB
sudo rm -rf /opt/ghc # ~5 GB
sudo rm -rf /usr/local/share/boost
sudo rm -rf "$AGENT_TOOLSDIRECTORY"
df -h /Docker layers
Build Cache is frequently the largest item and the least visible. BuildKit keeps every intermediate layer.
On an ephemeral runner, prune freely:
On a self-hosted runner this is destructive: it deletes caches other jobs depend on, and the next few builds are slow. Prune by age instead:
Better, cap the cache so it never needs manual attention:
BuildKit then garbage-collects itself and the problem stops recurring.
docker system df
Build caches in the job
Package manager caches restored from a CI cache action count against the disk too, and they grow monotonically unless keyed properly:
A cache keyed on a lockfile hash is bounded. One keyed on npm-${{ github.sha }} creates a new entry per commit and, on a self-hosted runner with a local cache directory, fills the disk over weeks.
GitLab has an explicit limit:
policy: pull on jobs that only read the cache avoids re-uploading it from every parallel job.
- uses: actions/cache@v4
with:
path: ~/.npm
key: npm-${{ hashFiles('package-lock.json') }}
restore-keys: npm-Self-hosted runners: what accumulates
Four things, in order:
Build directories. GitLab keeps one per project per concurrent slot. Enable cleanup:
The runner cache directory. Nothing expires it by default. A cron job is the usual answer:
Docker volumes from containers that were never removed.
Logs. Journald with no cap:
Add a monitoring alert on disk usage for self-hosted runners. Discovering it through a failed build is the expensive way.
du -sh /var/lib/docker /home/gitlab-runner/builds /home/gitlab-runner/cache /var/log
Larger runners
If you genuinely need more, both platforms sell it. GitHub's larger runner classes come with more disk, and self-hosted runners have whatever you give them. That is worth doing for genuinely large builds and is not a substitute for bounded caches, since an unbounded cache fills any disk eventually.
A checklist
Frequently Asked Questions
Why does a hosted runner run out of disk so quickly?
Because the image ships with a very large preinstalled toolchain, leaving only around 14 GB free on a 75 GB disk. Android SDKs, .NET, GHC, multiple language runtimes and a set of cached Docker images occupy most of it before your job starts. A Docker build pulling a few large base images can exhaust the remainder easily. Deleting the toolchains your job does not use reclaims 30 GB or more in about twenty seconds and is usually the whole fix.
Is docker system prune safe in CI?
On an ephemeral hosted runner, yes, since the machine is destroyed after the job and nothing is shared. On a self-hosted runner it is destructive: -a removes every image not currently in use and --volumes removes named volumes, so other jobs lose their caches and the next several builds are slow. There, prune by age with --filter until=24h, or better, configure BuildKit garbage collection with defaultKeepStorage so the cache stays bounded automatically.
Why does my self-hosted runner's disk fill up over weeks?
Four things accumulate and none of them expires by default: Docker images and build cache, per-project build directories, the runner's cache directory, and system logs. The cache directory in particular grows forever if your cache key includes the commit sha, since every commit creates a new entry. Cap the Docker builder, set GIT_CLEAN_FLAGS: -ffdx, add a cron job to delete cache files older than a week, and vacuum journald.
How should I key a CI cache so it does not grow without bound?
On a hash of the dependency lockfile, for example npm-${{ hashFiles('package-lock.json') }}, with a restore-keys prefix for partial matches. That produces a new cache entry only when dependencies actually change. Keying on the commit sha creates one per commit, which on hosted runners hits the platform's cache quota and on self-hosted runners fills the local disk. Also use a read-only cache policy on jobs that do not modify dependencies.
Will a larger runner fix this?
It buys headroom and does not fix an unbounded cache, which will fill any disk given enough time. Larger runner classes are genuinely worth it for builds with large images or heavy compilation, where even a clean runner is tight. Treat it as capacity planning rather than a remedy: apply the cache bounds and cleanup first, then size the runner to what the job actually needs.
Reference and practice
Learn the underlying concept
Other CI/CD errors
Free, comprehensive DevOps tutorials and learning roadmaps. Master Docker, Kubernetes, CI/CD, and more.
Get notified about new tutorials and features.
• Roadmaps
• Interview Prep
• Certifications
• Company
© 2026 DevOpsLesson. All rights reserved.
Need to practice this scenario?
Simulate this error condition and practice recovery commands safely in our in-browser terminal.