Linux OOM Killer: What It Is and How to Fix It

Overview

The Linux OOM (out of memory) killer is a kernel mechanism that kicks in when your server has exhausted all available RAM and swap space. Rather than let the entire system lock up, the kernel picks one or more processes to terminate — freeing memory so the system can keep running. From a user’s perspective, it looks like MySQL randomly crashed, or Apache just stopped, with no clear reason in the application logs.

On shared hosting you’ll rarely see this directly, but on a VPS or dedicated server it’s a common headache — especially if you’re running PHP-FPM, MySQL, Redis, and a Node.js app on a plan with 1–2 GB of RAM. The OOM killer isn’t a bug. It’s doing its job. The real problem is that your server is running out of memory, and the killer is just the last line of defence.

This article covers how to confirm the OOM killer fired, how to identify what’s consuming memory, and how to either tune the killer’s behaviour or address the underlying memory pressure.

Prerequisites

• Root or sudo access to your Linux server
• SSH access (PuTTY, Terminal, or similar)
• A basic understanding of Linux processes is helpful but not required
• Applies to: Ubuntu 20.04/22.04/24.04, Debian 11/12, AlmaLinux 8/9, Rocky Linux 8/9, CentOS 7/8

Step 1: Confirm the OOM Killer Actually Fired

Before you start tuning anything, confirm that the OOM killer is what killed your process. Don’t assume — check the kernel logs.

sudo dmesg | grep -i "oom|killed process"
# or for systemd-based systems:
sudo journalctl -k | grep -i "oom|killed process"

You’re looking for lines like this:

Out of memory: Killed process 14532 (mysqld) score 842 or sacrifice child
Oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=/,mems_allowed=0,global_oom,task_memcg=/system.slice/mysql.service,task=mysqld,pid=14532,uid=999

The “score” (or oom_score) is how the kernel ranked that process as a candidate for termination. Higher score = more likely to get killed. If you see your process name in these lines, the OOM killer is confirmed.

📝 Note: On servers with log rotation enabled, older OOM events may have been rotated out of dmesg. Check /var/log/kern.log (Debian/Ubuntu) or /var/log/messages (RHEL-based) for historical records.

Step 2: Check Current Memory Usage

Once you’ve confirmed the killer fired, get a clear picture of where memory is going right now.

free -h
ps aux --sort=-%mem | head -20
cat /proc/meminfo | grep -E "MemTotal|MemFree|MemAvailable|SwapTotal|SwapFree"

Pay attention to the MemAvailable line in /proc/meminfo — it’s a better real-world indicator than MemFree because it accounts for reclaimable cache. If it’s consistently under 10% of total RAM, your server is under genuine memory pressure.

Step 3: Identify the Memory Hogs

The process that gets killed isn’t always the one causing the problem. MySQL might get killed because PHP-FPM quietly consumed 800 MB of RAM over six hours.

# Sort by actual memory usage (RSS = resident set size in KB)
ps aux --sort=-%mem | awk 'NR<=15{print $0}'

For a more detailed breakdown per process:

smem -r -k | head -20

If smem isn’t installed:

# Ubuntu/Debian
sudo apt install smem

# AlmaLinux/Rocky/CentOS
sudo yum install smem

⚠ Warning: Don’t immediately kill the memory-hungry process until you understand why it’s using that much. A PHP process using 500 MB might be handling a legitimate large import job — killing it mid-run can corrupt data.

Step 4: Add or Expand Swap Space

If your server has little or no swap, that’s often the first thing to fix. Swap isn’t a replacement for RAM, but it gives the kernel somewhere to offload memory pages and buys you time. I’d recommend at least 1–2 GB of swap on any VPS with under 4 GB of RAM.

# Check existing swap
swapon --show

# Create a 2 GB swap file
sudo fallocate -l 2G /swapfile
sudo chmod 600 /swapfile
sudo mkswap /swapfile
sudo swapon /swapfile

# Make it persistent across reboots
echo '/swapfile none swap sw 0 0' | sudo tee -a /etc/fstab

📝 Note: On SSDs (which most of our VPS SSD Hosting plans use), swap performance is reasonable. On spinning disk it’s noticeably slower, but still better than the OOM killer terminating MySQL.

Step 5: Tune the OOM Score for Critical Processes

You can tell the kernel to protect specific processes from being killed first by lowering their OOM score adjustment. The range is -1000 (never kill) to +1000 (kill first).

# Check the current OOM score of a process (replace PID with the actual process ID)
cat /proc/<PID>/oom_score
cat /proc/<PID>/oom_score_adj

To protect MySQL from being killed before less critical processes:

# Find MySQL's PID
pgrep -a mysqld

# Lower its OOM score adjustment
echo -500 | sudo tee /proc/<PID>/oom_score_adj

That’s temporary and resets on reboot. To make it persistent via systemd (for MySQL on AlmaLinux/Ubuntu with systemd):

sudo systemctl edit mysql

Then add:

[Service]
OOMScoreAdjust=-500

⚠ Warning: Setting OOMScoreAdjust=-1000 (which disables OOM killing for that process entirely) can cause a full system hang if that process is the one actually consuming all the memory. Use -500 to -800 as a safer middle ground for critical services.

Step 6: Fix the Underlying Memory Problem

Tuning OOM scores is a band-aid. The real fix is reducing memory consumption or adding more RAM. Common culprits on hosting servers:

• MySQL innodb_buffer_pool_size too high — a common mistake is setting this to 70–80% of total RAM on a server running multiple services. Set it to 40–50% if MySQL isn’t the only service.
• PHP-FPM pm.max_children set too high — each child process uses 30–80 MB depending on your app. On a 2 GB server with WordPress, 20 children is often already too many.
• Memory leaks in long-running PHP or Node.js processes — monitor memory usage over time with ps or set up a cron to log it hourly.
• Redis maxmemory not configured — Redis will consume all available RAM if you haven’t set a cap in /etc/redis/redis.conf.

Common Issues and Troubleshooting

MySQL keeps getting killed despite having enough RAM

MySQL is often the highest-scoring process because it uses a lot of anonymous memory (the InnoDB buffer pool). Even on a server with available RAM, if innodb_buffer_pool_size is set aggressively, MySQL will have a high OOM score. Lower the buffer pool size in /etc/mysql/mysql.conf.d/mysqld.cnf (Debian/Ubuntu) or /etc/my.cnf (RHEL-based) and restart MySQL. Alternatively, adjust its OOM score as covered in Step 5.

OOM killer fires overnight with no obvious traffic spike

Scheduled tasks are almost always the cause. Cron jobs running backups, log processing, or database exports at 2 AM can spike memory hard and fast. Check /var/log/cron or sudo journalctl -u cron and cross-reference the timestamps with your OOM killer log entries. Staggering your cron jobs and adding nice and ionice prefixes can help.

Server is sluggish but the OOM killer hasn’t fired yet

This is often swap thrashing — the system is paging memory in and out so aggressively that the server grinds to a near-halt. Check with vmstat 1 10 and look at the si (swap in) and so (swap out) columns. If those numbers are consistently non-zero, you’re in a pre-OOM state. Lowering vm.swappiness to 10 can help: sudo sysctl vm.swappiness=10 and persist it in /etc/sysctl.conf.

The wrong process keeps getting killed

If a low-priority worker process has a high memory footprint, the kernel may prefer killing MySQL over it even though the worker is the real problem. Raise the OOM score of your worker processes (oom_score_adj closer to +500) so the kernel targets them first. This is especially relevant if you’re running custom queue workers or crawlers alongside production databases.

OOM errors appear in logs but free memory looks fine

This is a less obvious one. Some OOM kills happen not because of total memory exhaustion, but because of memory fragmentation or cgroup memory limits. If your server runs Docker containers or uses systemd slices with memory limits, a container can trigger its own OOM kill even when the host has free RAM. Check cgroup limits with systemctl status <service> and look for MemoryMax or MemoryLimit settings.

FAQ