Systems Administration

inotify vs Polling: Measured Wake Latency, Queue Overflow, and the Non-Recursive Trap

A blocked inotify read wakes in about 0.087 milliseconds. A 10 millisecond stat poller takes about 6.1 milliseconds on median, and a 100 millisecond poller about 40.9 milliseconds. The queue holds exactly 16,384 events before it overflows, and a directory watch never sees a grandchild change.

Written November 9, 2026 · Tested with GCC 13.3.0 · 14 min read

By the end of this guide
  • The wake latency table
  • The 1 second poll row is first-interval biased
  • Idle CPU: the poller pays even when nothing changes
  • Queue overflow: 20,000 creates, 16,384 events kept, the rest dropped
What you need
The probe is a small C program, compiled with gcc 13.3.0 with optimization level 2 and all warnings enabled, zero warnings. A parent process and a child watcher are forked from one process and share a memory page carrying the write time.

What this guide is

This guide times the two ways a Linux program learns that a file changed: a blocked read on an inotify file descriptor, and a loop that stats the file every few milliseconds. The manuals describe both mechanisms and state the rules, but they give no wake latency figures, so a builder choosing between them has only folklore: inotify is instant, polling is slow.

The headline is a ratio, not a slogan. A blocked inotify read wakes in about 0.087 milliseconds. A 10 millisecond poller takes about 6.1 milliseconds on median, about 70 times slower. A 100 millisecond poller takes about 40.9 milliseconds on median, about 470 times slower. The ratios are stable across both nights even though the absolute poll figures moved, and the directions never changed. The traps matter more than the ratios for a real watcher: the queue holds exactly 16,384 events and then drops the rest with a single overflow marker, and one watch covers one directory only, never its subdirectories.

The nearest existing pages do not publish a measured table. One copyprogramming page says inotify tools are 10 to 100 times faster than polling on large directories, per benchmarks it never cites, and its comparison table is qualitative: find and diff are slow, inotifywait is instant. An agent-island post and a dev.to post both make the same qualitative point with a 6 second poll interval: state can be 6 seconds late even when parsing takes only a few milliseconds. A second copyprogramming guide compares FileSystemWatcher with polling on .NET and calls the event path real time, with no timings. A knobs-dials page on file polling is an admitted stub. None of them times an inotify wake against a stat poll at three intervals, and none of them executes the overflow or the non-recursive case.

The setup

The probe is a small C program, compiled with gcc 13.3.0 with optimization level 2 and all warnings enabled, zero warnings. A parent process and a child watcher are forked from one process and share a memory page carrying the write time. The writer records the monotonic time immediately before the write syscall, then appends 1 byte and syncs the file. The watcher timestamps detection on the same monotonic clock, so latency is detection time minus write time on one clock, with no wall-clock drift in the number. The writer fires 50 to 500 milliseconds after the watcher starts, jittered uniformly, so the latency is a distribution across phase offsets rather than one fixed offset.

The poll watcher stats the file and reports the first stat where the modification time or the size differs from the baseline. The inotify watcher blocks in read on the descriptor with an IN_MODIFY watch on the file and reports when the event arrives. All 30 repetitions per case verified correct: the inotify event carried the right watch descriptor and an IN_MODIFY mask, and the poll watcher saw the change in 30 of 30 repetitions with no missed touch.

The inotify limits read 16,384 for max queued events and 524,288 for max user watches. Every number below is a WSL2 number, not a bare-metal number. Wherever the two nights differ, both figures are shown.

A minimal watcher looks like this, and it compiles and runs as shown:

C
#include <stdio.h>
#include <unistd.h>
#include <sys/inotify.h>

int main(void)
{
    int fd = inotify_init();
    int wd = inotify_add_watch(fd, "target", IN_MODIFY);
    char buf[4096];
    int n = read(fd, buf, sizeof buf);
    printf("events bytes=%d wd=%d\n", n, wd);
    return 0;
}

The wake latency table

Medians of 30 repetitions, milliseconds. The 90th percentile and the maximum are shown because a watcher that is fast on median and slow on the tail still wakes late sometimes.

Watcher Median 90th percentile Maximum
inotify, blocked read 0.087 0.103 0.117
poll, 10 millisecond interval 6.066 8.964 10.245
poll, 100 millisecond interval 40.900 85.049 100.432
poll, 1000 millisecond interval 768.216 903.683 948.370

Read the table as a poller paying about half its interval on average. The 10 millisecond poller averages 5.587 milliseconds, close to half its interval. The 100 millisecond poller averages 44.826 milliseconds, close to half its interval. The inotify median of 0.087 milliseconds is about 70 times faster than the 10 millisecond poller and about 470 times faster than the 100 millisecond poller, on median. The inotify maximum of 0.117 milliseconds is still about 50 times faster than the 10 millisecond poller median.

The night-before figures, measured with the same probe on the same box, were 0.095, 6.370, 54.695, and 768.240 milliseconds. The inotify median moved from 0.095 to 0.087 milliseconds, the 10 millisecond poller from 6.370 to 6.066, and the 1000 millisecond poller from 768.240 to 768.216, all inside run-to-run noise. The 100 millisecond poller moved more, from 54.695 to 40.900 milliseconds, with its mean moving from 54.537 to 44.826. That movement is the poller sampling a different phase mix across its interval, not a change in the mechanism: the mean stays near half the interval on both nights, and the maximum stays near the interval itself, 101.658 milliseconds the night before and 100.432 today. The poll maximum slightly exceeds the interval on both nights, by about 1 to 2 milliseconds, which is scheduler and timer delay on WSL2 rather than a clock error: no repetition anywhere in either run measured a negative latency.

The 1 second poll row is first-interval biased

The 1000 millisecond median of 768.216 milliseconds is not the uniform-phase median for a 1 second poller, and this guide does not present it as one. The writer fired 50 to 500 milliseconds after the watcher started, always inside the first 1000 millisecond interval, so the measured latency is 1000 milliseconds minus the jitter: the minimum of 505.209 milliseconds is about 1000 minus 495, and the maximum of 948.370 milliseconds is about 1000 minus 52. The true uniform-phase expectation for a 1000 millisecond poller is a mean and median of 500 milliseconds, with a worst case of 1000 milliseconds. The measured row is the first-interval case, and the theory is the number to plan against.

The 10 and 100 millisecond rows need no such caveat. The writer jitter spans 50 to 500 milliseconds, which covers many 10 millisecond intervals and several 100 millisecond intervals, so those rows sample phase uniformly. The 1000 millisecond row is the only one where the jitter window fits inside a single interval.

Idle CPU: the poller pays even when nothing changes

Ten seconds of quiet, CPU time from getrusage, user plus system time. The inotify watcher blocks in read and the kernel wakes it only on an event. The poller wakes on every tick whether or not the file changed.

Watcher CPU in 10 seconds idle
inotify, blocked on read 0.130 milliseconds
poll, 10 millisecond interval 18.783 milliseconds
poll, 100 millisecond interval 2.866 milliseconds
poll, 1000 millisecond interval 0.400 milliseconds

The 10 millisecond poller burns about 144 times the idle CPU of inotify, 18.783 milliseconds against 0.130 in 10 seconds. The 100 millisecond poller burns about 22 times, and the 1000 millisecond poller about 3 times. The night-before figures were 0.152, 19.522, 2.794, and 0.422 milliseconds, the same shape within noise. The cost scales with the tick rate, as it must: a 10 millisecond poller stats the file about 1,000 times in 10 seconds, and each stat costs a syscall even when the answer is unchanged.

Queue overflow: 20,000 creates, 16,384 events kept, the rest dropped

The flood creates 20,000 distinct files in a watched directory, at 12,110 creates per second over 1.651 seconds, while the inotify reader sleeps for 2 seconds and then drains the queue. The queue held exactly 16,384 IN_CREATE events, then delivered exactly 1 IN_Q_OVERFLOW event. The remaining 3,616 creates were silently dropped, with no per-file record of which files were lost. A stat-based poll of the same directory sees the change the overflow dropped: the directory entry count goes from 0 to 20,000, and the poll reports the change.

The limit is the box default, read from the kernel setting: 16,384 queued events. The manual states the rule the probe confirms: events in excess of the limit are dropped, but an IN_Q_OVERFLOW event is always generated. The overflow event carries a watch descriptor of negative 1, which is how the reader tells it apart from a file event. The design consequence is direct: a watcher that reads slowly during a burst does not get a late copy of the burst, it gets a single marker saying the burst was lost, and the only recovery is a full rescan.

Flooding appends to one file does not overflow. The fresh coalescing check appends 1 byte 20,000 times to a single watched file with no reader running, and the queue holds exactly 1 event with 0 overflows. The manual states why: successive identical events are coalesced into a single event while the older event has not yet been read. Overflow needs distinct files or distinct events, not repeated writes to the same file. A watcher that counts IN_MODIFY events as a count of writes is counting coalesced events, not writes.

The non-recursive trap: a grandchild change is invisible

Trap
The watch is IN_MODIFY on a directory named nr, with no log file inside it. Touching nr/sub/target, a grandchild, produces 0 inotify events in a full 2 second window, and the poll of the same path sees the change. Touching nr/direct, a direct child, as the control, produces exactly 1 event, mask 0x00000002, which is IN_MODIFY, name direct, not a directory. The manual states the rule the probe confirms: inotify monitoring of directories is not recursive, and monitoring subdirectories needs additional watches, one per subdirectory.

The failure mode is silent. Nothing errors, nothing warns, and the watch descriptor stays valid. The watcher simply never learns that the grandchild changed, and the only signal that something is wrong is the absence of events. A recursive watcher has to be built by adding a watch per subdirectory, and it has to rescan when a new subdirectory appears, because a subdirectory created after the scan starts has no watch until one is added for it. The poll fallback sees the grandchild because it stats the path directly, which is the same reason a periodic rescan repairs both the overflow case and the non-recursive case: the poll does not depend on the event stream at all.

What the manual already tells you

A blocking read on the inotify descriptor blocks until at least one event occurs. A watch on a directory returns events for the directory itself and for files inside it, with the event name identifying the file within the directory. Successive identical events are coalesced into a single event while the older event has not yet been read, which means an application cannot use inotify to reliably count file events. Monitoring of directories is not recursive: additional watches must be created to monitor subdirectories. The queue limit is set when the inotify instance is created, events in excess of the limit are dropped, and an IN_Q_OVERFLOW event is always generated. The probe adds the numbers the manual does not give: the blocked read wakes in about 0.087 milliseconds, the queue holds exactly 16,384 events before it overflows, and a grandchild touch produces zero events in a full 2 second window.

What this guide does not cover

The scope is WSL2 only: kernel 6.18.33.2-microsoft-standard, Ubuntu 24.04.4, gcc 13.3.0, 32 virtual CPUs. A bare-metal re-run would strengthen the numbers, and the directions are the portable part, not the exact latency figures. The scope is one file watched for modification, and one directory watched for creates and child modifications. Watching thousands of files spreads the queue across watches but shares the same 16,384 event limit per inotify instance, and this guide does not measure that shape. The scope is local filesystems only: the manual notes that inotify does not apply to mounted remote filesystems in most cases, because a remote change involves no local syscall, and this guide does not test a remote mount. The poller here stats one path; a poller that lists a whole directory tree pays a scan cost per tick that grows with the tree, and this guide does not measure that cost. The latency figures are wake latencies, not end-to-end reaction times: the time from event delivery to the application acting on it depends on what the application does next, and that part is unmeasured here.

Checklist

  • Before choosing a poll interval, divide it by two: a poller pays about half its interval on average, and the worst case is the full interval, so a 1 second poller can be 1 second late.
  • Before trusting a 1 second poll median, check the phase: a writer that fires inside the first interval biases the median toward the interval length, and the true uniform-phase median is half the interval.
  • Before counting IN_MODIFY events as writes, remember coalescing: 20,000 appends to one file produced 1 event, so the event count is not a write count.
  • Before relying on a directory watch during a burst, check the queue: 20,000 distinct creates kept exactly 16,384 events and dropped 3,616 with a single overflow marker, so a slow reader needs a rescan, not a longer queue.
  • Before watching a tree with one watch, add the subdirectories: a grandchild touch produced 0 events in 2 seconds, while the direct-child control produced 1, so one watch covers one directory only.
  • Before assuming idle is free, measure the tick rate: the 10 millisecond poller burned about 144 times the idle CPU of the blocked inotify read, while the 1 second poller burned about 3 times.
  • Before quoting a wake time from one night, run the probe again: the inotify median moved from 0.095 to 0.087 milliseconds and the 100 millisecond poll median from 54.695 to 40.900 milliseconds between nights, while every direction held.

Sources

Keep Exploring