inotify and directory change tracking | Linux System Programming

QUESTION

Can the current directory state always be reconstructed from inotify events alone?

inotify supplies an event stream for filesystem-object changes, but it is neither a complete transaction log nor a recursive snapshot. When the queue overflows, only one IN_Q_OVERFLOW remains; because the omitted changes are unknown, a full rescan is required.

A directory watch reports changes to the directory itself and events for its immediate entries; it does not automatically add watches to newly created child directories. A revalidation procedure must handle changes between event processing and watch addition.

STRUCTURE

Structure diagram

Figure 1. A watch tree and one inotify event queue

inotify instance fdgroup queue · max_queued_events · overflow flag

wd=1 /project

IN_CREATE
IN_MOVED_*
child name record

wd=2 /project/src

separate watch mark
inode event
rename cookie

new /project/build

CREATE|ISDIR
scan required
add_watch required

queue

wd/mask/cookie/name
variable-length record
IN_Q_OVERFLOW → rescan

Watches are not recursive. A new subdirectory needs its own mark, and the entire tree must be rescanned after queue overflow.

CALL PATH

Call path

Figure 2. From userspace code to observable results

inotify_add_watch connect path to wd

fsnotify mark register on inode/mount

filesystem change generate event

read records variable-length batch

reconcile recover overflow/rename

Do not use events as final state. Treat them as invalidation hints that reduce rescans, and define consistency requirements separately between the event sequence and the actual directory scan.

Figure 3. Major points along the kernel-internal path

fsnotify hook inode operation

inotify_handle_inode_event construct mask/name

group queue event merge/limit

inotify_read user buffer copy

wd lookup merge application state

This diagram is not for memorizing function names. Follow which return value, file descriptor, memory mapping, or wait queue is passed to the next stage.

SOURCE COORDINATES

Linux 6.18.37 LTS source locations

Go beyond the glibc function to the files where the syscall implementation meets kernel objects. Each link points to the original file at the same tag.

File	Function / structure	What to inspect
fs/notify/inotify/inotify_user.c	inotify_add_watch(), inotify_read()	Create a wd and return variable-length events
fs/notify/inotify/inotify_fsnotify.c	inotify_handle_inode_event()	Convert an fsnotify event to inotify format
fs/notify/notification.c	fsnotify_add_event()	Group queue limits and overflow handling

COMPLETE PROGRAM

Complete runnable example

The code below is not pseudocode with explanatory lines omitted. It is a minimal example that can be built and run as a file.

Buildcc -std=c17 -Wall -Wextra -O2 watch_dir.c -o watch_dir

01#define _GNU_SOURCE
02#include <errno.h>
03#include <stdio.h>
04#include <sys/inotify.h>
05#include <unistd.h>
06
07int main(int argc, char **argv)
08{
09    const char *path = argc > 1 ? argv[1] : ".";
10    int fd = inotify_init1(IN_CLOEXEC);
11    if (fd < 0)
12        return 1;
13    int wd = inotify_add_watch(fd, path,
14        IN_CREATE | IN_DELETE | IN_MOVED_FROM | IN_MOVED_TO | IN_Q_OVERFLOW);
15    if (wd < 0)
16        return 1;
17
18    _Alignas(struct inotify_event) char buffer[8192];
19    for (;;) {
20        ssize_t count = read(fd, buffer, sizeof(buffer));
21        if (count < 0 && errno == EINTR)
22            continue;
23        if (count <= 0)
24            break;
25        for (char *p = buffer; p < buffer + count; ) {
26            struct inotify_event *event = (struct inotify_event *)p;
27            printf("wd=%d mask=0x%x cookie=%u name=%s\n",
28                   event->wd, event->mask, event->cookie,
29                   event->len ? event->name : "-");
30            p += sizeof(*event) + event->len;
31        }
32    }
33    close(fd);
34    return 0;
35}

CODE NOTES

Code notes

Source line 10inotify_init1(IN_CLOEXEC)

Creates the event queue as an fd and prevents exec inheritance. When integrating with epoll, use IN_NONBLOCK as well.

Source line 14IN_MOVED_FROM | IN_MOVED_TO

Within one inotify instance, the two sides of a rename can be paired by cookie. A move across filesystems can look like create/delete.

Source line 18_Alignas(struct inotify_event)

Gives the char buffer the alignment required to read inotify_event records.

Source line 25p < buffer + count

One read contains several variable-length records, so walk them within the returned byte range.

Source line 30sizeof(*event) + event->len

event->len includes the name and padding. Advancing by strlen(name) alone loses alignment for the next record.

DETAILS

Detailed behavior

A wd is not a pathname

A watch descriptor is an integer key within an inotify instance. An inode watch may continue after the watched object is renamed, and the same wd number can be reused after IN_IGNORED.

Store wd, generation, and the currently estimated path together in the application map to distinguish stale events.

Rename pairing needs a timeout

An IN_MOVED_FROM may not have a corresponding IN_MOVED_TO in the same queue. The object may move outside the watched tree or overflow may occur.

Do not keep cookie-map entries forever; after a short timeout, finalize them as deletion or an out-of-tree move.

Overflow is a full-rescan boundary

After IN_Q_OVERFLOW, it is impossible to infer which entries changed. Drain the queue, rebuild state with an authoritative directory scan, and verify the watch set too.

Measure event-processing speed, max_queued_events, and bursty build output, but do not replace a correctness protocol merely by raising the limit.

OBJECTS

Objects and lifetimes

Object	Creation and release	Values to inspect
`inotify group`	Created by inotify_init1; the queue and marks are released when the fd closes	queue length, overflow state
`watch mark/wd`	Attached to an inode by add_watch and removed by rm_watch/object deletion/close	mask, inode, generation
`inotify_event record`	Copied from the kernel queue to the read buffer and consumed by the application	mask, cookie, len, name

FAILURE PATH

Failure conditions and common misconceptions

Observed symptom	Likely causes	How to verify
Changes are missing	IN_Q_OVERFLOW or a change before the watch was added	Handle overflow and perform a full rescan
A rename has no matching half	Movement outside the watched tree or across a queue boundary	cookie timeout policy
Changes in child directories are absent	Recursive watching was assumed to be automatic	Scan the new directory and call add_watch

LAB

Verify it yourself

Rename an entry with mv inside the watched directory and verify that the FROM/TO cookies match.
Create many files in a short period to induce queue overflow and test the rescan path.
Implement a recursive tracker that scans and adds a watch immediately after receiving creation of a new child directory.

Run./watch_dir .

Tracestrace -e trace=inotify_init1,inotify_add_watch,read,close ./watch_dir .

PRIMARY REFERENCES

Primary sources

inotify(7)inotify_add_watch(2)Linux fsnotify