Glossary

/proc: A pseudo-filesystem: proc(5) calls it "an interface to kernel data structures". The files in it are not stored on any disk. The kernel writes the text of a file at the moment you read it.; Reading /proc
/proc/<pid>: One directory per process, named by its PID, with files such as stat, status, cmdline, environ, limits and the fd/ directory (chapter 05). /proc/self points at the reader's own directory.; Reading /proc
%util: The share of the interval during which the device had at least one I/O request in flight. iostat computes it from the kernel's io_ticks counter.; Disk I/O and iostat
%wait: A pidstat column: the share of CPU time a task spent "waiting to run". It counts time in the run queue. Time waiting for disk shows up elsewhere, as wa in vmstat.; A slow server in 60 seconds
`/etc/sysctl.d`: A directory of .conf files with lines like net.core.somaxconn = 4096. At boot, systemd-sysctl.service applies them together with /run/sysctl.d, /usr/local/lib/sysctl.d and /usr/lib/sysctl.d.; Kernel parameters
`/proc/<pid>/ns`: A directory with one symbolic link per namespace of a process, such as net -> net:[4026531833]. Two processes are in the same namespace when the numbers match.; Namespaces
`cgroup.subtree_control`: The file where a cgroup turns controllers on for its children: echo "+cpu +memory" > cgroup.subtree_control. It creates the cpu.* and memory.* files in each child.; cgroups v2
`cpu.stat`: The cgroup file with the counters. nr_periods counts periods in which the group had work, nr_throttled counts the periods in which it was throttled, and throttled_usec adds up how long it was held back.; CPU limits and throttling
`errnoRet`: The error number a rule returns instead of running the call. Docker's profile sets it to ENOSYS (38, "Function not implemented") for clone3.; seccomp
128 + N: The rule bash, runc and tini use to fit a death by signal N into one exit code: SIGKILL (9) gives 137, SIGTERM (15) gives 143.; Exit codes
Anonymous memory: Pages that belong to no file: the heap, the stacks, mmap with MAP_ANONYMOUS, and private copies of file pages the process wrote to.; Finding a memory leak
Anonymous page: A page of memory with no file behind it: heap, stack, or mmap with MAP_ANONYMOUS. The kernel can free it only by writing it to swap.; Swap
apt: The front end that reads package lists from the sources in /etc/apt/sources.list.d/, works out which packages to install, upgrade or remove, downloads them and calls dpkg.; Packages
aqu-sz: The average number of requests in flight during the interval: the queue depth seen by the device. It was called avgqu-sz in older iostat versions.; Disk I/O and iostat
Arena: A pool of memory that glibc malloc hands out from, with its own lock. The main arena grows the [heap] with brk; other arenas live in mmaped regions of up to 64 MiB.; Finding a memory leak
await: The average time in milliseconds from the moment a request is created in the block layer to its completion: queue time plus device time. r_await is for reads, w_await for writes.; Disk I/O and iostat
Badness: The score oom_badness() gives each process: resident pages, plus swapped-out pages, plus page table pages, plus the oom_score_adj share. The highest score is killed.; The OOM killer
Bind mount: A second mount of a directory that is already mounted, created with mount --bind olddir newdir. Both paths show the same files.; Mounts and mount options
Boot loader: The program the firmware starts, such as GRUB or systemd-boot. It loads the kernel and the initramfs into memory and passes the kernel command line.; How Linux boots
Bounding set: A ceiling on the capabilities a thread can gain when it runs a new program with execve(2). CapBnd in /proc/<pid>/status shows it.; Capabilities
Buffer pool: A cache of file pages that a database keeps in its own memory, reading and writing with pread and pwrite. It is what mmap replaces.; mmap
Capability: One unit of root's power, named CAP_ plus a word: CAP_CHOWN changes file owners, CAP_NET_BIND_SERVICE binds ports below 1024. Each thread has its own sets of them.; Capabilities
cgroup: A directory under /sys/fs/cgroup that holds a group of processes, listed in its cgroup.procs file. cgroup v2 has a single tree, and each process is in exactly one cgroup of it.; cgroups v2
Controller: The kernel part that limits one resource for a cgroup: cpu, memory, io, pids and others. Its settings are files with its prefix, such as cpu.max or memory.max.; cgroups v2
Copy-up: The step where overlayfs copies a file from a lower directory to the upper directory before the first write to it, or before a change to its owner or mode.; Image layers and overlayfs
data=ordered: The ext4 default mode: only metadata goes through the journal, and a transaction's data blocks are written before its metadata commits.; Filesystems and journaling
Default action: What the profile does with a system call that no rule names. In Docker's default profile it is SCMP_ACT_ERRNO with EPERM: the call fails with "Operation not permitted" and does not run.; seccomp
Delayed allocation: write(2) only reserves space, and ext4 picks the disk blocks just before it writes the data out. The delalloc mount option, on by default.; Filesystems and journaling
Direct I/O: Reads and writes opened with O_DIRECT, which go between the program's buffer and the disk and skip the page cache. dd iflag=direct uses it, so every read reaches the device.; Disk I/O and iostat
Directory entry: One line in a directory that maps a name to an inode number. ls -i prints that number next to the name.; Inodes and links
Dirty page: A page cache page changed by a write and not yet written to disk. The kernel must write it back before it can reuse the page.; The page cache
Disposition: What a process does when a given signal arrives: the default action (often terminate), ignore it, or run a handler.; Signals
dpkg: The low-level tool that unpacks and configures .deb files and records each package's state in /var/lib/dpkg/status. It installs what you give it and does not download dependencies.; Packages
Durable: Still there after a crash or power loss. Data is durable once an fsync on it has returned 0.; fsync and durability
eBPF program: Bytecode loaded into the kernel with the bpf(2) system call and attached to an event, such as a tracepoint or the arrival of a network packet. It runs each time the event fires.; eBPF in practice
Effective set: The capabilities the kernel checks when the thread asks for a privileged action. CapEff in /proc/<pid>/status shows it as a hexadecimal mask.; Capabilities
EMFILE: Error 24, "Too many open files": the call needed a new descriptor and the process already uses every number below its soft limit.; File descriptors
ENOSPC: The error code a system call returns when the device has no room. Its message is No space left on device. write returns it when blocks run out, open when a new file cannot be created.; A full disk with free space
errno: The error code a failed system call leaves, such as ENOENT (no such file) or EACCES (permission denied). strace prints it after = -1.; Watching system calls with strace
execve(): The system call that replaces the program running in the calling process with a new one. The PID stays the same.; How a process starts
Exit status: The number a process passes to exit() when it ends. Only the low 8 bits reach the parent, so it is between 0 and 255. 0 means success.; Exit codes
File capability: Capabilities stored on an executable in the extended attribute security.capability, set with setcap. They let a non-root user's process gain a capability when it runs that file.; Capabilities
File descriptor: A small integer that indexes the process's table of open files, sockets and pipes. read(), write() and close() take it as their first argument.; File descriptors
File mapping: A range of a process's virtual addresses that mmap(2) ties to a file. Reading an address there reads the file's bytes at the matching offset.; mmap
fork(): The system call that copies the calling process. It returns 0 in the new child and the child's PID in the parent.; How a process starts
fsync: The system call that writes a file's dirty pages and metadata from the page cache to the device, and returns only when the device reports the transfer complete.; fsync and durability
Grace period: The seconds docker stop waits between the stop signal and SIGKILL. 10 by default, set with --stop-timeout or docker stop -t.; Signals
Handler: A function the program registers with sigaction(). The kernel calls it when the signal arrives. SIGKILL and SIGSTOP cannot have one.; Signals
Hard link: A second directory entry with the same inode number as the first. The inode's link count goes up by one.; Inodes and links
Hold: A dpkg selection state: the package stays at its version, with "no automatic new installs, upgrades or removals". apt-mark hold nginx sets it, and dpkg -l then shows hi.; Packages
Huge page: A page mapped from a higher level of the page table. On x86 that gives 2 MB or 1 GB pages next to the normal 4 KB ones.; Huge pages
Init process: PID 1 of a PID namespace. It adopts orphaned processes, and when it exits the kernel kills every process in the namespace with SIGKILL.; PID 1 in a container
initramfs: A compressed cpio archive that the kernel unpacks into memory at boot. Its /init runs first, finds the real root filesystem and mounts it.; How Linux boots
Inode: The record a filesystem keeps for one file: type and mode, owner, size, timestamps, link count and where the data blocks are. It has a number and no name.; Inodes and links
Inode: The record that holds a file's metadata: its owner, its mode, its link count. ext4 creates a fixed number of them at mke2fs time, so a disk can run out of inodes with gigabytes still free.; A full disk with free space
Journal: A reserved area of the filesystem, usually inode 8 on ext4, where jbd2 writes each metadata change before writing it to its final place.; Filesystems and journaling
Journal: The structured, indexed log files that systemd-journald writes under /var/log/journal/ (kept across reboots) or /run/log/journal/ (lost at reboot). Each entry is a set of fields such as MESSAGE=, PRIORITY= and _SYSTEMD_UNIT=.; The journal
Kept back: An installed package with a newer version that the chosen upgrade command will not install, because it would need a new package, a removal, or the package is on hold.; Packages
Kernel parameter: A setting the running kernel exposes as a file under /proc/sys, such as /proc/sys/net/core/somaxconn. Its sysctl name uses dots: net.core.somaxconn.; Kernel parameters
khugepaged: The kernel thread that scans memory and collapses runs of 512 small pages into one 2 MB page after the fact.; Huge pages
Link count: The number of directory entries (names) that point to an inode. rm lowers it by one. The kernel frees the inode and its blocks when the count is 0 and no process has the file open.; A full disk with free space
Listen backlog: The length of the queue of connections the kernel has accepted for a listening socket and the program has not yet taken with accept(). The program asks for it in listen(fd, backlog).; Kernel parameters
Load average: Three numbers in /proc/loadavg, also printed by uptime and top: a damped average of the tasks that are runnable or in uninterruptible sleep, with 1, 5 and 15 minute constants.; Load average and the scheduler
Logical volume (LV): A block device cut from a volume group, such as /dev/mapper/data-root. Device Mapper in the kernel maps each of its blocks to a PV. A filesystem lives on it like on a partition.; Disks, partitions and LVM
Lower directory: A read-only directory tree in an overlay mount. Each image layer is one, and the lowerdir= option lists them, top layer first, separated by :.; Image layers and overlayfs
Map: A key-value store in the kernel, shared between eBPF programs and the user process that loaded them. In bpftrace, a name starting with @ is a map.; eBPF in practice
MAP_PRIVATE: The flag for a copy-on-write mapping. The first write to a page gives the process its own copy, and the file never sees the change.; mmap
MAP_SHARED: The mapping flag under which writes go to the file's pages in the page cache. Other processes that map the same file see them, and the kernel writes them to the file.; mmap
MemAvailable: The kernel's estimate "of how much memory is available for starting new applications, without swapping": free pages, plus the page cache and slab memory it can reclaim. free prints it as available.; The page cache
MemFree: RAM that holds nothing at all. free prints it in the free column.; The page cache
memory.current: What the cgroup uses now: anonymous memory, page cache including tmpfs, and kernel memory such as dentries, inodes and socket buffers.; Memory limits in containers
memory.events: Counters for the cgroup: high and max count hits on each limit, oom counts allocations about to fail, oom_kill counts processes killed.; Memory limits in containers
memory.high: The cgroup v2 throttle limit. Above it the cgroup's processes are slowed down and made to reclaim, and the OOM killer never runs because of it.; Memory limits in containers
memory.max: The cgroup v2 hard limit, in bytes. When the cgroup's usage reaches it and reclaim cannot bring it down, the kernel runs the OOM killer inside that cgroup.; Memory limits in containers
memory.swap.max: The cgroup v2 file that caps how much swap one cgroup may use. At 0, the cgroup's anonymous pages are never swapped out.; Swap
Metadata: The filesystem's own records: inodes, directory entries, block and inode bitmaps. The bytes inside a file are its data.; Filesystems and journaling
Monotonic clock: CLOCK_MONOTONIC: on Linux, the time since boot, minus any time spent in suspend. Nobody can set it, and consecutive reads never go backwards.; Clocks and NTP
Mount: Attaching a filesystem to a directory, the mount point. After mount /dev/sdb1 /data, paths under /data lead into that filesystem. findmnt lists every mount with its options.; Mounts and mount options
Mount option: A flag set on one mount, such as ro, noexec, nosuid or nodev. The kernel checks it on each access through that mount. Two mounts of the same filesystem can have different options.; Mounts and mount options
Namespace: A kernel object that gives the processes inside it their own copy of one global resource: process IDs, mounts, network stack, hostname, user IDs, IPC, cgroup root or clocks. namespaces(7) lists 8 types.; Namespaces
nsenter: The util-linux command that runs a program inside the namespaces of another process, -t <pid>, by calling setns(2) on its /proc/<pid>/ns/* files.; Namespaces
OnCalendar=: A wall-clock schedule written as a calendar event, like Mon..Fri *-*-* 02:30:00. systemd-analyze calendar prints the next times it matches.; Timers instead of cron
OOM killer: Kernel code in mm/oom_kill.c that picks one process and sends it SIGKILL when an allocation fails after reclaim found no memory.; The OOM killer
oom_score: The read-only file /proc/<pid>/oom_score: the badness scaled for display. 0 means the process is exempt.; The OOM killer
oom_score_adj: A per-process value from -1000 to 1000 in /proc/<pid>/oom_score_adj. Each unit adds one thousandth of the allowed memory to the badness. -1000 means never pick this process.; The OOM killer
Ordering dependency: After= and Before=. After=b.service in a.service means: if both are being started, a waits until b has finished starting. It starts nothing by itself.; systemd units
Overcommit: Letting processes map more memory than RAM plus swap, on the bet that they will not touch all of it. vm.overcommit_memory picks the policy.; Virtual memory
Page: The unit the kernel maps memory in: 4096 bytes here (getconf PAGESIZE). A page of a process is either backed by a page of RAM or not yet.; Virtual memory
Page cache: RAM pages that hold the contents of files, filled when a process reads or writes a file. /proc/meminfo shows it as Cached, together with tmpfs and shared memory.; The page cache
Page table: The per-process table the CPU reads to turn a virtual address into a RAM address. The VmPTE line of status gives its size.; Virtual memory
Partition: A range of sectors on a disk, listed in the partition table. The kernel shows each one as its own block device, like nvme0n1p2.; Disks, partitions and LVM
Persistent=: With true, systemd stores the last trigger time on disk. If a run was due while the timer was inactive, for example with the machine off, the service starts as soon as the timer is active again.; Timers instead of cron
Physical volume (PV): A disk or partition that LVM manages, prepared with pvcreate. lsblk shows its type as LVM2_member.; Disks, partitions and LVM
PID: Process ID: the number the kernel gives each process. The PPID is the PID of its parent, the process that created it.; How a process starts
PID 1: The first user-space process, started by the kernel. On the test laptop it is systemd, which then starts every service.; How Linux boots
PID namespace: A separate numbering of process IDs. The first process in it gets PID 1, and processes inside cannot see processes in the parent namespace.; PID 1 in a container
Priority: The syslog level of an entry, from 0 (emerg) to 7 (debug), with 3 for err and 6 for info. journalctl -p err shows level 3 and every lower number.; The journal
Private_Dirty: Pages mapped by this process only and written since they were loaded. The kernel cannot drop them, only swap them out.; Finding a memory leak
Probe: The event a bpftrace program attaches to. tracepoint:syscalls:sys_enter_openat is a tracepoint, a fixed point in the kernel. kprobe:vfs_read is a kprobe, placed on any kernel function.; eBPF in practice
Propagation: Whether a mount or unmount under one mount point is repeated in other mount namespaces. The types are shared, slave, private and unbindable.; Mounts and mount options
ptrace: The kernel interface strace uses: a tracer process can stop another process, the tracee, and read its registers and memory while it is stopped.; Watching system calls with strace
Quota and period: The two numbers in cpu.max. In each period (microseconds), the group may use up to quota microseconds of CPU time, summed over all its threads and all CPUs. A Kubernetes limit of 1 CPU gives 100000 100000.; CPU limits and throttling
RandomizedDelaySec=: A random delay between 0 and the given time, drawn before each run and added to the scheduled time. It spreads timers that would all fire at the same moment.; Timers instead of cron
Rate limit: More than RateLimitBurst= messages from one service within RateLimitIntervalSec= are dropped until the interval ends. journald then logs how many it dropped.; The journal
Reaping: Calling wait() or waitpid() on a zombie: the parent reads the status and the kernel frees the PID.; PID 1 in a container
Requirement dependency: Wants= and Requires=. Starting the unit also starts the listed units. With Requires=, explicitly stopping or restarting a listed unit also stops or restarts this one.; systemd units
Reserved blocks: A share of an ext4 filesystem that only root (or the user and group set with tune2fs -u and -g) may allocate. mke2fs sets it to 5% by default.; A full disk with free space
RLIMIT_NOFILE: The per-process resource limit on descriptors: "one greater than the maximum file descriptor number that can be opened". ulimit -n reads and sets it in a shell.; File descriptors
RSS: Resident set size: the memory of a process that is in RAM right now, in kB in the VmRSS line of status. Chapter 09 compares it to the virtual size.; Reading /proc
RSS: Resident set size: the part of a process's mappings that is in RAM right now, the Rss: line of smaps and VmRSS in status.; Finding a memory leak
Runnable (R): A task that is running on a CPU or waiting in a CPU run queue for its turn. ps shows it as R.; Load average and the scheduler
Runnable task: A task running on a CPU or waiting for one. vmstat counts them in the r column, from procs_running in /proc/stat.; A slow server in 60 seconds
RuntimeDefault: The Kubernetes name for the container runtime's default profile, set in securityContext.seccompProfile.type.; seccomp
Saturation: Work that a resource cannot take yet, so it waits. The post: it can be seen "either as the length of a request queue, or time spent waiting". For CPUs, the queue is the runnable tasks.; A slow server in 60 seconds
seccomp filter: A small BPF program attached to a process. The kernel runs it on each system call, with the call number and arguments, before the call executes. Seccomp: 2 in /proc/<pid>/status means a filter is active.; seccomp
Set-group-ID bit: Mode bit 02000, shown as s in the group's execute slot. On a program it sets the effective group ID; on a directory, new files there take the directory's group.; Permissions beyond rwx
Set-user-ID bit: Mode bit 04000, shown as s in the owner's execute slot. When a program with it is executed, execve sets the process's effective user ID to the file owner's.; Permissions beyond rwx
Signal: A numbered notice the kernel delivers to a process, sent by another process with kill() or by the kernel itself. SIGTERM is 15, SIGKILL is 9 on x86.; Signals
Slew: Correcting the wall clock by running it slightly faster or slower until the offset is gone. chronyd does this by default, at most 83333 ppm (one twelfth).; Clocks and NTP
Slice: The amount of quota a CPU takes from the group's pool at a time, 5 ms by default (/proc/sys/kernel/sched_cfs_bandwidth_slice_us). Each CPU running the group's threads spends its own slice.; CPU limits and throttling
Slice and scope: systemd's names for cgroups it creates: a slice groups other units (system.slice, user.slice), and a scope holds processes started outside systemd, such as a Docker container.; cgroups v2
Soft and hard limit: The kernel enforces the soft limit. An unprivileged process may raise its soft limit up to the hard limit, which only a process with CAP_SYS_RESOURCE can raise.; File descriptors
Start limit: StartLimitBurst= starts allowed per StartLimitIntervalSec=. A unit that goes over it is refused with Start request repeated too quickly., and Restart= stops trying.; systemd units
Step: Setting the wall clock to a new value at once, so it jumps forward or back. chronyd steps only when makestep allows it, and the kernel steps back one second for an inserted leap second.; Clocks and NTP
Sticky bit: Mode bit 01000, shown as t in the others' execute slot. In a directory, only a file's owner, the directory's owner or a privileged process can delete or rename that file.; Permissions beyond rwx
sudoers rule: A line who where = (as_whom) tags: commands, like %sudo ALL=(ALL:ALL) ALL. % marks a group, and the (user:group) part says whom the command may run as.; Users, groups and sudo
Swap: A partition or file where the kernel writes anonymous pages to free RAM. swapon --show lists them, and /proc/meminfo shows SwapTotal and SwapFree.; Swap
Symbolic link: A separate file, with its own inode, whose content is a path string. The kernel follows that path when you open the link.; Inodes and links
sysctl: The procps-ng command that reads and writes kernel parameters: sysctl net.core.somaxconn reads, sysctl -w net.core.somaxconn=4096 writes, sysctl --system loads every config file.; Kernel parameters
System call: A request from a process to the kernel, such as openat to open a file or connect to open a socket. Each one returns a value, and -1 with an errno code on failure.; Watching system calls with strace
System user: An account for a daemon, created by useradd --system with a UID in the SYS_UID_MIN to SYS_UID_MAX range, below the first regular UID of 1000 by default, usually with /usr/sbin/nologin as its shell. www-data has UID 33.; Users, groups and sudo
SystemMaxUse=: The most disk space the journal files in /var/log/journal/ may take. Defaults to 10% of the file system, capped at 4G.; The journal
Target: A systemd unit that groups other units and marks a point in the boot, such as multi-user.target or network-online.target. default.target is the one systemd boots to.; How Linux boots
THP: Transparent Huge Pages: the kernel backs anonymous memory with 2 MB pages on its own, without the program asking. Set in /sys/kernel/mm/transparent_hugepage/enabled to always, madvise or never.; Huge pages
Throttling: What the scheduler does once the group's quota for the period is used up: its threads are taken off the CPU and "will not be able to run again until the next period when the quota is replenished".; CPU limits and throttling
Timer unit: A .timer file with a [Timer] section. When it elapses, it starts the service of the same name: backup.timer starts backup.service.; Timers instead of cron
TLB: The Translation Lookaside Buffer, the CPU's cache of recent virtual-to-physical address translations. The kernel docs call it "pretty scarce resource".; Huge pages
UID: The number the kernel uses for a user; names live in /etc/passwd. A process with effective UID 0, called root, bypasses all kernel permission checks.; Users, groups and sudo
umask: A per-process mask of permission bits that open and mkdir turn off on every new file. The usual value is 022.; Permissions beyond rwx
Uninterruptible sleep (D): A task blocked in the kernel that a signal cannot wake, usually waiting for disk I/O. ps shows it as D, "uninterruptible sleep (usually I/O)".; Load average and the scheduler
Unit file: An ini-style file such as /etc/systemd/system/app.service with a [Unit], a [Service] and an [Install] section. systemd reads it at boot and at systemctl daemon-reload.; systemd units
unshare: The unshare(2) system call moves the caller into new namespaces given by CLONE_NEW* flags. The util-linux command unshare runs a program that way.; Namespaces
Upper directory: The writable directory tree on top, set with upperdir=. Every change the container makes lands there. Docker calls it the container layer.; Image layers and overlayfs
USE method: Gregg's checklist: for every resource (CPUs, memory, disks, network), check utilization, saturation and errors. Each resource you clear narrows the search.; A slow server in 60 seconds
Verifier: The part of the kernel that checks an eBPF program at load time and refuses it if it could read an unset register, an unchecked pointer or memory out of bounds.; eBPF in practice
visudo: The editor wrapper for sudoers files. It locks the file, parses your edit and refuses to install it while it has a syntax error.; Users, groups and sudo
vm.swappiness: A sysctl from 0 to 200, default 60, that tells reclaim how costly swap I/O is compared to reading file pages back.; Swap
Volume group (VG): lvm(8): "a collection of one or more physical devices, each called a Physical Volume (PV)". Its space is a pool for logical volumes.; Disks, partitions and LVM
VSZ and RSS: VSZ (VmSize) is the size of every address range the process has mapped. RSS (VmRSS) is the part of it that is in RAM now. ps prints both, in KiB.; Virtual memory
wait: The family of calls (wait, waitpid, wait4) a parent uses to block until a child ends and read its exit status.; How a process starts
Wait status: The integer the kernel gives the parent through wait(). It records either "exited with code n" or "killed by signal N", read with WIFEXITED, WEXITSTATUS, WIFSIGNALED and WTERMSIG.; Exit codes
Wall clock: CLOCK_REALTIME: the date and time, counted in seconds since 1970-01-01 00:00 UTC without leap seconds. Root, date -s and NTP daemons can set it, forwards or backwards.; Clocks and NTP
Whiteout: An entry in the upper directory that hides a name from the lower directories. On disk it is a character device with device number 0/0; in an image layer tar it is an empty file named .wh.<name>.; Image layers and overlayfs
Writeback: The kernel writing dirty page cache pages to disk in the background, some time after write(2) has already returned.; fsync and durability
Zombie: A process that has exited but whose exit status no parent has read yet. It keeps its PID and a slot in the kernel's process table.; PID 1 in a container