System Calls
Syscall table dispatch, entry_SYSCALL_64, VDSO, seccomp BPF, and the audit framework — the kernel/user boundary dissected.
Subsystem Overview
System calls are the fundamental interface between user-space programs and the kernel. On x86-64, a syscall is initiated with the SYSCALL instruction, which atomically switches to ring 0, loads the kernel stack pointer from the per-CPU TSS, and jumps to entry_SYSCALL_64 — the single entry point for all 64-bit system calls.
The entry path saves the full register state (pt_regs), validates the syscall number against NR_syscalls, runs seccomp BPF filters, and dispatches through the sys_call_table array of function pointers. On return, the path restores registers and executes SYSRET to return to user space at ring 3.
Linux 7.2 extended the seccomp subsystem with SECCOMP_RET_USER_NOTIF_ADDFD for file descriptor injection into supervised processes, and hardened the VDSO against Spectre-v2 by switching to IBPB-based mitigation on affected microarchitectures.
// x86-64 syscall ABI — argument registers
Diagrams
// interactive · click nodes to explore
Syscall Entry Path
Step-by-step x86-64 path from SYSCALL instruction through entry_SYSCALL_64, do_syscall_64, and back via SYSRET.
sys_call_table Dispatch
How RAX indexes the sys_call_table array to dispatch to the correct kernel handler function.
Seccomp BPF Decision Tree
BPF filter evaluation flow: ALLOW, ERRNO, TRAP (SIGSYS), and KILL outcomes from seccomp_run_filters().
Key Concepts
SYSCALL / SYSRET
The fast syscall mechanism on x86-64. SYSCALL saves RIP into RCX and RFLAGS into R11, then jumps to the address in MSR_LSTAR (entry_SYSCALL_64). SYSRET restores RIP from RCX and RFLAGS from R11. This avoids the overhead of the legacy INT 0x80 / IRET path.
pt_regs
The register save area pushed onto the kernel stack at syscall entry. Contains all general-purpose registers, RIP, CS, RFLAGS, RSP, and SS. Syscall arguments are passed in RDI, RSI, RDX, R10, R8, R9 — note R10 instead of RCX (which is clobbered by SYSCALL).
sys_call_table
An array of function pointers indexed by syscall number, defined in arch/x86/entry/syscall_64.c. Each entry points to the kernel implementation (e.g. sys_read, sys_write). The table is read-only after boot and protected by CONFIG_STRICT_KERNEL_RWX.
seccomp BPF
Secure Computing mode 2. A BPF program attached to a thread that inspects each syscall's number and arguments before dispatch. Can return ALLOW, KILL_PROCESS, TRAP (SIGSYS), ERRNO, TRACE (ptrace), LOG, or USER_NOTIF. Used by container runtimes and sandboxes.
VDSO (Virtual Dynamic Shared Object)
A small shared library mapped into every process by the kernel. Implements frequently called syscalls (clock_gettime, gettimeofday, getcpu) entirely in user space by reading kernel-maintained data in the vvar page — avoiding the ring-transition overhead entirely.
Audit Framework
The kernel audit subsystem records syscall events (entry, exit, arguments, return value) into an in-kernel ring buffer consumed by auditd. Enabled per-thread via audit_context. Adds overhead only when rules match; the fast path is a single branch on audit_dummy_context.
Source References
// arch/x86/entry/ · kernel/ · annotated entry points
Linux 7.2 Changes
// recent arch/x86/entry/ · kernel/seccomp.c commits
// related tutorial
Syscall Entry Path — full annotated walkthrough
7 steps from SYSCALL instruction to SYSRET, with ARM64 comparison table and knowledge checks.