01
hardware
CPU delivers the #PF exception
When the CPU encounters a virtual address it cannot translate — because the page-table entry is not present, a permission bit is violated, or a reserved bit is set — it raises exception vector 14 (#PF). Before jumping to the kernel handler, the CPU saves the faulting virtual address in CR2, pushes an error code onto the kernel stack, and saves CS:RIP and RFLAGS. The error code encodes: P (present bit was set, so it was a protection fault, not a missing page), W/R (write or read access), U/S (user or supervisor mode), RSVD (reserved PTE bit set), and I/D (instruction fetch). The IDT entry for vector 14 points to asm_exc_page_fault in entry_64.S.
// Error code bits (arch/x86/include/asm/trap_pf.h)
#define X86_PF_PROT BIT(0) // 0=not-present fault, 1=protection fault
#define X86_PF_WRITE BIT(1) // 0=read, 1=write
#define X86_PF_USER BIT(2) // 0=kernel, 1=user-mode
#define X86_PF_RSVD BIT(3) // reserved PTE bit set
#define X86_PF_INSTR BIT(4) // instruction fetch (NX violation)
#define X86_PF_PK BIT(5) // protection-key violation
#define X86_PF_SHSTK BIT(6) // shadow-stack access
#define X86_PF_SGX BIT(15) // SGX-induced fault
// CR2 holds the faulting virtual address — read it immediately
// before any other fault can overwrite it
unsigned long address = read_cr2();
NOTECR2 is a per-CPU register — on SMP systems a second fault on another CPU will not overwrite it. However, the kernel reads CR2 as early as possible in the fault path to avoid any window where a kernel bug could trigger a nested fault and lose the address.
02
x86 entry
asm_exc_page_fault → exc_page_fault()
The assembly stub asm_exc_page_fault (generated by the DEFINE_IDTENTRY_RAW_ERRORCODE macro) saves registers, reads CR2, and calls exc_page_fault(). This C function first handles the special case of a fault inside the kernel's own code — if the faulting RIP is in a fixup table entry (set up by the __ex_table mechanism), the kernel patches the return address to the fixup handler and returns without killing anything. Otherwise it calls handle_page_fault() which dispatches to do_kern_addr_fault() or do_user_addr_fault() based on the address and error code.
// arch/x86/mm/fault.c (Linux 7.2, simplified)
DEFINE_IDTENTRY_RAW_ERRORCODE(exc_page_fault)
{
unsigned long address = read_cr2(); // faulting VA
irqentry_state_t state;
prefetchw(¤t->mm->mmap_lock);
/*
* KVM/hypervisor may want to handle the fault first.
*/
if (kvm_handle_async_pf(regs, (u32)address))
return;
state = irqentry_enter(regs);
instrumentation_begin();
handle_page_fault(regs, error_code, address);
instrumentation_end();
irqentry_exit(regs, state);
}
NOTEprefetchw() on mmap_lock is a performance hint — the kernel speculatively prefetches the lock into cache because the vast majority of user faults will need to acquire it shortly after.
03
mm entry
do_user_addr_fault(): acquire mmap_lock
For user-space faults, do_user_addr_fault() is called. It first checks for vsyscall emulation and perf event sampling faults, then tries to acquire mm->mmap_lock in read mode. This lock protects the VMA tree. On the fast path it uses mmap_read_trylock() — a non-blocking attempt. If that fails (another thread is modifying the VMA tree), it falls back to mmap_read_lock() which blocks. Once the lock is held, find_vma() searches the red-black tree of VMAs for the region that contains the faulting address.
// arch/x86/mm/fault.c
static void do_user_addr_fault(struct pt_regs *regs,
unsigned long error_code,
unsigned long address)
{
struct vm_area_struct *vma;
struct task_struct *tsk = current;
struct mm_struct *mm = tsk->mm;
...
if (unlikely(!mmap_read_trylock(mm))) {
if (!(error_code & X86_PF_USER) && !search_exception_tables(regs->ip))
goto bad_area_nosemaphore;
mmap_read_lock(mm); // blocking acquire
}
vma = find_vma(mm, address); // O(log n) rbtree search
if (unlikely(!vma))
goto bad_area; // no VMA → SIGSEGV
if (likely(vma->vm_start <= address))
goto good_area; // address is inside the VMA
if (!(vma->vm_flags & VM_GROWSDOWN))
goto bad_area; // gap between VMAs → SIGSEGV
...
}
NOTEfind_vma() returns the first VMA whose vm_end > address. If the address falls in a gap between VMAs, the returned VMA's vm_start will be above the address — the code checks for this and handles stack-growth (VM_GROWSDOWN) as a special case.
04
mm entry
VMA permission check
Once a covering VMA is found, the kernel checks whether the access is permitted by the VMA's flags. A write fault on a read-only mapping, or an instruction fetch on a non-executable mapping, is a protection violation — the VMA exists but the access is forbidden. The vm_flags field encodes VM_READ, VM_WRITE, VM_EXEC, VM_SHARED, VM_GROWSDOWN, and many others. If the access mode is not permitted, the kernel jumps to bad_area which sends SIGSEGV to the process.
// arch/x86/mm/fault.c — permission check (good_area label)
good_area:
if (unlikely(access_error(error_code, vma))) {
bad_area_access_error(regs, error_code, address, vma);
return;
}
/* Fault is legitimate — call the generic MM handler */
fault = handle_mm_fault(vma, address, flags, regs);
...
// access_error() checks:
// - write fault on read-only VMA → error
// - exec fault on non-exec VMA → error (NX)
// - user fault on kernel VMA → error
// - read fault on present page → should not happen (handled above)
static inline int access_error(unsigned long error_code,
struct vm_area_struct *vma)
{
if (error_code & X86_PF_WRITE)
return !(vma->vm_flags & VM_WRITE);
if (error_code & X86_PF_INSTR)
return !(vma->vm_flags & VM_EXEC);
return 0;
}
NOTEA write fault on a present, writable page that is mapped read-only in the PTE is a copy-on-write fault — X86_PF_PROT | X86_PF_WRITE with a present PTE. access_error() passes this through because VM_WRITE is set; the COW logic lives deeper in handle_mm_fault().
05
page tables
handle_mm_fault(): page table walk
handle_mm_fault() is the architecture-independent entry point into the page-fault resolution logic. It calls __handle_mm_fault() which walks the four-level page table hierarchy (PGD → P4D → PUD → PMD → PTE), allocating intermediate page-table pages as needed with pmd_alloc() and pte_alloc(). For huge pages (THP), it may resolve the fault at the PMD level. For normal 4 KiB pages it reaches the PTE level and calls handle_pte_fault().
// mm/memory.c (Linux 7.2, simplified)
static vm_fault_t __handle_mm_fault(struct vm_area_struct *vma,
unsigned long address, unsigned int flags)
{
struct vm_fault vmf = {
.vma = vma,
.address = address & PAGE_MASK,
.flags = flags,
.pgoff = linear_page_index(vma, address),
.gfp_mask = __get_fault_gfp_mask(vma),
};
struct mm_struct *mm = vma->vm_mm;
pgd_t *pgd = pgd_offset(mm, address); // level 4
p4d_t *p4d = p4d_alloc(mm, pgd, address); // level 3
pud_t *pud = pud_alloc(mm, p4d, address); // level 2
...
pmd_t *pmd = pmd_alloc(mm, pud, address); // level 1
...
return handle_pte_fault(&vmf); // PTE level
}
NOTEThe vm_fault struct is stack-allocated and passed by pointer through the entire fault chain. It accumulates state (the PTE pointer, the page pointer, fault flags) as the walk descends, avoiding repeated lookups.
06
page tables
handle_pte_fault(): four fault types
handle_pte_fault() dispatches to one of four handlers based on the PTE state. (1) Not-present, no swap entry, anonymous VMA → do_anonymous_page(): allocate a zeroed page. (2) Not-present, no swap entry, file-backed VMA → do_fault() → vma->vm_ops->fault(): read the page from the backing store (disk, tmpfs, etc.). (3) Not-present, swap entry → do_swap_page(): read the page back from swap. (4) Present but read-only, write fault → do_wp_page(): copy-on-write — allocate a new page, copy contents, install writable PTE.
// mm/memory.c
static vm_fault_t handle_pte_fault(struct vm_fault *vmf)
{
pte_t entry;
if (unlikely(pmd_none(*vmf->pmd))) {
// no PTE page yet
...
}
vmf->pte = pte_offset_map(vmf->pmd, vmf->address);
entry = *vmf->pte;
if (!pte_present(entry)) {
if (pte_none(entry)) {
if (vma_is_anonymous(vmf->vma))
return do_anonymous_page(vmf); // (1) anon demand page
else
return do_fault(vmf); // (2) file-backed
}
return do_swap_page(vmf); // (3) swap
}
if (pte_protnone(entry) && vma_is_accessible(vmf->vma))
return do_numa_page(vmf); // NUMA balancing
if (vmf->flags & FAULT_FLAG_WRITE) {
if (!pte_write(entry))
return do_wp_page(vmf); // (4) copy-on-write
}
return 0;
}
NOTEdo_numa_page() handles NUMA balancing faults — the kernel intentionally marks PTEs as prot_none to trap accesses and migrate pages to the NUMA node where they are most frequently accessed.
07
allocator
do_anonymous_page(): allocate and map
For a demand-paging fault on an anonymous VMA (heap, stack, mmap(MAP_ANONYMOUS)), do_anonymous_page() is called. It calls alloc_zeroed_user_highpage_movable() which ultimately calls the buddy allocator to get a single 4 KiB page, then zeroes it (security: never expose stale kernel data to user-space). It then builds a new PTE with mk_pte(), sets the dirty and accessed bits, and installs it with set_pte_at(). Finally it calls update_mmu_cache() to update any software TLB (needed on some architectures; a no-op on x86 which uses hardware page-table walking).
// mm/memory.c
static vm_fault_t do_anonymous_page(struct vm_fault *vmf)
{
struct vm_area_struct *vma = vmf->vma;
struct page *page;
pte_t entry;
/* Check for userfaultfd — user-space fault handling */
if (userfaultfd_missing(vma))
return handle_userfault(vmf, VM_UFFD_MISSING);
/* Allocate a zeroed page from the buddy allocator */
page = alloc_zeroed_user_highpage_movable(vma, vmf->address);
if (!page)
return VM_FAULT_OOM;
/* Build and install the PTE */
entry = mk_pte(page, vma->vm_page_prot);
entry = pte_sw_mkyoung(entry); // set Accessed bit
if (vma->vm_flags & VM_WRITE)
entry = pte_mkwrite(pte_mkdirty(entry)); // writable
vmf->pte = pte_offset_map_lock(vma->vm_mm, vmf->pmd,
vmf->address, &vmf->ptl);
set_pte_at(vma->vm_mm, vmf->address, vmf->pte, entry);
update_mmu_cache(vma, vmf->address, vmf->pte);
pte_unmap_unlock(vmf->pte, vmf->ptl);
return 0;
}
NOTEuserfaultfd allows a user-space process to handle its own page faults — used by CRIU (checkpoint/restore), live migration, and post-copy memory migration in VMs. If the VMA has UFFD_MISSING set, the fault is forwarded to the registered user-space handler instead of being resolved in the kernel.
08
exit
Return path: TLB, signals, and OOM
After handle_mm_fault() returns, do_user_addr_fault() releases mmap_lock and inspects the vm_fault_t return value. VM_FAULT_OOM triggers the OOM killer. VM_FAULT_SIGBUS sends SIGBUS (e.g. a file-backed mapping where the backing file was truncated). VM_FAULT_SIGSEGV sends SIGSEGV. On success (return 0 or VM_FAULT_MINOR/MAJOR), the kernel returns through the exception exit path. On x86 the hardware TLB is filled automatically by the page-table walker on the next memory access — no explicit TLB invalidation is needed for a new mapping. However, if a PTE was modified on one CPU (e.g. during COW), other CPUs holding a stale TLB entry must be shot down via an IPI — this is the TLB shootdown path triggered by flush_tlb_page().
// arch/x86/mm/fault.c — return path
fault = handle_mm_fault(vma, address, flags, regs);
mmap_read_unlock(mm);
if (fault_signal_pending(fault, regs)) {
if (!user_mode(regs))
no_context(regs, error_code, address, SIGBUS, BUS_ADRERR);
return;
}
if (unlikely(fault & VM_FAULT_ERROR)) {
if (fault & VM_FAULT_OOM)
pagefault_out_of_memory(); // invoke OOM killer
else if (fault & VM_FAULT_SIGBUS)
do_sigbus(regs, error_code, address, 0);
else if (fault & VM_FAULT_SIGSEGV)
bad_area_nosemaphore(regs, error_code, address);
return;
}
// Success — return to user-space, hardware fills TLB on retry
NOTEVM_FAULT_MAJOR is set when the fault required I/O (a page was read from disk or swap). The kernel accounts this in the process's maj_flt counter, visible in /proc/PID/stat. VM_FAULT_MINOR (no I/O needed) increments min_flt.