Building an Operating System in Rust: Part 3 - CPU Interrupts, IDT & Double Faults

Building an Operating System in Rust: Part 3 — CPU Interrupts, IDT & Double Faults

In Part 2: VGA Text Buffer Driver & Volatile Memory, we constructed a type-safe display driver and implemented println! to inspect the state of our bare-metal kernel.

However, computer systems do not execute in predictable isolation. The CPU constantly encounters unexpected runtime events: division by zero, invalid opcodes, page faults, and hardware device signals. When an unexpected event occurs, the CPU triggers an interrupt or CPU exception.

If our operating system does not possess a registered handler for an exception, the CPU triggers a catastrophic triple fault—which causes physical hardware to reset and reboot instantly.

In this third installment of our Building an Operating System in Rust series, you will learn how CPU interrupts work, how to construct the Interrupt Descriptor Table (IDT) in Rust, how to catch CPU exceptions safely, and how to defend against stack overflows using the Global Descriptor Table (GDT) and an Interrupt Stack Table (IST).

1. What are CPU Exceptions and Interrupts?

A CPU exception is a hardware mechanism through which the processor interrupts the current execution flow to report an urgent condition.

There are three broad classifications of CPU exceptions:

1. Faults: Recoverable exceptions where the CPU can resume execution after correcting the fault (e.g., Page Faults, Divide-by-Zero).
2. Traps: Exceptions reported immediately after the execution of the trapping instruction (e.g., Breakpoints for debuggers).
3. Aborts: Unrecoverable severe hardware failures that cannot pinpoint the exact instruction causing the error.

When an exception occurs, the CPU automatically pauses the running instructions, pushes the current execution context (instruction pointer, stack pointer, code segment, CPU flags) onto the active stack, and jumps to a specific handler function registered in the Interrupt Descriptor Table (IDT).

2. The Interrupt Descriptor Table (IDT)

The Interrupt Descriptor Table is a 256-entry array in memory that tells the processor which function to execute for each exception vector (from vector 0 to 255).

Each entry in the 64-bit IDT is a 16-byte descriptor containing:

• A 64-bit memory address pointing to the handler function.
• The Target Code Segment selector in the Global Descriptor Table.
• Descriptor Privilege Level (DPL): Specifies whether user-space (Ring 3) or kernel-space (Ring 0) code can trigger the interrupt.
• The Present bit: Indicates whether the descriptor is active.
• The Interrupt Stack Table (IST) index: Designates a dedicated stack for this interrupt.

Rather than manually constructing bitfields in raw assembly, we leverage the excellent x86_64 crate, which provides idiomatic Rust abstractions for CPU registers, table descriptors, and memory paging.

Add the crate to your Cargo.toml:

[dependencies]
x86_64 = "0.14.11"
lazy_static = { version = "1.5.0", features = ["spin_no_std"] }

3. Implementing the IDT in Rust

Let's create a dedicated module src/interrupts.rs to manage our interrupt descriptor table:

use x86_64::structures::idt::{InterruptDescriptorTable, InterruptStackFrame};
use crate::println;
use lazy_static::lazy_static;

lazy_static! {
    static ref IDT: InterruptDescriptorTable = {
        let mut idt = InterruptDescriptorTable::new();
        idt.breakpoint.set_handler_fn(breakpoint_handler);
        idt.double_fault.set_handler_fn(double_fault_handler);
        idt
    };
}

pub fn init_idt() {
    IDT.load();
}

The IDT.load() method executes the underlying x86 lidt assembly instruction, which loads the memory pointer of our IDT into the CPU's internal IDTR register.

4. Writing Exception Handlers with the x86-interrupt ABI

Exception handlers cannot use the standard C calling convention (extern "C"). Why?

In standard functions, the compiler only preserves "callee-saved" CPU registers. But an interrupt can occur at any arbitrary instruction during CPU execution. If an interrupt handler modifies "caller-saved" registers (like rax, rcx, rdx), the interrupted program's memory state will be silently corrupted upon return!

To solve this, the LLVM compiler provides a dedicated calling convention: the x86-interrupt ABI. Functions compiled with extern "x86-interrupt" automatically push all modified CPU registers onto the stack upon entry and pop them back off before calling the iretq (interrupt return) instruction.

Enable the feature in src/main.rs:

#![feature(abi_x86_interrupt)]

Now, implement the breakpoint exception handler:

extern "x86-interrupt" fn breakpoint_handler(
    stack_frame: InterruptStackFrame
) {
    println!("EXCEPTION: BREAKPOINT");
    println!("{:#?}", stack_frame);
}

When a breakpoint occurs, our handler displays the saved InterruptStackFrame, revealing the exact instruction pointer (rip), stack pointer (rsp), and CPU flags at the moment the interrupt was triggered.

5. Catching Double Faults and Preventing Triple Faults

What happens when an exception occurs, but the CPU encounters an error while attempting to call the exception handler?

For example:

1. The kernel attempts to write to an unmapped page of memory, triggering a Page Fault (Vector 14).
2. The CPU looks up the Page Fault handler in the IDT, but discovers the handler entry is missing or the kernel stack is exhausted.
3. Because the CPU cannot invoke the Page Fault handler, it escalates the failure to a Double Fault (Vector 8).
4. If the Double Fault handler also cannot be invoked, the CPU gives up completely and triggers a hardware Triple Fault, resetting the physical machine instantly!

To catch double faults, we define a dedicated handler:

extern "x86-interrupt" fn double_fault_handler(
    stack_frame: InterruptStackFrame,
    _error_code: u64,
) -> ! {
    panic!("EXCEPTION: DOUBLE FAULT\n{:#?}", stack_frame);
}

6. The Stack Overflow Problem & The Interrupt Stack Table (IST)

There is one critical scenario where a double fault handler still fails: a kernel stack overflow.

When a function recurses infinitely, the kernel stack grows downward until it hits a guard page:

Stack Growth:
[ Stack Frame 0 ]
[ Stack Frame 1 ]
[ Stack Frame 2 ]
...
[ Guard Page (Unmapped) ] <--- Stack pointer pushes here!

Hitting the unmapped guard page causes a Page Fault. The CPU attempts to push the InterruptStackFrame onto the stack to invoke the handler. But the stack pointer is already pointing to invalid memory! Pushing onto the stack causes another page fault, escalating immediately to a Double Fault.

The CPU tries to invoke the Double Fault handler—and again tries to push to the broken stack, triggering an instant Triple Fault reboot!

The Solution: Switching Stacks via the IST

The x86_64 architecture provides the Interrupt Stack Table (IST) mechanism. The IST allows us to define up to seven dedicated, known-good backup stacks inside the Task State Segment (TSS).

When a double fault occurs, the CPU automatically switches to the backup stack before pushing register frames, guaranteeing that stack overflows can be caught and diagnosed safely.

Let's configure the TSS and GDT in src/gdt.rs:

use x86_64::structures::tss::TaskStateSegment;
use x86_64::structures::gdt::{GlobalDescriptorTable, Descriptor, SegmentSelector};
use x86_64::VirtAddr;
use lazy_static::lazy_static;

pub const DOUBLE_FAULT_IST_INDEX: u16 = 0;

lazy_static! {
    static ref TSS: TaskStateSegment = {
        let mut tss = TaskStateSegment::new();
        tss.interrupt_stack_table[DOUBLE_FAULT_IST_INDEX as usize] = {
            const STACK_SIZE: usize = 4096 * 5; // 20 KB dedicated stack
            static mut STACK: [u8; STACK_SIZE] = [0; STACK_SIZE];

            let stack_start = VirtAddr::from_ptr(unsafe { &raw const STACK });
            let stack_end = stack_start + STACK_SIZE;
            stack_end // x86 stacks grow downwards!
        };
        tss
    };
}

struct Selectors {
    code_selector: SegmentSelector,
    tss_selector: SegmentSelector,
}

lazy_static! {
    static ref GDT: (GlobalDescriptorTable, Selectors) = {
        let mut gdt = GlobalDescriptorTable::new();
        let code_selector = gdt.add_entry(Descriptor::kernel_code_segment());
        let tss_selector = gdt.add_entry(Descriptor::tss_segment(&TSS));
        (gdt, Selectors { code_selector, tss_selector })
    };
}

pub fn init() {
    use x86_64::instructions::tables::load_tss;
    use x86_64::instructions::segmentation::{CS, Segment};

    GDT.0.load();
    unsafe {
        CS::set_reg(GDT.1.code_selector);
        load_tss(GDT.1.tss_selector);
    }
}

Now, connect the dedicated double fault stack to the IDT in src/interrupts.rs:

lazy_static! {
    static ref IDT: InterruptDescriptorTable = {
        let mut idt = InterruptDescriptorTable::new();
        idt.breakpoint.set_handler_fn(breakpoint_handler);
        unsafe {
            idt.double_fault.set_handler_fn(double_fault_handler)
                .set_stack_index(crate::gdt::DOUBLE_FAULT_IST_INDEX);
        }
        idt
    };
}

7. Testing Fault Catching in main.rs

Now, let's test our complete interrupt and stack isolation system:

#![no_std]
#![no_main]
#![feature(abi_x86_interrupt)]

mod vga_buffer;
mod gdt;
mod interrupts;

use core::panic::PanicInfo;

#[no_mangle]
pub extern "C" fn _start() -> ! {
    println!("Initializing Adoreka OS...");

    // Initialize GDT and TSS (Dedicated Interrupt Stacks)
    gdt::init();

    // Initialize IDT
    interrupts::init_idt();

    println!("Interrupt Descriptor Table (IDT) loaded successfully.");

    // Trigger a software breakpoint exception!
    x86_64::instructions::interrupts::int3();

    println!("Kernel resumed execution after breakpoint exception!");

    loop {}
}

#[panic_handler]
fn panic(info: &PanicInfo) -> ! {
    println!("{}", info);
    loop {}
}

When you boot the kernel in QEMU:

1. The kernel initializes the GDT, TSS, and IDT.
2. The int3() instruction triggers an architectural breakpoint exception.
3. The CPU jumps to our breakpoint_handler.
4. Our handler prints the saved instruction pointer and stack frame to the VGA screen.
5. The CPU resumes execution seamlessly!
6. If a stack overflow occurs anywhere in the kernel, the CPU switches to the dedicated IST stack and prints a clean double fault diagnostic instead of triple faulting!

Architectural Summary

In Part 4: Hardware Timer Interrupts & Keyboard Controllers, we will connect the Intel 8259 PIC controller to handle physical clock ticks and read keystrokes from the keyboard.