Building an Operating System in Rust: Part 2 - VGA Text Buffer Driver & Volatile Memory

Building an Operating System in Rust: Part 2 — VGA Text Buffer Driver & Volatile Memory

In Part 1: Zero-Cost Bare-Metal Freestanding Binary, we configured a #![no_std] Rust binary capable of executing directly on bare-metal x86_64 hardware. However, a kernel that merely loops in silence is difficult to debug. To observe the internal behavior of our operating system, we need a mechanism to display text on the physical monitor.

On x86 hardware, the simplest and most universal display mechanism available at boot time is the VGA text mode buffer.

In this second installment of our Building an Operating System in Rust series, you will learn how the VGA text hardware works, why naive pointer writes fail due to compiler optimizations, how to enforce memory safety using volatile writes, and how to implement Rust's famous print! and println! macros for your bare-metal kernel.

1. How the VGA Text Mode Buffer Works

The VGA text buffer is a specialized region of physical memory mapped directly to the computer's display hardware. When an x86 processor boots in standard compatibility mode, the video card maps a fixed physical memory address to the screen:

$\text{Physical Address: } \mathbf{0xB8000}$

Writing bytes to this memory address immediately renders characters to the connected display monitor.

Hardware Specification: The VGA text buffer dimensions span exactly 80 columns by 25 rows (2,000 character cells). With each character cell consuming 2 bytes in memory (Byte 0: ASCII code point, Byte 1: 4-bit foreground + 4-bit background color attribute), the total contiguous memory buffer occupies exactly 4,000 bytes (0xFA0) mapped at physical memory address 0xB8000.

The Structure of a 16-Bit Character Cell

In memory, bits 0–7 store the 8-bit ASCII character byte, bits 8–11 store the foreground color code (0–15), and bits 12–15 store the background color and blink flag.

1. Byte 0 (Lower 8 bits): The ASCII character code (e.g., 'A' = 0x41, ' ' = 0x20).
2. Byte 1 (Upper 8 bits): The color attribute byte:
   • Bits 0–3: Foreground color (16 available colors: Black, Blue, Green, Cyan, Red, Magenta, Brown, Light Gray, etc.)
   • Bits 4–7: Background color and blink flag.

2. The Danger of Compiler Optimization: Why We Need Volatile Writes

In C, programmers often write to the VGA buffer by casting an integer address directly to a pointer:

// Dangerous C approach
volatile char* vga = (volatile char*) 0xb8000;
vga[0] = 'H';
vga[1] = 0x0F; // White text on black background

In Rust, writing through a raw pointer without special precautions creates a severe bug. The optimizing compiler (LLVM) assumes that normal RAM behaves like regular memory: it assumes that if you write a value to an address, nothing external reads that value unless explicitly observed by software.

If you write a loop that updates the VGA buffer repeatedly, the compiler might optimize away earlier writes because it does not realize that writing to memory address 0xB8000 produces an external hardware side effect (rendering pixels on a CRT/LCD monitor)!

To prevent the compiler from optimizing away memory writes, we must use volatile access. Volatile reads and writes instruct the compiler that the memory location has hardware side effects and must never be reordered, cached, or eliminated.

3. Modeling Colors with Type-Safe Rust Enums

Let's begin writing our VGA driver in src/vga_buffer.rs. We model the 16 VGA colors using a C-like enum with a #[repr(u8)] representation:

#[allow(dead_code)]
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
#[repr(u8)]
pub enum Color {
    Black = 0,
    Blue = 1,
    Green = 2,
    Cyan = 3,
    Red = 4,
    Magenta = 5,
    Brown = 6,
    LightGray = 7,
    DarkGray = 8,
    LightBlue = 9,
    LightGreen = 10,
    LightCyan = 11,
    LightRed = 12,
    Pink = 13,
    Yellow = 14,
    White = 15,
}

#[derive(Debug, Clone, Copy, PartialEq, Eq)]
#[repr(transparent)]
struct ColorCode(u8);

impl ColorCode {
    fn new(foreground: Color, background: Color) -> ColorCode {
        ColorCode((background as u8) << 4 | (foreground as u8))
    }
}

The #[repr(transparent)] attribute ensures that ColorCode has the exact same in-memory representation as a primitive u8.

4. Modeling the Screen Character & Buffer

Next, we represent each 2-byte character cell and the 80x25 character matrix:

use volatile::Volatile;

#[derive(Debug, Clone, Copy, PartialEq, Eq)]
#[repr(C)]
struct ScreenChar {
    ascii_character: u8,
    color_code: ColorCode,
}

const BUFFER_HEIGHT: usize = 25;
const BUFFER_WIDTH: usize = 80;

#[repr(transparent)]
struct Buffer {
    chars: [[Volatile<ScreenChar>; BUFFER_WIDTH]; BUFFER_HEIGHT],
}

By wrapping ScreenChar in the Volatile wrapper (from the volatile crate), Rust guarantees that all reads and writes to this structure use volatile assembly instructions (mov instructions that LLVM cannot omit).

Add the dependency to your Cargo.toml:

[dependencies]
volatile = "0.2.6"
spin = "0.9.8"

5. The Writer Type: Scrolling & Line Breaks

Now we construct the Writer struct to manage the cursor position and handle line breaks, text scrolling, and color formatting:

pub struct Writer {
    column_position: usize,
    color_code: ColorCode,
    buffer: &'static mut Buffer,
}

impl Writer {
    pub fn write_byte(&mut self, byte: u8) {
        match byte {
            b'\n' => self.new_line(),
            byte => {
                if self.column_position >= BUFFER_WIDTH {
                    self.new_line();
                }

                let row = BUFFER_HEIGHT - 1;
                let col = self.column_position;

                let color_code = self.color_code;
                self.buffer.chars[row][col].write(ScreenChar {
                    ascii_character: byte,
                    color_code,
                });
                self.column_position += 1;
            }
        }
    }

    pub fn write_string(&mut self, s: &str) {
        for byte in s.bytes() {
            match byte {
                // Printable ASCII character or newline
                0x20..=0x7e | b'\n' => self.write_byte(byte),
                // Non-printable ASCII character: print a solid block (0xfe)
                _ => self.write_byte(0xfe),
            }
        }
    }

    fn new_line(&mut self) {
        for row in 1..BUFFER_HEIGHT {
            for col in 0..BUFFER_WIDTH {
                let character = self.buffer.chars[row][col].read();
                self.buffer.chars[row - 1][col].write(character);
            }
        }
        self.clear_row(BUFFER_HEIGHT - 1);
        self.column_position = 0;
    }

    fn clear_row(&mut self, row: usize) {
        let blank = ScreenChar {
            ascii_character: b' ',
            color_code: self.color_code,
        };
        for col in 0..BUFFER_WIDTH {
            self.buffer.chars[row][col].write(blank);
        }
    }
}

Notice the elegance of new_line(): when the screen fills up, it shifts every row up by one position (row - 1), clears the bottom-most line, and resets the column position to 0.

6. Implementing Global State with Spinlocks

To use our Writer from anywhere in the kernel without passing a pointer through every function call, we instantiate a global static instance. However, in Rust, mutable global variables (static mut) are unsafe because concurrent threads can trigger data races.

Because our kernel does not yet support threads or OS-level blocking mutexes, we use a Spinlock from the spin crate. A spinlock repeatedly spins in a tight loop until the lock becomes available:

use spin::Mutex;

pub static WRITER: Mutex<Writer> = Mutex::new(Writer {
    column_position: 0,
    color_code: ColorCode::new(Color::Yellow, Color::Black),
    buffer: unsafe { &mut *(0xb8000 as *mut Buffer) },
});

7. Implementing Rust's println! and print! Macros

To make text output ergonomic and familiar, we implement the standard core::fmt::Write trait for our Writer:

use core::fmt;

impl fmt::Write for Writer {
    fn write_str(&mut self, s: &str) -> fmt::Result {
        self.write_string(s);
        Ok(())
    }
}

Now, we can implement custom print! and println! macros that hook directly into Rust's formatting machinery:

#[macro_export]
macro_rules! print {
    ($($arg:tt)*) => ($crate::vga_buffer::_print(format_args!($($arg)*)));
}

#[macro_export]
macro_rules! println {
    () => ($crate::print!("\n"));
    ($($arg:tt)*) => ($crate::print!("{}\n", format_args!($($arg)*)));
}

#[doc(hidden)]
pub fn _print(args: fmt::Arguments) {
    use core::fmt::Write;
    WRITER.lock().write_fmt(args).unwrap();
}

8. Booting and Testing Our Kernel Output

Now, return to src/main.rs. We can test our brand-new VGA console driver:

#![no_std]
#![no_main]

mod vga_buffer;

use core::panic::PanicInfo;

#[no_mangle]
pub extern "C" fn _start() -> ! {
    println!("Hello World from Adoreka OS!");
    println!("Kernel initialized in bare-metal Rust.");
    println!("Active display mode: VGA 80x25 Text Matrix (0xB8000)");

    // Test formatted numbers
    let cores = 4;
    println!("Detected CPU cores: {}", cores);

    loop {}
}

#[panic_handler]
fn panic(info: &PanicInfo) -> ! {
    println!("{}", info);
    loop {}
}

When compiled and executed in an x86 emulator like QEMU:

cargo build --target x86_64-unknown-none

The QEMU virtual machine powers on, boots our raw binary, and immediately prints bright yellow text on a black screen! If a kernel panic occurs, our custom panic handler formats the error message, file location, and line number directly to the VGA display.

Summary of Architectural Milestones

In Part 3: CPU Interrupt Handling & The Global Descriptor Table (GDT), we will explore hardware exception handling, configure the CPU interrupt descriptor table (IDT), and protect our kernel stack against stack overflows using the Task State Segment.