the tape-kernel

the tape-kernel is a modular modern independent kernel

it is designed to be simple and un-abstracted to use

license

GPL-3.0-or-later - see the LICENSE file for details

quickstart

git clone https://codeberg.org/Druid520/tape-kernel.git
cd tape-kernel
make iso
make run

and at the '$' prompt run

help

minimum hardware specs

for running kernel on bare metal you will need recommended

• cpu: i486
• memory: 3.8mb~
• disk: 1.44mb~ formatted with fat12
• graphics: vga text mode
• input: ps/2 or at keyboard
• support: 32 bit protected mode and csm or bios

dependencies

to compile and run the kernel you will require

• qemu i386 with graphical support
• the gnu assembler
• the gnu C compiler
• the gnu linker
• make
• gnu binary utilitys
• gnu core utilitys (or equivalent)
• git
• xorriso (libisoburn)
• dosfstools

to develop on the kernel itself you will need in addition

• gdb (optionally gef but highly recommended)
• doxygen
• graphviz

how to install dependencies

to install on Arch Linux and Arch based systems install with:

sudo pacman -S --needed coreutils qemu-desktop gcc make dosfstools git xorriso binutils # gdb doxygen graphviz

to install on Debian and Debian based systems install with:

sudo apt-get install coreutils qemu-system-x86 gcc make dosfstools git xorriso binutils # gdb doxygen graphviz

to install on Fedora and Fedora based systems install with:

sudo dnf install --needed coreutils qemu-system-x86 gcc make dosfstools git xorriso binutils # gdb doxygen graphviz

to install on Alpine Linux and Alpine based systems install with:

sudo apk add coreutils qemu-system-i386 gcc make dosfstools git xorriso binutils # gdb doxygen graphviz

to install on OpenSUSE and OpenSUSE based systems install with:

sudo zypper install coreutils qemu-x86 gcc make dosfstools git xorriso binutils # gdb doxygen graphviz

to install on Void Linux and Void based systems install with:

sudo xbps-install coreutils qemu gcc make dosfstools git libisoburn binutils # gdb doxygen graphviz

to install on FreeBSD and BSD based systems install with:

sudo pkg install coreutils qemu gcc make dosfstools git xorriso binutils # gdb doxygen graphviz

to install on macOS using Homebrew install with:

brew install coreutils qemu gcc make dosfstools git xorriso binutils # gdb doxygen graphviz

note: on macOS using Homebrew gdb requires codesigning, consider using lldb

compiling

to compile the tape kernel you can use the following commands

make

which only makes the .elf and

make iso

which is recommended as it generates the entire iso

running

to run the tape kernel you can use the following commands

make run

which is recommended as it runs it normally

make debug

which debugs the kernel with gdb (see dev dependencies)

debugging

note: this will assume your using gef (seriously use it)

to connect to qemu using gdb begin with running

make debug

then in a separate terminal run from the tape-kernel directory

gdb build/tape.elf

then when in gdb run

gef-remote --qemu-user --qemu-binary build/tape.elf localhost 1234

gdb helps with not spamming assert, panic, or print statements every line, common gdb commands include:

break <function to stop at> # breakpoint
hbreak <function to stop at> # hardware assisted breakpoint
continue # continue
step # steps one process at the current break
next # step over function
finish # step out of current function
call <function>(<args/params>) # calls a function currently loaded
print <variable> # print variable value
print/x <variable> # print variable value in hex
x/<n>x <address> # examine memory, e.g. 'x/16xb 0x100000'
info registers # cpu register info
info breakpoints # breakpoint info
info locals # show local variables
backtrace # show call stack, sometimes called bt
disassemble # show assembly code
list # show source code around current line
delete <number> # delete breakpoint by number
disable <number> # disable breakpoint by number
enable <number> # enable breakpoint by number
watch <variable> # break when variable changes
condition <number> <expression> # conditional breakpoint, e.g. 'break cmd_alloc if size > 1000'
vmmap # show memory map
xinfo <address> # show info about memory address
dereference <address> # show pointer chain
heap # examine heap

you can also debug by viewing src/build/kernel.map with a text editor or pager of your choice (e.g. nano or cat)

in the kernel

when in the kernel at the shell prompt run

help

for a list of commands

notes:

if you click something and it doesnt type it most likely hasnt been registered as a switch case yet in function 'scntasci' inside of src/io/kb.c

troubleshooting

check all dependencys are installed

ensure you have

make iso

and not only generated elf

fixes:

gdb can't connect to qemu

• ensure make debug is running in another terminal
• check that port 1234 is not in use: sudo lsof -i :1234
• try adding -s manually to QEMU command

kernel panics with "out of memory"

• check heap size in main.c (default 1MB)
• run heap command to see used memory
• use hreset() or reboot to clear all allocations

disk writes dont persist

• disk image is build/disk.img - it persists across boots
• if corrupted, delete and rebuild: rm build/disk.img && make iso

filesystem shows "(empty)" but you wrote files

• run ls to list files, then read <filename> to read them
• note: files are stored by name, not by LBA
• use write <name> <msg> not the raw write command

keyboard doesnt respond or types garbage

• ensure qemu window has focus
• ps/2 emulation can be slow in some qemu versions, try restarting qemu

screen scrolling doesnt work properly

• the scrl() function scrolls lines 1-24, leaving row 0 untouched
• call clscr() to clear the screen completely

kernel triple-faults and qemu reboots

• this usually means a stack overflow or invalid memory access
• run make debug and use gdb to find the exact crash location
• check for buffer overflows in prt() or string functions

build fails with "no such instruction" in boot.s

• gas uses # or // for comments, not ;
• change ; comment to # comment in boot.s

disk image is corrupted after many writes

• delete build/disk.img and rebuild: rm build/disk.img && make iso
• this creates a fresh 2mb fat12 disk image

contributing

please use commit prefixes of

when purely adding content use, ADD:
when modifying, or both adding and removing use, MOD:
when purely removing content use, DEL:

to keep the repo clean, readable, and simple when committing/sending pull requests

keep the codebase clean, simplistic, uncomplicated, and well commented

coding style

when contributing the kernel please use simplistic code, dont add unless needed, dont cause breakage, and always verify it works BEFORE pushing

keep code compact and coherent, example:

if (result == 1) { //result 1 means it has passed verification
    char msg[7] = "passed";  //set message
    prt(0, 0, msg, 0x0F); //print at x and y 0 "passed" with white text on black bg
} else if (result == 0 ) { //result 0 means it failed
    char msg[7] = "failed";  //set message
    prt(0, 0, msg, 0x0F); //print at x and y 0 "failed" with white text on black bg
} else {
    panic("fatal error during verification: unknown value"); //panic if unknown value, should never happen
}

naming preface

please name functions with the following pattern

myfunc //public function, anything can call it
__myfunc //internal/private function, dont call this use the public macro
___myfunc //kernel core code like ___kmain, ___start, and ___init, do not call unless you know what your doing

and define public functions in the header with

#define myfunc __myfunc

FOR ADVANCED USERS/DEVELOPERS

note: for better complete documentation use

make docs

and open 'src/html/index.html' in your browser

project structure

the current src tree is consisted of

src # main source tree
├── boot # contains boot.s assembly, src/boot/boot.s
├── fs # file system code
├── io # input and output code
├── kernel # kernel codebase, includes things like 'kmain' in src/kernel/main.c
├── lib # librarys such as types.h and utils.h
├── mem # memory operations
└── usr # userspace, shell, etc

key data structures

arena_t

typedef struct {
    uint32_t *start; //heap start address
    uint32_t *current; //current allocation pointer
    uint32_t size; //total size in bytes
    void *next; //the list for the arena allocator
} arena_t;

file table (src/fs/fs.h and src/fs/fs.c only)

typedef struct {
    char *name; //filename (heap allocated)
    uint32_t lba; //starting sector
} fs_entry_t;

adding new shell commands

• adding a command handler in 'shell'

else if (strcmp(args[0], "mycmd") == 0) {
    cnb(&cx, &cy);
    prt(0, cy, "mycmd executed", 0x0F);
}

• adding help text in 'show_help'

prt(0, row++, "mycmd            - description", 0x0F);

• creating helper functions

void cmd_mycmd(char *arg) {
    //implementation
}

ffs flat file system

model

lba allocation: 0:

• boot sector 1: file table 2-99:
• reserved (unused) 100+: file
• content (every file uses 1 sector)

file entry layout (36 bytes)

• bytes 0-31: filename (null-terminated, max 31 chars)
• bytes 32-35: lba (uint32_t, little endian)

file table format (sector 1)

offset      size        field       description
0           1           magic[0]    'F' (0x46)
1           1           magic[1]    'S' (0x53)
2           1           file_count  number of files (0-32)
3+          36*n        entry[n]    file entry for file n

file entry layout (36 bytes)

• bytes 0-31: filename (null-terminated, max 31 chars)
• bytes 32-35: lba (uint32_t, little endian)

file content format

each file occupies exactly one sector (512 bytes):

• first 511 bytes: file content (null terminated string)
• byte 511: unused (reserved for null terminator if needed)

read path 'fsread'

• fsread(name) -> linear scan of 'files[]' array
• get lba from entry
• 'irsec(lba, sector)' -> read 512 bytes from disk
• copy sector to static buffer (up to 511 bytes)
• return pointer to buffer

write path 'fswrite'

• fsfind(name) -> check if file exists
• if exists: use existing lba
• if not: call fsnextlba() → find next free lba
• clear sector (512 bytes of zeros)
• copy content into sector (max 511 bytes)
• iwrt(lba, sector) → write to disk
• if new file: fsadd() → add entry to table and heap
• save table to sector 1

delete path 'fsdelete'

• fsfind(name) -> check if file exists
• if not: panic
• if exists: clear the file
• mark lba as free
• rebuild fs
• write the new fs

lba allocation (fsnextlba)

current implementation:

    //find first unused lba
    for (int i = 0; i < MAX_FILES; i++) {
    if (lba_map[i] == 0) {
            uint32_t new_lba = 100 + (i * 2); //start at sector 100, each file gets its own sector
            lba_map[i] = new_lba;
            return new_lba;
    }
}

example:

• file_0: lba 100
• file_1: lba 102
• file_2: lba 104 ...

memory map

• 0x00000000 - 0x000FFFFF: reserved (bios, bootloader, vga)
• 0x00100000 - 0x0010????: kernel .text, .data, .bss
• 0x0010???? - 0x001FFFFF: heap (bump allocator, 1MB default)
• 0x00200000 - 0x00FFFFFF: unused (free memory)
• 0x000B8000 - 0x000BFFFF: vga text buffer (80x25, 16-bit cells)

boot sequence

• bios loads isolinux (syslinux)
• isolinux reads multiboot via headers from 'src/boot/boot.s' using mboot.c32
• syslinux loads tape.elf at 0x100000
• syslinux jumps to '_start' (boot.s)
• '_start' sets up stack (16KB)
• '_start' calls 'kmain'
• 'kmain' calls 'init' (initializes other processes as a unified function)
• 'kmain' calls shell while active (subject to change)

memory layout

stack memory layout

src/boot/boot.s contains a 16kb stack at the .bss section

• stack_bottom: base
• stack_top: top (grows downward)

each function call pushes:

• return address (4 bytes)
• ebp (4 bytes if frame pointer enabled)
• local variables

heap memory layout

the heap allocator is done with a arena allocator using a bump allocators logic, it does allow child arenas

the heap is allocated by calling 'hinit' with your heap pointer, in the kernel the default for the system is kheap with 1MB of storage

allocating to the heap is done with

alc(&heap, size);

or

void *ptr = alc(&heap, size); //returns aligned memory

resetting the heap is done with

res(&heap);  //sets heap->current = heap->start

and making a new arena under one heap is done with

anew(&heap, size);

you cannot deallocate from the heap as free isnt and wont be implemented for now, you can only initialize, allocate to, and reset the heap

when making a arena under a heap you do not need to use the & prefix, you may treat it as a ptr

making a custom heap is done using 'arena_t' type, one example is:

#define HEAP_SIZE (1024 * 1024) //1mb heap
arena_t heap; //initialize heap and memory
static uint8_t mem[HEAP_SIZE];

each allocation:

• aligns size to 4 bytes (for uint32_t compatibility)
• checks if current + size exceeds heap bounds
• saves current pointer
• advances current pointer by size
• returns saved pointer

memory layout after boot

0x00100000: kernel .text, .data, .bss
0x0010????: kernel_end (start of heap)
0x0010????: heap->current (moves forward on each alloc)
0x001FFFFF: heap end (1MB mark)

file entries are cached in heap for fast access:

• each filename: 32 bytes (31 chars + null)
• file_count: up to 32 entries
• total heap used: ~1KB for file table cache

function call conventions

• 32 bit cdecl: parameters on stack, return in eax
• caller cleans stack
• registers: eax, ecx, edx are scratch (caller saved)
• ebx, esi, edi, ebp, esp are callee-saved

port i/o

all port functions are defined in src/io/io.h currently

uint8_t  inb(uint16_t port); //read byte
uint16_t inw(uint16_t port); //read word
void outb(uint16_t port, uint8_t val); //write byte
void outw(uint16_t port, uint16_t val); //write word

pit ports

• 0x40: main pit port
• 0x43: pit cmd port

ide ports

• 0x1F0: data register (16-bit)
• 0x1F1: error register
• 0x1F2: sector count
• 0x1F3: lba low (bits 0-7)
• 0x1F4: lba mid (bits 8-15)
• 0x1F5: lba high (bits 16-23)
• 0x1F6: drive select (bits 24-27)
• 0x1F7: command/status
• 0x3F6: alternate status/control

pit

pit in the kernel is rougly initialized with

psuedo code:

#define PIT_FREQUENCY 1193182
//set pit to 1000 hz (1ms ticks)
uint32_t divisor = PIT_FREQUENCY / 1000;
outb(PIT_CMD_PORT, 0x34);  //channel 0, lobyte/hibyte, rate gen
outb(PIT_PORT, divisor & 0xFF);
outb(PIT_PORT, (divisor >> 8) & 0xFF);

ide

ide command sequence (read)

psuedo code:

while (status & 0x80); //wait for bsy clear
outb(0x1F6, 0xE0 | (lba>>24)); //select master
outb(0x1F2, 1); //1 sector
outb(0x1F3, lba & 0xFF);
outb(0x1F4, (lba>>8) & 0xFF);
outb(0x1F5, (lba>>16) & 0xFF);
outb(0x1F7, 0x20); //read command
while (!(status & 0x08)); //wait for drq
for(i=0; i<256; i++) buffer[i] = inw(0x1F0); //read data

ide command sequence (write)

psuedo code:

while (status & 0x80); //wait for bsy clear
outb(0x1F6, 0xE0 | (lba>>24));
outb(0x1F2, 1);
outb(0x1F3, lba & 0xFF);
outb(0x1F4, (lba>>8) & 0xFF);
outb(0x1F5, (lba>>16) & 0xFF);
outb(0x1F7, 0x30); //write command
while (!(status & 0x08)); //wait for drq
for(i=0; i<256; i++) outw(0x1F0, buffer[i]); //write data
while (status & 0x80); //wait for completion

vga text mode

• memory: 0xB8000 - 0xBFFFF (32KB)
• cell format: [ attribute | character ]
• attribute byte: high nibble = background, low nibble = foreground

offset for cells can be calculated with:

int offset = y * 80 + x

common colors:

• 0x0F: white on black
• 0x1F: white on blue
• 0x0C: red on black
• 0x0E: yellow on black

ps/2 keyboard scancodes

• port 0x60: data register
• port 0x64: status register

scancode → ascii conversion (kb.c):

• scancode 0x02 → '1'
• scancode 0x03 → '2'
• scancode 0x1C → 'a' (unshifted) or 'A' (shifted)
• etc (full table in src/io/kb.c under the 'scntasci' function)

scancodes >= 0x80: key release (ignored)

'alc' internals

struct arena_t {
    uint32_t *start; //base address
    uint32_t *current; //next free address
    uint32_t size; //total bytes
    struct arena *next; //the list for the arena allocator  
};

allocation:

• align size to 4 bytes
• check (current + size) <= (start + size)
• ptr = current
• current += size
• return ptr
• reset: current = start (all allocations become invalid)

shell command parsing

• rdln() reads line into buffer
• pargs() splits into argv array
• strcmp(args[0], "command") dispatches
• atoi() converts numeric arguments

example: "write hello.txt Hello"

• → argc=3, args[0]="write", args[1]="hello.txt", args[2]="Hello"

error handling

printf-style debugging (prt, prtd, prth)

prt(x, y, "string", color); //print string
prtd(x, y, 123, color); //print decimal 123
prth(x, y, 0xDEAD, color); //print hex "DEAD"

assert and panic

assert(condition, "error message"); //if condition false → panic()
panic("message"); //print error, halt system

functions

primary functions for higher level work (e.g. shell) include:

• prt: print, prints text, used as:

prt(<x location>, <y location>, "<text>", <color);

and color is used as 0x<bg><col> (e.g. 0x1F is white text on a blue background)

• rdln: readline, used as:

char input[100]; //100 is the array size, so its the limit of characters
rdln(input, 100); //the variable for input and the array limit, in this case 100

• cob, set cursor pos, used as:

cob(<x>, <y>);

which sets the cursor postion to that

• cnb, give current cursor pos, used as:

cnb(&<x cursor var>, &<y cursor var>); //popular variables include cx and cy

• delay, delays a program, used as:

delay(ms);

so you could do:

if (a == b) {
    delay();
}

• strcmp, string comparison, used as:

strcmp(<string 1>, <string 2>); //if returns 0 they are the same, if returns else it isnt

and a common use of this is in src/usr/shell.c that uses a block of code similar to:

if (strcmp(args[0], "<command>") == 0) {
    <do something>
}