Developing your own 32-bit operating system

Chapter 1, Introduction
Overview
What This Book Is About
Who Should Use This Book
My Basic Design Goals (yours may vary)
A Brief Description of MMURTL
Uses for MMURTL
Similarities to Other Operating Systems
Hardware Requirements
Chapter 2, General Discussion and Background
Where Does One Begin?
You Are Where You Were When
The Development Platform and the Tools
The Chicken and the Egg
Early Hardware Investigation
The Real Task at Hand (Pun intended)
A Starting Point
The Critical Path
Working Attitude
Chapter 3, The Tasking Model
Terms We Should Agree On
TASK
KERNEL
PREEMPTIVE MULTITASKING
Resource Management
Task Management
Single vs. Multi User
The Hardware State
The Software State
CPU Time
Single Tasking
Multitasking
Cooperative Multitasking
Preemptive Multitasking
Task Scheduling
Who, When, and How Long?
Scheduling Techniques
Time Slicing
Simple Cooperative Queuing
Prioritized Cooperative Scheduling
Variable Time Slicing
Other Scheduling Factors
Mixed Scheduling Techniques
Fully Time Sliced, Some Cooperation
Time Sliced, More Cooperative
Primarily Cooperative, Limited Time Slicing
Primarily Cooperative, More Time Slicing
The Tasking Model I Chose
Interrupt Handling
Chapter 4, Interprocess Communications
Introduction
Messaging and Tasks
Synchronization
Semaphores
Pipes
Messages
Send and Wait
Request()
Respond()
Link Blocks
Reentrancy Issues
Interrupt Latency
Memory Management Issues
Chapter 5, Memory Management
Introduction
Basic Terms
Memory Model
Simple Flat Memory
Figure 5.1, Simple Flat Memory
Paged Flat Memory
Figure 5.2 - Paged Flat Memory
Demand-Paged Flat Memory
Figure 5.3 - Demand Paged Flat Memory
Virtual Paged Memory
Figure 5.4 - Virtual Paged Memory
Demand Paged Virtual Memory
Demand-Paged Memory Management
Segmented Memory
Memory Management
Tracking Linear Memory Allocation
Basic Memory Allocation Unit
Linked List Management
Memory Management With Tables
Tracking Physical Memory
Initialization of Memory Management
Memory Protection
An Intel Based Memory Management Implementation
A Few More Words On Segmentation
How MMURTL Uses Paging
Page Tables (PTs)
Page Directories (PDs)
The Memory Map
Table 5.1 - MMURTL Memory Map
Page Directory Entries (PDEs)
Table 5.2 Page Directory Example
Allocation of Linear Memory
Deallocation of Linear Memory
Allocation of Physical Memory
Loading Things Into Memory
Messaging and Address Aliases
Memory and Pointer Management for Messaging
Summary of MMURTL Memory Management
Chapter 6, Hardware Interface
Hardware Isolation
The CPU
The Bus Structure
Serial I/O
Parallel I/O
Block-Oriented Devices
Keyboard
Video
Direct Memory Access (DMA)
Timers
Priority Interrupt Controller Unit (PICU)
Chapter 7, OS Initialization
Getting Booted
Boot ROM
The Boot Sector
Operating System Boot Image
Other Boot Options
Basic Hardware Initialization
Static Tables and Structures
Initialization of Task Management
Initialization of Memory
Dynamic Tables and Structures
Chapter 8, Programming Interfaces
Application Programming Interface
Documenting Your API
Procedural Interfaces
Calling Conventions
Mechanical Details
Portability Considerations
Error Handling and Reporting
Systems Programming Interface
Internal Cooperation
Device Driver Interfaces
A Device-Driver Interface Example
Announcing the Driver to the OS
A Call to the driver
Chapter 9, Application Programming
Introduction
Terminology
Understanding 32-BIT Software
Table 9.1 - Memory Alignment
Operating System Calling Conventions
Stack Usage
Memory Management
Table 9.2 - Basic Memory Map
Operating System Protection
Application Messaging
Starting a New Thread
Job Control Block
Basic Keyboard
Basic Video
Table 9.3 - Foreground Colors (Low nibble)
Table 9.4 - Background Colors (High nibble)
Terminating Your Application
Replacing Your Application
Chapter 10, Systems Programming
Introduction
Writing Message-Based System Services
Initializing Your System Service
A Simple System Service Example
Listing 10.1 - A System Service.
The Request Block
Items In the Request Block
The Service Code
Caller Information in a Request
Asynchronous Services
System Service Error Handling
Writing Device Drivers
Device Driver Theory
Building Device Drivers
How Callers Reach Your Driver
Device Driver Setup and Installation
System Resources For Device Drivers
Interrupts
Direct Memory Access Device (DMA)
Timer Facilities
Message Facilities for Device Drivers
Detailed Device Interface Specification
Device Control Block Setup and Use
Table 10.1-. Device control block definition
Standard Device Call Definitions
DeviceOp Function Implementation
DeviceStat Function Implementation
DeviceInit Function Implementation
Initializing Your Driver
OS Functions for Device Drivers
Standard Device Error Codes
Chapter 11, The Monitor Program
Introduction
Active Job (Video & keyboard)
Initial Jobs
Listing 11.1. Sample Initial.Job file.
Monitor Function Keys
Table 11.1. Monitor Function Keys
Monitor Program Theory
Performance Monitoring
Monitor Source Listing
Listing 11.2.Monitor program source listing.
Chapter 12, Debugger
Introduction
Using the Debugger
Entering the Debugger
Exiting the Debugger
Debugger Display
Debugger Function Keys
Debugging Your Application
Debugger Theory
Table 12.1.Processor exceptions.
Chapter 13, Keyboard Service
Introduction
Available Services
Table 13.1.Available Keyboard Services
Read Keyboard
Notify On Global Keys
Cancel Notify On Global Keys
Assign Keyboard
Key codes and Status
Alpha-Numeric Key Values
Table 13.2 - .Shift State Bits
Lock-State Byte (third byte)
Table 13.3 - Lock State Bits
Numeric Pad Indicator
Key Codes
Table 13.4 - .Keys Values
Function Strip Key Codes
Table 13.5 - Function Key Strip Codes
Numeric Pad Key Codes
Table 13.6 - Numeric Pad Key Codes
Cursor, Edit, and Special Pad Key Codes
13.7.Additional Key Codes
Your Keyboard Implementation
Chapter 14, The File System Service
Introduction
File Specifications
Network File Request Routing
File Handles
File Open Modes
File Access Type
Block Mode
Stream Mode
Logical File Address
File System Requests
Listing 14.1 - Openfile request in C
Table 14.1 - File System Service Codes
Procedural Interfaces
Device Access Through the File System
Table 14.2 - Device Stream Access
File System Functions in Detail
OpenFile
CloseFile
ReadBlock
WriteBlock
ReadBytes
WriteBytes
GetFileLFA
SetFileLFA
GetFileSize
SetFileSize
CreateFile
RenameFile
DeleteFile
CreateDirectory
DeleteDirectory
GetDirSector
File System Theory
Internal File System Structures
File Control Blocks
File User Blocks
FAT Buffers
File Operations
Read
Write
Close
Chapter 15, API Specification
Introduction
Public Calls
Parameters to Calls (Args)
Table 15.1 - Size and Type Prefixes
Table 15.2 - Additional Prefixes
Table1 5.3 - Compound Prefixes
Stack Conventions
Calls Grouped by Functionality
Alphabetical Call Listing
AddCallGate
AddIDTGate
Alarm
AliasMem
AllocDMAPage
AllocExch
AllocOSPage
AllocPage
Beep
Chain
CheckMsg
ClrScr
Compare
CompareNCS
CopyData
CopyDataR
DeAliasMem
DeAllocExch
DeAllocPage
DeviceInit
DeviceOp
DeviceStat
DMASetUp
EditLine
EndOfIRQ
ExitJob
FillData
GetCmdLine
GetCMOSTime
GetCMOSDate
GetDMACount
GetExitJob
GetIRQVector
GetJobNum
GetNormVid
GetPath
GetPhyAdd
GetpJCB
GetSysIn
GetSysOut
GetTimerTick
GetTSSExch
GetUserName
GetVidChar
GetVidOwner
GetXY
InByte
InDWord
InWord
InWords
InitDevDr
ISendMsg
KillAlarm
LoadNewJob
MaskIRQ
MicroDelay
MoveRequest
NewTask
OutByte
OutDWord
OutWord
OutWords
PutVidAttrs
PutVidChars
QueryPages
ReadCMOS
ReadKbd
RegisterSvc
Request
Respond
ScrollVid
SendMsg
SetCmdLine
SetExitJob
SetIRQVector
Table 15.4 - Hardware IRQs
SetJobName
SetNormVid
SetPath
SetPriority
SetSysIn
SetSysOut
SetUserName
SetVidOwner
SetXY
Sleep
SpawnTask
Tone
TTYOut
UnMaskIRQ
UnRegisterSvc
WaitMsg
Chapter 16, MMURTL Sample Software
Introduction
Command Line Interpreter (CLI)
Internal Commands
Cls (Clear Screen)
Copy (Copy a file)
Dir (Directory listing)
Debug (Enter Debugger)
Del (Delete a File)
Dump (Hex dump of a file)
Exit (Exit the CLI)
Help (List internal commands)
Monitor (Return to Monitor)
MD (Make Directory)
Path (Set file access path)
RD (Remove Directory)
Rename (Rename a file)
Run (Execute a .RUN file)
Type (View a text file on the screen)
External Commands
Commands.CLI File
Global "Hot Keys"
CLI Source Listing
Listing 16.1.CLI Source Code Listing Ver. 1.0.
A Simple Editor
Editor Screen
Editor Commands
Table 16.1 - File Management and General Commands
Table 16.2 - Cursor and Screen Management Commands
Listing 16.2.Editor Source Code.
DumbTerm
Listing 16.3 - DumbTerm source code.
Print
Listing 16.4.Print source code.
System Service Example
Service.C Listing
Listing 16.5 - Simple Service Source Code.
TestSvc.C Listing
Listing 16.6 - TestSvc.C.
Chapter 17, Introduction to the Source Code
Comments In the Code
Calling Conventions
Organization
Building MMURTL
Listing 17.1 – Assembler Template File
Using Pieces of MMURTL in Your OS
Using Pieces of MMURTL in Other Projects
Chapter 18, The Kernel
Introduction
Naming Conventions
Kernel Data
Listing 18.1.Kernel data segment source.
Local Kernel Helper Functions
enQueueMsg
Listing 18.2.Queueing for messages at an exchange.
deQueueMsg
Listing 18.3.De-queueing a message from an exchange.
deQueueTSS
Listing 18.4.De-queueing a task from an exchange.
enQueueRdy
Listing 18.5.Adding a task to the ready queue.
deQueueRdy
Listing 18.6.De-queueing the highest priority task.
ChkRdyQ
Listing 18.7.Finding the highest priority task
Internal Public Helper Functions
RemoveRdyJob
Listing 18.8.Removing a terminated task from the ready queue.
GetExchOwner
Listing 18.9.Finding the owner of an exchange.
SetExchOwner
Listing 18.10.Changing the owner of an exchange.
SendAbort
Listing 18.11.Notifying services of a job’s demise.
Public Kernel Functions
Request()
Listing 18.12.Request kernel primitive code.
Respond()
Listing 18.13.Respond kernel primitive code.
MoveRequest()
Listing 18.14, MoveRequest kernel primitive code.
SendMsg()
Listing 18.15.SendMsg kernel primitive code.
ISendMsg()
Listing 18.16.IsendMsg kernel primitive code.
WaitMsg()
Listing 18.17.WaitMsg kernel primitive code.
CheckMsg()
Listing 18.18 -CheckMsg kernel primitive code.
NewTask()
Listing 18.19.NewTaskkernel primitive code.
SpawnTask()
Listing 18.20.SpawnTask kernel primitive code
AllocExch()
Listing 18.21.Allocate exchange code
DeAllocExch()
Listing 18.22.Deallocate exchange code
GetTSSExch()
Listing 18.23.GetTSS exchange code.
SetPriority()
Listing 18.24.Set priority code.
GetPriority()
Listing 18.25.Get priority code.
Chapter 19, Memory Management Code
Introduction
Memory Management Data
Listing 19.1.Memory management constants and data
Internal Code
InitMemMgmt
Listing 19.2.Memory management initialization code
Listing 19.3.Continuation of memory management init.
Listing 19.4.Continuation of memory management init.
Listing 19.5.Turning on paged memory management
Listing 19.6.Allocation of memory management exchange.
Listing 19.7.Finishing memory management initi.
FindHiPage
Listing 19.8.Find highest physical page code.
FindLoPage
Listing 19.9.Find lowest physical page code.
MarkPage
Listing 19.10.Code to mark a physical page in use.
UnMarkPage
Listing 19.11.Code to free a physical page ofr reuse.
LinToPhy
Listing 19.12.Code to convert linear to physical addresses.
FindRun
Listing 19.13.Code to find a free run of linear memory
AddRun
Listing 19.14.Adding a run of linear memory.
AddAliasRun
Listing 19.15.Adding an aliased run of linear memory.
AddUserPT
Listing 19.16.Adding a page table for user memory
AddOSPT
Listing 19.17.Adding an operating system page table.
Public Memory Management Calls
AddGDTCallGate
Listing 19.18.Adding a call gate to the GDT.
AddIDTGate
Listing 19.19.Adding entries to the IDT.
AllocOSPage
Listing 19.20.Allocating a page of linear memory.
AliasMem
Listing 19.21.Code to alias memory.
DeAliasMem
Listing 19.22.Code to remove alias PTEs.
DeAllocPage
Listing 19.23.Deallocating linear memory.
QueryMemPages
Listing 19.24.Code to find number of free pages
GetPhyAdd
Listing 19.25.Public for linear to physical conversion.
Chapter 20, Timer Management Source Code
Introduction
Choosing a Standard Interval
Listing 20.1.Loop in timer ISR
Timer Data
Listing 20.2 - Data and constants for timer code
Timer Code
Listing 20.3 - External functions for timer code
The Timer Interrupt
Listing 20.4 - The timer interrupt service routine
Sleep()
Listing 20.5 - Code for the Sleep function
Alarm()
Listing 20.6 - Code for the Alarm function
KillAlarm()
Listing 20.7 - Code for the KillAlarm function
MicroDelay()
Listing 20.8 - Code for the MicrDelay function
GetCMOSTime()
Listing 20.9.Code to read the CMOS time
GetCMOSDate()
Listing 20.10 - Code to read the CMOS date
GetTimerTick()
Listing 20.11.Code to return the timer tick
Beep_Work()
Listing 20.12.Helper function for Beep() and Tone()
Beep()
Listing 20.13.Code to produce a beep.
Tone()
Listing 20.14 - Code to produce a tone
Chapter 21, Initialization Code
Introduction
OS Global Data
Listing 21.1 - Main Operating System Data
Operating System Entry Point
Listing 21.2 - OS Initialization Code and Entry Point
Listing 21.3.Continuation of OS initialization.
Listing 21.4.Continuation of OS initialization
Listing 21.5 - End of Initialization Code
Initialization Helpers
Listing 21.6 - Initialization Subroutines
Listing 21.7 - Initialize call gates amd the IDT
Chapter 22, Job Management Code
Introduction
Reclamation of Resources
Job Management Helpers
Listing 22.1 - Job Management Subroutines
Job Management Listing
Listing 22.2 – Jobc.c Source code
Chapter 23, Debugger Code
Introduction
Debugger Interrupt Procedure
Listing 23.1 - Exception Handlers for Entering Debugger
Debugger Source Listing
Listing 23.2 – Debugger Data and Code
Chapter 24, Selected Device Driver Code
Introduction
IDE Disk Device Driver
Listing 24.1 - IDE Device Driver Data (Defines and Externs)
Listing 24.2 - Continuation of IDE Driver Data (protos and defines)
Listing 24.3 - Continuation of IDE Driver Data (data structures)
Listing 24.4 - IDE Device Driver Code (Initialization)
Listing 24.5 - Continuation of IDE Driver Code (code)
Listing 24.6 - Continuation of IDE Driver Code (Device Interface)
The RS-232 Asynchronous Device Driver
Listing 24.7 - RS-232.h Header File
Listing 24.8 - RS-232 Device Driver Code (defines and externs)
Listing 24.9 - RS-232 Device Driver Code Continued (Register bits)
Listing 24.10 - RS-232 Device Driver Code Continued (status record)
Listing 24.11 - RS-232 Device Driver Code Continued (DCB)
Listing 24.12 - RS-232 Device Driver Code Continued (DCB Init)
Listing 24.13 - RS-232 Device Driver Code Continued (ISR and code)
Listing 24.14 - RS-232 Device Driver Code Continued (Interface)
Chapter 25, Keyboard Source Code
Introduction
Keyboard Data
Listing 25.1 - Keyboard Data and Tables
Keyboard Code
Keyboard ISR
Listing 25.2 - Keyboard ISR Code
Keyboard Translation
Listing 25.3 - Keyboard Translation Code
Reading the Final Buffer
Listing 25.4.Read Keyboard Buffer Code
The Keyboard Service
Listing 25.5.Keyboard system service code.
Keyboard Procedural Interface
Listing 25.6 - Code for Blocking Read Keyboard Call
Debugger Keyboard
Listing 25.7 - Debugger Read Keyboard Code
Keyboard Hardware Set Up
Listing 25.8 - Code to Initialize the Keyboard Hardware
Keyboard Service Initialization
Listing 25.9 - Keyboard Service Initialization
Hardware Helpers
Listing 25.10 - Low-level Keyboard Control Code
Chapter 26, Video Code
Introduction
Virtual Video Concept
VGA Text Video
The Video Code
Listing 26.1.Video constants and data
Listing 26.2 - Code to Initialize Video
Listing 26.3 - Code to Set the Video Owner.
Listing 26.4 - Code to set Normal Video Attribute.
Listing 26.5 - Code to get the Normal Video Attribute
Listing 26.6 - Code to Get the Current Video Owner
Listing 26.7 - Code to Clear the Screen
Listing 26.8 - Code for TTY Stream Output to Screen.
Listing 26.9.Code to Screen Video Attributes
Listing 26.10 - Code to Place Video chars Anywhere on the Screen
Listing 26.11.Code to Get a Character and Attribute.
Listing 26.12.Code to Scroll an Area of the Screen
Listing 26.13 - Support Code to Set Hardware Cursor Position.
Listing 26.14.Code to Set Cursor Position
Listing 26.15.Code to Return Current Cursor Position
Listing 26.16 - Code to Edit a Line on the Screen
Relating MMURTL’s Video Code to Your Operating System
Chapter 27, File System Code
Introduction
How MMURTL Handles FAT
Physical (Absolute) Sector Numbers
Hidden Sectors
Logical Block Address (LBA)
MS-DOS Boot Sector
File System Initialization
File System Listing
Listing 27.1 - MS-DOS FAT-Compatible File System Code
Chapter 28, DASM; A 32-Bit Intel-Based Assembler
Introduction
DASM Concepts
Using DASM
Application Template File
Listing 28.1 - Simple Application Template File
Listing 28.2 - A More Complicated Template File
Command Line Options
List File Option
Errors Only Option
Dynamic Link Libraries
Device Drivers
DASM Source Files
Local (Non-Public) Declarations
Memory Model
DASM Program Structure
Listing 28.3 - Alternating data and code in a template file
DOT Commands
DASM versus DOS Assemblers
DASM Addressing Modes
Table 28.1 - .Specifying Memory Operands
Storage Declarations
DB (Define Byte)
DW (Define Word)
DD (Define Double Word)
DF (Define Far Word)
DASM Peculiar Syntax
PUBLIC Declarations
EXTRN Declarations
Labels and Address
Segment and Instruction Prefixes
CALL and JUMP Addresses
386 Instruction Set
Table 28.2 - Supported Instructions
Executable File Format
Table 28.3 - Tag Usage
Tag Descriptions
Error Codes from DASM
Your Own Assembler?
Chapter 29, CM32; A 32-Bit C Compiler
Introduction
Memory Usage and Addressing
Important Internal Differences
Variable Length Argument Lists
CM32 Language Specifics
FAR Calls
Interrupt Functions
Supported C Language Features
Library Header File Access
Limitations (compared to ANSI)
Structure Limitations
Far External Functions
Type Conversion and Pointers
Initialization of Static Variables
Other Implementation Dependent Information
Using CM32
Command Line Options
Library Functions
Listing 29.1 - Supported Library Functions
Library Code Examples
Listing 29.2 - Standard file I/O for MMURTL
Listing 29.3 - Standard String Functions for MMURTL