25KB
Developing Your Own 32-Bit Operating System — Front-Matter: Front matter (covers, title, copyright, contents, preface, acknowledgments, about the author)
- Source: /library/Developing Your Own 32-Bit Operating System/source-file.pdf
- PDF pages: 1–22
- Pages without text: none
SAMS PUBLISHING CD-ROM contains source code for a 32-bit operating system, a 32-bit assembler, a W 32-bit C compiler, and more! Master the details of 32-bit programming and operating systems Explore interprocess communications, hardware interfaces, systems programming, file system services, API specifications, and more Design and develop a feature- \ rich, 32-bit operating system Developing Your Own Operating System
Have you ever wanted to write your own computer operating system? Or have you ever wanted to find out what it would take? Or have you just wanted to see the source code with real comments? If so, then this book is for you! This book contains the ideas, sugges- tions, and tools you will need — plus a sample operating system, MMURTL. You will see these features of MMURTL highlighted: Micro-kernel Message based Real-time Multitasking Multithreaded Client/server oriented ISA 32-bit PC-compatible The CD-ROM contains: Complete MMURTL operating system source code Sample applications, including source code A 32-bit assembler, including source code A 32-bit C compiler, including source code
irO Developing Your Own 32-Bit Operating System Richard A. Burgess SAMS PUBLISHING 201 West 103rd Street Indianapolis, Indiana 46290
For Deborah, Valerie, and Heather Copyright © 1995 by Sams Publishing FIRST EDITION All rights reserved. No pan of this book shall be reproduced, stored in a retrieval system, or transmitted by any means, electronic, mechanical, photocopying, recording, or otherwise, without written permission from the publisher. No patent liability is assumed with respect to the use of the information contained herein. Although every precaution has been taken in the preparation of this book, the publisher and author assume no responsibility for errors or omissions. Neither is any liability assumed for damages resulting from the use of the information contained herein. For information, address Sams Publishing, 201 W. 103rd St., Indianapolis, IN 46290. International Standard Book Number: 0-672-30655-7 Library of Congress Catalog Card Number: 94-69271 98 97 96 95 4 3 2 1 Interpretation of the printing code: the rightmost double-digit number is the year of the book's printing; the rightmost single-digit, the number of the book's printing. For example, a printing code of 95-1 shows that the first printing of the book occurred in 1995. Composed in AGaramond and MCPdigital by Macmillan Computer Publishing Printed in the United States of America Trademarks All terms mentioned in this book that are known to be trademarks or service marks have been appropriately capitalized. Sams Publishing cannot attest to the accuracy of this information. Use of a term in this book should not be regarded as affecting the validity of any trademark or service mark.
Publisher Richard K. Swadley Acquisitions Manager Greg Wiegand Managing Editor Cindy Morrow Acquisitions Editors Chris Denny Stacy Hiquet Development Editors Todd Bumbalough Bradley L. Jones Software Development Specialist Timothy Wilson Production Editor Deborah Frisby Copy Editors Fran Blauw Johnna VanHoose Joe Williams Editorial Coordinator Bill Whitmer Editorial Assistants Carol Ackerman Sharon Cox Lynette Quinn Technical Reviewer David McLarty Marketing Manager Team Supervisor Gregg Bushyeager Katy Bodenmiller Assistant Marketing Manager Page Layout Michelle Milner Charlotte Clapp Cover Designer Mary Ann Cosby Judy Everly Tim Amrhein Shawn MacDonald Book Designer Tina Trettin Alyssa Yesh Proofreading Director of Production and Donna Martin Manufacturing Brian-Kent Proffitt JeffValkr Erich J. Richter Production Manager Indexer Kelly Dobbs GregEldred Manufacturing Coordinator Paul Gilchrist Production Analysts Angela D. Bannan Bobbi Satterfield Dennis Clay Hager Support Services Supervisor Mary Beth Wakefield Graphics Image Specialists Teresa Forrester Clint Lahnen Tim Montgomery Dennis Sheehan Greg Simsic Susan VandeWalle JeffYesh
Overview Preface I Introduction 1 Overview 2 General Discussion and Background II Architecture and General Theory 3 The Tasking Model Interprocess Communications Memory Management The Hardware Interface OS Initialization Programming Interfaces III MMURTL-Specific Details 9 Application Programming Systems Programming The MMURTL Monitor Program The Debugger Keyboard Service File System Service API Specification MMURTL Sample Software IV The Operating System Source Code 17 Introduction to the Source Code Kernel Code Memory Management Code Timer Management Code Initialization Code Job Management Code 10 11 12 13 14 15 16 18 19 20 21 22 3 13 25 41 55 81 99 115 129 141 159 181 189 201 221 265 327 335 377 405 419 439
23 Debugger Code 24 Selected Device Driver Code 25 Keyboard Code 26 Video Code 27 File System Code V Additional Software 28 DASM: A 32-Bit Intel-Based Assembler 29 CM32: A 32-Bit C Compiler Index 471 497 541 571 591 663 697 719
Contents
Part I Introduction
1 Overview 3
What This Book Is About '
5
Who Should Use This Book 6
My Basic Design Goals (Yours May Vary)
7
A Brief Description of MMURTL 9
Uses for MMURTL 9
Similarities to Other Operating Systems 10
Hardware Requirements 10
2 General Discussion and Background 13
Where Does One Begin? 14
You Are Where You Were When 14
The Development Platform and the Tools 1 5
The Chicken and the Egg 16
Early Hardware Investigation 17
The Real Task at Hand (Pun Intended) 20
A Starting Point 20
The Critical Path
21
Working Attitude 22
Part II Architecture and General Theory
3 The Tasking Model 25
Terms 26
Tasks
26
The Kernel 27
Preemptive Multitasking 27
Resource Management 27
Task Management 28
CPU Time 30
Single Tasking 30
Multitasking
31
Cooperative Multitasking 31
Preemptive Multitasking 32
Task Scheduling 32
Who, When, and How Long?
32
Scheduling Techniques 33
Other Scheduling Factors 35
Mixed Scheduling Techniques 36 The Tasking Model I Chose 38 Interrupt Handling 39 Interprocess Communications 41 Messaging and Tasks 42 Synchronization 42 Semaphores 43 Pipes 44 Messages 44 Reentrancy Issues 51 Interrupt Latency 53 Memory Management Issues 53 Memory Management 55 Basic Terms 56 Memory Model 57 Simple Flat Memory 57 Paged Flat Memory 58 Demand-Paged Flat Memory 59 Virtual Paged Memory 59 Demand Paged Virtual Memory 61 Segmented Memory 61 Memory Management 62 Tracking Linear Memory Allocation 63 Tracking Physical Memory 69 Initialization of Memory Management 69 Memory Protection 70 An Intel-Based Memory-Management Implementation 70 A Few More Words On Segmentation 71 How MMURTL Uses Paging 72 The Hardware Interface 81 Hardware Isolation 82 The CPU 82 The Bus Structure 83 Serial I/O 84 Parallel I/O 85 Block-Oriented Devices 85 The Keyboard 86 Video 87
Qjjj] Developing Your Own 32-Bit Operating System Direct Memory Access (DMA) 89 Timers 97 The Priority Interrupt Controller Unit (PICU) 98 7 The OS Initialization 99 Getting Booted „ 100 Boot ROM 100 The Boot Sector 101 Operating System Boot Image 110 Other Boot Options Ill Basic Hardware Initialization 1 1 1 Static Tables and Structures 112 Initialization of Task Management 112 Initialization of Memory 113 Dynamic Tables and Structures 1 14 8 Programming Interfaces 115 Application Programming Interface 116 Documenting Your API 116 Procedural Interfaces 116 Systems Programming Interface 120 Internal Cooperation 120 Device Driver Interfaces 120 A Device-Driver Interface Example 121 Part III MMURTL-Specific Details 9 Application Programming 129 Terminology 130 Understanding 32-Bit Software 130 Operating-System Calling Conventions 131 Stack Usage 132 Memory Management 132 Operating-System Protection 134 Application Messaging 135 Starting a New Thread 136 Job Control Block 136 Basic Keyboard 138 Basic Video 138 Terminating Your Application 140 Replacing Your Application 140
Contents H 10 Systems Programming 141 Writing Message-Based System Services 142 Initializing Your System Service 142 A Simple System Service Example 143 The Request Block 144 Items in the Request Block 144 Asynchronous Services 146 System Service Error Handling 146 Writing Device Drivers 147 Device Driver Theory 148 Building Device Drivers 148 How Callers Reach Your Driver 149 Device Driver Setup and Installation 149 System Resources for Device Drivers 150 Detailed Device Interface Specification 152 Initializing Your Driver 157 Operating System Functions for Device Drivers 158 Standard Device Error Codes 158 1 1 The MMURTL Monitor Program 1 59 Active Job (Video and Keyboard) 160 Initial Jobs 160 Monitor Function Keys 161 Monitor Program Theory 161 Performance Monitoring 162 Monitor Source Listing 163 12 The Debugger 181 Using the Debugger 182 Entering the Debugger 182 Exiting the Debugger 182 Debugger Display 183 Debugger Function Keys 183 Debugging Your Application 185 Debugger Theory 185 13 Keyboard Service 189 Available Service 190 Read Keyboard 191 Notify On Global Keys 191 Cancel Notify On Global Keys 192 Assign Keyboard 192
m
Developing Your Own 32-Bit Operating System
Key Codes and Status 193
Alphanumeric Key Values 193
Shift-State Byte (Second Byte) 193
Lock-State Byte (Third Byte)
194
Numeric Pad Indicator 194
Key Codes :
194
Function Strip Key Codes 196
Numeric Pad Key Codes 197
Cursor, Edit, and Special Pad Key Codes 198
Your Keyboard Implementation 198
14 File System Service 201
File Specifications 202
Network File Request Routing 202
File Handles 203
File Open Modes 203
File Access Type 203
Block Mode 203
Stream Mode 204
Logical File Address 204
File System Requests 204
Procedural Interfaces 206
Device Access Through the File System 206
File System Functions in Detail 207
OpenFile
207
CloseFile 207
ReadBlock 208
WriteBlock 209
ReadBytes
209
WriteBytes
210
GetFileLFA 211
SetFileLFA 211
GetFileSize 212
SetFileSize 213
CreateFile 213
RenameFile 214
DeleteFile
214
Create Directory 215
DeleteDirectory 216
GetDirSector 216
Contents H File System Theory 217 Internal File System Structures 217 File Operations 219 15 API Specification 221 Public Calls 222 Parameters to Calls (Args) 222 Stack Conventions 224 Calls Grouped by Functionality 224 Alphabetical Call Listing 227 AddCallGate 227 AddlDTGate 228 Alarm 229 AliasMem 229 AllocDMAPage 230 AllocExch 230 AllocOSPage 230 AllocPage 231 Beep 231 Chain 231 CheckMsg 232 ClrScr 232 Compare 233 CompareNCS 233 CopyData 233 CopyDataR 233 DeAliasMem 234 DeAllocExch 234 DeAllocPage 234 Devicelnit 235 DeviceOp 235 DeviceStat 236 DMASetUp 236 EditLine 237 EndOflRQ 237 Exitjob 238 FillData 238 GetCmdLine 238 GetCMOSTime 239 GetCMOSDate 239
H Developing Your Own 32-Bit Operating System GetDMACount 240 GetExitJob 240 GetlRQVector 240 GetJobNum 241 GetNormVid 241 GetPath .’ 241 GetPhyAdd 242 GetpJCB 242 GetSysIn 242 GetSysOut 243 GetTimerTick 243 GetTSSExch 243 GetUserName 243 GetVidChar 244 GetVidOwner 244 GetXY 244 InByte 244 InDWord 245 InWord 245 InWords 245 InitDevDr 245 ISendMsg 246 KillAlarm 247 LoadNewJob 247 MasklRQ 248 MicroDelay 248 MoveRequest 248 NewTask 249 OutByte 249 OutDWord 249 OutWord 250 OutWords 250 PutVidAttrs 250 PutVidChars 250 QueryPages 251 ReadCMOS 251 ReadKbd 251 RegisterSvc 252 Request 252 Respond 254 ScrollVid 254
Contents H SendMsg 254 SetCmdLine 255 SetExitJob 255 SetlRQVector 256 SetJobName 257 SetNormVid 257 SetPath 258 SetPriority 258 SetSysIn 258 SetSysOut 259 SetUserName 259 SetVidOwner 259 SetXY 260 Sleep 260 SpawnTask 261 Tone 261 TTYOut 262 UnMasklRQ 262 UnRegisterSvc 263 WaitMsg 263 16 MMURTL Sample Software 265 Command-Line Interpreter (CLI) 266 Internal Commands 266 External Commands 270 Global “Hot Keys” 271 CLI Source Listing 271 A Simple Editor 288 Editor Screen 288 Editor Commands 288 Editor Source Listing 290 DumbTerm 314 Print 317 Print Source Listing 317 System Service Example 322 Service.C Listing 322 TestSvc.C Listing 324 Part IV The Operating System Source Code 17 Introduction to the Source Code 327 Comments in the Code 328
H Developing Your Own 32-Bit Operating System Calling Conventions 328 Organization 328 Building MMURTL 331 Using Pieces of MMURTL in Your OS 332 Using Pieces of MMURTL in Other Projects 333 18 Kernel Code 335 Naming Conventions 336 Kernel Data 336 Local Kernel Helper Functions 337 enQueueMsg 337 deQueueMsg 338 deQueueTSS 339 enQueueRdy 340 deQueueRdy 340 ChkRdyQ 341 Internal Public Helper Functions 342 RemoveRdyJob 342 GetExchOwner 344 SetExchOwner 345 SendAbort 346 Public Kernel Functions 348 Request() 348 RespondQ 351 MoveRequest() 354 SendMsgQ 355 ISendMsgQ 357 WaitMsgf) 359 CheckMsgQ 363 NewTaskQ 366 SpawnTaskQ 368 AllocExchQ 370 DeAlbcExchO 371 GetTSSExchQ 373 SetPriorityO 374 GetPriorityO 374 19 Memory Management Code 377 Memory Management Data 378 Internal Code 378 InitMemMgmt 379 FindHiPage 381 FindLoPage 382
Contents H
MarkPage 383
UnMarkPage 384
LinToPhy 385
FindRun 385
AddRun 387
AddAliasRun 388
AddUserPT 390
AddOSPT 392
Public Memory Management Calls 394
AddGDTCallGate 394
AddlDTGate 395
AllocOSPage 396
AliasMem 397
DeAliasMem 400
DeAlbcPage 401
QueryMemPages 403
GetPhyAdd 404
20 Timer Management Code 405
Choosing a Standard Interval 406
Timer Data 407
Timer Code 408
The Timer Interrupt 408
SleepQ 410
AlarmQ 411
KillAUrmQ 412
MicroDelayO 413
GetCMOSTimeO 414
GetCMOSDateQ 415
GetTimerTickQ 416
Beep_Work() 416
BeepQ
417
Tone() 418
21 Initialization Code 419
OS Global Data 420
Operating System Entry Point 422
Initialization Helpers 429
22 Job Management Code 439
Reclamation of Resources
440
Job Management Helpers 440
H Developing Your Own 32-Bit Operating System Job Management Listing 448 23 Debugger Code 471 Debugger Interrupt Procedure 472 Debugger Source Listing 473 24 Selected Device Driver Code 497 IDE Disk Device Driver 498 The RS-232 Asynchronous Device Driver 519 RS232.h 519 The Device Driver, RS-232.C 520 25 Keyboard Code 541 Keyboard Data 542 Keyboard Code 550 Keyboard ISR 550 Keyboard Translation 551 Reading the Final Buffer 557 The Keyboard Service 557 Keyboard Procedural Interface 563 Debugger Keyboard 565 Keyboard Hardware Set Up 566 Keyboard Service Initialization 567 Hardware Helpers 568 26 Video Code 571 Virtual Video Concept 572 VGA Text Video 572 The Video Code 573 Relating MMURTL's Video Code to Your Operating System 589 27 File System Code 591 How MMURTL Handles FAT 592 Physical (Absolute) Sector Numbers 592 Hidden Sectors 592 Logical Block Address (LBA) 592 MS-DOS Boot Sector 593 File System Initialization 593 File System Listing 593 Part V Additional Software 28 DASM: A 32-Bit Intel-Based Assembler 663 DASM Concepts 664
Contents
xvii
Using DASM 665
Command Line Options 667
DASM Source Files 668
Local (Nonpublic) Declarations 669
Memory Model
669
DASM Program Structure 669
DOT Commands 670
DASM Versus DOS Assemblers 672
DASM Addressing Modes 673
Storage Declarations 675
DASM Peculiar Syntax 676
386 Instruction Set 682
Executable File Format 688
Error Codes From DASM 693
Your Own Assembler? 696
29 CM32: A 32-Bit C Compiler 697
Memory Usage and Addressing 699
Important Internal Differences 699
Variable-Length Argument Lists 699
CM32 Language Specifics 700
Far Calls
700
Interrupt Functions 700
Supported C Language Features 701
Library Header File Access 702
Limitations as Compared to ANSI 702
Far External Functions 703
Type Conversion and Pointers 704
Initialization of Static Variables 704
Other Implementation-Dependent Information 706
Using CM32 707
Command Line Options 707
Library Functions 708
Library Code Examples 709
Index 719
Preface I'm taking this opportunity to give you some insight into the frame of mind I've been in for the past few years while writing my own operating system. For years I have developed software under the direction of others, using tools designed by others, eventually for others to use. The tools I used were top-quality, the operating systems I used all worked as advertised for the most part, and the jobs got done. One of the more perplexing things I ran across was the tendency to over-design something. When this tendency was added to the requirements for backward compatibility, the resulting products could become endless projects, or even nightmares, if not carefully watched. I wanted to break free of the bonds of backward compatibility. I wanted to produce a system that was powerful, yet elegant and simple in its implementation. I also wanted to make use of the more interesting designs and ideas I have run across over the years. When an avenue was presented to me that could reduce the complexity of the code and sacrifice only a small amount of power, it was my obvious path. Quite often, I refer to Albert Einstein's words on simplicity: “Everything should be as simple as possible, but not simpler.” You'll run across this idea or a reference to it in several chapters of this book. Simplicity was also the best way to provide you with a foundation of information and ideas for your own operating system, especially if you've never undertaken such a gargan- tuan project. Here's one other important note: Version 1.0 (or the first-released version) of every operating system I've ever used was not to be trusted with valuable information and was not a good basis on which to build a time-critical project. I put MMURTL, the operating system included with this book, in this same class. It's new, untested, and still under construction to a large extent. Good luck, and have fun! Richard A. Burgess
Acknowledgments This has been a long five years. I have had a tremendous amount of help along the way. In Chapter 2, “General Discussion and Background,” I mention a friend who initially started with me on this project. He is Reginald B. Carie, code-cutter extraordinaire. Very early in the game, Reggie went on to make more important things than an operating system. He and his lovely wife had a baby. Now that's work. Thanks Reg — I couldn't have gotten this far without you. Not a whole lot of what we did together is left, but your enthusiasm will remain forever. I was on active duty in the U.S. Coast Guard when I started MMURTL. Many people at the U.S. Coast Guard Information Systems Center (now called the Telecommunications and In- formation Systems Command) in Alexandria, Virginia, were backboards for my ideas and were sources of information. Tom Clark (the one-man think tank) and Dan Haynes (the wunderkind of programming) lost a few lunch hours to my babbling questions and ideas. Commander Scott Bair laughed at me when I needed it. Thanks guys. Dave McLarty of Convergent Consultants of Atlanta made some serious contributions while listening to my ideas (and paying the phone bill most of the time). Dave knows real-time multitasking and has taken it places I've never even been. Thanks Dave. I had learned a lot at the 4 1/2-year mark, and I decided to share what I had learned in the form of a book. I initially had delusions of grandeur for a commercial product, but that's too much work. (Hey! I'm retired....) When I submitted the proposal to a publisher, I immedi- ately got a positive response from Stacy Hiquet of Sams Publishing. Stacy was briefly my acquisitions editor and was promoted off my book on to bigger things. Congratulations Stacy and thanks. Chris Denny took over as acquisitions editor and has been so easy to work with that it's almost scary. Thanks Chris. Brad Jones (famous author and all-around great guy) has been one of my development editors. He has given me some serious insight into the publishing industry. I have been doing my best not to be a thorn in his side, and he hasn't once destroyed my will to live. Thanks Brad. Thanks also to Todd Bumbalough, development editor, who helped guide the project in its later stages. Thanks to the production editor, Deborah Frisby and to the copy editors, Fran Blauw, Johnna VanHoose, and Joe Williams. I must also apologize to the copy editors and proofreaders. I do know the difference between there, their, and they're, and also between its and it's. While typing at 60 WPM, listening to the Crash Test Dummies and Boingo at about 100 decibels, and gulping down coffee and sodas like nobody's business, anything can happen and usually does. Hundreds of pages in six months. . .never again. Special thanks to Nita, for some serious scanning of the mental manuscript. Special thanks to Dave Dunfield of Dunfield Development Systems; Nepean, Ontario; Canada, for his contributions to the CD-ROM and for the beginnings of the CM-32 compiler.
About the Author Richard A. Burgess is a retired U.S. Coast Guard commissioned officer (1993). His first 10 years in the Coast Guard were in electronics technology and maintenance. His last 10 years of service were all computer-science related; he was the primary software design engineer for the Coast Guard's Communication Station Control System, and he headed the Coast Guard's Software Development Standards Team. He taught computer electronics technology at the Computer Learning Center ofWashington, D.C., for two years, and he consulted in and around Washington under the guise of The D-Group. He is now the Chief Information Officer with Optipat, Inc. of Arlington, Virginia, a rapidly growing, high-technology, patent-trademark- and-copyright imaging firm. He would rather be on a beach somewhere in Florida and claims he is working to that end.