25KB
Developing Your Own 32-Bit Operating System — Chapter-01: Overview
- Source: /library/Developing Your Own 32-Bit Operating System/source-file.pdf
- PDF pages: 23–34
- Pages without text: 23, 24, 25, 34
- Note: Includes the Part I divider plate and its blank verso before the chapter.
Introduction
Overview
ra Part I 1 1 Introduction Computer programmers and software engineers work with computer operating systems every day. They use them, they work with them, they even work “around” them if that's what it takes to get their jobs done. If you're an experienced programmer, I'm willing to bet you've pondered changes you would make to the operating systems you use, or you've even thought about what you would build and design into an operating system if you were to write your own. You don't have to be Albert Einstein to figure out that many years of labor can go into writing a computer operating system. You also don't have to be Einstein to write one, although I'm sure it would have increased my productivity a little while writing my own. Unfortunately, I have only Albert's absent-mindedness, not his IQ. Late in 1989, 1 undertook the task of writing a computer operating system. What I discovered was that most of the books about operating-system design were very heavy on general theory and existing designs, but very light on specifics, code, and advice on the seemingly endless tasks facing someone who really wants to sit down and create one. I'm not knocking the authors of these books; I've learned a lot from all that I've read. It just seemed there should be a more “down to earth” book that documents what you need to know to get the job done. Writing my own operating system has turned into a very personal thing for me. I have been involved in many large software projects where the deliverables were conceived by others and I was to turn them into working code and documentation. In the case of my own operating system, I set the ground rules. I decided what stayed and what went. I determined the specifi- cations for the final product. When I started out, I thought the “specs” would be the easy part. I would just make a list, do some research on each of the desirables, and begin coding. Not so fast there, bucko! I may not have to tell you this if you've done some serious “code cutting” yourself, but the simplest items on the wish list for a piece of software can add years to the critical path for re- search and development, especially if you have a programming team comprising just one per- son. My first list of desirables looked like a kid's letter to Santa Claus. It turned out that I hadn't been a good-enough boy to get all those things (and the average life expectancy of a male hu- man being wouldn't allow it either). Realizing this, I whittled away to find a basic set of ingre- dients that constitute the real make-up of the “guts” of operating systems. The most obvious of these, as viewed by the outside programmer, are the tasking model, the memory model, and the programming interface. As coding began, certain not-so-obvious factors came into play, such as the OS-to-hardware interface, portability issues, size, and speed. One of the most obvious concerns for commercial operating-system developers is compatibil- ity. If you write something that no one can use, it's unlikely they'll use it. Each of the versions of MS-DOS that were produced had to be able to run software written for previous versions. When OS/2 was first developed, the “DOS-BOX” was a bad joke. It didn't provide the needed
Chapter 1 M Overview compatibility to run the programs that business depended on. In the early days, UNIX had a somewhat more luxurious life, because source-code compatibility was the main objective; you were pretty much expected to recompile something if you wanted it to run on your particular flavor of the system. You may have noticed that this has changed substantially. I did not intend to let compatibility issues drive my design decisions. I was not out to compete with the likes of Microsoft, IBM, or Sun. You, however, may want compatibility if you intend to write your own operating system. It's up to you. If you desire compatibility, the existing interfaces are all well documented, so you don't need to design your own. But I wanted a small API. That was part of my wish list. Along my somewhat twisted journey of operating-system design and implementation, I have documented what it takes to really write one, and I've included the “unfinished” product for you to use with very few limitations or restrictions. Even though I refer to it as unfinished, it's a complete system. I believe that when a piece of software is “finished,” it's probably time to replace it because of obsolescence or lack of maintenance. m What This Book Is About In this book I discuss the major topics of designing your own OS: the tasking model, the memory model, programming interfaces, hardware interface, portability issues, size, and speed. I back up these discussions by showing you what I decided on (for my own system), and then finally I discuss my code in great detail. Your wish list may not be exactly like mine. In fact, I'm sure it won't be. For instance, you'll notice the ever-so-popular term “object oriented” conspicu- ously missing from any of my discussions. This was completely intentional. This term clouds far too many real issues these days. Don't get me wrong — I'm an avid C++ and OOP user. But that's not what this book is about, so I've removed the mystique. Once again, your goals will be your own. Throughout this book, I use the operating system I developed (MMURTL) to help explain some of the topics I discuss. You can use pieces of it, ideas from it, or even all of it if you wish to write your own. I have included the complete source code to my 32-bit, message-based, multitasking, real-time, operating system (MMURTL), which is designed for the Intel 386/486/Pentium processors on the PC Industry Standard Architecture (ISA) platforms. The source code, written in 32-bit, Intel-based assembly language and C, along with all the tools necessary to build, modify, and use the operating system are included on the accompany- ing CD-ROM. The hardware requirements section below tells you what you need to run MMURTL and to use the tools I have developed. It's not really required if you decide on a different platform and simply use this book as a reference.
m Introduction One thing I didn't put on my wish list was documentation. I always thought that many sys- tems lacked adequate documentation (and most of it was poorly written). I wanted to know more about how the system worked internally, which I felt would help me write better appli- cation software for it. For this reason, I have completely documented every facet of my com- puter operating system and included it here. You may or may not want to do this for your system, but it was a “must” for me. I think you will find that this will help you if you intend to write your own. Well-commented code examples are worth ten times their weight in generic theory or simple text-based pseudo-algorithms. The “Architecture and General Theory” part of this book, along with the sections that are specific to the MMURTL operating system, will provide the necessary information for you to use this system as is, to redesign it to meet your requirements, or to write your own. If you don't like the way something in my system is designed or implemented, change it to suit your needs. The only legal restriction placed on your use of the source code is that you may not sell it or give it away. Who Should Use This Book This book is for you if you are a professional programmer or a serious hobbyist, and you have an interest in any of the following topics: H Writing a 32-bit microcomputer operating system. ■ 80386/486/Pentium 32-bit assembly language. (An assembler is included.) H The C programming language. (A compiler for C is included.) ■ Intel 80386/486/Pentium paged memory operation and management using the processor paging hardware. ■ Intel 80386/486/Pentium 32-bit hardware and software task management using the processor-hardware task-management facilities. ■ Embedded or dedicated systems using the 32-bit PC ISA architecture. ■ Real-time, message-based operating systems. ■ PC Industry Standard Architecture hardware management — including: DMA controllers, hardware timers, priority interrupt controller units, serial and parallel ports, and hard or floppy disk controllers. ■ File systems. (A DOS File Allocation Table-compatible file system in C is included.) You may note that throughout this book I refer to the 386, 486, and Pentium processors as if they were all the same. If you know a fair amount about the Intel 32-bit processors, you know that a quantum leap was made between the 286 and 386 processors. This was the 16- to 32-bit jump.
Chapter 1 ■ Overview The 486 and Pentium series have maintained compatibility with the 386 instruction set. Even though these newer processors have added speed, a few more instructions, and some serious “turbo” modifications, they are effectively super-386 processors. In my opinion, from a programmer's standpoint the next quantum leap really hasn't been made, even though the Pentium provides 64-bit data access. My Basic Design Goals (Yours May Vary) A review of my ambitions when I was designing the MMURTL operating system may help you understand what I faced, my end results, and what you may face. If you're going to write your own system, I recommend that you avoid an overly extensive wish list. You may start with my list and modify it, or you may start one from scratch. From my own wish list, you may be able to get a good handle on your own list. Chapter 2, “General Discussion and Background,” may also help by explaining in some detail how my list came to be. Here was my wish list: True multitasking: Not just task switching between programs, or even sharing the processor between multiple programs loaded in memory, but real multithreading — the ability for a single program to create several threads of execution that could communi- cate and synchronize with each other while carrying out individual tasks. I'll discuss some of your options, and also what I decided on, in Chapter 3, “The Tasking Model,” and Chapter 4, “Interprocess Communications.” Real-time operation: The ability to react to outside events in real time. This design goal mandated that MMURTL be message based. The messaging system would be the heart of the kernel. Synchronization of the messages would be the basis for effective CPU utilization (the tasking model). Real-time operation may not even be on your list of requirements, but I think you'll see that the push in the industry is towards real- time systems, and for a very good reason. We, the humans, can be thought of as “outside events.” We want the system to react to us. This is also discussed in Chapters 3 and 4. Client/server design: The ability to share services with multiple client applications, whether on the same machine or across a network. A message-based operating system seemed the best way to do this. I added the Request and Respond message types to the more common Send and Wait found on most message-based systems. Message agents could be added to any platform. If you don't have this requirement, the Request and Respond primitives could be removed, but I don't recommend it. I discuss this in Chapter 4. A common, affordable hardware platform with minimal hardware requirements: The most common 32-bit platform in the world is the PC ISA platform. How many do you think are out there? Millions! How much does a 386SX cost these days? A m
ra Part I ■ Introduction little more than dirt (or maybe a little less). Of course, you may have a different platform in mind. The hardware interface may radically change, based on your choice. I discuss this in Chapter 6, “The Hardware Interface.” Flat 32-bit virtual memory model: Those of you that program on the Intel proces- sors know all about segmentation and the headaches it can cause (not to mention Expanded, Extended, LIM, UMBs, HIMEM, LOMEM, YOURMOTHERSMEM, and who knows what other memory schemes and definitions out there). Computers using the Intel-compatible 32-bit processors are cheap, and most of them are grossly underused. MMURTL would use the memory paging capabilities of 386/486 proces- sors to provide an easy 32-bit flat address space for each application running on the system. Chapter 5, “Memory Management,” covers memory management but not all of the options. Many detailed books have been written on schemes for memory management, and I stuck with a very simple paged model that made use of the Intel hardware. You could go overboard here and throw in the kitchen sink, if desired. Easy programming: The easiest way to interface to an operating system from a procedural language, such as C or Pascal, is with a procedural interface. A procedural interface directly into the operating system (with no intermediate library) is the easiest there is. Simple descriptive names of procedures and structures are used throughout the system. I also wanted to maintain a small, uncluttered Applications Programming Interface (API) specification, adding only the necessary calls to get the job done — power without bloat, the right mix of capability and simplicity. I would shoot for less than 200 basic public functions. I discuss some of your options in Chapter 8, “Programming Interfaces.” Protection from other programs: But not the programmer. I wanted an OS that would prevent another program from taking the system down, yet allow us the power to do what we wanted, if we knew how. This could be a function of the hardware as well as the software, and the techniques may change based on your platform and processor of choice. Use of the CPU instruction set as designed: Many languages and operating systems ported between processors tend to ignore many of the finer capabilities of the target processor. I didn't want to do this. I use the stack, calling conventions, hardware paging, and task management that is native to the Intel 32-bit x86 series of processors. Even with this goal in mind, I have attempted to isolate the API from the hardware as much as possible (for future source-code portability). You may want your operating system to run on another platform or even another processor. If it is for another processor, many of the items that I use and discuss in this book may not apply. I take a generic look at these issues in Chapters 6 and 8. Simplicity: Keep the system as simple as possible, yet powerful enough to get the job done. I also wanted to reduce the “jargon” level and minimize the number of terse, archaic names and labels so often used in the computer industry.
Chapter 1 M Overview A Brief Description of MMURTL From the wish list, I put together the MMURTL operating system. I will describe it to you here so that you can see what I ended up with, based on the list. This may help you with your own list. MMURTL (pronounced like the girl's name Myrtle) is a 32-bit, message-based, multitasking, real-time operating system designed around the Intel 80386 and 80486 processors on the PC Industry Standard Architecture (ISA) platforms. The name is an acronym for Message-based, Multitasking, Real-Time kerneL. If you don't like my acronym, make up your own. But I warn you: Acronyms are in short supply in the computer industry. MMURTL is designed to run on most 32-bit ISA PCs in existence. Yes, this means it will even run on an 80386SX with one megabyte of RAM (although I recommend 2MB). Then again, it runs rather well on a Pentium with 24MB of RAM, too. If you intend to run MMURTL or to use any of the tools I included, see the “Hardware Requirements” section later in this chap- ter. MMURTL is not designed to be compatible with today's popular operating systems, nor is it intended to directly replace them. It is radically different, in that simplicity was my most im- portant goal. If you don't want to start from scratch or you have an embedded application for MMURTL, the tools, the code, and the information you need are all contained here so you can make MMURTL into exactly what you want. Sections of the source code will, no doubt, be of value in your other programming projects as well. ra Uses for MMURTL If you are not interested in writing your own operating system (it can be a serious time-sink) and you want to use MMURTL as is (or with minor modifications), here's what MMURTL can be used for: ■ MMURTL can be considered a general-purpose operating system, and dedicated vertical applications are an ideal use. ■ It is an ideal learning and reference tool for programmers working in 32-bit environ- ments on the Intel 32-bit, x86/Pentium processors, even if they aren't on ISA plat- forms or using message-based operating systems. ■ MMURTL can be the foundation for a powerful, dedicated communications system, or can be used as a dedicated interface between existing systems. The real-time nature of MMURTL makes it ideal for handling large numbers of interrupts and communi- cations tasks very efficiently. ■ Dedicated, complex equipment control is not beyond MMURTL's capabilities.
H Part I ■ Introduction Vertical applications of any kind can use MMURTL where a character-based color or monochrome VGA text interface is suitable. MMURTL would also be suitable for ROM-based embedded systems (with a few minor changes). One of my goals was to keep the basic system under 192K (a maxi- mum of 128K of code and 64K of data), excluding dynamic allocation of system structures after loading. The OS could be moved from ROM to RAM, and the initialization entry point could be jumped to. If you're not worried about memory consumption, expand it to your heart's desire. In the educational field, MMURTL can be used to teach multitasking theory. The heavily commented source code and in-depth theory covered in this book make it ideal for a learning/teaching tool or for general reference. And of course, MMURTL can be the basis or a good starting point for your very own microcomputer operating system. Similarities to Other Operating Systems MMURTL is not really like any other OS that I can think of. Its closest relative is CTOS (Unisys), but that is only a distant cousin, and only so because of the similar kernel-messaging schemes. Your creation also will take on a personality of its own, I'm sure. The flat memory-model implementation is a little like that of OS/2 2.x (IBM), but not enough alike that you could ever confuse the two. Some of the public and internal calls may resemble UNIX a little (or even MS-DOS), but still, it is not at all the same. The file system included with MMURTL uses the MS-DOS disk structures, but only out of convenience. This certainly wasn't because I liked the design or filename limitations. There are some 100 million+ disks out there with MS-DOS FAT formats. This makes it easy to use MMURTL, and more importantly, it eases the development burden. No reformatting or par- titioning your disks — you simply run the MMURTL OS loader from DOS, and the system boots. Rebooting back to DOS is a few keystrokes away — if you also want to run DOS, that is. MMURTL has its own loader to boot from a disk, which eliminates the need for MS-DOS altogether. If you don't want to write a completely new operating system, some serious fun can be had by writing a much better file system than the one I have included. Hardware Requirements This section describes the hardware required to run MMURTL (as included) and to work with the code and tools I have provided. If you intend to build on a platform other than that de- scribed here, you can ignore this section.
Chapter 1 The hardware (computer motherboard and bus design) must be PC ISA (Industry Standard Architecture) or EISA. Other 16 or 32-bit bus designs may also work, but minor changes to specialized interface hardware may be required. The processor must be an Intel 80386, 80486, Pentium, or one of the many clones in exist- ence that execute the full 80386 instruction set. These include Cyrix, AMD, IBM, and other clone processors. VGA video text (monochrome or color) is required. MMURTL accesses the VGA text memory in color text mode (which is the default mode set up in the boot ROM if you have a VGA adapter). One megabyte of RAM is required; 2MB (or more) are recommended. MMURTL will handle up to 64MB of RAM. MMURTL itself uses about 300K after complete initialization. The floppy disk controller must be compatible with the original IBM AT controller specifica- tion. (Most are.) Both 5.25 inch and 3.5 inch are supported. The hard-disk controller must be MFM or IDE (Integrated Drive Electronics). IDE is pre- ferred. Some RLL controllers will not work properly with the current hard-disk device driver. A standard 101-key AT keyboard and controller (or equivalent) is required. The A20 address-gate line must be controllable, as documented in the IBM-PC AT hardware manual, via the keyboard serial controller. Most are, but a few are not. If yours isn't, you can change the code as needed, based on your system manufacturer's documentation (if you can get your hands on it). 8250, 16450, or 16550 serial controllers are required for the serial ports (if used). The parallel port should follow the standard IBM AT parallel port specification. Most I have found exceed this specification and are fully compatible. MMURTL does not use the BIOS code on these machines. Full 32-bit device drivers control all hardware. Hence, BIOS types and versions do not matter. m