Quick look under the hood
Original article written by Kudzu
Last year, I wrote my second Cosmos article which introduced readers to Cosmos (C# Open Source Managed Operating System). The article however was a high level article which only introduced Cosmos and how to build a custom kernel. This article will demonstrate a brief description of how Cosmos does its magic, how it works under the hood.
Cosmos is an operating system development kit which uses Visual Studio as its development environment. Despite C# in the name, any .NET based language can be used including VB.NET, Fortran, Delphi Prism, IronPython, F# and more. Cosmos itself and the kernel routines are primarily written in C#, and thus the Cosmos name. Besides that, NOSMOS (.NET Open Source Managed Operating System) sounds stupid.
Cosmos is not an operating system in the traditional sense, but instead it is an "Operating System Kit", or as I like to say "Operating System Legos". Cosmos lets you create operating systems just as Visual Studio and C# normally let you create applications. Most users can write and boot their own operating system in just a few minutes, all using Visual Studio. Cosmos supports integrated project types in Visual Studio, and an integrated debugger, breakpoints, watches, and more. You can debug your operating system the same way that you debug a normal C# or VB.NET application.
While Cosmos can boot on real hardware via USB, Ethernet, DVD, or even a real hard disk, most users use VMWare because it's faster during development. Given the following source code:
When you press F5, your code will be turned into a full operating system and boot in VMWare:
That might seem like magic to many. There certainly is a lot of code that had to be written to make this work as seamlessly as seen here. Here are the basic steps of what occurred.
Visual Studio compiles .NET languages such as C# to Intermediate Language (IL) using a Cosmos utility named IL2CPU. IL is normally handled on Windows by the JIT (Just in Time) compiler when the executable is run. The executable produced by Visual Studio actually is not native code, but IL byte code. The only native code that normally exists in a .NET executable is a small bit of bootstrap code which allows Windows to run it like a normal executable. This bootstrap code invokes the .NET JIT, which then reads and compiles the IL inside the executable. IL can be seen using many tools such as ILDasm, or reflector.
.method family hidebysig virtual instance void Run() cil managed
{
.maxstack 1
.locals init (
[0] int32 i,
[1] class BreakpointsKernel.Test xTest)
L_0000: nop
L_0001: ldc.i4.0
L_0002: stloc.0
L_0003: ret
} This is then passed to an assembler called NASM which turns it into binary format.
System_Void__BreakpointsKernel_BreakpointsOS__ctor__:
push dword EBP
mov dword EBP, ESP
System_Void__BreakpointsKernel_BreakpointsOS__ctor____DOT__00000000:
call DebugStub_TracerEntry
; [Cosmos.IL2CPU.X86.IL.Ldarg]
; Ldarg
; Arg idx = 0
; Arg type = BreakpointsKernel.BreakpointsOS
; Arg size = 4
push dword [EBP + 8]
; Stack contains 1 items: (4)
System_Void__BreakpointsKernel_BreakpointsOS__ctor____DOT__00000001:
; [Cosmos.IL2CPU.X86.IL.Call]
call System_Void__Cosmos_System_Kernel__ctor__
test dword ECX, 0x2
je near System_Void__BreakpointsKernel_BreakpointsOS__ctor____DOT__00000006
jne near System_Void__BreakpointsKernel_BreakpointsOS__ctor____DOT__END__OF__METHOD_EXCEPTION
; Stack contains 0 items: () A variety of tools are then used to link the binary output and convert it into ISO format.
Now we have a binary file, but we need to boot it. To do this, Cosmos uses a bootloader. All operating systems use a bootloader including Windows and Linux. The BIOS of a computer looks for bootstrap code, but this bootstrap code must be very small. This bootstrap code then loads a slightly larger bootloader, which then initializes memory and loads a larger operating system. After transfer to the operating system, the bootstrap and bootloader are removed from memory. To do this, Cosmos uses a bootloader called Syslinux. Despite the name, it is NOT Linux, and Cosmos is not built on Linux. Syslinux has roots involving older Linux bootloaders, and thus the name. Syslinux is only used to get the BIOS to boot Cosmos code, and once Cosmos code is up and running, neither the BIOS nor Syslinux are used.
Cosmos supports normal source debugging allowing tracing, breakpoints, and even watches. Cosmos also supports an experimental assembly level debugger, as well as GDB. To support debugging in Visual Studio, Cosmos has a small handwritten assembly method called the DebugStub. The DebugStub is called repeatedly during Cosmos execution and automatically inserted by the Cosmos compiler. The DebugStub uses a serial port to communicate with the DebugClient which is part of the Cosmos Visual Studio debug package. On VMWare, the serial port is mapped to a pipe, and the DebugClient communicates with the pipe. When debugging on physical hardware, a serial port is used on both sides. In the future, debug over ethernet will be supported as well.
Many of the Framework Class Libraries use icalls, or pinvokes. For example, when .NET needs to draw to the screen, it has no code for this. It uses pinvoke to directly call the Windows API. icalls are similar, but instead map to internal functions in the .NET runtime. Because Cosmos is running on real hardware, without the .NET runtime and without the Windows API, such code has to be implemented. Cosmos uses plugs to implement these. A plug is a section of code which marks a class and/or method that it will replace during the IL2CPU stage. Plugs can be written in C# (or any .NET language), assembly, or X#. Plugs can also be used to interface C# code to assembly code. Cosmos works to minimize the need to write assembly code, however when interacting directly with hardware in the kernel, assembly must be used. Plugs are used to hide these deep details away from the developer by creating C# classes which actually run assembly. The IOPort class is an example of this.
public AtaPio(Core.IOGroup.ATA aIO,
Ata.ControllerIdEnum aControllerId, Ata.BusPositionEnum aBusPosition) {
IO = aIO;
mControllerID = aControllerId;
mBusPosition = aBusPosition;
// Disable IRQs, we use polling currently
IO.Control.Byte = 0x02;
mDriveType = DiscoverDrive();
if (mDriveType != SpecLevel.Null) {
InitDrive();
}
} This is a small excerpt from the PATA (hard disk access) class. It is able to communicate directly to the CPU IO bus using C# code. Although parts of the IOPort class (IO variable in this code) are written in C#, parts of it are plugged with X86 assembly code.
Cosmos is written in C#. Cosmos developers including kernel developers stay in C#. However our IL2CPU libraries have to deal with assembly, and the few of us who deal with the compiler code of course muck about with X86. Previously we had our own class based "inline compiler". It worked reasonably well, but we always wanted something more. We affectionately call our solution X# (from X86). The idea is to keep all of our source in one place, and keep things very type safe. Our previous class based inline compiler did that. Typical code looked something like this:
DebugStub_Const_VidBase equ 0xB8000
DebugStub_Cls:
Mov ESI, DebugStub_Const_VidBase
DebugStub_Cls_Block1_Begin:
Cmp ESI, 0xB8FA0
JAE DebugStub_Cls_Block1_End
Mov dword [ESI + 0], 0x00
Inc ESI
Mov dword [ESI + 0], 0x0A
Inc ESI
jmp DebugStub_Cls_Block1_Begin
DebugStub_Cls_Block1_End:
DebugStub_Cls_Exit:
mov dword [static_field__Cosmos_Core_INTs_mLastKnownAddress], DebugStub_Cls_Exit
Ret It's type safe, and not too bad to read. However compare that now with X#:
const VidBase = $B8000
function Cls {
ESI = #VidBase
// End of Video Area
// VidBase + 25 * 80 * 2 = B8FA0
while ESI < $B8FA0 {
// Text
ESI[0] = $00
ESI++
// Colour
ESI[0] = $0A
ESI++
}
}It's actually all C# and compilable. When it executes, the execution of these statements causes it to output X86 code which can then be built using an assembler. How is it done? Well, very similarly to how I did SQL in C# before LINQ came along.