This may seem like a small accomplishment to many, but I now have PwnOS boot code that takes one CPU core from 16-bit mode to 32-bit mode to 64-bit mode using code developed and assembled/linked with Inventor IDE.  Until now, so far as I’ve been able to find, no linker has supported 16-bit, 32-bit, and 64-bit code smoothly linked together in one output file.  If that’s the case, Inventor IDE is the first linker to support this, and it makes developing an operating system bootloader an absolute breeze compared with the alternative, which is to:

1. The 16-bit Master Boot Record (MBR) or Boot Sector (BS) loads the rest of the 16-bit bootloader.
2. The 16-bit bootloader finds and reads in a file on the drive containing the 32-bit bootloader with an entry point at a hard-coded address.
3. Switch to 32-bit mode and jump to the hard-coded address.
4. The 32-bit bootloader finds and reads in a file on the drive containing the 64-bit bootloader with an entry point at a hard-coded address.
5. Switch to 64-bit mode and jump to the hard-coded address.
6. The 64-bit bootloader actually starts loading the OS after starting the other CPU cores.

Now it’s as simple as:

1. The 16-bit Master Boot Record (MBR) or Boot Sector (BS) load the entire rest of the bootloader.
2. Switch to 32-bit mode and jump to the 32-bit starting function (address determined automatically by the linker)
3. Switch to 64-bit mode and jump to the 64-bit starting function (ditto)
4. The 64-bit function actually starts loading the OS after starting the other CPU cores.

Eliminating the need to find and read in files from a filesystem reduces the size and complexity of the code many-fold.  Plus, now the entire OS should easily fit in the 30KB of disk space I’ve got for the bootloader, so that first step will load the entire OS from the drive.  I would show a screenshot or photo, but it doesn’t write to the harddrive yet (it’s probably for the best), and I don’t have a working camera anymore.  Nonetheless, here’s roughly (just in a different font) what was output:

Output showing the PwnOS bootloader switching from 16-bit mode to 32-bit mode to 64-bit mode

There have been similar linkers for just 16-bit and 32-bit code, most notably JLOC.  I would link to the original site, but the site vanished many years ago, which explains why there’s no support for 64-bit code.  There’s also the standard linker for linux, LD.  Although it supports 64-bit code and has extensive scripting options, I can’t seem to find any evidence that it can link code of different modes together, not to mention that I’d still need to have a custom assembler to output data to some format it supports that keeps the necessary information.  Also, if LD supported this, linux could get to 64-bit mode in less than 1KB of code, and that certainly isn’t the case right now.

Hopefully other people will be up for trying this out with Inventor IDE Alpha 5b once it’s out (just the settings dialog box left).  Anyway, I’ve written some code for more extensive debugging that I’m anxious to test out, and I still need to get the other CPU cores up and running, but that should be much easier now. 

Yesterday, several friends and I, especially Calvin, set up 4 computers I bought 68 days ago and received 4 days ago (moral of the story, DO NOT BUY FROM CANADACOMPUTERS!)  Here are the specs:

• 4 x Intel Core 2 Quad Q8200 2.33GHz
• 4 x 4GB of 667MHz RAM
• 4 x 160GB SATA harddrive
• 5-port Gigabit Ethernet Switch
• $2000 after taxes and shipping

At the recommendation of my friend Ben, I’m naming the small cluster “Ensemble”, a reference to Quarantine by Greg Egan.  As a further reference to Quarantine, and to D-Wave, I’ve named the 4 nodes “identity”, “sigma-x”, “sigma-y”, “sigma-z”.

With these, and with Inventor IDE Alpha 5, I can finally start building PwnOS after months of delays.  It will be awesome to have 16 CPU cores and 16 GB of RAM at my disposal. 

That’s an exaggeration, of course, but what I’m really trying to say is that working alone on a project for an extended period of time really sucks the life out of me.  With my busy schedule this term, I’m finding it really hard to find time to work on Inventor IDE.  Next term I’ll be working on it as part of a directed study, and I do have a hard deadline of the end of 2008 for getting built-in assembling fully functional, but progress has been slow.

I got asked about teamwork in an interview with Microsoft’s Photosynth group, so that got me really thinking about team dynamics.  At the very least, on any sizable project, having someone else to bounce ideas off of is a must.  Since for assignments 1 and 2, the professor of the course I’m helping with hasn’t posted my alternate assignments, until I get built-in assembling working and start posting about it widely, I won’t even have many users off of which  to bounce ideas about Inventor IDE.

Beyond that, assuming there’s an actual team, there must be some sort of division of labour such that what needs to get done gets done without losing the members.  I’ve started organizing the team for the next big Code Cortex project, codenamed PwnOS, and it’s interesting trying to figure out how to split up the work and to keep people interested.  Conveniently, the team came up with a great idea: each person investigates a device protocol and goes through developing support for that device.  It works perfectly, since getting a device to work is one of the really satisfying parts of OS development, and it only depends on having as much of a bootloader as I’ve already got.  I can then focus on getting some of the other parts of the OS up and running without everyone waiting for me.

Other good news includes that I’ve found someone quite eager to get a third (unnamed) project off the ground, and I may have found another person to help with Inventor IDE next term.  It’ll take lots of work to get things going, but I’m optimistic that we can find a way to make things work, together.

I’ve now completed the first version of the PwnOS design document.  Development can move forward now that the design has been fairly well cemented.

The design in the document does not reflect the current state of the code.  Significant differences include that the design is to be for 64-bit code, not 32-bit, and page memory will be managed in 2MB pages by disabling the lowest level of page tables instead of 64KB simulated pages made by handling real pages in groups of 16.  Also, there will be a large, read-only area devoted to libraries common to all processes.

Also included in the document are a brief analysis of high-performance computing and a comparison with other operating systems.

After being informed by a third party that RIM probably prefers to keep interview questions private, (okay, so it was actually much less polite than that, but I won’t hold that against anyone), I’ve decided to post about two questions other than those in my interview, but with slightly similar components to how I answered the otherwise-unrelated questions in my RIM interviews. I haven’t checked whether these answers would compile, since I wouldn’t be able to in an interview.

Question: Write a C function to select k random elements in a random order from a set of n pointers in O(k) time. The original set can be reordered.

void** selectRandom(void** pSet,DWORD n,DWORD k) {
    // Create the array to be returned, null-terminated
    void** pSelected = (void**)malloc((k+1)*sizeof(void*));
    pSelected[k] = NULL;
    DWORD selectedIndex = 0;
    // Select elements until no more needed
    while (k--) {
        DWORD index = (DWORD)((rand()*((QWORD)n))>>32);	// This is more uniform and faster than using modulus
        void*const element = pSet[index];		// Select the edge from the list
        pSet[index] = pSet[--n];			// Remove it by replacing with the last element
        pSelected[selectedIndex++] = element;
    }
    return pSelected;
}

Question: Write a C function to sort an array of floats (i.e. 4-byte floating-point numbers in the appropriate IEEE format) in O(n) time, where n is the number of floats in the array.

The answer needs a bit of explanation first. Positive floating-point numbers when interpreted as integers are in the same numerical order in both cases. e.g. 2.0 > 1.0 and the integer interpretation of 2.0 is greater than the integer interpretation of 1.0. Negative floating-point numbers when interpreted as integers are in the opposite numerical order. e.g. -2.0 < -1.0, but the integer interpretation of -2.0 is less than the integer interpretation of -1.0. Mixed sets of negative and positive numbers results in other special cases. What would be ideal is a constant-time bijective mapping from a floating-point number to an unsigned integer that will maintain perfect ordering through positive and negative numbers. It can be shown that there is exactly one such mapping, and it is simple enough to be constant time.

The mapping is: take the bitwise not of negative numbers, and flip the sign bit of positive numbers. After applying this to all numbers in the array, one can use 4-pass, base-256 radix sort on the numbers as unsigned 32-bit integers, then use the reverse mapping back to floating-point numbers. This works with 6 passes through the array. There is a way to do the full thing with 4 passes through the array, but it’s unsure which would be faster.

Context-sensitive auto-complete now works for some contexts.  Contexts that it doesn’t handle yet will just give you an apology message, since I’m leaving it at that for the Alpha.  It’s also not as flashy as I’m hoping to have it for the Beta, but it’s plenty sufficient for the Alpha.

I’m going to have to take a small break from PwnIDE to work on PwnOS if I’m planning to use PwnOS for my Operating Systems class project, ’cause PwnOS needs a lot more work to get anything up and running.  However, the first draft of the paper on PwnIDE is due at the end of October, so it’ll still be in my mind.  The remaining functionality for PwnIDE Alpha is mostly fixing loose ends, whereas the remaining functionality for PwnOS is a lot.  I’ll make it all work somehow.

It may seem a bit premature to be writing a user interface for PwnOS before it’s fully functional, but thanks to a simple wrapper library I’ve written, it’s now fairly simple to make and test PwnOS interface applications under Windows.

The rudimentary interface application I’ve written (mostly an interface library) is completely in C++, in case anyone was wondering whether you’d be able to make applications for PwnOS in something other than assembly language. I’m considering making a more formal interface library out of it so that:

1. It’ll be worth putting in the (or a new) repository, and
2. Other interfaces and applications will be able to use the same interface library

Here’s a screenshot of the current test interface:

Everything in this is simulated, including the cursor. When running in Windows, the program “thinks” that it’s writing to video memory, when really there’s a full-screen window reading that chunk of memory and writing it to the window 20 times a second. One of the funny things with that is that it’d be quite simple to instead take that chunk of memory and apply it to something like a cube, for Beryl fans.

The button will show 4 different images, one each for up, hover, down, and disabled. The cursor image is decoded from a GIF file, though there’s still a bug decoding GIF files that are much larger, so the background and button images are bitmaps.

I’m more focused on PwnIDE than on PwnOS at the moment, but I’ll try to figure out what to do with this interface library in case anyone’s interested in writing interface or other applications for PwnOS. The classes are quite reminiscent of Java’s interface API, so it shouldn’t be to hard to use and expand.

This blog now updates through wordpress.com, so people can actually post comments.   It also means that it’ll be easier to update, and that I can separate entries into categories for the front, PwnOS, and PwnIDE pages.  Now I’ll be able to give updates on each of the projects more often, and people can subscribe to one or more of them without subscribing to all of them.

There’s still a bug that I might not be able to fix any time soon, which is that when you post a comment, you’ll be redirected to the blog on wordpress.com instead of back here.  The issue is that the SCS web server doesn’t have the HTTP extension to PHP installed, so I can’t make POST requests to wordpress.com.  Hopefully that’s not too much of an issue.  The only other thing is that the dates on older blog entries were lost, so I put them in the message text.

Overall, this should be really handy.