Microkernel/Project planning

Research

This is the text of the first meeting which was on the Puredarwin IRC (freenode). It is also an example of wiki deconstruction, a converstion about the correct pathway for open and free OS development. This will be pulled apart and turned into a flow chart of development options: XNU (or OSX Darwin microk seems like the choice of choice, but has barriers. GNU Mach also looks good. The developed base microkernel (MK) will be the development platform for purer MKs especially L4, which boasts a billion copies out there. Others that may be applicable but have languished are LibOS and ExOS. Beyond that are paradigms to differentiate the project from the convoluted mess we deal with now, such as data centricity over disconnected applications.

Puredarwin.org IRC

<johnbessa> <machdev> I was ty'n whoever can get xnu to boot on x86, so I can get started w/ gcc

<johnbessa> dumb question, is I/O kit compile in gcc c++?

<johnbessa> sorry, does I/O kit..

<vmlemon_> I believe that G++ is used to compile XNU in its entirety.

<johnbessa> good to know, thanks

<johnbessa> your honest opinion, what is the easiest way to get xnu running as an independent desktop?

<vmlemon_> No problem. It's been a long time since I've attempted to build it, though.

<johnbessa> I am attempting a microkernel project, and it would be good to have something to show

<vmlemon_> Bootstrapping it from a Mac OS X system using the PD scripts is probably the easiest method, at the moment.

<johnbessa> what would be the earliest model Mac to do this? (ebay)

<vmlemon_> Adapting one of the ISOs or VMDKs might work, though - but you're sure to hit limitations.

<vmlemon_> It might be possible to cross-compile from a PowerPC-based Mac, but it's a case of trial-and-error.

<johnbessa> actually x86 might be better

<vmlemon_> An early MacBook or x86-based Mac Mini running 10.5.x should do the trick, though.

<johnbessa> ty

<vmlemon_> If it works under 10.4, then you might be able to obtain something even cheaper.

<johnbessa> I was fine with x and Toms wm on less than 300mz and 1/4 gig mem

<johnbessa> gcc too, I suppose

<johnbessa> compiles can always swap while I visit my "reality" of psychopathology

<johnbessa> 3 weeks to go

<vmlemon_> If you're willing to strip out the optimisations for S/SSE/2/3 and workaround the CPU checks, you could probably run it on a fairly modest system.

<vmlemon_> *work around

<johnbessa> its a project-course to introduce microkernel, and I still believe in supporting the 3rd world!

<vmlemon_> (The Voodoo fork of XNU may hold some clues with regards to that). :)

<johnbessa> what about these guys in Malaysia with the pirate logos?

<vmlemon_> If all else fails, there's always Minix...

<johnbessa> does minix swap yet?

<vmlemon_> Minix 3 does, apparently.

<johnbessa> been, like only 22 yrs, ugh

<vmlemon_> Or at least there's talk of swap partition support.

<johnbessa> during my last IT incarnation, I was looking at ExOS

<vmlemon_> Bah. http://groups.google.com/group/minix3/browse_thread/thread/2ff644ae70ce3782?pli=1 seems to suggest rudimentary paging support, but not "swap partition" support.

<johnbessa> rudimentary does not inspire one

<johnbessa> LibOS too, and I was interested in the Perl VM as it uses registers instead of stacks

<johnbessa> and a compiler writer told me that a register-only system would be factors more efficient

<vmlemon_> Until they decided to close access to the codebase again, QNX Neutrino might have been a nice fit.

<johnbessa> yeah, that angered me, QNX is a nice OS

<johnbessa> All things considered L4 seems a good path, as it is marginally related to XNU and Mac

<vmlemon_> Lots of microkernel-based OSes seem to implement support for demand-paging (or some other paging technique), but not swapping to disk.

<vmlemon_> :(

<vmlemon_> For what it's worth, we were contemplating adding swapping/disk-bound demand paging to Symbian OS; but the consensus was that it'd be a bad idea, since NAND Flash chips would quickly burn out under the workload.

<vmlemon_> (Not what you'd want to happen in an expensive smartphone, of course!) :)

<vmlemon_> Still, at least a demand paging mechanism made its way in, prior to the Open Source release.

<johnbessa> heh, as it happens I have an early Mako that is the nicest piece of hardware I own, beautiful construction

<johnbessa> who is "we?" are you in the symbian sphere?

<vmlemon_> "We" was the Symbian Foundation and the community that formed around it, prior to its recent closure.

<johnbessa> yet another "bites the dust"

• duvjones has quit (Quit: Leaving)

• machdev (~machdev@ip68-4-241-106.oc.oc.cox.net) has joined #puredarwin

<johnbessa> my hardware scope is a little larger that smart phones, possibly recycled laptop equipment, as I know my audience has that lying around

<vmlemon_> (The SF ran an "Ideas" site where folks both internally and externally could throw around thoughts on improving the Symbian Platform - although it eventually devolved into a bunch of spamming idiots, and whiners who wanted us to make all but an iPhone OS clone; and both the staff and the visitor community abandoned it quickly after the signal-to-noise ratio plummeted).

<vmlemon_> A lot of high-quality ideas and debate was generated when it was in private beta, though.

<vmlemon_> The moral of the story is that letting in the unwashed masses quickly ruins things...

<johnbessa> sounds like the Wikiversity, except the non-whiners have sufficent critical mass

<vmlemon_> Heh

<johnbessa> the fact that it is a virtual mental hospital is benefiting me now that I am studying to be a counselor

<vmlemon_> Interesting stuff

<johnbessa> my better half and I want to open a group home, and knowing most kids are gaming junkies only becuase they are frustrated OS hacks

<johnbessa> I will make IT a component of it

<vmlemon_> Aah

• vmlemon_ never really got into gaming

<johnbessa> the other component will be working shelter animals

<vmlemon_> Nice

<johnbessa> me neither, though I liked Sonic the Hedgehog

<johnbessa> being in canada, my fantasy is to hitch the dogs up to sleds, and go from there!

<vmlemon_> The last games console we bought was a Wii - which is now seldom used, and the last one I played with any regularity was the PlayStation 2.

<johnbessa> <== we are talking sega dreamcast

<vmlemon_> I sometimes play the odd puzzle game on my phone, though. (I've got Snakes and a Dutch Tetris clone installed on it, at the moment).

<johnbessa> I actually text now that I have a keyboard on a phone, though I swore I never would

<johnbessa> and I tried snowboarding

<johnbessa> looks like the gen II is winning me over

<johnbessa> so the least I can do is give them a decent OS to get acquainted with

<vmlemon_> Make sense

• vmlemon_ had a GameBoy Advance emulator installed on the Flash card in his old phone, until he lost it somewhere

<johnbessa> hate it when that happens

<vmlemon_> What about Haiku? It seems to have a fairly minimalistic kernel, and a UNIX-like userland.

<johnbessa> I actually lost a pair of steel edge XC skis

<johnbessa> OK, noted

<vmlemon_> Bah

<machdev> If you don't mind me jumping in, what are you looking for?

<vmlemon_> Internally, it seems to be implemented in a very different manner to "classical UNIX clones", though. :)

<johnbessa> student project that can work on old x86 hardware that seems to have a future

<johnbessa> L4 seems to have the most "future" potential

<vmlemon_> (Like the Symbian OS, it's mostly written in C++, and has its own set of APIs that are preferred over the use of POSIX ones, where possible).

<machdev> I use to teach CS classes, my advice is to either use linux (which runs decently on old hardware) or to turn those x86 machines into dumb terminals and connect to a TS such as WTS (Windows), LTSP (Linux), ACTS (Mac)

<johnbessa> did I forget to mention microkernel?

<johnbessa> also interested in I/O kit

<machdev> lol

<machdev> Then in that case there are 2 options still:

<vmlemon_> An L4-derived microkernel and Genode might be worth a look, FWIW.

<machdev> 1. Run GNU Mach (bootable now)

<vmlemon_> (I managed to get an L4Ka::Pistachio-based system built fairly quickly using it; and it supports running Linux applications when using OKL4 instead - but that's a PITA to build).

<johnbessa> GNU has switched strategies so many times...

<machdev> 2. ACTS which can be used to develeop Mac apps/kernel stuff (using virtualbox and xcode)

<machdev> True but the GNU Mach system is 80% the same as the XNU's Mach System (both based on Mach 3.0)

<johnbessa> this is a constructivist project, which means starting with the most basic components and then building up to something usable

<machdev> just my opinion of course

<johnbessa> usable meaning something a kid can hold and say "this does something useful"

<machdev> If thats the case then I would avoid the Darwin and related (XNU) platforms

<machdev> The darwin platform (no offense) is not complete enough for usefulness and Apple won't really let you build a XNU platform (OSX)

<johnbessa> on the surface, darwin and XNU seem the best route

<johnbessa> and/or

<johnbessa> OK, dumb(er) question: If XNU is open source, why do they think they can prevent an x86 boot?

<johnbessa> if I phrased the correctly

<johnbessa> that <machdev> thats not it, Apple's "opensource" projects are in complete and even if they were would still not yield a very useful platfrom since all the really good pieces are not opensoured

<johnbessa> actually, apple being a corp, why does "it" think that?

<vmlemon_> They're not actively preventing it, but they're not focussing on it as much as they used to, since they've already got a Darwin-based x86-platform that works for them internally.

<vmlemon_> (Mac OS X, of course).

<vmlemon_> Darwin was probably considered to be a proof-of-concept/Open Source experiment, in my opinion.

<johnbessa> what do you mean by "really good pieces?"

<machdev> CoreGraphics, CoreAudio, CoreAnimation, CoreVideo, CoreImage, QuartzCore, etc...

<machdev> For example

<vmlemon_> A case in point being that there isn't even a working audio subsystem in Darwin, unlike Linux.

<machdev> Exactly

<vmlemon_> (Same deal with Bluetooth and 802.11, too).

<johnbessa> but that OSX, and I would go with Xwin

<johnbessa> oh 802.11, thats a biggie

<johnbessa> and sound

<machdev> doesn't matter, the User Land daemons and kernel extensions are far from complete and some components are out right missing

<johnbessa> is I/O kit portable?

<vmlemon_> Heck, some of the most critical components are essentially "reimplementations for Open Source".

<vmlemon_> (CoreFoundation comes to mind, there).

<machdev> For example the IOAudioFamily kernel extension is there but it can't hook in to the kernel properly since a dependency is missing between the IOAudioFamily and the XNU kernel

<machdev> IOKit is found in XNU/Darwin, FreeBSD (latest), GNU Mach

<vmlemon_> Some folks from NICTA managed to weld it onto an experimental hybrid XNU/L4-based system (Darbat), if I remember correctly.

<machdev> I heard OpenBSD was also porting IOKit but not sure

<johnbessa> and BSD is now a debian release

<machdev> FreeBSD is NOT a microkernel though

<johnbessa> but a step along the way

<machdev> but it does have an version of IOKit now

<machdev> FreeBSD won't ever become a microkernel (FYI)

<machdev> hey vmlemon_, who is running the things for puredarwin anyways?

<johnbessa> it would have to become mach, which I think* is where mach came from (??)

<vmlemon_> machdev: probono's mostly running things, but he's busy nowadays.

<machdev> ahh

<vmlemon_> Heh, yes. XNU could probably be considered a "family reunion" of the two. ;)

<machdev> I jumped on here a day or 2 ago to see what "blockers" are being worked on so I could start on ones that are not being worked on

<machdev> I got an ok from my work to do some OSS work on the side and since my work is 90% having to deal with low-level Apple components I though this would be a good place to start

<johnbessa> thanks, I have to read about girls who "cut" who have been abused by religious figures (not plaster ones)

<johnbessa> I will look into your suggests, I think IOkit is a key component

<machdev> good luck johnbessa

<johnbessa> xnu/darwin seemed a perfect world

<machdev> I would agree if the project was complete enough

<johnbessa> seeing how many would benefit from it, possibly there is a pathway to create open versions of all missing or disconnected pieces

<machdev> thats the point of puredarwin (I think)

<johnbessa> there is no reason that it has to comply with the OSX model

<machdev> vmlemon_: has anyone thought about porting GNUStep for the AppKit related pieces?

<johnbessa> bing!

<johnbessa> or bingo

<vmlemon_> Good question. I tried to build it under Darwin 8, and didn't get very far.

<vmlemon_> I haven't reattempted it, since.

<machdev> vmlemon_: currently you are unaware of anyone working on any current blockers, correct?

<vmlemon_> Most of the work done so far as been ad-hoc stuff on getting booting under VirtualBox working.

<vmlemon_> Theoretically, there's nothing to prevent you from extracting the I/OKit components from XNU, and implementing a compatibility layer on top of another kernel/OS, in order to use said components...

<vmlemon_> machdev: Of course, anyone's free to work on stuff that they find interesting.

<btQuark> greetings guys - have you, by chance had any chance of trying the new vbox 4.x additions for osx server?

<machdev> vmlemon_: I'm just trying to see where my time would be best spent, all of the work in the blockers are not interesting to me but it has to be done. (I spend all day doing OSX Low Level dev work)

<johnbessa> I think that a criteria for developing strategies is showing that the ideas have potential for long-term value

<johnbessa> in other words, are popular

<vmlemon_> That said, we've had a successful boot on real non-Apple hardware.

<machdev> btQuark: I have not tried it, however early vbox 4.0 seemed to be very flaky for me

<vmlemon_> (Of Xmas, which is one of the most complex PureDarwin-based systems so far).

<machdev> thats great

<machdev> would access to a remote OSX box help?

<johnbessa> you mean, via the Internet?

<vmlemon_> Pertinent question, since we've been trying to get a build machine of any sort running, for ages.

<johnbessa> SSH?

<vmlemon_> (probono came close, but he had to take said machine down, and couldn't find a new location to host it in).

<machdev> I probably could rig up a Mac Terminal Server (Graphical) / SSH server

<vmlemon_> Bart- has also investigated various things, but hasn't quite brought anything to fruition.

<btQuark> machdev the 4.0 seems quite ok

<btQuark> the final one it may help with our vm we could provide anyhow, it could be run tool-less if needed too

<btQuark> anyhow it needs some testing

<machdev> I could put an extra MacBook Pro (Core2Duo - 4gb RAM - OSX 10.6 Server + ACTS) I have, probably on Tuesday or Wed (after I get it from my lab at work).

<vmlemon_> I'm sure that probono would appreciate any solution that you can come up with. <machdev> k <johnbessa> if you are not running apache, then using a DSL connection is hardly a bother

<johnbessa> or cable modem

<machdev> The box will have RDP and SSH access

<vmlemon_> Great! :) <machdev> It is literally sitting in the corner collecting dust, I would have to host it at either my home or a friends (via cable modem) since I won't be allowed to host it at the office

<machdev> How would I get the information to you guys when I have it set?

<johnbessa> well, be thankful you have an IT job (I assume it is, I wound up driving big rigs for $$)

<machdev> I haven't seen probono

<machdev> lol

<machdev> My job is nice but the pay sucks

<vmlemon_> Just a case of setting up Dynamic DNS; and maybe adding a CNAME record to the PureDarwin.org DNS server, I guess.

<vmlemon_> (Although I'm not responsible for that).

<machdev> lol, no I mean how do I get the login credentials to you guys (I don't want the box used for downloading porn, etc)

<johnbessa> my psych fork may actually be a business model, esp if we can cure some rich kid with grateful parents

<machdev> so I don't want to "publish" it

<vmlemon_> Giving them to probono via e-mail is probably the best way.

<machdev> k

<vmlemon_> He can then redistribute them to genuinely needy folks.

<vmlemon_> :)

<machdev> lol

<machdev> The box won't be that powerful so it could handle only like 5 to 10 remote RDP sessions at 1 time

<vmlemon_> (I trust that most folks here won't abuse the box).

<machdev> RDP = Remote Desktop Protocol (btw)

<machdev> Each RDP session would have its own desktop so each can run xcode, etc

<johnbessa> Ahhh, speaking of abuse... I really have to research cutting, inserting, injecting...

<machdev> lol, johnbessa, that just sounds bad

<vmlemon_> Creation of FrankenKids? ;)

<johnbessa> better someone else!

• vmlemon_ hides

<machdev> lol

<johnbessa> you should see the xrays

<johnbessa> whole tool boxes inside humans

<vmlemon_> Ugh

<machdev> I would rather not

<johnbessa> but they survive!

<johnbessa> humans are really resiliant, and so is DNA

<vmlemon_> And I thought that the story about the guy with the knife lodged in his skull for years was grotesque...

<johnbessa> probably a story

<machdev> omg you saw that! It gave me the creeps

<vmlemon_> http://www.spokesman.com/blogs/hbo/2011/feb/18/man-discovers-knife-lodged-skull/

<machdev> I kept scratching my head for hours

<vmlemon_> It made one of the national newspapers here.

<vmlemon_> (In the UK, that is).

<vmlemon_> *at least one

<johnbessa> self-inserted?

<vmlemon_> No clue.

<machdev> no idea

<johnbessa> still, probably needs therapy

<johnbessa> seems traumatizing

<johnbessa> Canada hands therapy out for free

<machdev> not here (USA)

<johnbessa> makes a big difference

<johnbessa> yup, I am just 5 miles past the end of Interstate I-95

<johnbessa> and it makes a big difference

<johnbessa> like the difference between Detroit and nearby Windsor/London

<machdev> lol

<machdev> I'm near mexico

• vmlemon_ notes that Nokia Remote Device Access is a nice idea, but the implementation could do with some improvement

<machdev> in the USA still

<machdev> haven't seen it vmlemon_

<vmlemon_> Aah. I'm working on a Qt-based application, but I don't have a handset capable of running Qt. :(

<johnbessa> I hear Mexico is suffering from drug saturation worse than the US

<machdev> Mexico is in a state of civil war (just no one wants to admit it)

<johnbessa> well, anyway, thanks for the material, this provides a good start towards creating a project path, I will edit it down to wiki bytecode

<vmlemon_> Good luck :)

<machdev> good luck

<machdev> Yeah Qt doesn't work well in an ACTS session

<machdev> so I couldn't help you there