Why must it be so hard to cluster?

In the past, I’ve gotten pretty upset by how difficult it is to take advantage of multiple computers on a network for general tasks. In this age of advanced Linux software, I’m still shocked at how hard it is to cluster machines.

Let’s say I have three or so machines on my local network. If the task is something commonplace like encoding audio or compiling, I can use either distmp3 or distcc, respectively.

Alternatively, if I want to share disk space among nodes, I could use a clustered file system such as Lustre or GFS. After that, I’d have to put together a more formal cluster like OpenMosix (now abandoned), OpenSSI, Kerreghed (comparison paper, PDF) or some other option. The next step is to write my own applications to do something explicitly parallel using any number of options like OpenMP, PVM along with trendy stuff like hadoop and MapReduce. I can always opt for just doing it by hand using distributed objects for a given language. Apropos, Ruby has positively stellar support for distributed objects indcluding Rinda, an implementation of tuple-spaces (ala Linda) which provides nifty things (auto-discovery, among other features). Still, these options don’t help me build a general usage cluster out of machines.

Then there are the tools to control the actions of the machines remotely like clusterssh, dsh and gsh.

So far, my options are:

  1. Settle for the limited capabilities except for select tasks.
  2. Write my own app to do something (or everything, which is a bad idea).
  3. Deal with it and control actions using a remote, group-admin tool.

I understand how the landscape could reach such a state, but I don’t like the fact that this is the same set of options I’ve had for the last five years or so. Are there options I’m overlooking? Is there something I don’t know about? The only thing I can see down the pipeline is GNU Queue (got a tipoff from mct) which might very well be exactly what I’ve been dreaming of. Unfortunately, no releases have yet been made, so certainly no chance of using it now.

Palm should go Android

Palm would benefit from adopting Android for so many reasons!

  • Stop funneling profits to Microsoft
  • Get enthusiastic Android developers from the community
  • Support free software and change the business model enough to ensure future solidity. How long before someone does a *really* good clone of HotSync? Why not just move to an open standard like SyncML and monetize the transition?
  • Focus on kickin hardware and stop worrying about the OS. Remember the fiasco with PalmSource?
  • Contributing the upcoming Linux-based new Palm OS to the community would be a kickin way to give back, a great PR stunt and it would free up talent assets to work on the coolness that is Android

Just one person’s opinion but I remember my old Palm hardware and wouldn’t think twice before dropping some cash on a Palm device that ran Android.

Linux boots even faster!

I have always been a super-huge fan of Linux for it’s quick and elegant startup. Ok, so various versions of init aren’t exactly the most elegant things but projects like upstart enrich the landscape all the same. Anyway, I got wind of efforts to make Linux boot faster by way of a Slashdot post or two which pointed me to some major coolness. Basically, there are numerous individuals trying to track the boot process, reduce the resource consumption of the whole process and set stronger goals for the whole affair. All in all, they’ve shaved boot time down to 5 seconds. Whoa….

One-time passwords for SSH and more

Great post by Pablo Lorenzzoni about using one-time passwords for SSH logins (and more through PAM).

Switch to Linux, make it easier for admins

Great article about twenty reasons to switch to Linux.

Ten tips to make Linux administration easier.

Making things happen with the JOLIE programming language

I caught this really interesting interview over at the KDE news site which talks about a programming language called JOLIE which is used for what seems to be called service-oriented computing. Basically, it seems the admirable aim is to use JOLIE to increase compatibility by replacing inter-application frameworks like DCOP, DCOM and D-Bus. From the KDE perspective, it looks like there is work being done toward integrating JOLIE with Plasma, the KDE desktop shell which sounds to me like it would add a whole new level of programmable richness to Plasmoids. The interview from the KDE site and the JOLIE project page have more details and the concepts seem pretty cool.

Wait, so Firefox in Qt is not just a dream?

Well, it would seem that my tiny-little-private dream is not just a fantasy! It looks like Nokia is helping to port Firefox to Qt instead of GTK+, which I personally believe to be butt-ugly in most instances. This is most likely because I’m a proud KDE user…

IMHO, Qt is a better technical platform but I wonder if this is just a port of Firefox or whether it’s a complete Qt version of XUL and such. A complete port of XUL would actually be amazing and would have some serious potential! Even if it’s just an Firefox thing, I’d like to have my favorite browser fit in with my desktop a little better!

A $200 tablet?

A recent blog post on TechCrunch discusses their plans to try for the construction of a $200 tablet for light browsing with a modified Firefox 3 (in a special kiosk mode) and VOIP thanks to Skype. Now, we’ve heard rumblings of low-cost, touch-based computing devices before. Remember concept stuff from the XO-2 which was supposed to be the second OLPC? The idea was to have it be an ebook-style device with dual touchscreens and a sub-$100 price tag. Shortly following the announcement by the OLPC guys, some researchers from Maryland and Berkeley showed of a prototype ebook reader that responded to hand-generated movement like opening and closing the leaves to advance the displayed pages.

While the dream of a lightweight, $200 tablet seems a little far-fetched, the blog post does show that some serious thought went into this. They are very insistent that the proposed device run a stripped down version of Linux that will boot right into Firefox instead of a traditional desktop. From there, the plan is to have a special start page with “large buttons for bookmarked services” linking to sites of great interest and/or utility.

With this in mind, it stands to reason that SproutCore would be an excellent way to build this start page. By designing the startup page as a SproutCore application, the whole experience can be constructed in a completely customized way while still maintaining whatever parts of the traditional application experience are desired. Furthermore, SproutCore apps can be cached can be automagically and transparently upgraded as well as cached on the client so that basic functionality can be maintained even in the absence of a WiFi link.

After a little bit of thought, I think that the main interface (with the big buttons) should be a draggable grid of icons so that a user may reposition applications according to their preferences and usage patterns. Additionally, there should be a basic configuration interface allowing users to add or remove sites from the startup page. After that, anything else seems like an added complication even if it’s really, really interesting.

Planet Plug goes live!

After much tweaking and messing around, Planet Plug has finally gone live! It doesn’t have many active subscriptions right now but that should change as more people sign up. If you’re a member of the Philadelphia Linux community, you should get onto the #plug IRC channel on irc.freenode.net and ask to have your feed added. Hooray!

Link to it with this button: Planet Plug button

Open Source in Schools

This afternoon I read a really thoughtful paper about Free/Open Source Software in schools published in the Canadian Journal of Learning and Technology. As an aside, the CJLT has a really solid collection of papers and I look forward to paging through them in the coming months. Anyway, this paper, Open Source Software and Schools: New Opportunities and Directions by Gary Hepburn, presents a tasty discussion of the benefits and advantages to using F/OSS in schools.

After a brief overview of the proprietary software model, Hepburn devotes a solid page examining the issues surrounding using proprietary software in schools. He cites the cost of doing business with Microsoft (according to the Microsoft School Agreement), as of Winter 2005 with a school of 400 machines, to be US$7200 for the OS alone and US$19,200 for a complete desktop package. Additionally, he notes that, in general, “…using proprietary software requires a high degree of accountability” on account of license compliance issues and other legal liabilities. Alongside the financial downsides, Hepburn makes a very good point when examining “…school’s complicity in exposing students to commercial products and corporate interests”. The idea of a classroom as a corporate marketing machine is rather unappealing.

While it’s not hard to find literature extolling the virtues of F/OSS, Hepburn does a nice job of emphasizing the potential benefits for schools. One particular point that caught my eye was his comment that “Schools can also burn OSS on to a compact disc and give it to staff or students to take home and do what they wish with it”. Granted, projects like OpenDisc have been leveraging the flexibility of F/OSS licenses for some time now, but I have been toying with the idea of using virtual machines as distributable workspaces. Personally, I find the idea of being able to send a student home with their own system saved on a thumb drive very compelling.

After touching briefly on the subject of Linux, Hepburn goes on to discuss some of the neater software packages from the F/OSS world. In particular, OpenOffice.org gets a shout out though I don’t know if I agree with his insinuation that OO.org requires fewer system resources (specifically hardware) than MS Office. Overall, I do agree with his other major points which support the ideas that the F/OSS community is better equipped to respond to change and correct errors. Moreover, I fully support the idea that more diverse exposure to different types of software will lead to an overall greater computing literacy.

With a healthy sprinkling of Lawrence Lessig quotes and shameless support for F/OSS, this paper is definitely worth a read.

Next Page »