Best platform for distributed processing?

[dhicks]

Hello All,

 

We'll (hopefully) be getting a bunch of nice new PCs turning up this summer. They'll be running Windows XP, but I'll be installing our own imaging system on them that consists of a small Linux distribution (SystemRescueCD) that can copy disk images off a central server. I was thinking that as I'd be setting up each workstation with this system anyway, I could go one step further and add a couple more Linux distributions to do some distributed processing of some kind overnight.

 

Does anyone have any recommendations? With the release of Big Bug Bunny, we might try and get people interested in using Blender, so it might be an idea to have a Blender render farm available for use - is DrQueue the best/only choice? There's also BOINC, which would seem to be a generic platform for running various projects on top of. Is BOINC the best generic distributed processing platform? Is there, by any chance, a Blender render farm that will run on top of BOINC? Would I be best mixing the two by having two separate Linux distributions installed on our machines, one with BOINC installed and one with DrQueue?

 

--

David Hicks

[SYNACK]

I can't find a download link but this:

Penguin Computing - Linux Servers & Clusters - Scyld ClusterWare™ HPC

may be something that could work. It is only installed on the master computer and just uses a small image to boot the other machines into slaves.

 

extra link:

CDROM with Scyld Beowulf - Linux Central the /root for Linux resources

 

Edit:

This page looks directly at the clustering of blender but is quite old:

http://blenderartists.org/forum/showthread.php?t=10018

Edited June 1, 2008 by SYNACK

[dhicks]

Scyld ClusterWare may be something that could work. It is only installed on the master computer and just uses a small image to boot the other machines into slaves.

 

Thanks, looks interesting. Beowulf - of course, how could I forget that one? I also found OpenMosix via your links, but that project seems to now be closed. Ideally I guess I want something that lets our computers be used for something useful for as much time as possible.

 

--

David Hicks

[Ric_]

I believe that Solaris and 'N1 Grid' is also meant to be good and easy to use - although the N1 stuff might have license implications if you aren't using Sun hardware.

[CyberNerd]

Parallel Knoppix can boot over the network, by PXE

 

If you could automatically update the dhcp configuration and have the XP boxes reboot by PXE into linux, you'd have yourself a server farm

 

Ideally the script would change the dhcp config back and reboot them into XP before the teachers come into school.

 

ParallelKnoppix: Create a HPC cluster in 5 minutes

[SYNACK]

Ok, if you are after maximum usability while doing it these should work for you. There are client background app options that work with Windows, OSX and Linux. They also include all the management stuff to split up the jobs.

 

SourceForge.net: Distriblend

Farmerjoe - The render Farmer

Yadra - Yet Another Distributed Rendering App at BlenderNation

ATPM 10.06 - How To: Distributed Blender Rendering