Open source render farm

[BJG]

Having been playing around with some 3D/VFX apps recently, I was starting to wonder about configuring the school's IT suites as a render farm.

 

There was a post about this a couple of years ago, but I'm surprised there hasn't been any other discussion. Programs like Blender and DrQueue seem to offer great potential.

 

Here's a couple of articles to get you thinking. And when you've figured it out, you can write me an idiot guide.

 

http://mos.futurenet.com/resources/3dworld/TDW94.s_farm.pdf

Build Your Own Render Farm - How To by ExtremeTech

 

(Watched How to Train Your Dragon last night. 50 million render hours using a 25,000 core render farm. That's some serious pixel crunching.)

 

http://www.hp.com/hpinfo/newsroom/press/2010/100326xa.html

Edited November 9, 2010 by BJG

[dhicks]

50 million render hours using a 25,000 core render farm.

 

How much call is there for large-scale rendering or general-purpose computing like that? Is there any potential for us selling computing power overnight?

 

I think Fog might be the best thing able to sort out the remote rebooting and selecting of different PXE images, just got to get a server set up...

 

--

David Hicks

[Ulkary]

Dont know about open source but our contractor did a pretty cool job in our Visualization lab:

 

KAUST | Research | Core Labs & Major Facilities | Advanced Computation & Visualization Facility - Cornea

 

I have some buddies still working on servicing it if you want me to ask anything specific.

[BJG]

How much call is there for large-scale rendering or general-purpose computing like that? Is there any potential for us selling computing power overnight?

 

Although there are doubtless some design agencies who would like to get their hands on a couple of hundred PCs, I think it would get too complicated. As far as I can understand, if you wanted to do rendering for commercial apps like Maya, you'd need to buy the specific software and licences for that, and rendering rates are fairly cheap unless you have huge numbers of cores.

 

(Here's a typical example - $.036/GHz-Hour at this French place.)

 

Our contractor did a pretty cool job in our Visualization lab

 

Hmm, not bad...!

Edited November 9, 2010 by BJG

[Ulkary]

There is defiantly a market for this, I know that there are a few Universities in the states renting out their systems for a few thousand dollars an hour to companies like boeing and even branches of the government.

[vikpaw]

@ulkary, why don't you sort out some kind of rental facility and lease some of your enormous processing power on the weekends

[Ulkary]

Because that would mean that I cant watch my 3D BluRays on the weekends.

 

Fully immersive avatar ROCKS

 

Just joking.

[dhicks]

Although there are doubtless some design agencies who would like to get their hands on a couple of hundred PCs, I think it would get too complicated.

 

But I was thinking the other day: the main waste by-product of processor-heavy computing seems to be heat, so why not use computers designed as radiators to heat your house? This doesn't work so well in summer, obviously, so you'd need to have machines globally available to spread the computing load to wherever it was cold at the time. You could sell all-in-one computing radiators that, connected to power and network of some kind, joined up to a computing grid, ready to accept jobs. You probably wouldn't even have to subsidise the purchase price of the hardware, it'd be a way of paying less for electric for heating (i.e. the cost of electricy would be partially offset by getting some money back for doing processing). You could even include a license for something like Maya in with the purchase price. The main issue would be finding a real-world use for the computing power available - is global-scale rendering that much in demand?

 

--

David Hicks

[dhicks]

Fully immersive avatar ROCKS

 

But seriously... if there was enough 3D rendering power on-tap globally, would immersive 3D environments be realisticly possible using a distributed computing/rendering model, or are thing like 3D games always going to need as much computing power locally as possible?

 

--

David Hicks

[Ulkary]

From my understanding the network infrastructure is vital, unless we are all on a 10gbps fiber link the communication between machines would be too slow.

 

If we are talking 3D applications for the consumer market, games, movies etc local processing power is enough.

 

Render farms like the one we have at my Uni are for visualizing MASSIVE data sets and models and if the global infrastructure was fast and robust enough you could do global processing I guess... anything is possible.

 

Thats just from my understanding, please correct me if I'm wrong.

[BJG]

...I guess we're getting into the realms of cloud computing here; not something I know a lot about. Just found this, but there's no payment system yet...

 

vSwarm: Free Render Farm - Cloud Computing for Rendering

[dhicks]

...I guess we're getting into the realms of cloud computing here

 

Annoyingly, the people who seem to be able to do the best distributed-computing-for-sale model seem to be bot farms, created from virus-infected PCs. I don't know exactly what the going rate is for computation on a bot farm, but I think it's pretty cheap (although you probably want to be careful as to what payment method you use...).

 

--

David Hicks

[SYNACK]

But seriously... if there was enough 3D rendering power on-tap globally, would immersive 3D environments be realisticly possible using a distributed computing/rendering model, or are thing like 3D games always going to need as much computing power locally as possible?

 

I thought the lag would preclude it but then I read more about how long the average game actually takes to respond to user input. I now think it is totally possible within current envelopes assuming your not stuck with NZ grad internet with local ping times of 70ms+.

[dhicks]

You could sell all-in-one computing radiators

 

Scratch that - make them modular, so you can have as many processor/radiator units as you like (or, at least, however many your power supply can handle) daisy-chained together. Sell an end-unit that takes care of job allocation, network access and has thermostat controls - crank up the thermostat to run more cores at a faster clock rate. Make each module so that if it fails it won't cause other further down the chain to fail. People would simply buy as many modules as were appropriate to the space to be heated.

 

--

David Hicks

[dhicks]

Sell an end-unit that takes care of job allocation, network access and has thermostat controls

 

Might as well make it a decent home server / firewall / wireless access point / etc while you're at it, make it do something useful locally. That bit would have to be low-powered as it'd be on all the time, including during summer.

 

You could register as an energy supplier, then sell people electricity for their heating at a rate subsidised by the fees your collect from selling computational time. Hmm... Google are registered as an energy supplier, I wonder if that's what they're planning?

 

--

David Hicks

[powdarrmonkey]

But I was thinking the other day: the main waste by-product of processor-heavy computing seems to be heat, so why not use computers designed as radiators to heat your house? [...] You could sell all-in-one computing radiators that, connected to power and network of some kind, joined up to a computing grid, ready to accept jobs.

 

I already do this

[SYNACK]

I already do this

 

I have also owned a Pentium 4