+ Post New Thread
Page 1 of 2 12 LastLast
Results 1 to 15 of 22
Group Project Thread, distributed storage array in Coding and Web Development; years ago I had the idea that all these computers in our school, sitting there using 1/4 or less now ...
  1. #1
    browolf's Avatar
    Join Date
    Jun 2005
    Location
    Mars
    Posts
    1,364
    Blog Entries
    36
    Thank Post
    73
    Thanked 76 Times in 63 Posts
    Rep Power
    30

    distributed storage array

    years ago I had the idea that all these computers in our school, sitting there using 1/4 or less now of their drivespace, could be turned into a distributed storage array. maybe to run somekind of vast shadowcopy array or storing video feeds from cameras. I envisaged each pc would have a driver than made the diskspace available to the array but invisible to users to the pc. I dont know how you'd make a backup array of many parts function, there would certainly be room for many raid component. It was just a crazy idea, light-years beyond my ability to create. any thoughts?

  2. #2
    tmcd35's Avatar
    Join Date
    Jul 2005
    Location
    Norfolk
    Posts
    3,057
    Blog Entries
    9
    Thank Post
    441
    Thanked 410 Times in 356 Posts
    Rep Power
    103
    I had the same idea a while back. Windows does include the facility for creating such a distributed file system. The idea was shot down when I explained it to a colleague who then asked...

    What if the file you want is on a PC in Room 10 that hasn't been turned on yet?

  3. #3
    browolf's Avatar
    Join Date
    Jun 2005
    Location
    Mars
    Posts
    1,364
    Blog Entries
    36
    Thank Post
    73
    Thanked 76 Times in 63 Posts
    Rep Power
    30
    yeah it neads to be like that but feature raid capability and better security

  4. #4

    Domino's Avatar
    Join Date
    Oct 2006
    Location
    Bromley
    Posts
    3,163
    Blog Entries
    4
    Thank Post
    99
    Thanked 655 Times in 462 Posts
    Rep Power
    217
    I think the problems and complexity arising from this far outweighs the benefits. Its not like disk is especially expensive is it..?

    Plus it'll have an impact on your network infrastructure to have many machines running DFS-like around the school

  5. #5

    dhicks's Avatar
    Join Date
    Aug 2005
    Location
    Knightsbridge
    Posts
    4,596
    Thank Post
    941
    Thanked 573 Times in 501 Posts
    Rep Power
    193
    Quote Originally Posted by browolf View Post
    years ago I had the idea that all these computers in our school, sitting there using 1/4 or less now of their drivespace, could be turned into a distributed storage array.
    My thoughts exactly - I start sorting ours out on Monday, hopefully. We have a bunch of Windows XP machines (SCL all-in-ones) that we're going to fit second harddrives in. These machines also have a second network card each to avoid network traffic problems - these machines basically have their own dedicated SAN. The idea is to get all of those machines to each share their second harddrive as a network-available block device of some sort, then have a fast file server aggregate those shares into a RAID array which can then be shared back out as a Samba file share.

    The details of how, exactly, we share the harddrives out are a little hazy - I could have sworn I found a Windows XP driver at some point for a block-sharing protocol of some kind, but that's about all I remember about it.

    --
    David Hicks
    Last edited by dhicks; 13th February 2009 at 08:36 PM.

  6. #6

    Join Date
    Feb 2009
    Posts
    94
    Thank Post
    3
    Thanked 33 Times in 32 Posts
    Rep Power
    12
    Quote Originally Posted by dhicks View Post
    I could have sworn I found a Windows XP driver at some point for a block-sharing protocol of some kind, but that's about all I remember about it.
    It may have been the windows DFS drivers you found (Distributed File System: Frequently Asked Questions)

    Otherwise I'd take a look at systems like Gluster (Gluster - Home) for a distributed file system or perhaps Cloudstore (WELCOME TO CLOUDSTORE) - been a while since I deployed one. Most are *NIX based systems I'm afraid and my experiences were all with deploying into *NIX computation cluster environments.

  7. #7

    dhicks's Avatar
    Join Date
    Aug 2005
    Location
    Knightsbridge
    Posts
    4,596
    Thank Post
    941
    Thanked 573 Times in 501 Posts
    Rep Power
    193
    Quote Originally Posted by Chillibear View Post
    Most are *NIX based systems I'm afraid and my experiences were all with deploying into *NIX computation cluster environments.
    Nope - I definitely found a driver for Windows XP that could share a spare partition from a Windows machine to a Linux server as a block device. I can't remember the name of the driver, or even the protocol used, as I'm on holiday at the moment and not on my work PC to look stuff up.

    --
    David Hicks

  8. #8
    matt40k's Avatar
    Join Date
    Jun 2008
    Location
    Ipswich
    Posts
    2,859
    Thank Post
    201
    Thanked 323 Times in 274 Posts
    Rep Power
    78
    Would hate to try and recover any lost data!! Sounds like a crazy idea. If you want loads of disc space, remove the hdds, get USB pens and make the workstations thin client and make a load of file servers.

  9. #9
    budgester's Avatar
    Join Date
    Jan 2006
    Location
    Enfield, Middlesex
    Posts
    438
    Thank Post
    3
    Thanked 30 Times in 27 Posts
    Rep Power
    18
    Quote Originally Posted by matt40k View Post
    Would hate to try and recover any lost data!! Sounds like a crazy idea. If you want loads of disc space, remove the hdds, get USB pens and make the workstations thin client and make a load of file servers.
    Bah, now your just being sensible, where is the fun in that I ask you.

  10. #10
    matt40k's Avatar
    Join Date
    Jun 2008
    Location
    Ipswich
    Posts
    2,859
    Thank Post
    201
    Thanked 323 Times in 274 Posts
    Rep Power
    78
    Quote Originally Posted by budgester View Post
    Bah, now your just being sensible, where is the fun in that I ask you.
    If you want to do something fun, use RAM as storage.

  11. #11
    tmcd35's Avatar
    Join Date
    Jul 2005
    Location
    Norfolk
    Posts
    3,057
    Blog Entries
    9
    Thank Post
    441
    Thanked 410 Times in 356 Posts
    Rep Power
    103
    Quote Originally Posted by matt40k View Post
    If you want to do something fun, use RAM as storage.
    That's dynamic RAM not static RAM

    Watches as the cleaner unplugs the file server to plug the hoover in

  12. #12

    SYNACK's Avatar
    Join Date
    Oct 2007
    Location
    EOL
    Posts
    6,967
    Blog Entries
    9
    Thank Post
    403
    Thanked 1,779 Times in 1,517 Posts
    Rep Power
    415
    For real fun use the network itself as storage, racetrack memory over utp all fun and games till someone touches a patch lead.

  13. #13

    dhicks's Avatar
    Join Date
    Aug 2005
    Location
    Knightsbridge
    Posts
    4,596
    Thank Post
    941
    Thanked 573 Times in 501 Posts
    Rep Power
    193
    Quote Originally Posted by matt40k View Post
    Would hate to try and recover any lost data!! Sounds like a crazy idea. If you want loads of disc space, remove the hdds, get USB pens and make the workstations thin client and make a load of file servers.
    That's the idea of exporting the extra harddrives as block devices, so I can create a RAID array with them. I actually wanted to make a RAID-1 array but mirroring between three drives instead of two, just to be safe, but I don't think mdadm will let me do that. This storage network is intended for storing large video files, nothing else - more important files go on the normal file servers.

    --
    David Hicks

  14. #14
    browolf's Avatar
    Join Date
    Jun 2005
    Location
    Mars
    Posts
    1,364
    Blog Entries
    36
    Thank Post
    73
    Thanked 76 Times in 63 Posts
    Rep Power
    30
    Quote Originally Posted by dhicks View Post
    This storage network is intended for storing large video files, nothing else - more important files go on the normal file servers.
    -
    David Hicks
    I like that idea. I'm intrigued about this driver you found. how long before you get back!?

  15. #15

    dhicks's Avatar
    Join Date
    Aug 2005
    Location
    Knightsbridge
    Posts
    4,596
    Thank Post
    941
    Thanked 573 Times in 501 Posts
    Rep Power
    193
    Quote Originally Posted by browolf View Post
    I'm intrigued about this driver you found.
    Ah - found the ServerFault post I did:

    Cluster file systems - Server Fault

    --
    David Hicks

SHARE:
+ Post New Thread
Page 1 of 2 12 LastLast

Similar Threads

  1. Distributed File System help.
    By boomam in forum Windows
    Replies: 5
    Last Post: 7th August 2008, 03:39 PM
  2. Best platform for distributed processing?
    By dhicks in forum Networks
    Replies: 5
    Last Post: 2nd June 2008, 10:41 PM
  3. Distributed Gameing for Science
    By SYNACK in forum General Chat
    Replies: 0
    Last Post: 11th May 2008, 02:44 AM
  4. 12 bay esata storage array
    By fooby in forum Hardware
    Replies: 8
    Last Post: 22nd June 2007, 08:35 PM
  5. Anyone running any distributed computing projects?
    By ChrisH in forum General Chat
    Replies: 5
    Last Post: 15th February 2006, 07:21 PM

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •