+ Post New Thread
Page 1 of 2 12 LastLast
Results 1 to 15 of 22
Group Project Thread, distributed storage array in Coding and Web Development; years ago I had the idea that all these computers in our school, sitting there using 1/4 or less now ...
  1. #1
    browolf's Avatar
    Join Date
    Jun 2005
    Location
    Mars
    Posts
    1,524
    Thank Post
    106
    Thanked 88 Times in 74 Posts
    Blog Entries
    46
    Rep Power
    40

    distributed storage array

    years ago I had the idea that all these computers in our school, sitting there using 1/4 or less now of their drivespace, could be turned into a distributed storage array. maybe to run somekind of vast shadowcopy array or storing video feeds from cameras. I envisaged each pc would have a driver than made the diskspace available to the array but invisible to users to the pc. I dont know how you'd make a backup array of many parts function, there would certainly be room for many raid component. It was just a crazy idea, light-years beyond my ability to create. any thoughts?

  2. #2

    tmcd35's Avatar
    Join Date
    Jul 2005
    Location
    Norfolk
    Posts
    5,617
    Thank Post
    845
    Thanked 880 Times in 730 Posts
    Blog Entries
    9
    Rep Power
    326
    I had the same idea a while back. Windows does include the facility for creating such a distributed file system. The idea was shot down when I explained it to a colleague who then asked...

    What if the file you want is on a PC in Room 10 that hasn't been turned on yet?

  3. #3
    browolf's Avatar
    Join Date
    Jun 2005
    Location
    Mars
    Posts
    1,524
    Thank Post
    106
    Thanked 88 Times in 74 Posts
    Blog Entries
    46
    Rep Power
    40
    yeah it neads to be like that but feature raid capability and better security

  4. #4

    Domino's Avatar
    Join Date
    Oct 2006
    Location
    Bromley
    Posts
    4,126
    Thank Post
    215
    Thanked 1,255 Times in 786 Posts
    Blog Entries
    4
    Rep Power
    505
    I think the problems and complexity arising from this far outweighs the benefits. Its not like disk is especially expensive is it..?

    Plus it'll have an impact on your network infrastructure to have many machines running DFS-like around the school

  5. #5

    dhicks's Avatar
    Join Date
    Aug 2005
    Location
    Knightsbridge
    Posts
    5,618
    Thank Post
    1,230
    Thanked 776 Times in 673 Posts
    Rep Power
    235
    Quote Originally Posted by browolf View Post
    years ago I had the idea that all these computers in our school, sitting there using 1/4 or less now of their drivespace, could be turned into a distributed storage array.
    My thoughts exactly - I start sorting ours out on Monday, hopefully. We have a bunch of Windows XP machines (SCL all-in-ones) that we're going to fit second harddrives in. These machines also have a second network card each to avoid network traffic problems - these machines basically have their own dedicated SAN. The idea is to get all of those machines to each share their second harddrive as a network-available block device of some sort, then have a fast file server aggregate those shares into a RAID array which can then be shared back out as a Samba file share.

    The details of how, exactly, we share the harddrives out are a little hazy - I could have sworn I found a Windows XP driver at some point for a block-sharing protocol of some kind, but that's about all I remember about it.

    --
    David Hicks
    Last edited by dhicks; 13th February 2009 at 07:36 PM.

  6. #6

    Join Date
    Feb 2009
    Posts
    95
    Thank Post
    3
    Thanked 33 Times in 32 Posts
    Rep Power
    17
    Quote Originally Posted by dhicks View Post
    I could have sworn I found a Windows XP driver at some point for a block-sharing protocol of some kind, but that's about all I remember about it.
    It may have been the windows DFS drivers you found (Distributed File System: Frequently Asked Questions)

    Otherwise I'd take a look at systems like Gluster (Gluster - Home) for a distributed file system or perhaps Cloudstore (WELCOME TO CLOUDSTORE) - been a while since I deployed one. Most are *NIX based systems I'm afraid and my experiences were all with deploying into *NIX computation cluster environments.

  7. #7

    dhicks's Avatar
    Join Date
    Aug 2005
    Location
    Knightsbridge
    Posts
    5,618
    Thank Post
    1,230
    Thanked 776 Times in 673 Posts
    Rep Power
    235
    Quote Originally Posted by Chillibear View Post
    Most are *NIX based systems I'm afraid and my experiences were all with deploying into *NIX computation cluster environments.
    Nope - I definitely found a driver for Windows XP that could share a spare partition from a Windows machine to a Linux server as a block device. I can't remember the name of the driver, or even the protocol used, as I'm on holiday at the moment and not on my work PC to look stuff up.

    --
    David Hicks

  8. #8

    matt40k's Avatar
    Join Date
    Jun 2008
    Location
    Ipswich
    Posts
    4,377
    Thank Post
    368
    Thanked 637 Times in 519 Posts
    Rep Power
    158
    Would hate to try and recover any lost data!! Sounds like a crazy idea. If you want loads of disc space, remove the hdds, get USB pens and make the workstations thin client and make a load of file servers.

  9. #9
    budgester's Avatar
    Join Date
    Jan 2006
    Location
    Enfield, Middlesex
    Posts
    486
    Thank Post
    4
    Thanked 37 Times in 30 Posts
    Rep Power
    24
    Quote Originally Posted by matt40k View Post
    Would hate to try and recover any lost data!! Sounds like a crazy idea. If you want loads of disc space, remove the hdds, get USB pens and make the workstations thin client and make a load of file servers.
    Bah, now your just being sensible, where is the fun in that I ask you.

  10. #10

    matt40k's Avatar
    Join Date
    Jun 2008
    Location
    Ipswich
    Posts
    4,377
    Thank Post
    368
    Thanked 637 Times in 519 Posts
    Rep Power
    158
    Quote Originally Posted by budgester View Post
    Bah, now your just being sensible, where is the fun in that I ask you.
    If you want to do something fun, use RAM as storage.

  11. #11

    tmcd35's Avatar
    Join Date
    Jul 2005
    Location
    Norfolk
    Posts
    5,617
    Thank Post
    845
    Thanked 880 Times in 730 Posts
    Blog Entries
    9
    Rep Power
    326
    Quote Originally Posted by matt40k View Post
    If you want to do something fun, use RAM as storage.
    That's dynamic RAM not static RAM

    Watches as the cleaner unplugs the file server to plug the hoover in

  12. #12

    SYNACK's Avatar
    Join Date
    Oct 2007
    Posts
    11,039
    Thank Post
    852
    Thanked 2,666 Times in 2,263 Posts
    Blog Entries
    9
    Rep Power
    767
    For real fun use the network itself as storage, racetrack memory over utp all fun and games till someone touches a patch lead.

  13. #13

    dhicks's Avatar
    Join Date
    Aug 2005
    Location
    Knightsbridge
    Posts
    5,618
    Thank Post
    1,230
    Thanked 776 Times in 673 Posts
    Rep Power
    235
    Quote Originally Posted by matt40k View Post
    Would hate to try and recover any lost data!! Sounds like a crazy idea. If you want loads of disc space, remove the hdds, get USB pens and make the workstations thin client and make a load of file servers.
    That's the idea of exporting the extra harddrives as block devices, so I can create a RAID array with them. I actually wanted to make a RAID-1 array but mirroring between three drives instead of two, just to be safe, but I don't think mdadm will let me do that. This storage network is intended for storing large video files, nothing else - more important files go on the normal file servers.

    --
    David Hicks

  14. #14
    browolf's Avatar
    Join Date
    Jun 2005
    Location
    Mars
    Posts
    1,524
    Thank Post
    106
    Thanked 88 Times in 74 Posts
    Blog Entries
    46
    Rep Power
    40
    Quote Originally Posted by dhicks View Post
    This storage network is intended for storing large video files, nothing else - more important files go on the normal file servers.
    -
    David Hicks
    I like that idea. I'm intrigued about this driver you found. how long before you get back!?

  15. #15

    dhicks's Avatar
    Join Date
    Aug 2005
    Location
    Knightsbridge
    Posts
    5,618
    Thank Post
    1,230
    Thanked 776 Times in 673 Posts
    Rep Power
    235
    Quote Originally Posted by browolf View Post
    I'm intrigued about this driver you found.
    Ah - found the ServerFault post I did:

    Cluster file systems - Server Fault

    --
    David Hicks

SHARE:
+ Post New Thread
Page 1 of 2 12 LastLast

Similar Threads

  1. Distributed File System help.
    By boomam in forum Windows
    Replies: 5
    Last Post: 7th August 2008, 03:39 PM
  2. Best platform for distributed processing?
    By dhicks in forum Wireless Networks
    Replies: 5
    Last Post: 2nd June 2008, 10:41 PM
  3. Distributed Gameing for Science
    By SYNACK in forum General Chat
    Replies: 0
    Last Post: 11th May 2008, 02:44 AM
  4. 12 bay esata storage array
    By fooby in forum Hardware
    Replies: 8
    Last Post: 22nd June 2007, 08:35 PM
  5. Anyone running any distributed computing projects?
    By ChrisH in forum General Chat
    Replies: 5
    Last Post: 15th February 2006, 06:21 PM

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •