+ Post New Thread
Page 1 of 2 12 LastLast
Results 1 to 15 of 22
Group Project Thread, distributed storage array in Coding and Web Development; years ago I had the idea that all these computers in our school, sitting there using 1/4 or less now ...
  1. #1
    browolf's Avatar
    Join Date
    Jun 2005
    Location
    Mars
    Posts
    1,525
    Thank Post
    106
    Thanked 89 Times in 75 Posts
    Blog Entries
    46
    Rep Power
    40

    distributed storage array

    years ago I had the idea that all these computers in our school, sitting there using 1/4 or less now of their drivespace, could be turned into a distributed storage array. maybe to run somekind of vast shadowcopy array or storing video feeds from cameras. I envisaged each pc would have a driver than made the diskspace available to the array but invisible to users to the pc. I dont know how you'd make a backup array of many parts function, there would certainly be room for many raid component. It was just a crazy idea, light-years beyond my ability to create. any thoughts?

  2. #2

    tmcd35's Avatar
    Join Date
    Jul 2005
    Location
    Norfolk
    Posts
    5,665
    Thank Post
    850
    Thanked 893 Times in 738 Posts
    Blog Entries
    9
    Rep Power
    328
    I had the same idea a while back. Windows does include the facility for creating such a distributed file system. The idea was shot down when I explained it to a colleague who then asked...

    What if the file you want is on a PC in Room 10 that hasn't been turned on yet?

  3. #3
    browolf's Avatar
    Join Date
    Jun 2005
    Location
    Mars
    Posts
    1,525
    Thank Post
    106
    Thanked 89 Times in 75 Posts
    Blog Entries
    46
    Rep Power
    40
    yeah it neads to be like that but feature raid capability and better security

  4. #4

    Domino's Avatar
    Join Date
    Oct 2006
    Location
    Bromley
    Posts
    4,156
    Thank Post
    215
    Thanked 1,260 Times in 791 Posts
    Blog Entries
    4
    Rep Power
    507
    I think the problems and complexity arising from this far outweighs the benefits. Its not like disk is especially expensive is it..?

    Plus it'll have an impact on your network infrastructure to have many machines running DFS-like around the school

  5. #5

    dhicks's Avatar
    Join Date
    Aug 2005
    Location
    Knightsbridge
    Posts
    5,624
    Thank Post
    1,240
    Thanked 778 Times in 675 Posts
    Rep Power
    235
    Quote Originally Posted by browolf View Post
    years ago I had the idea that all these computers in our school, sitting there using 1/4 or less now of their drivespace, could be turned into a distributed storage array.
    My thoughts exactly - I start sorting ours out on Monday, hopefully. We have a bunch of Windows XP machines (SCL all-in-ones) that we're going to fit second harddrives in. These machines also have a second network card each to avoid network traffic problems - these machines basically have their own dedicated SAN. The idea is to get all of those machines to each share their second harddrive as a network-available block device of some sort, then have a fast file server aggregate those shares into a RAID array which can then be shared back out as a Samba file share.

    The details of how, exactly, we share the harddrives out are a little hazy - I could have sworn I found a Windows XP driver at some point for a block-sharing protocol of some kind, but that's about all I remember about it.

    --
    David Hicks
    Last edited by dhicks; 13th February 2009 at 07:36 PM.

  6. #6

    Join Date
    Feb 2009
    Posts
    95
    Thank Post
    3
    Thanked 33 Times in 32 Posts
    Rep Power
    17
    Quote Originally Posted by dhicks View Post
    I could have sworn I found a Windows XP driver at some point for a block-sharing protocol of some kind, but that's about all I remember about it.
    It may have been the windows DFS drivers you found (Distributed File System: Frequently Asked Questions)

    Otherwise I'd take a look at systems like Gluster (Gluster - Home) for a distributed file system or perhaps Cloudstore (WELCOME TO CLOUDSTORE) - been a while since I deployed one. Most are *NIX based systems I'm afraid and my experiences were all with deploying into *NIX computation cluster environments.

  7. #7

    dhicks's Avatar
    Join Date
    Aug 2005
    Location
    Knightsbridge
    Posts
    5,624
    Thank Post
    1,240
    Thanked 778 Times in 675 Posts
    Rep Power
    235
    Quote Originally Posted by Chillibear View Post
    Most are *NIX based systems I'm afraid and my experiences were all with deploying into *NIX computation cluster environments.
    Nope - I definitely found a driver for Windows XP that could share a spare partition from a Windows machine to a Linux server as a block device. I can't remember the name of the driver, or even the protocol used, as I'm on holiday at the moment and not on my work PC to look stuff up.

    --
    David Hicks

  8. #8

    matt40k's Avatar
    Join Date
    Jun 2008
    Location
    Ipswich
    Posts
    4,410
    Thank Post
    368
    Thanked 639 Times in 521 Posts
    Rep Power
    158
    Would hate to try and recover any lost data!! Sounds like a crazy idea. If you want loads of disc space, remove the hdds, get USB pens and make the workstations thin client and make a load of file servers.

  9. #9
    budgester's Avatar
    Join Date
    Jan 2006
    Location
    Enfield, Middlesex
    Posts
    486
    Thank Post
    4
    Thanked 37 Times in 30 Posts
    Rep Power
    24
    Quote Originally Posted by matt40k View Post
    Would hate to try and recover any lost data!! Sounds like a crazy idea. If you want loads of disc space, remove the hdds, get USB pens and make the workstations thin client and make a load of file servers.
    Bah, now your just being sensible, where is the fun in that I ask you.

  10. #10

    matt40k's Avatar
    Join Date
    Jun 2008
    Location
    Ipswich
    Posts
    4,410
    Thank Post
    368
    Thanked 639 Times in 521 Posts
    Rep Power
    158
    Quote Originally Posted by budgester View Post
    Bah, now your just being sensible, where is the fun in that I ask you.
    If you want to do something fun, use RAM as storage.

  11. #11

    tmcd35's Avatar
    Join Date
    Jul 2005
    Location
    Norfolk
    Posts
    5,665
    Thank Post
    850
    Thanked 893 Times in 738 Posts
    Blog Entries
    9
    Rep Power
    328
    Quote Originally Posted by matt40k View Post
    If you want to do something fun, use RAM as storage.
    That's dynamic RAM not static RAM

    Watches as the cleaner unplugs the file server to plug the hoover in

  12. #12

    SYNACK's Avatar
    Join Date
    Oct 2007
    Posts
    11,174
    Thank Post
    868
    Thanked 2,702 Times in 2,289 Posts
    Blog Entries
    11
    Rep Power
    773
    For real fun use the network itself as storage, racetrack memory over utp all fun and games till someone touches a patch lead.

  13. #13

    dhicks's Avatar
    Join Date
    Aug 2005
    Location
    Knightsbridge
    Posts
    5,624
    Thank Post
    1,240
    Thanked 778 Times in 675 Posts
    Rep Power
    235
    Quote Originally Posted by matt40k View Post
    Would hate to try and recover any lost data!! Sounds like a crazy idea. If you want loads of disc space, remove the hdds, get USB pens and make the workstations thin client and make a load of file servers.
    That's the idea of exporting the extra harddrives as block devices, so I can create a RAID array with them. I actually wanted to make a RAID-1 array but mirroring between three drives instead of two, just to be safe, but I don't think mdadm will let me do that. This storage network is intended for storing large video files, nothing else - more important files go on the normal file servers.

    --
    David Hicks

  14. #14
    browolf's Avatar
    Join Date
    Jun 2005
    Location
    Mars
    Posts
    1,525
    Thank Post
    106
    Thanked 89 Times in 75 Posts
    Blog Entries
    46
    Rep Power
    40
    Quote Originally Posted by dhicks View Post
    This storage network is intended for storing large video files, nothing else - more important files go on the normal file servers.
    -
    David Hicks
    I like that idea. I'm intrigued about this driver you found. how long before you get back!?

  15. #15

    dhicks's Avatar
    Join Date
    Aug 2005
    Location
    Knightsbridge
    Posts
    5,624
    Thank Post
    1,240
    Thanked 778 Times in 675 Posts
    Rep Power
    235
    Quote Originally Posted by browolf View Post
    I'm intrigued about this driver you found.
    Ah - found the ServerFault post I did:

    Cluster file systems - Server Fault

    --
    David Hicks

SHARE:
+ Post New Thread
Page 1 of 2 12 LastLast

Similar Threads

  1. Distributed File System help.
    By boomam in forum Windows
    Replies: 5
    Last Post: 7th August 2008, 03:39 PM
  2. Best platform for distributed processing?
    By dhicks in forum Wireless Networks
    Replies: 5
    Last Post: 2nd June 2008, 10:41 PM
  3. Distributed Gameing for Science
    By SYNACK in forum General Chat
    Replies: 0
    Last Post: 11th May 2008, 02:44 AM
  4. 12 bay esata storage array
    By fooby in forum Hardware
    Replies: 8
    Last Post: 22nd June 2007, 08:35 PM
  5. Anyone running any distributed computing projects?
    By ChrisH in forum General Chat
    Replies: 5
    Last Post: 15th February 2006, 06:21 PM

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •