Teachers storage space - how much is enough?!

[AngryTechnician]

I'm still at a loss as to why so many people on this forum are happy to wait, and wait, for vaporware rather than just choose the best system to do the job in the first place. Deduplication has been around for a long time, if it's something you need it is worth learning how to do it, even if that means using an unfamiliar system.

TCO for adding a non-Windows server in an all-Windows environment goes well beyond the initial 'learning how it works' phase. When you have experience of Solaris/BSD/Samba and all the rest its very easy to recommend, but when you have no experience of any of those things, introducing them for the sake of a single server is not nearly as straightforward or cheap as you seem to be implying. Time is money and not everyone has the time to commit to training and additional maintenance burden.

 

There may also be limitations on supported OS environments dictated by other things like the hypervisor you've standardised on - for example, Hyper-V (rightly or wrongly) only officially supports a very small number of Linux distros.

 

All of these things must be considered when deciding what the 'best' system for your environment is. You cant just look at the technology in isolation and say "that one".

[gshaw]

I'm still at a loss as to why so many people on this forum are happy to wait, and wait, for vaporware rather than just choose the best system to do the job in the first place. Deduplication has been around for a long time, if it's something you need it is worth learning how to do it, even if that means using an unfamiliar system.

 

Because I already have my primary storage and servers in the form of a SAN and VMWare infrastructure and don't feel the need to buy another box just to get a single feature. Btw are you talking file or block level dedupe? Looking at the reports from 2008 R2 FSRM file level saves a bit of space but the efficiency gains very much depend on how your data is being used. In our case I can see loads of similar (but not identical files) which really needs block level dedupe to do... NetApp make a big deal of this but then they also charge the £££ to match

 

I'd like dedupe... however I'm not going to throw away stable systems or chuck bundles of £££ at the solution to get it...

 

As shared drive on the VLE isn't quite what I'd be hoping for, the management of the content needs to be better via metadata on the initial upload vs a bunch of randomly named files in a randomly named folder... all imho of course

Edited January 6, 2012 by gshaw

[Flakes]

our teachers get 500MB of space, our students get 30MB of space which we increase as they need it to a limit of about 300MB.

[GrumbleDook]

Comments about deduplication and using a VLE are pretty spot on.

 

Duplication of files (from group shares to home shares, duplicates of files on different home shares, etc) is a big problem now that people have come to expect unlimited storage in some places ... the fact that most of the unlimited storage actually does have a limit and people have not hit it yet is a separate issue.

 

Moving over to a VLE to replicate shares is not a good thing. you are just changing the location of the problem. When needs to happen is to take a look at document management in general ... and this will mean a fair chunk of user training. This should be costed in at the same time ... and not delivered as an IT thing, but as part of how the school operates and runs. MS have some good stuff on the SharePoint pages which can be translated over to other VLEs so it might be worth looking at that. This is not specifically an IT issue ... I still see the same in the council office with physical storage where people want more copies of the same pieces of paper I have in my files (now archived off-site but I have digital copies of most things and people in other teams insist on printing copies out and filing things away!)

[CyberNerd]

Because I already have my primary storage and servers in the form of a SAN and VMWare infrastructure and don't feel the need to buy another box just to get a single feature. Btw are you talking file or block level dedupe?

 

You wound't need to with vmware - we have linux file and webservers running on vmware with SAN and a similar setup to you, all integrated into a windows domain.

ZFS has block level deduplication, and file level prefetching.

 

 

I'd like dedupe... however I'm not going to throw away stable systems or chuck bundles of £££ at the solution to get it...

 

 

Some pretty large companies run BSD/Solaris. they are stable and you'd be saving money with storage space. You don't need to throw anything away, it would be an upgrade.

 

TCO for adding a non-Windows server in an all-Windows environment goes well beyond the initial 'learning how it works' phase. When you have experience of Solaris/BSD/Samba and all the rest its very easy to recommend, but when you have no experience of any of those things, introducing them for the sake of a single server is not nearly as straightforward or cheap as you seem to be implying. Time is money and not everyone has the time to commit to training and additional maintenance burden..

 

I agree to some extent. I'm no solaris guru, but I know that setting up samba is pi** easy - it's literally a couple of text files and there is tonnes of documentation.

It's not that hard to integrate these days, maintenance is as minimal as on a windows server and once setup they tend to just work. It's just about weighing up the increased hardware costs and the reduced functionality that you get with windows vs's the TCO's of linux/unix. I reckon that in most circumstances it is worth investing the time to become competent in running multiple OS's - you'll eventually end up with an increased skillset and save money in the medium to long term. Choosing windows for windows sake isn't necessarily the smart option.

 

 

All of these things must be considered when deciding what the 'best' system for your environment is. You cant just look at the technology in isolation and say "that one".

 

more like choose the best tool for the job if it integrates properly with a windows domain (which they do) and provide features that you need then choose solution that will do it. Windows isn't always the best tool for the job - and neither is linux/unix, but sometimes there are massive advantages to running mixed systems.

 

There may also be limitations on supported OS environments dictated by other things like the hypervisor you've standardised on - for example, Hyper-V (rightly or wrongly) only officially supports a very small number of Linux distros.

 

Hyper-v was never the best system around! mosy only put it in because of an MS mindset, rather than comparing features and pricing compared to Xen, vmware, KVM. Kind of my original point.

[CyberNerd]

Moving over to a VLE to replicate shares is not a good thing. you are just changing the location of the problem.

 

If your shared drives are the same repository as the VLE repository then it isn't changing the location it is consolidating the repositories to the same location. Moodle 2.0 does this.

[plexer]

Opendedup

 

Ben

[localzuk]

Opendedup

 

Ben

 

Nice, but definitely not one to run under Windows as it has the following limitation "Memory Restrained - Only 32 bit version is supported on Windows currently. This will limit the size of the files and the amount of data you will able to store to about 200 GB."

[Domino]

Nice, but definitely not one to run under Windows as it has the following limitation "Memory Restrained - Only 32 bit version is supported on Windows currently. This will limit the size of the files and the amount of data you will able to store to about 200 GB."

 

But, they do provide a vm template machine to deploy - which should make setup for the linux nervous a lot easier. I may even have a butchers....

[localzuk]

But, they do provide a vm template machine to deploy - which should make setup for the linux nervous a lot easier. I may even have a butchers....

 

Doesn't really help people who already have a windows environment though - that would require a major shift in the way they operate! Dabbling in Linux for a hobby is one thing if you don't have training or experience, but rolling it out to host your mission critical data without any knowledge is A Bad Idea ™ IMO.

[Domino]

Doesn't really help people who already have a windows environment though - that would require a major shift in the way they operate! Dabbling in Linux for a hobby is one thing if you don't have training or experience, but rolling it out to host your mission critical data without any knowledge is A Bad Idea ™ IMO.

 

I didn't say it resolved all the issues did I? It just makes it easier to have a look at a working model and see what it can do

[gshaw]

Some pretty large companies run BSD/Solaris. they are stable and you'd be saving money with storage space. You don't need to throw anything away, it would be an upgrade.

 

The throwing away comment was re: OEM physical servers for Storage Server 2008 R2 but I get your point

 

Also if you use a proper repository system linked with the VLE it only stores a single copy of the resource across many courses using it. If it's then stored in that repository as primary storage location for teaching materials then you've also got it off the network and it's indexed... win-win

Edited January 6, 2012 by gshaw

[GrumbleDook]

If your shared drives are the same repository as the VLE repository then it isn't changing the location it is consolidating the repositories to the same location. Moodle 2.0 does this.

 

A school has a file share with 1.2 TB in it ... and a vle using 0.1 ... it consolidates them / gets rid of the files share to force the use of the VLE and it becomes 1.3 TB ... this is not the same as document management, stopping the duplication of files and helping people understand things like version control, using wikis instead of lots of word files, etc. The use of File Shares in Moodle 2.0 just masks a problem, it doesn't fix it. It might be a stepping stone to better adoption of the VLE ... but will still be an issue. Other VLEs and LPs have similar issues ... the use of document libraries in SharePoint to replicate what is done with files shares is stupid ... no use of meta-data, no workflow management ... it makes a mockery of how powerful tools within VLEs can be.

[plexer]

Nice, but definitely not one to run under Windows as it has the following limitation "Memory Restrained - Only 32 bit version is supported on Windows currently. This will limit the size of the files and the amount of data you will able to store to about 200 GB."

 

Currently yes in each mounted filesystem.

 

Ben

[CyberNerd]

A school has a file share with 1.2 TB in it ... and a vle using 0.1 ... it consolidates them / gets rid of the files share to force the use of the VLE and it becomes 1.3 TB ... this is not the same as document management, stopping the duplication of files and helping people understand things like version control, using wikis instead of lots of word files, etc. The use of File Shares in Moodle 2.0 just masks a problem, it doesn't fix it. It might be a stepping stone to better adoption of the VLE ... but will still be an issue. Other VLEs and LPs have similar issues ... the use of document libraries in SharePoint to replicate what is done with files shares is stupid ... no use of meta-data, no workflow management ... it makes a mockery of how powerful tools within VLEs can be.

 

I think you're misunderstanding what I was saying. If the repository for the VLE and the shared drive use the same deduplicated file system then it's not really masking a problem, if the problem is multiple identical files across different systems. For things like office files, google apps is a much more powerful solution for collaboration and VLE integration, but for other files I don't see the issue with this methodology.

Edited January 6, 2012 by CyberNerd

[dhicks]

I found the key to this was a large number of specific share areas.

 

Agreed. I'd aim to give users the same amount of storage as an average, cheap USB stick - 8GB per pupil, probably the same for staff, but then staff should also have access to a bunch of other store areas. Things like photos and videos tend to get taken once and used but not changed by several people, so they're best off on their own server that can deduplicate them at the file level. As I think has been pointed out, if your staff have a tendancy to keep copies of shared resources in their own file areas then a file system that can do block-level deuplication (assuming your staff file areas and general shared areas are shared from the same file system, or that you have a file system that can deduplicate between storage devices) will at least cut down your disk space usage, if not actually organise your files better. All the deduplicating file systems I've tried on Linux, including ZFS at the time (a couple of years ago) had various issues, mostly performance - file systems implemented via FUSE are always going to be slower (I've not tried ZFS on BSD, though, where I understand it works well).

 

Deduplicating replicated data is a technical problem that can be solved relativly easily to the problem of getting users to accuratly index and label files and other content for easier retrival. The average user is simply not going to add meta-data to each file they upload to a file repository. Google, of course, have much the same problem - they can't expect (or trust) every web page to have accurate meta-data, so their whole business is centred around searching through data and indexing it as best they can.

 

This works better with a large dataset (i.e. The Web), so I don't know how practical it is to run a similar process on a local storage server, but it might make for a system that works more the way people are used to these days - simply have a search box that you type keywords into and see what results you get.