Storing an ever increasing amount of digital data!

[Disease]

With Dance, Drama, Music Technology and Photography taking off in the school, not to mention the requirement for departments to take large amounts of photos and videos on trips, I am struggling to back up the vast amount of data being generated. What solutions have you ladies and gents had to put in place to:

 

A. Store this data

 

B. Back up this data on a regular basis.

 

Storage nightmare!

[john]

This is one thing that I could never get people to listen to or understand. They didn't see why I was cross when SEN bought a HD Camcorder as that could be another 20 or 30Gb a year of storage just for that and then all the flip cams in PE and then decent camcorder, then the drama ones etc... I kept saying it and all I got was we gave you ££ for storage 2 years ago and I was yes and IF we keep adding cameras and Camcorders we won't make 5 years before needing more and we cannot manage to back all this up anymore in a weekend.... they just don't make the connection between enterprise storage V the USB HDD in the newspapers and then backing it all up and the time limitations

[timbo343]

If it is only for evidence we grab the video off them and convert it to a much smaller size then stick it to CD/DVD. Is it then upto them to look after it.

[jamesfed]

There are plenty of options here going from a DIY storage server (as some on here have made) all the way up to NAS boxes which host iSCSI mounted storage (cheaply).

 

Some things I made myself a while back-

Server for CCTV storage (although I wish I went with smaller faster drives in the end) - Inside a new Storage Server | my world of IT

A make shift backup server - Time for a new backup server | my world of IT

 

IMO storage using this kind of setup (at least for non mission critial things) is cheap and cheerful so why not just keep piling it up

Edited July 9, 2013 by jamesfed

[pete]

Tier your storage into "stuff that people use every day" and "stuff that gets written once and then looked at once in a blue moon".

 

The former goes on fast disks, the latter goes on bulk SATA.

 

Regarding backups - there's no reason to be backing up files on the bulk SATA that haven't changed since the last backup. By splitting it off (and depending on how your backup network works) you can a) back it up in the daytime b) reduce the time it takes to do the "stuff everyone uses daily" backups.

[dhicks]

What solutions have you ladies and gents had to put in place to:

A. Store this data

B. Back up this data on a regular basis.

 

For our server hardware update this summer, I've asked for a bunch of 4TB harddrives, some of which will be dedicated just to media storage. I'm currently planning for a 12TB media storage area, with a 6TB general file store and a 3TB VM image store. Our backup server will have 12TB, allowing us to back up the VM images (i.e. have runnable copies of VM disk images) and the 6TB general file area, with space for historical copies. That means I've still got to find a sensible way of backing up that 12TB of media storage, for which I'm inclined to look at something like Amazon's Glacier storage service. It works out around 2p per GB per month, which is reasonable for cloud storage, and images and movies tend to be write-once.

 

At some point we will need a decent way to manage all this, and I'm still planning to sort out a "cloud backed" Media Asset Management (MDM) server. Whether this winds up being something based on an existing solution or written from the ground up for schools I'm still not sure - hopefully when we have some decent hardware in place I can spend some time developing this system. I've set up my own OpenStack server with 5TB of storage for testing cloud-style storage, but I don't know if I'd prefer to host my own cloud storage or use Amazon / Rackspace - third parties are hassle-free, but seem to charge huge amounts for cloud storage, especially when you start getting to the terrabyte range that we're talking about here.

[rad]

We have a NAS drive in Media for their use, Languages have HDD but the others use the main server.

 

I attack pictures with an XP addin. Its worth keeping a decent PC on XP just for that. I do get annoyed reguarly when you end up with copies here there and everywhere and videos of random class activities they look at for 30 minutes and leave there on the server for ever. I archive to our ICT Office NAS and 95% of the time, it is never asked for again.