Great big mess of a network share!

[TC-1927]

I've been putting it off for too long now, it's time to tackle the hulking great mess that is one of school's staff shares.

 

I don't know where to start really - it's around 1.5TB of data and ticks all the boxes:

 

- photo and video evidence, spread out nicely all over the share

- lots of uncompressed photos with lots of publisher/office files relying on current file paths

- users creating their 'own' folders (and moaning when stuff goes missing), in no particular location.

- hilariously long file paths

 

etc!

 

I've been running regular backups on it, but rather than buying more space I need to get it organised first.

 

My first step is probably to pull all the multimedia files out and move them to a different location, so any scripts would be much appreciated - but does anyone have any similar experiences/stories and more importantly some advice on where to start?

 

 

Cheers!

[elsiegee40]

We started by creating a Media drive for photos, videos, etc which helped enormously. However, 3 years down the line we have all the photos and videos separated from other file types and 2 disorganised staff drives. Unless staff are prepared to take ownership themselves, and clear up themselves, there is little you can do. It is their data and their problem.

 

All you can do is compression. If you start moving or deleting stuff yourself then someone will scream.

[featured_spectre]

Step 1 - Create staff folders, accessible only to that staff member and administrator

Step 2 - Map the drive with the staff log on policies

Step 3 - Email around to all staff, asking them to slowly start migrating their items into the folders

Step 4 - Create a department folder, accessible only to department members

Step 5 - Map the drive with the staff log on policies

Step 6 - Email around to all staff, asking them to slowly start migrating their departments items into the department folders

Step 7 - Make a media area for all staff

Step 8 - Map the drive with the staff log on policies

Step 9 - Email around to all staff, informing duplicate files WILL be deleted

Step 10 - Make a backup of everything as it is

Step 11 - Start removing duplicated items which are not in staff areas

Step 12 - Inform staff when the removals have started

Step 13 - Inform staff when the removals have finished

 

This process should take a maximum of 1 academic term, I know it seems like a long time, but you need to get staff used to things slowly or you will end up in a world of hurt!

 

*edit* I did this, and it was quite successful, but your mileage may vary!

[elsiegee40]

I can't see how it helps nephilim. However you tie stuff up with rules, staff will just bung it in any old how and then copy it to the department file just in case. You cannot be certain which is the correct version of anything to delete "duplicates".

 

We had it organised for a couple of terms, but it didn't last... and it won't last. Some people seem incapable of thinking in logically ordered file structures to save things where they, and others, can find it again. All you end up with is total chaos within the folders and a massive amount of data as you had before. There is not time in the week to spend all our time housekeeping for the other staff.

[fiza]

@nephilim - With steps 1 and 4 what happens when other staff need access to those folders for cover lessons etc.

[TC-1927]

Thanks for the replies! It's still easter term here at the moment, and they have inspection coming up so I'm not moving anything just yet lest I become swiftly unemployed. Hoping that I can crack it over the summer having given them the summer term to move stuff!

 

I'm going to get a media share up and running anyway to 'encourage' staff to at least attempt to differentiate - I neglected to mention that I already have redirected my docs in place as I'm sure you're aware with teaching staff we have horse>water issues.

 

Also - I see both sides of the argument for/against departmental restrictions but I probably won't implement them in this school - at least not for the forseeable. It depends on the size/needs of the school and staff in fairness.

Edited April 17, 2014 by TC-1927

[featured_spectre]

@nephilim - With steps 1 and 4 what happens when other staff need access to those folders for cover lessons etc.

 

Sorry, should have clarified. There was also a space for cover lessons.

[localzuk]

I'd suggest creating a "loose" structure - splitting it into too many folders will just drive people crazy and they'll start dumping stuff anywhere they want again.

 

Next, instigate a proper photograph storage plan - ie. All photographs MUST be stored in the "photographs" share, in the year they were taken, in a folder with a relevant name (that's how we do it here). All other photos must either be moved there by X date, or they'll be deleted. Then just do a periodic check of the drives for photos and shout at people who don't follow the rules.

 

Finally, ensure you have de-duplication on your storage. It doesn't matter then if they decide they need a copy in their home directory and on the shared drive, put both shares on the same disk and you'll still only have 1 copy stored. Saves us 50% of our storage!

 

Other than that, there is pretty much nothing you can do. Staff will not stick to complicated structures. All they want to do is store things somewhere they can retrieve them from quickly.

 

The only other thing I'd recommend is advising staff that folders with 200 characters in their names, and then files with 200 characters in the name are BAD.

[clareq]

I got my boss to e-mail all staff, telling them that any files not accessed in the last two years would be archived, unless I was told about it first. A week later I ran a robocopy script that moved all old data into an archive folder (on the same drive, but hidden from staff) I left it there for a few weeks, and when nothing had been requested from it, moved it all to a NAS. Our issue want hard drive space, but back up - it was taking far too long.

[Trapper]

I admire your ability to tune out the complaints!

 

The most I've managed is to archive anything not accessed in +>3 years.

 

The single worst part of this for me is my bloody techie is the worst culprit for creating random folders in the staff common drive. Class 4B have taken photos last Tuesday? Fine! New top level folder.

 

AAAAAAAAAAARRRRRRRRRRRRRRRRRRGH!

[clareq]

E-mail came from my boss, complaints went to my boss, easy.

[p858snake]

Not tested my self but a idea,

 

4 drives (mapped locations):

 

1. & 2. Staff Common (Staff only) and Common (Staff & Students), Lock down the root directories so new ones can't be created by any old one, Create folders for each department/teaching area and other associated clubs/projects etc and let them go nuts in there

3. Multimedia, Top level folders by year, then maybe auto create a similar layout as above but not locked down

4. Archives, Whole drive read only, Move staff and common over at the end of year, so they can still view old stuff and move over as needed

[synaesthesia]

What server operating systems and backup setups do you have? For instance if you are running 2012 then deduplication will help with the duplicate files massively and seems to typically mean drive space savings around 40% (ymmv). Also this is a feature built into quite a lot of backup systems so also worth looking out for there. It's not a direct answer to your questions but it may help how you plan to move forwards.

[pettit87]

Can I ask what robocopy command you used to archive the old data?

 

Sorry to highjack the thread.

[SYNACK]

Dedupe plus staff broadcast policy plus fire type restrictions and you may get on top of it a little in a few years.

[clareq]

Can I ask what robocopy command you used to archive the old data?

 

Sorry to highjack the thread.

 

it was one I found here somewhere. Looking an the syntax, you'll need the /move /midlad:YYYYMMDD /s switches.

[seawolf]

The only real way you can solve this problem is by using a Wiki or DMS (Sharepoint, Alfresco, KTDMS, Mindtouch, Confluence), structuring it well at the start, having an agreed upon document management strategy that is adhered to, and training staff. I managed to accomplish this at a previous company and it worked extremely well, but the COO spearheaded adoption of it and would admonish people for not using it (and properly). Full support and use of anything such as this at the exec level is essential.

 

Failing that, the best you can do is implement a good and easy to use Wiki or DMS, and slowly start forcing users to move their data to it. You may still have a disorganised mass of documents, but at least they can be structured using tags, labels, and easily searched. It will still be important to get the idea across of one file, multiple versions (versioning) and not saving multiple,copies of the same document in different locations and names.