File server to Office 365 / SharePoint / Azure?

[mb2k01]

Hi All,

 

I tried to keep the title fairly vague, but essentially I'm hoping to get some advice from those who have gone through a local > cloud data migration so that we can avoid the pitfalls we're experiencing in our early testing.

 

What I've got my team looking in to at the minute is a migration of traditional 'shared areas' on our local network up to SharePoint document libraries in Office 365.

The set up at the minute is O365 Sharepoint Site > Document Library per subject area - with us then using a 3rd party util to upload local subject area folders up to their respective Document Libraries.

We than have CDM in use which, with customised permissions on the Document Libraries, enables subject-specific staff to only be able to access their own folders.

 

The issue we've run in to is around the 5000 threshold in SharePoint, as well as the URLs generated in SharePoint, resulting in CDM not showing the files/folders which have been uploaded.

(I.E. We were hoping to migrate the shared areas in their current state - not particularly cleansed - but the result has been huge amounts of files (problem 1 - threshold) but also having huge amounts of sub folders within sub folders (resulting in the SharePoint URLs being too long)

 

My question is - what have others done in this scenario?

Is our approach of SharePoint Site > Document Libraries the correct one, or should we be looking at SharePoint Document Centre instead (does that remove some of the above limitations / issues?) - or - should we be looking to do something else entirely... maybe Azure file storage?

 

Any comments / advice gratefully received!

[mb2k01]

Anyone? (I know I'm asking at a poor time - you're all either enjoying the sun or knee-deep in your own projects!)

[Steve21]

Although we haven't done it yet, from what I researched last year it seems most people ignore SharePoint and use OneDrive shared folders for things. So say Admin shared would be one folder etc etc. There's a few huge threads on here about people trying to get it to work and no-one seemed to fully manage (I say fully as in without major reworking of how their areas are setup)

 

As you say due to the limitations and then issues with uploading/downloading files to SharePoint etc it didn't seem a nice way to do it unless you're going to go through and modify all the areas so they are small enough to fit inside it. e.g. Not a shared area, but an English one, a maths etc etc

 

Using OneDrive still allows the web based access via OWA etc, but also the client to be used to map the drives locally if you wanted.

 

Steve

[chaplic]

I think you've answered your own question - the idea of just 'lifting and shifting' content from a fileserver to SharePoint sounds attractive, but they are not quite the same thing and as you've discovered exposes sharepoints weaknesses pretty quickly. You need to have a design for this, or at the very least scan the fileshares for noncompliant filenames, over long files, etc etc.

 

Azure Files will not likely meet your needs as it's really meant for applications which happen to need file storage.

 

There is another solution called 'storsimple' - your files still look the same (i.e. fileshare) but it's only cached and the content is stored in azure blob storage. I don't know how much space you need to determine if this is viable. I think I'd take the pain and go for sharepoint, file servers are very 1970s..

[pcstru]

There is a throttle on sharepoint lists that limits output to (n) number of rows. That may be what is affecting the lists. You can alter that setting but it would be better to design the list so that the number of files selected for the view is always much lower. How you go about that might depend on the access you want (do you want to be able to map the library as a drive or will it always be access via the website?).

 

In terms of URL's being too long, what exactly is too long? Any web hosted system has no particular limit on how long URL's can be. Functionally a long URL copies as well as a short one and in most cases (unless you are pasting them into notepad or some such), they can be displayed as a link which can be as short as you like. Is the problem there perhaps user education?

 

Given both these, I'd probably look at populating metadata and then having views onto the list that filter on that metadata. So maybe taking folder names out of the path and using those as part of the taxonomy rather than having them as an actual path. However, that all involves work up-front if not cleaning the data, then prepping it. IMO that can be time well spent if the documents are valuable to the business, but if it is a bunch of documents which departments say "might be useful but I can't be bothered to actually look at them to say", then I'd probably dump them on a lump of dumb read only storage and tell them to move them over manually as they need to use or update them.

[chaplic]

Until a few weeks ago, sharepoint did not even support '#' in filenames, and definitely has problems with file/folder length. If you have

 

\\server1\docs\stuff\really\long\path\yada\yada\Big long important name for a word doc.docx

 

And your goal is just to lift to that to the same URL in sharepoint online e.g. https://mytennant.sharepointonline.com/docs/stuff/really/long/path/yada/yada/Big long important name for a word doc.docx

 

Then you are going to have 'fun'

[apearce]

Have you tried moving to the new modern document library view (can change this in the advanced area within the library settings). I recently did some testing around this in which i uploaded ~20k items into it with no folders. With the paging of the document lbirary, i could navigate down the page to that 20,000th file Edited August 15, 2017 by apearce

[Michael]

OneDrive's due to have a big upgrade later this year, so this kind of thing may very well be possible. I've had numerous requests for the same thing, but when looking at limitations it instantly puts myself and end users off.

 

Unfortunately Microsoft give you a wealth of space to work with, but it's all about what you can do with it. You can't have one and not the other.

 

It wasn't that long ago (with OneDrive for Business), that you couldn't download multiple files/folders at once, yet you could upload multiple by drag and drop!

[gshaw]

OneDrive's due to have a big upgrade later this year, so this kind of thing may very well be possible. I've had numerous requests for the same thing, but when looking at limitations it instantly puts myself and end users off.

 

Unfortunately Microsoft give you a wealth of space to work with, but it's all about what you can do with it. You can't have one and not the other.

 

It wasn't that long ago (with OneDrive for Business), that you couldn't download multiple files/folders at once, yet you could upload multiple by drag and drop!

And until recently couldn't move items from a folder to another folder further up the tree, madness.

 

The Windows 10 Fall Creators Update should make OneDrive usable at last but by then MS may have already lost a lot of users to Google Drive, which seems to have less issues in comparison.

[AlanD]

"but by then MS may have already lost a lot of users to Google Drive, which seems to have less issues in comparison"

 

The problem with google drive is that you can't "map" it to a drive letter - the best you can do is to sync it with a local folder - and that is poor for school networks with users logging on and off.

 

I like the RM "OneDrive" where every desktop user gets an "O:" drive that behaves like a penstick....but one they have access to anywhere - including mobile devices. I think IAM cloud does something similar. No local caching. Increasingly users make this their "default" storage location.

 

Slightly alarmed to hear that Microsoft are changing OneDrive...I wouldn't want to lose this functionality.

[nickbro]

We started using the onedrive mapper which is free from http://www.lieben.nu/liebensraum/onedrivemapper/

 

Just did a copy job with all the user files up to their personal onedrive for business accounts using http://www.lieben.nu/liebensraum/o365migrator/

 

Something for you to look at. It works 9/10. The latest version works a lot better than the old ones.

[gshaw]

"but by then MS may have already lost a lot of users to Google Drive, which seems to have less issues in comparison"

 

The problem with google drive is that you can't "map" it to a drive letter - the best you can do is to sync it with a local folder - and that is poor for school networks with users logging on and off.

 

I like the RM "OneDrive" where every desktop user gets an "O:" drive that behaves like a penstick....but one they have access to anywhere - including mobile devices. I think IAM cloud does something similar. No local caching. Increasingly users make this their "default" storage location.

 

Slightly alarmed to hear that Microsoft are changing OneDrive...I wouldn't want to lose this functionality.

Check out Google Drive File Stream, that's where the future lies...

[mukz]

We started using the onedrive mapper which is free from http://www.lieben.nu/liebensraum/onedrivemapper/

 

Just did a copy job with all the user files up to their personal onedrive for business accounts using http://www.lieben.nu/liebensraum/o365migrator/

 

Something for you to look at. It works 9/10. The latest version works a lot better than the old ones.

 

+1 for this too.

Trialled it with our Sixth Formers and only hitch we had was when our internet fibre was cut in the main road for 3 days.

[nickbro]

Yep, that would be our worst case, but we still have on site servers as well. I only did a copy. Going to try it, we aren't that far from the exchange (300m)

[mukz]

We had to move to online because of our Sixth Form setup, students roam between 4 schools with 4 separate networks.

[jslate1980]

To the schools that have gone down this route. What are your schools internet speeds and how many users do you have connecting. This does sound appealing but I am concerned that my 100mb over 100mb line isn't going to be enough to cover the 900 students or 300 so devices connecting.

[snagrat]

Using it here on a 1:1 setup with Surfaces. About 400 devices connected. Upgraded from 100/100 to 300/1GB soon as the 100 is not enough

[mb2k01]

We started using the onedrive mapper which is free from OneDriveMapper | Liebensraum

 

Just did a copy job with all the user files up to their personal onedrive for business accounts using O365Migrator | Liebensraum

 

Something for you to look at. It works 9/10. The latest version works a lot better than the old ones.

 

I've actually been using this very tool to do the data migration I mentioned. The difference is that I'm looking to migrate large amount of shared data rather than just user data.

The upload process goes fine via Lieben mapper but once the files are up there, we run in to the threshold and url issues related to SharePoint.

 

The way we're doing user data is to use CDM to map an O: for users, which presents them with their personal OneDrive folder - and enables them to manage their own data migration from N: to O:

What we wanted to do in addition to that was have a new shared drive mapped which would take them to a 'new' (aka migrated) shared drive - but they end up not seeing all the files that have been uploaded due to the restrictions.

[AlanD]

Using it here on a 1:1 setup with Surfaces. About 400 devices connected. Upgraded from 100/100 to 300/1GB soon as the 100 is not enough

 

I'd be interested to know what your traffic graphs actually look like. Do you find that "upload" is as much as "download"?

[snagrat]

I'd be interested to know what your traffic graphs actually look like. Do you find that "upload" is as much as "download"?

 

At times, it will peek at 100meg

 

However, it's the download the we struggle with