Paper to Digital

[Homer]

Morning all

 

The SLT are looking into ways of retaining all paper documentation electronically and destroying the paper copies.

 

Is this a thing? What *has* to stay paper, what can be digitalised? Or is this even a thing? SIMS is in use for a lot of documentation however there is a lot of legacy (leavers) paperwork taking up precious storage space.

 

Looking for any guidance on this, if anybody has come across this one already it would be brilliant to hear your thoughts on this conundrum.

 

If it is possible I'd rather stay away from companies offering this and putting it into a proprietary DMS as once it's in it'll be a pain to ever get out if they fold in the future. But if somebody has a reputable provider, I am ears.

 

Cheers!

[mavhc]

Step 1 is to scan it to PDF, Step 2 is to input metadata about the scan, Step 3 is to OCR it and store the text in the PDF, Step 4 is to index and search it.

 

Step 0 is to have a file storage system that's indestructible, immutable, encrypted, and secure.

Step 0.5 is to have an authentication system to make sure the right people can access the right files

 

https://github.com/paperless-ngx/paperless-ngx is the simplest

https://www.papermerge.com/ has more features

https://www.mayan-edms.com/ is the more complex system

[Homer]

Spot on. Familiar with the PaperlessNGX

 

Steps 0/0.5 are the trickiest as well as the labour in steps 1 & 2. Is there anything in the guidance that says this cannot be done, or it as simple as ensuring step 0 is absolutely on point to satisfy their requirements?

[mavhc]

It can be done, question is how much money. You can always store disaster recovery backups in AWS Glacier etc.

 

Firstly I'd start with ZFS, sync snapshots to your local ZFS backup server, so now you have an immutable clone. Then sync to https://www.rsync.net/products/borg.html or something for offsite backups.

 

Having multiple copies is better than paper except in the case where the russians encrypt all your files.

[Homer]

Really appreciate your input, thank you. I'll start putting some suggestions together for SLT.

[dhicks]

Step 0 is to have a file storage system that's indestructible, immutable, encrypted, and secure.

Step 0.5 is to have an authentication system to make sure the right people can access the right files

 

For a school, Google Drive / OneDrive might be the most available / cheapest option for file storage and access control, although I'd want at least one other backup / mirror of the document store. I'm pretty sure Google Drive will automatically OCR a scanned PDF, too, OneDrive probably does similar. If that OCR mechanism isn't suitible, I recently saw this Python project:

 

https://pypi.org/project/ocrmypdf/

 

It looks pretty comprehensive, with options to fine-tune how the OCR is done.

[GrumbleDook]

If you are storing on a cloud service, ensure that you cover the areas such as transfer to 3rd countries within your DPIA. Ensure that anything stored has adequate encryption and that you control the keys. Also remember that anything converted to paperless still has to conform with yout retention schedule, so it is not a case of scan to PDF and forget.

[mavhc]

Yep, that's why you need good metadata, who the file is about, why you're retaining it, and what date it needs to be destroyed, for example

[Roberto]

Governance is a key issue as Tony points out. If you are an O365 school, I'd be looking at Sharepoint for this. Sharepoint online can very easily be structured to allow good use of metadata to retrieve all relevant docs and takes care of storing the data redundantly and is enough of a 'defacto standard' these days for storing documents not to be considered an odd choice. Not sure if there's something similar in the Google world but if there is I'd look at that if you are a google shop. Your biggest issue may be figuring out what metadata to store for each document and how to create that metadata efficiently.

 

Other questions to ask yourself: How will new documents be added? How will old documents be used? How would you deal with a FOI or Data Protection request or similar and feel confident you'd retrieved all the docs necessary to comply?