Encryption of disk volumes

[dhicks]

Hello All,

Right, so with all this stuff around about computer equipment with important data on it being stolen from various government offices, I figure I'd better get our data encrypted.

Most of our servers are now virtual machines running under Xen. We install CentOS on the physical machine, then use mdadm to create a RAID volume - i.e. we wind up with a single block device (/dev/md0). We then chop that RAID volume up into logical volumes with LVM, so we get a bunch of block devices along the lines of /dev/xenLV/xxx. We then use DRBD to mirror block devices between physical machines, so we might get something like /dev/drbd1 on two machines, both backed by a logical volume on each machine. The Xen virtual machine then runs on top of the DRBD volume.

So, we have a disk "stack" something like:

    Machine 1               Machine 2

  Physical Disks          Physical Disks
        |                       |
RAID with mdadm           RAID with mdadm
        |                       |
       LVM                    LVM
        |                       |
       DRBD - - - - - - - - - DRBD
        |
       Xen

So, if I understand this right, I can add disk encryption by using dm-crypt on top of DRBD:

     Machine 1               Machine 2

   Physical Disks          Physical Disks
        |                       |
 RAID with mdadm           RAID with mdadm
        |                       |
        LVM                    LVM
        |                       |
       DRBD - - - - - - - - - DRBD
         |
     dm-crypt
        |
       Xen

Have I got that about right? So before I can start a virtual machine I have to start the appropriate dm-crypt device by typing a password at the command line or providing an encryption key (like an SSH key kind of thing). The dm-crypt volume will be mirrored accross to the second machine in real-time - if the first physical machine goes down I need to go and start the dm-crypt device on the second physical machine with the password or key.

Right, so: what happens if someone walks off with a server or two? If dm-crypt uses some kind of encryption key stored on a local drive on each server, what's to stop someone simply using that key to start our servers up? Is a password secure enough - is, say, a 10-character password with words and numbers going to take a suitably long time to break, or is a 128/256/1024/2048-bit key the only thing that's going to be long enough? If a key is the only solution, where do I store the key so that encrypted volumes aren't too much palaver to start but no-one who shouldn't do can get at them?

--
David Hicks

[SYNACK]

Not sure about the actual implementations as I have not played with these software packages but one thing to consider would be the performance hit. Encrypting and decrypting everything on every virtual machine is going to burn up a lot of CPU time as it must do this with the nonsensitive stuff as well like the OS.

I would suggest implementing the encryption a couple of levels higher on only the sensitive data. This will give you the security but help to minimize the CPU toll on your servers.

As to the keys I would use public/private keys 1024bit and have these password protected. I would then think of storing the keys on a micro USB pen drive for each of the people who are allowed to start it and one in the safe.

You could also possibly put a copy in a secured online storage area that could be accessed in an emergency if you needed to direct someone else that did not have the key to do it over the phone.

Just random suggestions, hope that they are helpful.

[ICT_GUY]

How much data is considered sensitive? Not a hell of a lot I bet, even as far as the teachers stuff goes, 99% is crud in their user areas 1% might just be considered sensitive.

The truly sensitive stuff is on the admin network. Besides, encrypting everything is going to cause extra hassle if ever anything ever goes mammary glands up.

[dhicks]

I would suggest implementing the encryption a couple of levels higher on only the sensitive data. This will give you the security but help to minimize the CPU toll on your servers.

Thanks, excellent point that. Actually, all our VMs use three disk images - OS, data and swap space, so I guess I just need to encrypt the data disk. I'm not sure that CPU time is really such a massive problem these days - our main file server is currently running at a CPU utilisation of 1%, it's disk performance and/or network performance that holds us back.

I would then think of storing the keys on a micro USB pen drive for each of the people who are allowed to start it and one in the safe.

Definitely an idea, although it does mean we have to go along to the server cupboard and plug a USB stick in every time we want to start a virtual machine. Did have a thought, though: we could have a small web server sat somewhere on the network, just serving up that encryption key. To start a virtual machine we just need to get the encryption key from the web server each time - easy enough to write a script that does wget key, mount encrypted volume, rm key, start VM.

"Small" webserver could mean PoE device small enough to be hidden in the roofspace somewhere. Ideally, it needs some kind of self-destruct mechanism so that if it's moved or taken out of the building it's useless... Hmm, have to think about that one...

--
David Hicks

[dhicks]

How much data is considered sensitive? Not a hell of a lot I bet, even as far as the teachers stuff goes, 99% is crud in their user areas 1% might just be considered sensitive.

Indeed - but it's sorting the 1% from everything else that's the problem :-) It just strikes me that a lot of the fuss made about "sensitive data" being stolen from various places could well mean a list of names and addresses in a Word file in someone's home directory.

Besides, encrypting everything is going to cause extra hassle if ever anything ever goes mammary glands up.

Well, yes, so I'm trying to figure out an encryption system that is robust enough to be able to handle disk crashes / power cuts / server disappearances / fires / etc.

--
David Hicks

[ICT_GUY]

They are going to have their work cut out for them at our school, the new rack server weighs in at 27KGs + another 2 KGS for disks and is going to be bolted to the wall and lock up in a secure area. Oh right hot swap bays might make it easier to run off with them doh!

[dhicks]

Right, so next question: anyone know if I want to get Xen to use a dm-crypt volume as a virtual disk, do I have to use a different keyword thingy in the Xen config file? So, for instance, when Xen uses a DRBD volume I write:

disk = ['drbd:ACSFILES001,sda1,w']

or for a standard LVM volume, Xen knows that it's a "physical" volume:

disk = ['phy:xenVG/ACSFILES001SWAP,sda3,2']

For a dm-crypt volume do I have to use ['dm-crypt:xxx'] or something, or will "phy" do?

--
David Hicks

[SYNACK]

Just something to consider, apparently:

Loop-AES is more secure than dm-crypt (and possibly faster), although it requires a custom kernel module and is more work to install than dm-crypt

Loop-AES riseup labs

Edit: Ignore that there is a reason everyone used dm-crypt:

Loop-AES used to be the only secure option for encrypting linux filesystems, it is also the most stable and mature, and possibly the fastest. All in all, it ought to be the leader of the pack, except for one thing: Through politics and in-fighting the Loop-AES developers have alienated the Linux kernel team, including Linus himself. As a result loop-AES is not supported by the mainline Linux kernel, nor is it supported by the Fedora Core 5 package manager. If you want to run Loop-AES you have to install custom versions of mount, losetup, and the loop.ko kernel module.

http://www.shimari.com/dm-crypt-on-raid/#why_dmcrypt

[dhicks]

Ignore that there is a reason everyone used dm-crypt:

I had enough trouble getting DRBD to work alongside Xen - using the CentOS package manager to install one clobbers the kernel needed for the other. In the end I wound up installing DRBD on a separate server and copying the relevant kernel module accross. The idea of recompiling kernels makes me come over all shivery.

--
David Hicks