Home NAS With Raid1

[tickmike]

I'm building a home NAS / with remote automatic daily backup from my daughters laptop while she is at uni.

 

I will use an old PenIV with two 2TB sata drives and a cheap raid card from Amazon

Neewer 4 Port Serial ATA SATA to RAID PCI Card Adapter: Amazon.co.uk: Electronics

 

It will be booted from an SD card/IDE adapter with PCLinuxOS Minime on it.

 

It will run 24/7.

 

I have used info from

Linux Software Raid 1 Setup

Create RAID 1

Linux Create Software RAID 1 (Mirror) Array

 

After 11 hours the drive md0 was formed but I can not see it in 'Dolphin'

It does show up in PCC>diskdrake.

 

Mount point: /raid1

Device: md0

Type: Journalised FS: XFS

Size: 1.8TB

Formatted

Level 1

Chunk size 64 KiB

RAID-disks sdb1, sdc1

 

 

[root@localhost ~]# fdisk -l

 

Disk /dev/sda: 7948 MB, 7948206080 bytes

255 heads, 63 sectors/track, 966 cylinders, total 15523840 sectors

Units = sectors of 1 * 512 = 512 bytes

Sector size (logical/physical): 512 bytes / 512 bytes

I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk identifier: 0x00085ba0

 

Device Boot Start End Blocks Id System

/dev/sda1 * 2048 10233404 5115678+ 83 Linux

/dev/sda2 10233405 15518789 2642692+ 5 Extended

/dev/sda5 10233468 10426184 96358+ 82 Linux swap / Solaris

/dev/sda6 10426248 15518789 2546271 83 Linux

 

Disk /dev/sdb: 2000.4 GB, 2000398934016 bytes

255 heads, 63 sectors/track, 243201 cylinders, total 3907029168 sectors

Units = sectors of 1 * 512 = 512 bytes

Sector size (logical/physical): 512 bytes / 4096 bytes

I/O size (minimum/optimal): 4096 bytes / 4096 bytes

Disk identifier: 0x00045fe6

 

Device Boot Start End Blocks Id System

/dev/sdb1 2048 3907028991 1953513472 83 Linux

 

Disk /dev/sdc: 2000.4 GB, 2000398934016 bytes

255 heads, 63 sectors/track, 243201 cylinders, total 3907029168 sectors

Units = sectors of 1 * 512 = 512 bytes

Sector size (logical/physical): 512 bytes / 4096 bytes

I/O size (minimum/optimal): 4096 bytes / 4096 bytes

Disk identifier: 0x000348a1

 

Device Boot Start End Blocks Id System

/dev/sdc1 2048 3907028991 1953513472 83 Linux

 

Disk /dev/md0: 2000.4 GB, 2000397729792 bytes

2 heads, 4 sectors/track, 488378352 cylinders, total 3907026816 sectors

Units = sectors of 1 * 512 = 512 bytes

Sector size (logical/physical): 512 bytes / 4096 bytes

I/O size (minimum/optimal): 4096 bytes / 4096 bytes

Disk identifier: 0x00000000

 

Disk /dev/md0 doesn't contain a valid partition table

 

[root@localhost ~]# mdadm --detail /dev/md0

/dev/md0:

Version : 0.90

Creation Time : Sun Jun 24 22:09:48 2012

Raid Level : raid1

Array Size : 1953513408 (1863.02 GiB 2000.40 GB)

Used Dev Size : 1953513408 (1863.02 GiB 2000.40 GB)

Raid Devices : 2

Total Devices : 2

Preferred Minor : 0

Persistence : Superblock is persistent

 

Update Time : Tue Jun 26 14:50:03 2012

State : clean

Active Devices : 2

Working Devices : 2

Failed Devices : 0

Spare Devices : 0

 

UUID : 8e1d6643:1239dd50:bfe78010:bc810f04 (local to host localhost.localdomain)

Events : 0.18

 

Number Major Minor RaidDevice State

0 8 17 0 active sync /dev/sdb1

1 8 33 1 active sync /dev/sdc1

 

 

Any help getting this going please.

[nile_c]

It's been a very long time since I've used anything Mandrake/Mandriva related, so I don't know much about diskdrake.

 

Can you run, df -h at the terminal?

 

If you can see /raid1 there, then we'll know the array was definitely formatted and is mounted. If all's well, then the next step will be to edit /etc/fstab to ensure the volume (md0) is mounted at boot. (Or perhaps you can do it in diskdrake)

[tickmike]

Hi, nile_c Thanks for replying. (I'm also just building a new 'smoothwall' box with a DMZ to connect this NAS to the internet , my last one has been running 24/7 for 6 years.)

 

[root@localhost ~]# df -h

Filesystem Size Used Avail Use% Mounted on

/dev/sda1 4.9G 2.0G 3.0G 41% /

/dev/sda6 2.5G 208M 2.3G 9% /home

 

[root@localhost ~]# mount /dev/md0

mount: /dev/md0: can't read superblock

 

 

I tried in 'diskdrake' to mount it but got

"mounting partition /dev/md0 in directory /raid1... Failed"

 

This is my 'fstab' But not sure if I did it correctly.

As you can see I also tried to use the 'uuid' but that did not work,( but edited it out when I found another post that said it could change when the raid is re-built) ..

 

# Entry for raid1 added 24.6.2012

# UUID=8e1d6643-1239dd50-bfe78010-bc810f04 /raid1 xfs noatime,rw 0 0

/dev/md0 /raid1 xfs noatime,rw 0 0

# Entry for /dev/sda1 :

UUID=09db2276-132f-4593-9e1c-e3da366b317a / reiserfs defaults 1 1

# Entry for /dev/sda6 :

UUID=9a88e253-64bc-4343-b360-c523f6085d56 /home reiserfs defaults 1 2

none /proc proc defaults 0 0

# Entry for /dev/sda5 :

UUID=3e74b30c-2f7f-44b2-b133-163e83af8f2e swap swap defaults 0 0

none /dev/pts devpts defaults 0 0

[nile_c]

Hi tickmike,

 

Good luck with your smoothie! As for the NAS...

 

df is showing it's not currently mounted, and the 'mount: /dev/md0: can't read superblock' is telling us that it isn't formatted.

So you'll need to go ahead and do that. As always with partitions and formatting, here be dragons.

 

Now you have a partition on each disk (/dev/sdb1 and /dev/sdc1) which are "real" block devices. mdraid has built your RAID1 atop both of these, and now you've got the "virtual" block device /dev/md0.

From here on we can forget about the real disks and refer to '/dev/md0'. The kernel will handle the RAID stuff behind the curtain. If we play with sdb or sdc the data gods will harm us.

 

In the simplest case you can format with:

mkfs -t xfs /dev/md0

 

Did you really want XFS? If not you'll need to change the -t parameter to reflect that.

[tickmike]

mkfs -t xfs /dev/md0

mkfs.xfs: /dev/md0 appears to contain an existing filesystem (xfs).

mkfs.xfs: Use the -f option to force overwrite.

 

 

[root@localhost ~]# mkfs -t xfs -f /dev/md0

meta-data=/dev/md0 isize=256 agcount=32, agsize=15261824 blks

= sectsz=512 attr=2, projid32bit=0

data = bsize=4096 blocks=488378352, imaxpct=5

= sunit=0 swidth=0 blks

naming =version 2 bsize=4096 ascii-ci=0

log =internal log bsize=4096 blocks=238465, version=2

= sectsz=512 sunit=0 blks, lazy-count=1

realtime =none extsz=4096 blocks=0, rtextents=0

 

df -h

Filesystem Size Used Avail Use% Mounted on

/dev/sda1 4.9G 2.0G 3.0G 41% /

/dev/sda6 2.5G 213M 2.3G 9% /home

/dev/md0 1.9T 34M 1.9T 1% /raid1

 

I'm going to re-boot now.

[tickmike]

(can not get the forum 'Edit' button to work )

 

After the reboot

 

 

[root@localhost ~]# df -h

Filesystem Size Used Avail Use% Mounted on

/dev/sda1 4.9G 2.0G 3.0G 41% /

/dev/md0 1.9T 34M 1.9T 1% /raid1

/dev/sda6 2.5G 213M 2.3G 9% /home

 

But it still does not show in 'Dolphin' file manager (should it ?), what I'm looking for is a a symbol like I can see for my 'Root' And 'Home' on the LHS of Dolphin, If I go up two levels to the / , I can see a folder(directory) called 'raid1'

it,s permissions are Root + Root and it is 1.8TB.

How are you supposed to use md0 ? .

It's a long time since I used a Raid card .

 

re.. "Did you really want XFS?" I have a live post going on the PCLinuxOS forum and one of the top guys recommended it or ex4

[tickmike]

Because of this very odd forum rule that the 'EDIT' button disappears after a time I'm re-posting because of a 'typo'

 

The last line above..

re.. "Did you really want XFS?" I have a live post going on the PCLinuxOS forum and one of the top guys recommended it or ex4.

 

Should be ..

re.. "Did you really want XFS?" I have a live post going on the PCLinuxOS forum and one of the top guys recommended it or ext4

[CAWJames]

It looks like you have mounted md0 as /raid1, so that 'directory' you can see, that 1.8tb is the personification of your RAID drive.

 

Just treat /raid1/ as the root of md0

 

James

[nile_c]

That all sounds good. It is now mounted and available at /raid1

 

As you say, modern file managers will generally have handy links for Home and Root, etc. If you plug in a removable storage device, or insert a CD/DVD these also show up and you can mount/unmount them as a regular user.

 

Dolphin isn't likely to show your raid array here, because it's already configured in your fstab and mounted. However you can navigate to /raid1 and use it as you would any other directory. The permissions might need to be made more permissive

 

 

EDIT: What CAWJAmes said! I got distracted before I hit 'post' on mine

Edited June 27, 2012 by nile_c

[tickmike]

Thank you for your help It now takes data ok.

I added an icon with a link in Dolphin.

 

/raid1 has 'root,root' permissions , being I'm going to send it data from 'Luckybackup Super User' (root) shall I keep those permissions ?.

[AdamToone]

Could you not just look to buy a Laptop backup and encryption service?

 

I'm building a home NAS / with remote automatic daily backup from my daughters laptop while she is at uni.

 

I will use an old PenIV with two 2TB sata drives and a cheap raid card from Amazon

Neewer 4 Port Serial ATA SATA to RAID PCI Card Adapter: Amazon.co.uk: Electronics

 

It will be booted from an SD card/IDE adapter with PCLinuxOS Minime on it.

 

It will run 24/7.

 

I have used info from

Linux Software Raid 1 Setup

Create RAID 1

Linux Create Software RAID 1 (Mirror) Array

 

After 11 hours the drive md0 was formed but I can not see it in 'Dolphin'

It does show up in PCC>diskdrake.

 

Mount point: /raid1

Device: md0

Type: Journalised FS: XFS

Size: 1.8TB

Formatted

Level 1

Chunk size 64 KiB

RAID-disks sdb1, sdc1

 

 

[root@localhost ~]# fdisk -l

 

Disk /dev/sda: 7948 MB, 7948206080 bytes

255 heads, 63 sectors/track, 966 cylinders, total 15523840 sectors

Units = sectors of 1 * 512 = 512 bytes

Sector size (logical/physical): 512 bytes / 512 bytes

I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk identifier: 0x00085ba0

 

Device Boot Start End Blocks Id System

/dev/sda1 * 2048 10233404 5115678+ 83 Linux

/dev/sda2 10233405 15518789 2642692+ 5 Extended

/dev/sda5 10233468 10426184 96358+ 82 Linux swap / Solaris

/dev/sda6 10426248 15518789 2546271 83 Linux

 

Disk /dev/sdb: 2000.4 GB, 2000398934016 bytes

255 heads, 63 sectors/track, 243201 cylinders, total 3907029168 sectors

Units = sectors of 1 * 512 = 512 bytes

Sector size (logical/physical): 512 bytes / 4096 bytes

I/O size (minimum/optimal): 4096 bytes / 4096 bytes

Disk identifier: 0x00045fe6

 

Device Boot Start End Blocks Id System

/dev/sdb1 2048 3907028991 1953513472 83 Linux

 

Disk /dev/sdc: 2000.4 GB, 2000398934016 bytes

255 heads, 63 sectors/track, 243201 cylinders, total 3907029168 sectors

Units = sectors of 1 * 512 = 512 bytes

Sector size (logical/physical): 512 bytes / 4096 bytes

I/O size (minimum/optimal): 4096 bytes / 4096 bytes

Disk identifier: 0x000348a1

 

Device Boot Start End Blocks Id System

/dev/sdc1 2048 3907028991 1953513472 83 Linux

 

Disk /dev/md0: 2000.4 GB, 2000397729792 bytes

2 heads, 4 sectors/track, 488378352 cylinders, total 3907026816 sectors

Units = sectors of 1 * 512 = 512 bytes

Sector size (logical/physical): 512 bytes / 4096 bytes

I/O size (minimum/optimal): 4096 bytes / 4096 bytes

Disk identifier: 0x00000000

 

Disk /dev/md0 doesn't contain a valid partition table

 

[root@localhost ~]# mdadm --detail /dev/md0

/dev/md0:

Version : 0.90

Creation Time : Sun Jun 24 22:09:48 2012

Raid Level : raid1

Array Size : 1953513408 (1863.02 GiB 2000.40 GB)

Used Dev Size : 1953513408 (1863.02 GiB 2000.40 GB)

Raid Devices : 2

Total Devices : 2

Preferred Minor : 0

Persistence : Superblock is persistent

 

Update Time : Tue Jun 26 14:50:03 2012

State : clean

Active Devices : 2

Working Devices : 2

Failed Devices : 0

Spare Devices : 0

 

UUID : 8e1d6643:1239dd50:bfe78010:bc810f04 (local to host localhost.localdomain)

Events : 0.18

 

Number Major Minor RaidDevice State

0 8 17 0 active sync /dev/sdb1

1 8 33 1 active sync /dev/sdc1

 

 

Any help getting this going please.

[tickmike]

Re.. "Could you not just look to buy a Laptop backup and encryption service?" Buy !!!! ...

[JaSauders]

I'm not sure if I can help, but I'll throw some things on the table here (some of which that were already covered) that I ran into with my Linux RAID setup... While I did this on Ubuntu, what I'm seeing from your thread sounds familiar enough.

 

Basically I created a RAID 1 mirror using mdadm software RAID built within Linux. While /dev/sdb and /dev/sdc are "clients" to the RAID, you do not have any usable file system until you format /dev/md0. /dev/md0 acts as the combined disk array... What does any disk need to function? A file system, and of course a mount point. So I popped into GParted, selected /dev/md0, and created an EXT4 file system. Once done, I had to edit /etc/fstab (as notated above) to be available on boot. Then once I rebooted, presto - it was there.

 

I did this for both my server (where the RAID 1 is mounted as /media/NAS) and my personal desktop (where the RAID 1 is mounted as /home).

 

Now, about permissions... there's 100 ways you could do this... So let's say your mount point is /media/raid1...

 

root:root 755 permissions = root owner has full reign, while root group has read/execute, and of course your user would fall under the "others" category, giving you only read/execute. But don't worry... if you create (as root) as directory underneath the mount point, such as /media/raid1/daughtersname, you could assign permissions to that folder.... and basically do becky:becky with proper read/write permissions, and she'd be fine.

 

You could also block off the array completely from other users... What I did, for no other reason than "why not", was I created a group called users... and within that group I added every system user to it. Then I changed the group of the array so it reflected users and also set myself (jason) to be owner, so overall it looked like jason:users. Then I set the permissions to be 750... That means jason had full reign, users had read/execute, and lastly NOBODY else had access. That way, if you're me (jason), you can create folders all you want right in the top level, such as /media/raid1/becky, /media/raid1/fred, etc... meanwhile, for folders beneath it I assigned permissions for that specific user so they had proper read/write permissions. That basically means:

 

/media/raid1 = jason:users 750 (jason can do anything, users can read/execute, and others have no access)

*** becky and fred were added to the group "users" ***

/media/raid1/becky = becky:becky 755 (becky owner can do anything, becky group has read/execute, others have read/execute)

/media/raid1/fred = fred:fred 755 (fred owner can do anything, fred group has read/execute, others have read/execute)

 

Then of course if you want becky's documents to be 100% locked down so NOBODY else could get in, you can set the permissions to 750 so all others were blocked out. Lots of choice here.

 

With me being owner of the main folder, it allows me to create folders all I want without needing to be root. Not to say creating folders as root is hard to do or bad or anything like that... no not at all... but I think what we're getting into here is a bit of personal preference.

 

You could always 777 the mount point as well. Foolish? Quite. But would it work? Oh yes, it would work. I think it'd be more worthwhile to just assign permissions in a way that works for you. Remember, for good security I'd aim to be as locked down as possible while still allowing your users to get their job done.

 

Now on the backup topic, I am curious as to how your daughter is going to back up. I assume you've got DDNS set up so she can point to a dynamic web address instead of a public IP address that could change over time? What operating system is your daughter using? What application is doing the actual backups? Is it going to be regular data backup, like with using rsync that just has a 1:1 raw file backup, or is it going to be an incremental backup where it does a fresh backup every so often? I would assume a university would have a decent upload speed, so she could likely be firing out documents to you without too much struggle... as long as your download is half decent (even my basic DSL is 12-15 mbps down, which I found surprising). How often is this backup going to run?

Edited July 10, 2012 by JaSauders

[tickmike]

@ JaSauders

Thanks for taking the time to help me, your post is very helpful.

 

We use PCLinuxOS ( three 'w' dot pclinuxos.com) at home, also my daughter has it on her laptop and is quite at home with it (I made it dual boot but she does not use windows !).

 

I have 'Luckybackup' installed on her laptop and it will run a 'cron' job at boot and with a ' script' it will only do one backup per day (no matter how many times she boots it up ).

It will run a 'pass word less' 'ssh' connection (with the uni's permission) over the internet to one of my fixed IP addresses, via my 'hardware firewall DMZ' (smoothwall) to my backup server.

 

Today I have been testing my new smoothie box (but not tested the DMZ yet !) and have run for the first time the first backup of my daughters laptop home directory over our home network to the 'backup server' and all seems ok.

 

Next stage to follow your advice to set up the permissions and directory's on the raid1, test the DMZ, set up the ssh ports and security.

 

Do some more tests with backups and also some file recovery from the server.

The uni IT manager has said that I can take the laptop with us on our next visit before she starts in September to do some 'live' tests which will be useful.

[jamesfed]

Phew all I did is install SkyDrive on my girlfriends notebook and shes been doing great with it.

Not a single script in sight

[Arthur]

I will use an old PenIV / it will run 24/7.

Your electricity bill is going to be huge!

 

Depending upon the spec, the PC is going to cost you £100-£200+ per year to run!

[tickmike]

re , "Your electricity bill is going to be huge! " Yes I had thought about that and may have it running for say six hours in the evening , but my Smoothwall firewall and router are running 24/7 as well. !