User home folders being emptied randomly

Hi,
I've inherited a real mess of a network and, amongst all the other things going wrong in new employment and ripping out all the Comms Cab patching and doing it again, I've come across a real head scratcher.

I'm running a mainly 2003 Server environment with one 2008 box as a member server. I have three 2003 DCs, two of which are cluster nodes (don't ask my why because I do not know). The cluster has iSCSI ports to a SAN where all user data, profiles, home folders etc are stored.

My problem is that, randomly, a user will log on and access their home folder (which is mapped to the S:\ drive in AD) and all their files and folders will be gone. When I look on the SAN via the Cluster, the home folder looks like it is a new User folder which automatically gets 4 folders created when they first log on. I will restore the files using Previous Versions and then a random amount of time later it will happen again. If the user keeps themselves logged on overnight, they do not lose their files. If they log off then, every now and then all the files will be gone.

I cannot link this down to a single OU problem or a specific group of workstations. There are no patterns it is just completely random. Many may shout at me about having DCs as Cluster nodes is a BAD IDEA but I know at least four other Network Managers who have a similar environment but whose users do not lose data.

If anyone can shed any light on this or help me out in any way I will be really grateful because I am one step away from ripping the Cluster to pieces and starting again and if I can, I really want to avoid doing that if at all possible!

Many Thanks,
Al

Is there no-one out there who has either had this problem or who has any clue as to the cause?

Cheers,
Al

I think I'm getting somewhere with this. I've worked out that each time a user loses their files, the log on via a particular DC on the Cluster. Now this server cannot see the iSCSI connections to the SAN but I'm not sure if this would cause a problem. Does anyone out there know Cluster Services inside out alongside SAN iSCSI connections?

Cheers,
Al

Hi

I am stabbing in the dark here a lot but.

Is it a connection problem or is it that group policy is not applying. We had problems when we had a lot of broadcast traffic and bad packs running round the network and group policy did not apply correctly as the switches and nic cards were being overloaded.

Have a look at the event logs of the servers and see if anything shows up. Check the event logs of the client machine and see if that gives you any clues.

Richard

The shares on the ISCSI SAN, should just appear as a normal share to non-cluster devices. Usual you give the cluster a name and anything on it a name too. So the fileserver on the cluster maybe called "clusterfileserver1" and this name will remain no matter what node it is attached to. Essentially when you get a file from the cluster share, your client device contacts the node that it is currently on, and the node then talks to the san. It is best practise not to have cluster installed on a domain controller, so this should not be causing the issue you have.

Hi Ricki,

There are a lot of ARPs coming from both our Sharepoint server and Application server but the company whom we pay to support our AD and switch config say that it's fine even though I'm sure something keeps surging the network randomly though the day.

Will Post back what I find but I'm pretty sure now that the issue is the fact that one of the CLuster nodes cannot see the iSCSI harddrives in My Computer..


Thanks,
Al

Quote:

---

Originally Posted by TronXP

The shares on the ISCSI SAN, should just appear as a normal share to non-cluster devices. Usual you give the cluster a name and anything on it a name too. So the fileserver on the cluster maybe called "clusterfileserver1" and this name will remain no matter what node it is attached to. Essentially when you get a file from the cluster share, your client device contacts the node that it is currently on, and the node then talks to the san. It is best practise not to have cluster installed on a domain controller, so this should not be causing the issue you have.

---

Hi Mate. I get the basics of how the Cluster worked with other Members servers on the network, it's the two cluster nodes (DC02 & DC03) that I'm puzzled with. On DC02 in My Computer, all the iSCSI drives are listed (one being UserData) in DC03 only the local HDD of the Server is listed, not the iSCSI drives. I know that DCs shouldn't really be in Clusters but this is the system I've inherited (only been in role 4 weeks, baptism of fire or what!). On the up side, going full VM in Summer, but got to fix this problem as we cannot carry on like this til Summer!!

Thanks for your help so far!
Al

Have you checked the ISCSI initiator settings in control panel? to see if it is seeing the SAN on DC3?

Yes mate, and it is. Both the iSCSI IP addresses are listed in there as being connected. I have just forced a client to Authenticate with DC03 with lmhosts file and the problem appeared immediately so I know this is the problem now. Is there anyway to stop DC03 authenticating users but still load balance between DC01 and DC02?

Thanks for all your help so far.
Cheers,
Al

Quote:

---

Originally Posted by ALR

Yes mate, and it is. Both the iSCSI IP addresses are listed in there as being connected. I have just forced a client to Authenticate with DC03 with lmhosts file and the problem appeared immediately so I know this is the problem now. Is there anyway to stop DC03 authenticating users but still load balance between DC01 and DC02?

Thanks for all your help so far.
Cheers,
Al

---

demote it to a member server?

Cannot demote it, it is a Cluster node, there is too much risk of losing data, especially being as the previous guy messed up the Back-ups and no back-ups have been taken for about 4 months!

First things first, take a backup...

Then i might be confusing myself here, but don't you use MS clustering for fault tolerance as well as load balancing, so shouldn't you just be able to switch server #03 off and the other 2 nodes continue on regardless?

I have already ordered a NAS box and I am backing up to this being as the tape backups are complete screwed. This was one of the first things I did when I took role. The previous Manager also decided not to set folder size limits so, currently, the staff are using over 2TB of space and the students almost 1TB, so backing up is a nightmare, even with a 16 tape Quantum (which doesn't work proerly anyway). Secondly, there are only two nodes in the Cluster, DC02 and DC03 and in theory DC02 should carry on regardless if I pulled the plug on DC03 but "the powers that be" will not allow me to take DC03 offline because one of the "IT Consultants" from higher up said data may get corrupted. My experience and certifications don't seem to count for nothing by the sounds of it...

This is why I'm trying to find a way to stop users authenticating with DC03, hence stop the Home Folder problem, but keep the server online until said powers that be allow me to trash the damn thing and start again.

I love my job, I love my job, I love my job.....

Stop the net logon service.

How to Prevent User Validation on a Specific Domain Controller

It's been a while since I updated this thread but I'm happy to say we have found the cause at long last!!

The issue was the permissions on the home folder share. All our staff members home folders were being redirected to \\server\users$\staff, but all our staff accounts had Full Control of both the Users$ share and the staff share. I removed these permissions, then renamed their home folder to "username.old", in the user account settings in AD I changed the home folder setting to "%username%", thus allowing Server 2003 to create the home folder itself rather than manually creating it (as the ICT staff here were doing) and, after getting the user to log on and re-create the home folder and then moving all the data across, this seems to have solved the problem.

If Server 2003 creates the home folder itself, it doesn't replace it at logon as it was doing. This gave the impression that the home folders were emptying themselves when, in actual fact, Server 2003 was replacing them with new empty ones. Removal of the permissions to the root share and setting the home folder up correctly using %username% has rectified the problem!

Wanted to update in case anyone else had this problem as I have never seen this before...

Many Thanks,
Al