Veeam Replica Testing
I've been given the task of documenting how to use replicas and test them to ensure our servers are being backup up correctly. The reason being that when we my boss is not so sure that we are doing it correctly. We have been creaing replica's and have tested them by bringing them up and ensuring they functioned in a sandbox environment. However, we have since discovered that it has caused some corruption on some of our replica jobs. We have re-created all of them now, but we need to document how to test properly and ensure we don't make a similar mistake in the future. I have been told that we should be booting the replica's in Veeam in order to test them and I have also read an article Testing Veeam Backup about taking a snapshot before the test to restore the replica from.
So can any of you guys tell me how you do it or point me in the right direction about how we should be testing our replica's or any good resources I can use?
Personally What I do when doing DR tests with Veeam is essentially what is mentioned in that Blog post.
1. Disable any Veeam job which may run during the process of testing (this depends if you are doing hourly/daily replicas).
2. Ensure any Replica is in a Sandboxed network.
3. Create Snapshot of any VM's to be switched on.
4. Switch on VM's and perform tasks required to verify integrity.
5. Shutdown Vm's and delete snapshot that was created in step 3.
6. Enable Veeam jobs.
What gave you the impression it corrupted your Veeam jobs?
Sorry it wasn't the jobs that was corrupted, but the replicas. It was noticed when one of the other techs tried to do a full test & restore. Although when it was brought up initially it looked ok, we found later that there were errors. This may have been due to not creating a snapshot first. We were told that unless you boot the replica up in Veeam it can cause corruption in the replication. As a result we were all asked to created our jobs again in case someone powered on the server without doing it through Veeam.
What I've been tasked with doing now is documenting a full DR procedure so I'm starting with how to test the replicated servers and then need to progress onto restoring from a replicated server. From my understanding I can do this ok if I follow your steps above, but if I don't do the snapshot then there is a possibility of errors being created on the replicated version.
So I can now. Create replica. Test (as outlined above). Once I'm happy with the replica I can recover this to the live network and replace the existing server.
that sounds right if the snapshots were not created before - veeam would get confused with the Change block tracking errors as they hosts would not be consistent.
I can get you an example Veeam dr test document if that helps? I am actually re-writing a process this afternoon for one of my clients.
Just one thing to mention - do you also do veeam backups along with Veeam replica's?
replica's alone do not provide sufficient backups.
An example would be great. We currently do replicas of the server and backups to tape. We do not currently do Backups through Veeam. From what I read previously I thought they both did similar thing but changed the way you recovered them? I take it this is not the case then?
Ah ok, well you may or may not know, veeam backup to tape is coming in the near future.
Originally Posted by penfold
Personally I setup the following with my clients.
1. Veeam backups to disk.
2. Veeam replica's (off site)
3. Put the veeam backups into tape
We use Mozy pro to put critical data in the cloud also
It does have a different way to backup servers but you can do Instant restores through the system and also slower host restores.
I've just tried the procedure of creating a snapshot and reverting back but when I try to revert back to the snapshot I have taken I get the following error.
"A general system error has occurred. Could not delete change tracking file". As a result, I can't revert back to the snapshot I've taken. Any ideas?
There is no veeam jobs running?
I would stop all Veeam jobs, go into services stop them all and make sure there is no veeam processes running. then try and delete it again. Could be a process still running in the background.
Else I would raise a case with Veeam/Vmware.
OK Thanks. I've had an email to say someone is coming in tomorrow so if we are still having problems I'll raise it with them. Thanks for pointing me in the right direction.
A quick update on this. The problem I had with the snapshot error has been fixed in an update, apparently we had issues with the versions of the software we were using. I was then able to test a VM using snapshots and bring the replica up and discard any changes I made at that point. We are now on 6.5 and I have read that you don't need to do snapshots anymore but I'm looking for some good documentation to take me through creatation of replicas and full testing. I've been on the Veeam Forums but as there is lots of information there I was hoping someone could provide me with a link to somthing they have used and save me hours of searching?
@glennda did you ever get that DR test document done? I'm picking this up after a couple of weeks on leave, and I can see that the upgrade from our old version 5 to 6.5 has brought in some changes. We do not currently do backups but I think that was because it didn't work. In all honesty I think the way we have been using it previously was wrong. Some of the things we thought we should be able to do were failing because of the version we were using even though we were told they would work.
As a result I need to check to see if we are doing the right thing now and if there is an easier way that what we are currently doing. At the moment with Veeam we only do replica's.
Originally Posted by penfold
I do indeed - I require to remove my customers details first but if you PM me an email address I will get something sent over.