Backup via Internet - best practice
Hey there,
I hope that the collective intelligence helps me in what I intend to do.
Infrastructure
Site A:
- physical Windows2012R2 server with 2 fileservers
- Backup Advanced 12.5 installed on the machines
- 2 NAS as backup targets (BUT1 and BUT 2)
Site B:
- 1 NAS as backup target (BUT3)
Between the sites is a secured VPN with nominal transfer speed of 20Mbit.
The sites are connected to the internet via cable with 100Mbit downlink and 20Mbit uplink, so the nominal speed for a transfer between the sites is 20Mbit.
Status Quo
urrenty I run backups from my fileservers to BUT1 and BUT2.
The retention rules are set to keep full, differential and incremental backups in a suitable manner.
Plan
To increase data security I want to shift one backup to an external site, site 2.
Questions
a) Detect changes
If the backup starts and an incremental backup should be done, must Acronis read the data in the existing backup file to find the changes or how does Acronis detect whether there is a change?
b) Speed
Which scheme is faster?
If I use the agent on the fileserver on Site 1 that uses BUT3 as storage location or if I install an agent on site 2 that uses the net shares from site 1 and uses BUT3 as storage locations?
In which scheme is the lower data transfer?
c) compression
Does higher compression helps in my case to reduce the necessary time?
d) best practice
What would be the best practice to do the planned job with as less data transfer via the VPN as possible?
e) validation
As mentioned I have machines with agents on both sides.
Is it possible to use an agent on site 2 (that is not registered to the management server on site 1!) to validate the backups?
Or must all agents be registered to the same management server?
Thanks in advance for any help
Sven
I'm not sure if I understand your use case correctly, so I have a few general questions to answer more thoroughly:
- Are you using a file backup or the entire server?
- Do you want to move or replicate backups to Site B?
- Do you want to move/replicate all or only some of your backups at Site A to Site B?
- Do you have a storage node with those NASes as location or are they unmanaged?
- If you have a stroage node are the locations deduplicated?
- Also which backup format are you using?
a) Detect changes
If the backup starts and an incremental backup should be done, must Acronis read the data in the existing backup file to find the changes or how does Acronis detect whether there is a change?
I'm expecting that the CBT tracking(A setting in the backup plan) eliminates the need to read existing backups whatever the target location is. If the backup target is deduplicated though, it would send even less data while querying a storage node for hash values.
b) Speed
Which scheme is faster?
If I use the agent on the fileserver on Site 1 that uses BUT3 as storage location or if I install an agent on site 2 that uses the net shares from site 1 and uses BUT3 as storage locations?
In which scheme is the lower data transfer?
I'd separate the backup and replication and create separate plans for each. Then set up a storage node for each site and have Site-A replicate a deduplicated vault to Site-B's deduplicated vault. This way the backup can work as quickly as it can without using VPN bandwidth, and you only send unique data to Site B (plus some overhead of querying the target vault)
If you were thinking that agent on site2 could backup a net share to BUT3 I don't think that's possible. (not disk level backup anyway)
c) compression
Does higher compression helps in my case to reduce the necessary time?
Do you mean the neccessary time to transfer data or time to complete the backup?
If minimizing VPN traffic is a priority I'd set a higher compression, how high would be up for testing since it will slow down making the backups.
d) best practice
What would be the best practice to do the planned job with as less data transfer via the VPN as possible?
See b)
e) validation
As mentioned I have machines with agents on both sides.
Is it possible to use an agent on site 2 (that is not registered to the management server on site 1!) to validate the backups?
Or must all agents be registered to the same management server?
I'll test this tomorrow. Maybe if the target is an unmanaged network share it's possible. If it's managed it probably can't be done, becase the site 2 agent would only be able to validate a location known to site 2's AMS and a location has only one owning AMS.
Is there a particular reason you want them to be separate? Maybe there is an alternate solution.
-- Peter
Hey Peter,
thanks for your reply.
Péter Szatmári wrote:
- Are you using a file backup or the entire server?
- Do you want to move or replicate backups to Site B?
- Do you want to move/replicate all or only some of your backups at Site A to Site B?
- Do you have a storage node with those NASes as location or are they unmanaged?
- If you have a stroage node are the locations deduplicated?
- Also which backup format are you using?
So let´s start:
1. the regarded backups are file level backups.>
2. I want replicate them to site B to have 3 copies: BUT1/2 on site1 and one on BUT3 on site 2 to have one offsite.
3. Only the filebackups have to be replicated on daily base.
The server and machine backups are replicated via 3 rolling HDDs that are transported between site1 and site2 as the data amount is much more.
4./5. No I have no storage node as I made extremely worse experience with it since ABA12.5.
I gave the storage node 3 tries but each ended in non working backups and each time several hours and sessions with support, log files and so on and the result was always the same:
Support has no explanation and the backup chain was broken and no restore was possible....
So back to unmanaged locations.....
One of the big drawbacks going from 11.7 to 12.5 as in 11.7 everything went fine....
6. I use Version 12 format.
The other points you mention:
I do not want a storage node for the upper mentioned reasons.
Keeping this in mind, what would be your preferred scheme?
S.
Hello Sven!
Sorry for the late reply, it took some time to setup another acronis environment to simulate 2 sites.
Based on these tests I'd suggest the following:
- If BUT3 is accessible as a network share from SiteA, have SiteA replicate backups at off peak hours to BUT3. This way you can still leverage retention rules over a NAS sync.
- I guess you could configure the reverse as well: Have SiteB agent replicate BUT1 and BUT2 to BUT3.
- Would make sense to keep the first option though as this way you'd have to have the replication plan at SiteB, although you'd also move the replication load to SiteB.
- I checked that you can verify these replciated backups with an agent that's registered on SiteB.
-- Peter
Wow thanks a lot for your tests.
That helps me a lot.
What I figured out in the meanwhile is, that the NAS have the possibility to transfer data between the sites via rsync.
So I think I will also try the "plan B" to do the data replication with the "on-board" features of the NAS and then validate the transferred data on site B and eventually apply retention rules with a site B´s agent.
What I know up to now:
This replication via WAN is okay for fileserver-data that does not exceed 100Gig / day.
It is not suitable for replication of the machines-backups.
The full backups of the workstations (once per week) are about 300Gig.
Maybe the servers can be backupped in that way....
The backup of the servers (once per week full, then Diffl) has about 100 Gig for all...
Regards
S.
Hello SvenT,
thanks for posting this topic on Acronis forums!
Even though you have sorted everything out with the help of Péter, I should also emphasize the main advantage of using off-host data processing plans - you can create separate replication, validation and cleanup plans with Agent installed on Site B (you will not be required a separate license in this case since you will not back up NAS on BUT3) - which will allow you to do these operations locally on Site B (however, what concerns the speed, it should be tested prior whether the replication plan should be run by the Agent on Site A or Site B).
Of course, using higher compression reduces the amount of the data transferred to Site B.
Hello Maria,
Maria Belinskaya wrote:[...]
you can create separate replication, validation and cleanup plans with Agent installed on Site B (you will not be required a separate license in this case since you will not back up NAS on BUT3)
[...]
Do I understand it right, that an agent that does not run backups but only replication, validation and cleanup jobs does not need a license?
So I could run a virtual machine that handles only these tasks and this machine does not need a license but can be managed by the central Management console?
Regards
Sven
Hello Sven!
Do I understand it right, that an agent that does not run backups but only replication, validation and cleanup jobs does not need a license?
I've never noticed this either, but apparently it's true. The storage node I have is regularly running validation and replication tasks, yet this agent doesn't appear in the licence list.
So I could run a virtual machine that handles only these tasks and this machine does not need a license but can be managed by the central Management console?
I also have my storage node installed on a VM and is working just as you described.
-- Peter
Hmm this opens some new usecases..... ;-)
Currently the machines itself handle the replication, validation and retention stuff and therefore some of them run for hours to finish this.
As you may know the "command after backup" cannot be used to shutdown a machine as the other mentioned tasks take place afterwards...
But with the new knowledge it might be an option to run only the backups on the machines itself and all other stuff is done afterwards by a seperate machine.
I assume that replication, validation and retention rule tasks are executed one after another?
Regards
Sven
Yes you are right, offhost processing exists for this reason. Backup whenever possible then do the other time consuming tasks at night or other offpeak hours.
I assume that replication, validation and retention rule tasks are executed one after another?
I seem to remember finding a detailed article about replication and cleanup that stated replication and cleanup involves validation to ensure consistency. It's a shame I can't find the source of this information now. Replication task can also incorporate retention rules into itself.
The plans itself are quite independent (each running on their own schedule), but there's no obvious indication they will be run in a defined order if they overlap.
Hello SvenT.
Do I understand it right, that an agent that does not run backups but only replication, validation and cleanup jobs does not need a license?
Yes. Licensing of Acronis Backup is based on the number of backed-up physical machines and virtualization hosts. Other operations do not require licenses.
I assume that replication, validation and retention rule tasks are executed one after another?
You can schedule these tasks as you wish. Please note that the off-host data processing plans run according to the time settings of the management server machine.
Maria Belinskaya wrote:You can schedule these tasks as you wish. Please note that the off-host data processing plans run according to the time settings of the management server machine.
I know that I can schedule the tasks.
The question was slightly different:
Currently I have all tasks executed by the agent and the sequence is :
- backup to first location
- copy backup to second location
- retention rules at first location
- retention rules at second location
- validation at second (!!) location
- validation at first location
If I configure the machines with agents in a way that thay only backup to the first storage location I have the remaining tasks to be executed off-host.
So first I need a replication plan and then a validation plan.
The question now are:
- If I have several replication plans can they be executed in parallel by one host or is it the same as for a backup plan where an agent can only execute one backup plan at a time.
- How can I control that the validation plan starts after the replication tasks finished?
Regards
S.
Hello Sven.
So first I need a replication plan and then a validation plan.
It is better to double the validation: 1) on Site A to enable Validation (in a Backup plan) right after backup creation on local NASes 2) on Site B after the backup been replicated to BUT3 as a separate off-host mini-plan.
Here is a good feature request to add Validation option in Replication plan (as in Backup plan). Thank you for this idea. I will forward it to the respective team.
If I have several replication plans can they be executed in parallel by one host or is it the same as for a backup plan where an agent can only execute one backup plan at a time.
I will post a reply after consulting with experts.
How can I control that the validation plan starts after the replication tasks finished?
We suggest that you schedule validation and replication operations at different time. Please calculate it so that validation starts after replication because validation takes almost as much time as backup.
Hello Maria
It is better to double the validation: 1) on Site A to enable Validation (in a Backup plan) right after backup creation on local NASes 2) on Site B after the backup been replicated to BUT3 as a separate off-host mini-plan.
This is okay for the 2-site stuff thanks.
But maybe I was not clear enough: what I described is the status-quo with no replication to site B.
So there are several machines that run agents to backup the machines.
The current scheme with backup, replication and then 2 times retention and 2 times validation has the big disadvantage that I cannot use the "command after backup" as this command is executed right after the backup.
A shutdown command must then be stuffed with a delay that must be "estimated" to guarantee that the shutdown does not occur during the tasks.
For me as an admin, "estimation" and "calculation" of times is not professional as I typically work with reliable facts and deterministic things.
And a few dozen machines that cannot be shutdown are also an energy issue.
If I backup workstations in that way and can execute the shutdown one hour earlier (as I do not need any margin in my estimation) that is a sum of about 200Euros of energy cost...
Here is a good feature request to add Validation option in Replication plan (as in Backup plan). Thank you for this idea. I will forward it to the respective team.
This would be perfect as then the staging of replication and validation is deterministic and reiable.will forward it to the respective team.
If I have several replication plans can they be executed in parallel by one host or is it the same as for a backup plan where an agent can only execute one backup plan at a time.
I will post a reply after consulting with experts.
I will be on standby for the answer. Thanks in advance.
Why is there the limitation of "one task at one time" ?
For me in the age of multicore machines, very fast HDD/SDD and also LAN connections there is no explanation why a host cannot handle parallel replication or validation tasks.
Parallel Backups might be problematic if one host backups its own data.
Maybe the developers at Acronis pay more attention on performance and using the ressources (LAN, cores...) and not so much in fancy GUI or web based consoles..... ;-)
We suggest that you schedule validation and replication operations at different time. Please calculate it so that validation starts after replication because validation takes almost as much time as backup.
As mentioned above estimation is not very reliable and I would highly appreciate some kind of possibility to control the sequencing.
Other tools support this by making tasks dependent on each other - why does a tool like ABA (that claims to be a professional tool) does not support sequencing ?
My long-term-plan is:
- each workstation does only do the initial backup to BUT1 (a QNAP NAS) with retention rules on Site A and then shutdown after the backup
- one virtual machine per LAN-connection on my NAS does then the replication and cleanup to BUT2 (also a QNAP NAS) on Site A,
- after that the validation is done (as currently it cannot be combined with the off-host replication) also by the VMs
- than the backups on BUT1 are replicated to BUT3 on Site B and are also validated there off-host.
One VM / LAN to benefit from the multiple LAN ports of the 2 NAS that are aggregated and can handle multiple data streams.
But this plan only works if I have a possibility do have a proper sequencing.
Regards
Sven
Hello Sven.
If I have several replication plans can they be executed in parallel by one host or is it the same as for a backup plan where an agent can only execute one backup plan at a time.
These replication plans will be executed consistently. If even they could be run in parallel on multicore machines with very fast HDD/SDD, it would cause a bottleneck due to network bandwidth capacity limitations. Otherwise, these plans would be executed extremely slow.
As mentioned above estimation is not very reliable and I would highly appreciate some kind of possibility to control the sequencing.
I suggest that you test it in your environment to know the exact numbers.