High I/O at the end of a backup cycle
Hello,
We have the following issue. During a backup of a machine we observere extremely high I/O on it. The backup start on arund 5:15 and starts with deleting the old backups. Around 7:00am the Block reads are increase a lot. This is the timeframe on which the Backup is writing to the Storagenode.
Do you know is there an Option to Limit the taken I/O from the operating System for the Acronis process?
On this machine we have additional a program running which uses a Round Robin Database (RRD). This writes every 15minutes around 600000files (45Gb) into the System. During this backup cycle the writing of the RRD file increase 10times. Normally it takes 120seconds and during a backup Job is running it increase to 1200 seconds.
Is there a way to reduce this high I/O for a backup Job?
Below you can see a graph:
br
Michael
Hi Michael,
The first thing to check is "Performance" option in Backup Options for particular backup plan where you can set the process priority and network bandwidth which can be consumed during backup (since you're backing up to Acronis Storage Node, this option shold be applicable).
Another potential reason is interaction with Acronis Storage Node (ASN) and applying retention rules in particular which consume resources from the Agent side since it implies severe manipulations with existing data in the deduplicating storage. To overcome this you can do the following:
1) Disable retention rules in existing backup plan applied to problematic machine.
2) In Acronis Backup web console interface go to Plans->Cleanup and create a dedicated cleanup plan which will run on its own independent schedule. Remember to set the agent to execute this plan, different to agent running on problematic machine. Typical setup would be to use Agent for Windows installed alongside ASN, e.g. choose the agent which is running locally with ASN.
If the above doesn't help then the issue should be investigated with help from our support team.
Thank you.
Hi,
Thanks for your comment.
The Retention rule is configured that the deletion of old backups will start before the backup is executed.
During the delete process we see no Performance issue. If you match the graph we see that the the I/O increase after the inital backup write process starts. The deletion of old backups starts before and during that we have no Performance issue.
Any other hint?
Thank you for the clarifications - I probably misread the first post.
Another thing to try is to use 2-step backup method, where 1st send the backup to some temporary storage (e.g. some network share) and then create a separate replication plan in Plans->"Backup replication" which will use this temporary storage as source and ASN as target destination (you can map it to existing archive). This new backup replication plan can be executed by Agent for Windows running on ASN machine.
Important (!) when creating backup plan for the source machine (which will point to temporary storage as backup destination), make sure to change Backup Options->"Backup format" to version 11 or otherwise replication from this archive to ASN would fail, since ASN can work with version 11 format backups only.
The above approach would exclude ASN-related impact if such impact actually exists (e.g. if my guess is right).
Thank you.
Does it mean that the High I/O is related to replication only?
We first do backup on 1st storagenode and the replicate it to a second storage node.
br
Michael
The high I/O could be related to Agent->Storage Node communications, especially if agent needs to replicate data from 1 ASN to another - this would add more impact since the agent needs to first unpack the data while reading it from 1st ASN, then pack it back (dedup) before sending to 2nd ASN.
Thank you.
Hi,
So does i understand it correct?
On the first storagenode i create an additional volume. in the Initial plan i configure the backup plan to use this volume as a backup Destination.
Then i add a second plan which is running on the storagenode itself and put this volume on which the backup reside to first and 2nd Storage node.
br
Michael
Would it be easier to deploy 2 Backup plan for this Machine?
Backup plan 1: Machine backup to storagenode 1
Backup plan2: Machine backup to storagenode 2
Then this Performance issue with this replication is not taking effect, isn't it?
br
Michael
Hi Michael,
Both options you listed should reduce the performance impact, e.g. either 2 backup plans (each to different Storage Node) OR 1st backup plan to Storage Node 1 and 2nd separate backup replication plan for replicating from Storage Node 1 to Storage Node 2 (this plan is executed by agent running on Storage Node 1). The 2nd way would be actually better since the original source machine won't be read twice.
Thank you.
Thanks for this info.
Can you give me a hint how can i replicate a alredy written Archive from storagenode 1 to storagenode with an Agent on storagenode1?
I configure a nfs share a different machine with a different Agent. The backup is created there as a dedicated Directory. But if i create a plan then i can use this Directory for backup it to a second replication. But if i would like to restore it how can i Access this in the same Kind of other backups which are stored on the storagenode.
Hi,
You should go to Plans->Backup Replication and create a new plan. There you'll be able to browse existing archives after selecting the agent which will be used for backup replication. So in your case the "Items to replicate" should be Storage Node 1 (entire location or selected backup archives) and "Destination" would be Storage Node 2:
The backups after replication should be browsed from "Backups" section which lists all locations from all the agents registered on the management server.
Thank you.
Hi,
Thanks i get it now!
So i think this would be the best way to reduce the high I/O on this machine i mentioned at the beginning.
br
Michael
Hi,
Just one question. If i select "all backups" to replicate and already 10 backups are replicated it will replicated the only the newer backups right? Or does it replicate always the full set of backup? Otherwise we have to set the Option to replicate each day the lastet backup.
The replication engine takes into account existing recovery points in the target location, so it will replicate only the difference. To confirm you can select to replicate only one existing archive for example:
1) Select to replicate Backups instead of entire location:
2) Browse and select specific archive:
Thank you.
Hi,
In the options of a backup plan you can specify the compression level of a backup. Does this option have an impact on the I/O consumption of running agent?
br
Michael
Hi Michael,
Yes, compression option does effect the I/O consumption but only when compression is set to "maximum" level. With default "normal" level the impact from compression should be minimal.
Thank you.