Backup deletion takes 12 hours?
Had success with a full backup and a number of incrementals. Now that they are > 14 days old, I'm expecting a new full. It seems to know that's due, but spends 12 hours deleting the old backup sets. Looking at disk activity, it's reading/writing constantly during this 12 hours - on the destination media.
I chose Simple as the method.
Hello,
This is most probably consolidation yes, but 12 hours seems excessive. Could you clarify the size of your archive and the exact retention rules you configured?
Thank you.
Trying to imbed image not working here. I get the dialog, upload the file, but insert file does nothing...scratching head. Attaching instead.
The retention is just 15 days on all the jobs I have at the moment. Below is for a different drive but with the same settings.
Also attached a listing of the backup folder for Win10Main. I kept getting out of space so I chose cancel, wiped out the vault and started the backup again. The full backup completed 5 hours later.
I have a much smaller backup that's only 50Gb - drive N file listing also attached. It's almost reached the retention limit. According to the log, last night the backup completed for N: in 2m 32s and the "cleanup" took 1h 14m.
I guess consolidation is automatic and can't be controlled or avoided?
Hello,
Using the "Simple" schema with incremental backup type means that all backups are always incremental.
This means you physically cannot delete a backup slice without breaking the Full-Incremental chain.
Imagine this scenario: your archive consists of F-I-I slices. You create a new Incremental slice and the Full slice is set to be deleted by retention rules. If you simply delete it, you will now have I-I-I slices, or three incrementals without a full or completely uselss incrementals that can't be used to restore data.
So with the Simple-Incremental schema, consolidation is used because there is no other way to remove old data.
With a Simple-Full schema, fulls are created everytime so backups are always deleted instead of consolidated. If you have multiple fulls, you can remove any number of them without affecting the rest.
If you want to have a backup schema where old backups are deleted but still use incrementals, you have to use something other than the "Simple" Schema. For example, you can create a full backup every weekend and incrementals during the workweek. So you will have, for example, the following backups on your disk: F-I-I-I-I-I-F. When it is time to delete the first full, it will be marked for deletion (but not removed yet): F-I-I-I-I-I-F. Eventually, all the dependend backups will be marked for deletion: F-I-I-I-I-I-F-I-I-I-I-I
Once this happens, the whole first chain is removed from disk (because there are no dependend backups) and you are left with the original size F-I-I-I-I-I chain.
Good explanation! That explains what I'm seeing.
When I think of consolidation, I imagine many files being consolidated into a single file. But after the consolidation had completed, there were still what appeared to be the full and many incrementals lying around.
Given the time consumed in consolidation, I can't imagine a scenario where it would be useful. Unless in a remote scenario where another system running an agent does the work - thereby avoiding the need to do perodic fulls across a wire.
The last example you gave is what my brain considers logical and simple :-). That's what I'm used to with TIH. But I'm a nut for log files and history - hence the migration to ABPC.
Ataching another image that presumably shows a consolidation in progress. The backup completes in 23 seconds. After 6 hours, the cleanup was at 100%. Looking back at the logs I can see that cleanup finished after 12h 32m. I can only assume that this "cleanup" is a consolidation.
Since none of my full backups take more than 5 hours, I should probably avoid methods that invoke consolidation.
Looking at your example of the Full followed by incrmentals. In the case where I have F-I-I-I-I-I-F, are you saying that the first F and dependant incrementals will not be actually deleted until the last incremental reaches the retention trigger?
I had always thought (and observed in TIH) that upon complettion of a full backup, the last full and all incrementals were deleted. Maybe ABPC is different in that perspective.
Hello,
Yes, no backup is deleted until all dependeded backups are marked for deletion. What I described above is the case if you set the retention to 5-6 days.
If you set it to one day, every backup will lead to the previous day's backup being marked for deletion (you can still use them for restoration). In this case, as soon as a new full is created, the previous chain will all be marked for deletion in its entirety and removed, resulting in just the new F.
Hmmm, let me see if I understand ABPC incrementals.
TIH logic in my head - was thinking that I'd just do incrementals with 7 day retention that deletes old backups. The first incremental is automatically a full. So far have nearly 30 days of incrementals. Thought the old ones would be delted after 7 days and another full would automatically.
NOW thinking that I need TWO schedules for the job. A full say on Monday with incrementals all other days - with the same 7 day retention logic. Just set that up last night - attached below. So I guess next Monday I'll get another full and seven days after the last incremental of the first full all of that set will be deleted?
Hello,
Yes, the previous full will be marked for deletion after the next full backup.
BUT. The previous full AND all dependents will not be actually deleted until 7 days after the last "dependent" incremental?
I'm trying to develop a strategy since I'm limited on space. One that will do a full followed by incrementals for some time period. Then, just before the next full kicks off, wipe out all prior backups - not just mark them for deletion.
With TIH in the past, I've written scripts to delete backups that were beyond a certain age. ABPC seems more sensitive to having those files removed behind its back. The result being the UI is not capable of representing what backups are actually there.
Hello again,
You can refresh the UI by pressing the "refresh" button on the vault (this will reload the current meta-data).
As for the scheme, if you have space for 2 full backups, I would recommend setting the retention to 1 day. This will remove the older full and all it's dependends right after creating the new one.
Thank you.