"Store no more than '1' Recent Versions" left 2 versions

Coyote wrote: "About once a week set Backup Method to Full; in between I switch to Incremental (with a huge number of versions, since I don't want to be surprised by a Full, I want to 'manually' pick the days Fulls run."

One of the side effects of doing the above, i.e. changing the Backup Scheme settings is that it can also reset the version chain count and may explain why your automatic cleanup settings are not working correctly.

The log file does not show any entry for 'Consolidation' which is where automatic cleanup is being actioned.

I don't like your method of repeatedly changing the backup task settings between Full and Incremental. You can confuse the database. Also, depending upon your cleanup/retention rules, I worry that you might somehow end up with incrementals that lack their original full backup, such that the incrementals would be useless.

I suggest that you create separate backup tasks for each type of backup that you want.

You wrote: "I don't want to be surprised by a Full, I want to 'manually' pick the days Fulls run". That's fine. You can set each task to be run manually, at your convenience.

As always, thank you very much for your reply, Steve!  I figured that it might be something along those lines.

(I miss Ghost's option to force a new chain.)

My schedule is so erratic, and this job runs so long, that I might need to stick to my practice of switching Methods, and accept that the version chain count will not be of use for automatic cleanup.

(And even if "Delete versions older than X days" auto-cleanup functionality would still work, my schedule really doesn't accommodate that well either.)

So I'm thinking I'll forget about auto-cleanup.  Given that, is there significant reason I shouldn't just delete a chain each week manually USING WINDOWS EXPLORER.  Specifically...

I recall you explaining last month in another thread that "Acronis stores information about your backup chains in a metadata database, which is managed correctly when cleanup runs but gets out of sync with the actual files available if you delete files manually outside of Acronis, i.e. using Windows Explorer"

In other words, how important is that metadata database for more than auto-cleanup?  (Such as for creating backups and for boot media-restores.)  I don't care that much if the ATIH2017 backup tab displays the wrong Total Size for backups.  It would even be OK with me if I needed to do my own manual re-validations.

I really don't want to have to use boot media weekly to delete a chain for both jobs.  And honestly, after my thread https://forum.acronis.com/forum/acronis-true-image-2017-forum/delete-backup-chains-versions-ati-windows about deleting versions, I found I didn't understand doing manual deletions properly well enough not to find the prospect dauntingly.

Thank you very much for your reply, tuttle!  Please let me see if I understand.

For backup of a given hard drive:

I understand that I can create a job that only creates Fulls.  But I suspect that if I create another job that only creates Incrementals, it's first image will be it's OWN Full (not one that modifies the first job's last Full).  In which case I'd (1) never need the other job, and (2) this plan wouldn't achieve what I'm seeking to achieve.

Coyote, rather than going round in circles about automatic cleanup, backup chains etc.., perhaps it might be simpler if you can give us an idea of what you are wanting to try to do here, in terms of your backup strategy / aims?

Given that you say the backup 'runs so long' when creating a full backup image, what is being included in the backup and can you split this down in to separate, more manageable chunks based on what is most important, what changes most often, what is essential for recovery etc?

My aim is simple: backup the data, and it's changes daily.

My 2 backups have simple architecture.  Each are complete drive backups, each FROM a different physical HDD, and each TO a different (dedicated) HDD.  In other words, 2 HDD are backup sources, 2 more HDD are backup targets.

I never create more than one partition per HDD.

The long-running job's source HDD's contents doesn't change much.  (But it is my data drive, so I do regularly store new stuff on it in many different places.)  It contains 740GB (I need to add 400GB more, but won't until I have a chance to increase the job's target HDD from 2TB to at least 6TB).

Honestly every part of the source HDD changes with about the same modest frequency.  But I could split it into parts (one indivisible part now about 500GB) for separate backup jobs; but honestly the only thing that now motivates me is making sure it can complete overnight, and now it can.  However, with the additional 400GB it won't, so I think you anticipated a need I was going to recognize when I add that data!!  

Which reminds me; it's so easy now that I have separate targets each 100% dedicated to a single job.  I guess I need to accept that getting them to run overnight is more important (I'm not sure I'll be willing, or need, to add another HDD for the source HDD's 2nd target).

Anyway, while I'm not sure where you're going with this, I'm very intrigued.

Ok, thanks for the further details of your backup requirements.

Next question(s):  What type of backups are you doing here?  It seems that you are doing Disk & Partition backups but perhaps your source drives may lend themselves better to being backed up using Files & Folders, where you can target areas of the data more specifically.

With Disk & Partition backups, changes are being detected at a sector / cluster level, whereas with Files & Folders backups, you are looking a changes at the files & folders level.  You could also potentially reduce the need to do a full backup as often, assuming that the core data on that source drive remains largely unchanged?

Oh, right, I am doing full Disk backups (I've figured why not, since I can grab files off them too).

But of course I realize that splitting the data drive into backups will give me a good reason to switch to File backups.

"With Disk & Partition backups, changes are being detected at a sector / cluster level, whereas with Files & Folders backups, you are looking a changes at the files & folders level."

Very interesting; I'm not sure I can predict the outcome (other than that I guess with File backup I wouldn't need to refrain from defrag between Fulls), but it would be interesting!

I do still like the idea of not going "too long" between Fulls, but if "too long" is a more generous threshold with File backup, cool!

p.s.

regardless, I still would rather not schedule my backups, I'd rather continue to manually 'invoke' them

should I infer that the answer to my earlier question about deleting manually w/Windows Explorer (instead of boot media) is that it's a bad idea even though I can accept forgoing auto-cleanup? 

I doubt that all of that data changes frequently. Likely much of it stays pretty constant. Backing up all of it into disk images is, of course, very time consuming and perhaps not necessary. There's more than one way to skin a cat, and not everything needs to be in a .tib image. I also don't understand your aversion to partitioning your drives. Partitioning can make things more efficient, and make system restores easier and quicker.

I'll describe a bit of what I do, which may help you decide your own best course.

My laptop contains a single hard drive. To allow for quicker and smaller OS backups and quicker restores, I partitioned my drive. I have C: for the OS, D: for most user data, and E: for music.

E: contains about 550 GB of music files, so if I included those files in every system backup the backups would take hours and .tib files would be huge. When I create Acronis True Image backups, I use Disk Mode and select the checkbox for the entire drive which therefore includes all partitions and hidden partitions. That gives me .tib files that I know can fully restore a working system. But, I exclude the music files by file type: *.flac, *.m4a, *.mp3. Note that the partition is included, as I have selected to backup the entire disk, but those music files are excluded from the .tib backups

I backup my music collection (over 25,000 large lossless files, about 550 GB) to two external HDs using Robocopy and a command line I wrote. That backup is quick, because Robocopy compares the source with the existing backup, and backs up only the files that have changed, been newly added, and deletes any that I have deleted from the source.

When I have the need to restore the OS, such as if a newly installed application causes problems, I restore just the C: partition. If I ever had need to restore everything, the entire disk, such as if I had totall drive failure and had to replace the drive, I would first restore the entire disk from the .tib image, and then I would restore the music files from the file-based backup using Robocopy.

@tuttle

yes as a whole it is pretty constant

re: no partitions.  I like simplicity and flexibility.

Command line: I have an aversion to it, though I am a programmer.

I guess your definition of simplicity differs from mine. I like that I can create a backup image much more quickly. That encourages me to create one just before I install new software, so I can restore just the OS partition to "roll back" if the new install causes issues.

Re. command line, there is a free Robocopy GUI available which provides a GUI interface to the same, powerful functionality.

I guess it's really flexibility I adore; I'd hate to have walls inside my HDD.

As for simplicity: it's true, my wish to manually launch backups isn't easy; I wish my schedule was more conducive to scheduling backups.

Oh, right I forgot about the Robocopy GUI.  I'm still hesitant because I adore boot media restore functionality.  (And not that I'm sure it's relevant to me, but: do you manually Robocopy, or do you schedule it somehow?)

I write each Robocopy backup scheme into a batch file, which can be run either manually or as a scheduled task.

I used to feel as you do about partitioning. But, it allows the disk to be used more efficiently by the system, and gives more flexibility regarding backup and restore. If partition sizes ever need to be adjusted, it's easy and quick to do so. With the size of modern drives, it makes great sense to partition at least the drive that includes the OS.

Backing up everything into a .tib file is the simplest method. But, you seem to be concerned about the amount of time it takes, so I'm offering you alternatives.

All the other interesting and cherished parts of this thread aside, I still have the following question I've asked twice before.  I promise only this one last try, then I'll refrain from asking further:

Given that no matter how I re-configure my backup strategy, (due to my erratic schedule) I'm inclined to continue to run my backups in such a way that auto-cleanup doesn't work properly:  If I thus remain decided to resign myself to manual cleanup, are there functional purposes beyond auto-cleanup for which I should do that manual cleanup with boot media instead of in Windows Explorer?

Coyote, I have never needed to use the rescue media to delete unwanted backup files. The important action if you delete files manually using Explorer is to run a Validation for the task that created the files - this is needed to reconcile the Acronis database information for the task.

Ah yes, I see that the task "Validate" command.  Sweet.  Thank you very much, Steve!

Am I correct in assuming that that command will do Validation on the images it finds for the Backup task?

Validation will work for the Backup Version Chain(s) for your Backup task, so if you have a version chain with say 6 files (1 x Full plus 5 x Incremental) and you delete that version chain, then Validation will throw up 6 error messages, one for each deleted file.  The Ignore option for the error messages will tell Validation that that file has been deleted and references to it will be removed from the database.

Validation for existing files compares the checksum value embedded in the file with the new checksum value calculated for the file - if these match, then the file is as it was at the point when it was created.

1.

Thank you; I see that once I run task Validate I can click Ignore for each missing file...and now it's Validating the existing files.  (Not that I guess I actually needed to do this, since a couple hours ago I deleted the missing files with boot media; man I'm thrilled I won't be doing that regularly!)

2.

Please let me apologize, Steve.  I see now that you've answered what I thought was my remaining question MANY times.  I'm afraid I at the time I was too overwhelmed and confused to see that.  Sorry!

Hey, I just realized

I see that once I run task Validate I can click Ignore for each missing file...and now it's Validating the existing files.  (Not that I guess I actually needed to do this, since a couple hours ago I deleted the missing files with boot media

that even though I had deleted the filed with boot media, the ATIH task Validate put up the Error messages (which I clicked Ignore on) anyway!  

Conveniently, then one clicks Ignore the database is apparently updated immediately, since once actual image validation started I clicked "Stop".  Then I re-launched task Validate and got no error messages, and let the Validate complete.

Not sure what you mean by "anyway", because this is entirely expected and logical behaviour. Yes, of course ATI reports errors after you deleted backup files, because it cannot find backup files that are still listed in its database. By clicking "Ignore", you tell the database to ignore or purge the listings for those deleted files.

Not sure what you mean by "anyway"

I thought I read somewhere (but after a bit of digging through forum posts and the manual I'm surprised that I can't find where) that images deleted with the boot media got removed from the database.

(I know that sounds surprising, but I accepted it.)

But maybe I mis-remembered; that would certainly explain why, when I posted a bunch of times asking questions based upon what I thought I remembered, the questions tended to fall flat on their faces

p.s.

I have a speculation on how I came to my (apparently incorrect) conclusion.  Before installation I studied the ATIH2017 manual; in section 4.5.6 there's a paragraph explaining that one needs to avoid using Windows Explorer to delete images since Win Explorer won't update the database. 

Immediately after that the (yet again as I've repeatedly noted in other posts, IMO, problematic) manual tells about the proper method of deleting a whole backup (in the GUI) and specific backup versions (in boot media).  Since the manual fails to mention that the later doesn't (like Win Explorer doesn't) update the database (and fails to instruct users to run task|Validate), I believe it is reasonable to conclude that it does.

The manual has pissed me off again.  Grrr.  

Additionally, why even tell people to use the boot media when Windows Explorer would do the same bloody thing!

Coyote, I think you have got it now!  

If backups are created within Windows, then information about them is stored in the database, thus validation is needed if you delete files outside of using automatic cleanup.

If backups are created using the Rescue media then there is no database involvement so no issues with deleting those backup files either within Windows using Explorer or otherwise.

The unfortunate side of this topic is that it causes users like yourself lots of headaches trying to work it all out especially when the manual can be misinterpreted too.

"If backups are created using the Rescue media no database involvement so no issues with deleting those backup files either within Windows using Explorer or otherwise."

I'm intrigued by the ability to create backups with the Rescue media.  I don't think I could do that with Ghost; certainly not of the system drive.  Still, while I always imagined wanting to, not sure I'll ever make use of this functionality (unless my OS becomes unstable).

"the manual can be misinterpreted too."

I agree that the manual isn't dead wrong in this case (it's just badly lacking).  I admire that you're more diplomatic that I am, Steve.  

I wonder, does anyone from Acronis read these threads?  It would be great if they did.  In one thread I pointed out a place the manual is dead wrong; in another thread I pointed out discrepancies between the manual and the GUI.  Don't get me wrong, what the manual does right it does brilliantly and thoroughly; but like most things it isn't perfect.

The Rescue Media can be used for both backups, recovery, cloning and regardless of whether Windows (or other OS's) are installed or whether it is a bare-metal situation, i.e. new unformatted hard drive with no OS.

For some users, that is how they always want to use the product, completely outside of any involvement by Windows or other installed applications such as antivirus etc.

With regards to errors in the manuals, then please use the Feedback tool to tell Acronis about such things so that they can at least consider correcting the issues in the online / web versions of the manual and for future versions.

I'm one of the users that Steve describes. On my primary laptop, I make backups using the ATI Rescue Media. I like that it's a completely "clean" environment, without Windows running, background processes, antivirus, etc.

OTOH, on my father's PC I set up the ATI Windows applications to create automated backups on a schedule. That way he doesn't have to mess with any settings, and the schedule ensures that backups are made even if he forgets.

@tuttle

I can relate.  When it comes time to do a Full Drive Restore I'd never consider doing so anywhere but with Rescue Media, precisely because "it's a completely 'clean' environment".

I guess like the convenience of creating backups in Windows, though.  (Even though at the moments I'm launching them relatively manually, it is easy to do so with multiple jobs in Windows.)

Incidentally, I am continuing to percolate a strategy to allow me to automate backups on a schedule.  (In the past I've been concerned that otherwise I wouldn't do them daily or even at all, in which case I really wouldn't even have a backup solution.)

• "I'm one of the users that Steve describes. On my primary laptop, I make backups using the ATI Rescue Media. I like that it's a completely "clean" environment, without Windows running, background processes, antivirus, etc."

I wonder (perhaps tuttle or someone else knows): If one makes a Full using the ATI Rescue Media, might ATIH-in-Windows be able to build a chain of Incrementals upon that Full (which was made with the Rescue Media)?

(p.s. is there some way to quote previous posts that I'm missing?)

You would need to use the 'Add existing backup' option for the full backup made with the rescue media, then configure that new task to make incremental backups, which hopefully should then start that chain off, though you would need to test this to prove that this would work correctly.

The old forum had the option of using

xxx

but this hasn't been implemented in this new forum yet, but hopefully will be reinstated at some point!

Thank you very much for your reply, Steve!

1.  Ah yes I see.  So each time I run a Full (made with the rescue media) would be a new task for Windows to build Incrementals upon.  I think I'll try this approach with an old, slow PC I want to run big backups on.  I'll report my results.

2.  No problem.  I'm happy enough now with pasting, adding "", italicizing, and a bullet-indent for quotes.  (At first I tried the Block Quote button, and that didn't work after it was Saved.)