critical failure after reboot, missing config files and backups not working
Has anybody experienced something similar?
we have acronis cybe backup 12.5, windows server 2012 R2, storage node on a disk on the same server and agent based backups
here's what happened.
after a reboot the management service was down, the service wouldn't start because it was missing the file:
C:\ProgramData\Acronis\AMS\dml.config
following a support suggestion we repaired the install and the service came back up, but the storage node was "offline" even tho the service was running.
backups were not running with error "can't find the location [location name]".
following another support suggestion we updated to the lastes build, the storage node came back up but backups still fails mentioning "can't find the specified path" and it's missing files in this directory:
C:\Program Data\Acronis\AMS\AccessVault\profiles\[ID]
and in fact that directory only contains one profile, and that's doesn't correpond to the profile it's searching for.
suggestions?
Maria Belinskaya wrote:Hello Daniel,
Welcome to Acronis forums!
Please check whether this backup location is accessible from the Agent which backup is created.
If it is accessible, please edit your backup plan and re-add this backup location.
Otherwise, please collect Acronis System Information from the Storage Node and share it with your Support Engineer for further investigation (Acronis Storage Node logs analysis).
Hi Maria!
we have followed up this issue with the support and we've had a number of support calls and remote sessions on the machine, still no luck and the company hasn't had working backups for 6 days now.
I'm writing this on the forum as well to have a wider perspective, not just a single support engineer.
the situation has evolved as following:
- backup locations managed by the storagenode disappeared after a restart of the storagenode services
- following a second update of the server to the latest build agents cannot register anymore, when registering on the server during install the error printed is "Internal server 500".
- we manually recreated the directory and lock file missing from the directory C:\Program Data\Acronis\AMS\AccessVault\profiles\[ID] according to the error message, upon doing this we got a "can't contact the credential store service" and only after deleting everything from the CredentialStore directory and restarted the services acronis recreated the content of that directory and backup plans can be created (on a test location which is a shared directory), backups still hang upon running tho and now no errors are being printed.
- random "internal server error" http code 500 appear as one navigates the management web interface
We do believe that the underlying error is a windows error, after a reboot windows seems to have deleted a number of files and there's a strange behaviour in reading/writing in the acronis application directory, windows doesn't let you cut files from there (as administrator, permissions on the file/folder are ok) to another location, one can only create a directory there, move the files there and then one can cut/paste them in a different location, that's what we had to do to empty the CredStore directory.
we are exploring the option of redoing the server from scratch, my question is, if we were to do that is there anyway we could re-import the backup files we already have (4/5 TB) in the new installation?
I suppose there's a db within the acronis server application that holds all that data, can we dump and then reimport that information into a new installation?
please consider the backups are encrypted.
I can provide a case number if needed
thanks in advance!
Daniel
Hello Daniel.
I have found your case 04574111 and analyzed your system logs uploaded there. The issue is in cred-store.exe process which is a child process for asm.exe:
2020-08-27 14:38:15.297 [C] Encryptor initialization failed err="encryption/config: file with master key not found: C:\\ProgramData\\Acronis\\CredStore\\masterkey.local" caller=main.go:121 caller=main.go:121
Please do Repair Installation for Acronis Management Server in order to solve this issue.