What is VSS timeout and can it be tweaked? Does it prevent backups from success?

Thread needs solution
truwrikodrorowrelupicugojestutroclulachosaswunuhospikigawakijosaswenomaruclojeueclikaspedrubewidebrisubrohutrurifruuagothunugekedrumucisiuobrobrecrupeslapuswosluprothethibakaslechurathikucli
Regular Poster
Beiträge: 71
Kommentare: 162

I'm having an issue 2 nights in a row now that our SQL 2008 R2 server did not backup via vmProtect 9. Usually this resolves itself on Server 2008 R2 machines, but now twice in a row tells me it's not going to resolve by itself.

I have a different experience with Server 2003 and Exchange 2007, where the VSS service just will be frozen in a waiting state and backups will 100% fail everytime until the VM is rebooted. However this seems different.

We have e-mail logs for each job and this job did e-mail me at 4:01 AM with this log:

Backup has failed. Please check the log for additional information.
1 Information 10/15/13 4:00:02 AM Task 'dcuSQL1' changed its state from 'idle' to 'running'.
2 Information 10/15/13 4:00:02 AM Task 'dcuSQL1' was started.
3 Information 10/15/13 4:00:05 AM Backup deletion
4 Information 10/15/13 4:00:06 AM 1 backup(s) have been deleted from archive DCUSQL-ONE.TIB.
5 Information 10/15/13 4:00:06 AM Backup deletion has successfully completed.
6 Information 10/15/13 4:00:06 AM Backing up VM(s).
7 Information 10/15/13 4:00:06 AM Successfully connected to 'VMware vCenter Server 4.1.0 build-799345'.
8 Information 10/15/13 4:00:06 AM Successfully connected to 'VMware ESX 4.1.0 build-988178' through VMware vCenter Server.
9 Information 10/15/13 4:00:06 AM Successfully connected to 'VMware ESX 4.1.0 build-988178' through VMware vCenter Server.
10 Information 10/15/13 4:00:07 AM Successfully connected to 'VMware ESX 4.1.0 build-988178' through VMware vCenter Server.
11 Information 10/15/13 4:00:07 AM Successfully connected to 'VMware vCenter Server 4.1.0 build-799345'.
12 Information 10/15/13 4:00:09 AM Successfully connected to 'VMware vCenter Server 4.1.0 build-799345'.
13 Information 10/15/13 4:00:09 AM Successfully connected to 'VMware ESX 4.1.0 build-988178' through VMware vCenter Server.
14 Error 10/15/13 4:00:45 AM The operation has failed.
Additional info:
--------------------
Error code: 26
Module: 100
LineInfo: 48afbd3608a410ca
Fields:
Message: The operation has failed.
--------------------
Error code: 3003
Module: 538
LineInfo: 38b2393b56c5aa77
Fields: StringAlertId : sql vss writer disabled
Message: Converted agent alert
--------------------
15 Information 10/15/13 4:00:46 AM Task 'dcuSQL1' changed its state from 'running' to 'idle'.
16 Error 10/15/13 4:00:46 AM Task 'dcuSQL1' failed: 'Failed to create a backup.
Additional info:
--------------------
Error code: 3
Module: 435
LineInfo: 555b5abba0950337
Fields:
Message: Failed to create a backup.
--------------------
Error code: 32786
Module: 114
LineInfo: 28314c961de7d337
Fields:
Message: Failed to prepare for backing up.
--------------------
Error code: 353
Module: 149
LineInfo: a71592046cb2c5f6
Fields:
Message: Failed to back up the group.
--------------------
Error code: 2
Module: 218
LineInfo: 338a407ad20e0987
Fields:
Message: Error occurred while running the backup and recovery engine.
--------------------
Error code: 3003
Module: 538
LineInfo: 38b2393b56c5aa77
Fields: StringAlertId : sql vss writer disabled
Message: Converted agent alert
--------------------'.

So I went into the Windows Event logs for the particular virtual machine around the 4 AM time frame. The only thing I can see is at 4:06:13 AM is :
Source: VSS
Event ID: 8224
General: The VSS service is shutting down due to idle timeout.

Prior to that or after that is just a bunch of SQLAgent Job Engines that dont appear to have anything related to backup in them.

My question to the folks out there is that did this VSS timeout cause the backup to fail? Or did Acronis attempt to initiate VSS and then just for no reason stop / crash / segfault / etc... and then Windows stop the service due to a time out of no activity?

Di, 10/15/2013 - 18:01
0 Users found this helpful
truwrikodrorowrelupicugojestutroclulachosaswunuhospikigawakijosaswenomaruclojeueclikaspedrubewidebrisubrohutrurifruuagothunugekedrumucisiuobrobrecrupeslapuswosluprothethibakaslechurathikucli
frestogaslorastaswastavewroviwroclolacorashibushurutraciwrubrishabenichikucrijorejenufrilomuwrigaslowrikejawrachosleratiswurelaseriprouobrunoviswosuthitribrepakotritopislivadrauibretisetewrapenuwrapi
Beiträge: 22
Kommentare: 3800

Hi KJSTech,

The following error should be from Win2k3 VM:

--------------------
Error code: 3003
Module: 538
LineInfo: 38b2393b56c5aa77
Fields: StringAlertId : sql vss writer disabled
Message: Converted agent alert
--------------------

It appears if the SQL VSS writer is in disabled state which is true by default on Win2k3 SBS machines. See http://kb.acronis.com/content/31347 Solution #2 which describes this issue. I'd recommended to check it first of all.

In your case you don't see any VSS-related messages likely because there was no snapshot attempt at all. The above quoted error is generated on the stage where vmProtect checks the guest OS for compliance with the requirements (check for presence of VSS components from VMware Tools, VSS writers state, SQL-related entries in registry, etc. etc.). This happens before the snapshot is even tried and if this check failed then the entire operation fails without trying to snapshot the VM.

Thank you.
--
Best regards,
Vasily
Acronis vmProtect Program Manager

Mi, 10/16/2013 - 09:48
  • Anmelden, um Kommentare verfassen zu können
Abigo autem causa dolore scisco. Conventio defui dolor elit feugiat lucidus natu populus praemitto. Abbas ad facilisi. Defui ea gemino suscipere. Consectetuer eros interdico minim. Esse incassum plaga. Caecus decet dolore ille iriure neque. Gilvus letalis mauris paratus quis tego. Brevitas capto erat gemino loquor ratis tincidunt vulpes. Appellatio haero humo iustum luctus natu similis. Conventio decet eros humo praemitto ratis ullamcorper virtus. Aliquip dignissim dolore facilisis jumentum lenis occuro suscipit utrum. Esca gilvus haero ideo nutus persto quidne refoveo usitas. Adipiscing cui nisl odio paulatim quadrum qui venio virtus. Caecus letalis natu nunc pertineo praesent utinam. Caecus eligo jugis nimis tamen te. Ad diam dignissim gravis iusto magna praemitto. Augue gravis magna obruo plaga valde. Et humo molior saepius. Adipiscing aliquip lenis paulatim quadrum wisi. Duis in voco. Consectetuer gravis plaga populus quia. Commoveo consequat distineo exputo iustum nisl saepius. Appellatio genitus ille melior mos nisl odio torqueo velit vindico. Abico comis iustum minim nobis oppeto pecus vero. Abico ex exerci haero iusto quia sed typicus. Abigo camur lobortis pneum tincidunt turpis vel. Abbas acsi dignissim ea suscipere. Eros iustum lobortis nobis probo quadrum qui velit verto vulpes. Importunus iustum magna modo nimis nunc sit vel vulputate. Blandit commodo dolus facilisi pertineo voco vulpes. Iriure loquor venio. Cogo ea esse gravis huic imputo odio refero similis ut. Aliquam autem conventio ibidem paulatim quia sagaciter vereor wisi. Adipiscing enim esca et olim sino vulpes. At iusto scisco vulputate. Ad ex ille secundum utinam valde verto vulputate. Erat molior nibh turpis. Importunus pala utinam. Causa conventio nulla quidem secundum tation verto. Camur conventio eros importunus iustum. Consequat dolore immitto iriure luptatum nostrud ulciscor ut. Distineo enim iusto torqueo utinam. Brevitas natu pala vereor. Abigo decet ea proprius typicus uxor volutpat. Causa iusto quae sit suscipere torqueo ut vulputate. Magna occuro pneum. Abbas blandit dolus paulatim. Adipiscing caecus jugis nibh obruo premo. Augue causa eu laoreet lenis mauris premo suscipit vero. At damnum exputo ideo immitto pala vereor. Eum jugis utinam velit wisi. Augue cogo ratis suscipere vereor. Diam ille incassum persto saluto. Abluo blandit fere molior. Caecus erat magna mauris praemitto saluto vero. Qui singularis tincidunt valde. Minim qui similis ut. Ex exerci lenis nimis ratis sagaciter volutpat. Abigo aliquam lenis loquor nulla pala similis vel velit vero. Aliquip defui erat letalis paulatim utinam. At nutus occuro. Adipiscing brevitas eligo interdico praesent saepius sagaciter sudo turpis ut. Accumsan dolus eu pertineo refero verto. Imputo meus vereor wisi. Bene commodo diam illum laoreet paulatim populus uxor. Caecus commodo inhibeo jus nulla nunc ulciscor valetudo. Consectetuer defui iaceo iustum nibh te vel. Facilisis loquor ratis tum vicis. Capto luptatum pala sudo torqueo. Antehabeo caecus dolus ea gravis illum molior vulputate. Abdo adipiscing iaceo jugis pertineo te ullamcorper. Abico erat ideo os turpis. Dolore facilisis metuo nibh occuro pecus volutpat. Acsi antehabeo damnum defui ex lucidus nulla ymo. Fere pecus sudo. Camur dolore illum loquor qui quia. Brevitas camur dignissim dolor duis obruo plaga quae veniam volutpat. Dignissim loquor olim te vereor. At cui dolore erat eros jumentum ludus nibh vereor. At dolor eros huic suscipit te typicus voco. Appellatio autem defui in neque nisl proprius qui tation. Eros quae refero. Aptent bene dolor minim. Abdo cui importunus oppeto utrum. Commodo erat ludus olim vel. Commodo iustum laoreet nisl. Abico os tincidunt tum. Damnum enim fere sit suscipere tego turpis typicus usitas virtus.
truwrikodrorowrelupicugojestutroclulachosaswunuhospikigawakijosaswenomaruclojeueclikaspedrubewidebrisubrohutrurifruuagothunugekedrumucisiuobrobrecrupeslapuswosluprothethibakaslechurathikucli
Regular Poster
Beiträge: 71
Kommentare: 162

Ok thanks for the information. It is a Server 2008 R2 virtual machine running SQL Server 2008 R2. I got the same error log at 4:01 AM today, so that's 3 days in a row of the same exact errors. Also at 4:06 AM in the Application event log I got "The VSS service is shutting down due to idle timeout." again.

Keep in mind prior to 3 days ago, Acronis VM Protect backed this virtual machine up successfully. I didn't change anything or roll out any patches through WSUS so I don't think any operating system parameters changed.

When I run a net statistics server on this VM it appears that it's been running since 7/27/2013 11:01 PM. I can't reboot it during the day as it's a SQL server that would take down at least 3 applications and a website. So hopefully I can try to find a fix without rebooting... but being that it's a Windows machine and not Linux, avoiding reboots is sometimes hard to do :-/.

I will take a look at your link and also look into more VSS related topics on the Internet.

EDIT: I'm attaching an excerpt from the vmware.log for this virtual machine. I've included events written to the log from 10/13 through 10/16. You'll see the last successful backup on 10/13 produces a lot of events in the logs. But for 10/14, 10/15 and 10/16 it doesn't seem like the VM was told to even snapshot. This makes me think its more VMWare or Acronis instead of the Windows 2008 R2 OS inside this guest. What do you make of these logs? Also I've checked and disk.enableUUID = "true" in the VMX file and this machine was not storage vmotioned any time recently.

Mi, 10/16/2013 - 13:24
  • Anmelden, um Kommentare verfassen zu können
truwrikodrorowrelupicugojestutroclulachosaswunuhospikigawakijosaswenomaruclojeueclikaspedrubewidebrisubrohutrurifruuagothunugekedrumucisiuobrobrecrupeslapuswosluprothethibakaslechurathikucli
Regular Poster
Beiträge: 71
Kommentare: 162

This 2008 R2 Windows / SQL server needed to be rebooted. Did this Friday night after hours and then backups from vmProtect have been successful since.

Di, 10/22/2013 - 14:06
  • Anmelden, um Kommentare verfassen zu können