Overview

This article describes the process to increase the timeout values for Microsoft SQL Server, WCF and ASPNET in Archiver.

If requests are not responded to in expected time frames from one module to another (for example in regard of WCF connections) or from Microsoft SQL Server, the requesting module will not wait indefinitely and hit a defined timeout.

The most common default timeout values within Archiver are:

• 2 min for querying the Microsoft SQL Server.
• 5 min for WCF connections (This is used heavily for internal communication between GFI Archiver's own modules).

The common symptoms for these applications to hit timeouts are:

• Indexing of an archive store gets stuck at a certain number of items.
• No results are returned while searching.
• The system responds and loads slowly in general.

The most common debug logs are: 

..\Search\Debuglogs\index.log:
"error ","IndexManager","error: Store Synchronisation failed:
System.TimeoutException: This request operation sent to net.pipe://localhost/Store/EmailMessageRetrieval did not receive a reply within the configured timeout  (00:05:00).
The time allotted to this operation may have been a portion of a longer timeout.
This may be because the service is still processing the operation or because the service was unable to send a reply message.
Please consider increasing the operation timeout (by casting the channel/proxy to IContextChannel and setting the OperationTimeout property) and ensure that the service is able to connect to the client.

..\Store\Debuglogs\errorsdal.log:
"error ","ErrorsDal","error: (MsSql) GetBulkDetailedMessages() FA message:
Timeout expired. The timeout period elapsed prior to completion of the operation or the server is not responding.

Environment

• GFI Archiver
• All supported environments
  
  

Root Cause

Underlying issues are normally resourcing issues including under-dimensioned servers and most often I/O bottlenecks (especially on the Microsoft SQL Server side).

An indicator for an I/O bottleneck is the Average Disk Queue Length performance counter. If it has a value higher than 1 constantly, it could indicate an I/O bottleneck. The Windows Performance Monitor (perfmon.msc, Physical disk > Average disk queue length) or the Windows Resource Monitor can be used to check this counter.

Resolution

If the systems which form and serve GFI Archiver (incl. the Microsoft SQL Server) are overloaded, do not have sufficient resources or are in general under-dimensioned the best approach is to address this root cause and get the systems up-to-specs and fit for the tasks they are supposed to handle.

Additionally, work through the Best Practices guide for GFI Archiver as it contains helpful information on how the GFI Archiver system should be installed and configured for optimal performance.

A less desired approach is to increase the timeout values within GFI Archiver. This can allow the system including Microsoft SQL Server enough time to complete the query. This approach should not be seen as a solution, but more as a temporary attempt to try and cater to an overloaded system which does not cope with the required load.

NOTE: This procedure requires to edit files manually. If edited incorrectly it can leave the server in a non-operational state. Please keep backups of any file which is edited throughout this article before saving any changes to them.

1. Stop all GFI Archiver services.
2. Based on the application causing bottleneck, change the timeout value(s).
   
   

   • Store/Microsoft SQL Server:

     1. Open the ..\GFI\Archiver\Store\Data\product.config file in Notepad.
        
     2. Add the following key within <appSettings> section: <add key="CommandTimeout" value="1200"/>
        NOTE: The unit of asyncTimeout and executionTimeout is seconds.
        
        

   • WCF:

     1. Open each of the following files in Notepad:
        • ..\GFI\Archiver\ASPNET\Services\WCFClientSettings.xml
        • ..\GFI\Archiver\ASPNET\UI\Settings\WCFClientSettings.xml
        • ..\GFI\Archiver\BulkSchemaUpgrader\WCFClientSettings.xml
        • ..\GFI\Archiver\Core\Data\WCFClientSettings.xml
        • ..\GFI\Archiver\Core2\Data\WCFClientSettings.xml (version 2013 or older only)
        • ..\GFI\Archiver\IMAP\Data\WCFClientSettings.xml
        • ..\GFI\Archiver\MailboxRestore\WCFClientSettings.xml
        • ..\GFI\Archiver\MAIS\Data\WCFClientSettings.xml
        • ..\GFI\Archiver\Search\Data\WCFClientSettings.xml
        • ..\GFI\Archiver\Store\Data\WCFClientSettings.xml
     2. Edit the value within the line <OperationTimeoutMinutes>5</OperationTimeoutMinutes> to the desired value (for example 20).
        NOTE: The unit of OperationTimeoutMinutes is minutes.
        
        

   • ASPNET:

     1. Open the following file in Notepad: ..\GFI\Archiver\ASPNET\UI\web.config
     2. Find the line which begins with: <pages asyncTimeout="300" and edit it to: <pages asyncTimeout="600"
     3. Find the line which reads : <httpRuntime requestValidationMode="2.0" maxRequestLength="4096" executionTimeout="600"/> and edit it to: <httpRuntime requestValidationMode="2.0" maxRequestLength="4096" executionTimeout="1200"/>
        NOTE: The unit of asyncTimeout and executionTimeout is seconds.
        
        
3. Save the file(s).
4. Start all GFI Archiver services.

NOTE: When upgrading to a newer version of GFI Archiver, installation files mentioned in this procedure will be overwritten with default versions making the changes void and ineffective. It is therefore suggested to keep a record of this procedure and follow it once again directly after upgrading to keep this functionality intact.