Windows Monitor Data collection is running too slow and it is timed out

Windows Monitor Data collection is running too slow and it is timed out

 

Applications Manager uses VB scripts for connecting and fetching Telemetry data from remote servers. This error occurs when the script execution times out (default timeout value is 300 seconds).
Possible Reasons:
  • Data collection script may be running too slow and can possibly time out due to some network issues in Windows.

  • Target remote server did not respond because it was hung or performing some resource intensive operation.

  • WMI classes got corrupted in the target server.

  • Recent Firewall / AV upgrade / policy changes are blocking traffic between Applications Manager and the target server.

Troubleshooting :

To verify the error,  please run the data collection script in both the Applications Manager installed server and the actual server to check the time taken for the WMI query execution.
Script Usage:
From Applications Manager Installed Server :
  1. Open command prompt as Admin User.
  2. Go to the path <APM_HOME>\working\conf\application\scripts in the command line.
  3. set start=%time%
  4. cscript windowsScripts.vbs "hostname" "username" "password" "memory.vbs,cpucore.vbs,cpu_2008.vbs,disk.vbs:,NetworkInterface.vbs,uptime.vbs,PhyMemCpuImportProduct.vbs,services.vbs"
  5. set end=%time%
  6. echo %start%
  7. echo %end%
If the time taken is higher for script execution, we need to identify the problem and fix it.

Fix:

  1. Check for network issues. Check the Receive Side Scaling and the TCP Chimney registry settings. If you have enabled TCP Chimney along with RSS setting, WMI may fail.

  2. Meanwhile, in the target server, test if the local basic WMI query also slows by the Administrator account.

    1. Run WBEMTEST, connect the namespace root\cimv2

    2. Click query and enter query as "Select * from Win32_ComputerSystem"

    3. Note down the number of seconds the query takes to execute.

f this also takes time, WMI could be corrupted. 

            Kindly perform the WMI reconstruction operation based on the suggestion given by Microsoft in the below URL:

https://techcommunity.microsoft.com/t5/ask-the-performance-team/wmi-repository-corruption-or-not/ba-p/375484

  1. Test if the remote basic WMI query from Applications Manager server is slow by using the same account used for monitoring.

    1. WBEMTEST, connect the namespace \\machinename\root\cimv2

    2. Provide the username and password for the remote machine

    3. Click query and enter query as "Select * from Win32_ComputerSystem"

    4. Note down how many seconds the query takes to run

If this query takes a longer time to run, the problem might be in the network between Applications Manager server and the target monitored server. So check for any changes in AV / Firewall settings.  

  1. If the server is expected to respond slow, increase the script timeout to avoid the Data Collection Timeout issue (this is generally not recommended as this increases the time taken to fetch metrics and can cause slowness in data collection).

Below are the troubleshooting steps to increase the timeout and check for the issue.

‚Äč

Steps:

  1. Navigate to Windows->Monitor Details Page->Snapshot View->Edit Monitor.

  2. Add 300 more seconds to the Timeout value.

Now poll the problematic monitor and check for the issue.