Issue description   

• Service interruptions
• Incomplete log collection or parsing
• Failures in report generation
• Application crashes

Possible causes   

1. High log ingestion rate: Sudden spikes in log flow from integrated sources.
2. Improper storage allocation: Limited disk space allocated during initial installation.
3. Retained archives: Log archives and backup files accumulate over time.
4. Improper cleanup configuration: Lack of clearing up the stale/unwanted log sources from the product.

Resolution steps   

• Navigate to the installation directory (e.g., /opt/ManageEngine/EventLogAnalyzer or C:\ManageEngine\EventLogAnalyzer) and use tools like:

• du -sh * (Linux)
• TreeSize or WinDirStat (Windows)

• Identify large folders: typically archive/elasticsearch/logs/

•   To understand the current log flow (Events Per Second) collected in your instance, please refer to the below instructions.

1. Login to your EventLog Analyzer UI as a built-in default admin.
   
2. Go to the Search tab > Click pick log sources, select all sources > All log types drop-down > Specify the date range at the top right corner as last 7 days > Hit search. In the resulting graph (based on days), click the highest bar available > continue to click on the highest bar available till the graph displays results based on seconds. Hover over the highest bar to understand the peak log flow and EPS. The below image is for your reference.
   

• Based on the figure above, you can understand how the log flow is segregated between log sources as per the following references:

• Based on the above log flow, you can assign the system resources as displayed below.

• Configure live log retention for the limited number of days exactly for the period of analysis requirement. The live logs retention is directly applicable to the elasticsearch and it is responsible for displayed the log data in the product UI (say in Dashboard, Reports, Search, etc.), so if your requirement is to perform analysis on the collected logs for 32 or 60 days, set the same in the following settings: Retention Settings
  
• Since we have the Archive retention in our product, we can retrieve the logs from the installation directory (by default) anytime based on our convenience to perform the analysis. For example, if your organization is required to keep the logs for a year, we can set the live logs to 32 or 60 days and then set the Archives accordingly. This will help to sustain the storage space since the archives will be stored in a much compressed manner. i.e. 40:1 GB.
  
  Reference: Archive

• Relocate archives to a different disk/partition: Migrate Archive Data
• Change log storage directory:

• For Elasticsearch indices: relocate using custom configuration in elasticsearch.yml: Migrate Elasticsearch Data

• Please disable or delete the unwanted sources to delete the logs associated with it. Refer to: How to enable/disable a device?
• Employ log collection filters to collect and process only the necessary logs in your instance. Refer to: Log Collection Filter

• Expand disk capacity (Preferred approach):

• For VMs: Increase allocated disk size via hypervisor settings. Reference: System Requirements
• For physical servers: Add additional disks or partitions.

Tips  

1. Proactive monitoring:

• Configure disk usage monitoring alerts at the OS and application levels.
  
  Reference: Product Settings
  

2. Retention policies:

• Define and enforce appropriate log retention periods based on compliance requirements.

3. Capacity planning:

• Estimate storage needs based on daily ingestion rate, retention period, and future growth.

4. Virtualization best practices (if applicable):

• Use thick provisioned, Eager Zeroed disks for VMware.
• Avoid thin provisioning and snapshots where possible.

5. Regular maintenance windows:

• Schedule regular maintenance to clean up, archive, and optimize database and index storage.

6. Documentation:

• Maintain detailed documentation of all storage-related configurations for future reference.

 Related topics and articles   

• ManageEngine EventLog Analyzer Help Documentation
• Log Management - Analysis, Archive, Forensics, Compliance, SIEM

How to reach support