Disk Space Alert: EventLog Analyzer Installation Drive Reaching Capacity Threshold
Issue description
This document provides a technical overview, possible causes, recommended resolution steps, and best practices for handling the "Disk Space Alert: EventLog Analyzer Installation Drive Reaching Capacity Threshold" notification. This alert indicates that the disk space allocated to the EventLog Analyzer installation directory is approaching critical capacity, potentially affecting product performance, stability, and log processing.
The EventLog Analyzer generates, collects, parses, and stores large volumes of log data from versatile log sources. As data retention grows over time, the installation drive may run out of available space. The application proactively raises disk space alerts when predefined thresholds (e.g., 80%, 90%, 95%) are breached.
If left unaddressed, insufficient disk space can lead to:
- Service interruptions
- Incomplete log collection or parsing
- Failures in report generation
- Application crashes
Possible causes
- High log ingestion rate: Sudden spikes in log flow from integrated sources.
- Improper storage allocation: Limited disk space allocated during initial installation.
- Retained archives: Log archives and backup files accumulate over time.
- Improper cleanup configuration: Lack of clearing up the stale/unwanted log sources from the product.
Resolution steps
Step 1: Assess current sage
- Navigate to the installation directory (e.g., /opt/ManageEngine/EventLogAnalyzer or C:\ManageEngine\EventLogAnalyzer) and use tools like:
- du -sh * (Linux)
- TreeSize or WinDirStat (Windows)
- Identify large folders: typically archive/elasticsearch/logs/
Step 2: Check the current log flow and the allocated system resources
- To understand the current log flow (Events Per Second) collected in your instance, please refer to the below instructions.
- Login to your EventLog Analyzer UI as a built-in default admin.
- Go to the Search tab > Click pick log sources, select all sources > All log types drop-down > Specify the date range at the top right corner as last 7 days > Hit search. In the resulting graph (based on days), click the highest bar available > continue to click on the highest bar available till the graph displays results based on seconds. Hover over the highest bar to understand the peak log flow and EPS. The below image is for your reference.
- Based on the figure above, you can understand how the log flow is segregated between log sources as per the following references:
Note:
A single-installation server can handle either a maximum of 3,000 Windows logs or any of the high flow values mentioned for each log type in the table above.
For log types which are not mentioned in the above table, choose the appropriate category based on the log size. For example, in the case of SQL Server logs when the byte size is 900 bytes, and EPS is 3,000, it should be considered as high flow.
If the combined flow is higher than what a single node can handle, it is recommended to implement distributed setup.
It is recommended to choose the next higher band if advanced threat analytics and a many correlation rules have been used.
- Based on the above log flow, you can assign the system resources as displayed below.
Reference guide: System Requirements
Step 3: Define log retention precisely to meet your objectives
- Configure live log retention for the limited number of days exactly for the period of analysis requirement. The live logs retention is directly applicable to the elasticsearch and it is responsible for displayed the log data in the product UI (say in Dashboard, Reports, Search, etc.), so if your requirement is to perform analysis on the collected logs for 32 or 60 days, set the same in the following settings: Retention Settings
- Since we have the Archive retention in our product, we can retrieve the logs from the installation directory (by default) anytime based on our convenience to perform the analysis. For example, if your organization is required to keep the logs for a year, we can set the live logs to 32 or 60 days and then set the Archives accordingly. This will help to sustain the storage space since the archives will be stored in a much compressed manner. i.e. 40:1 GB.
Reference: Archive
Step 4: Review and reconfigure storage location
- Change log storage directory:
- For Elasticsearch indices: relocate using custom configuration in elasticsearch.yml: Migrate Elasticsearch Data
Step 5: Log collection filter and remove stale/unwanted sources
- Please disable or delete the unwanted sources to delete the logs associated with it. Refer to: How to enable/disable a device?
- Employ log collection filters to collect and process only the necessary logs in your instance. Refer to: Log Collection Filter
Step 6: Long-term solution
- Expand disk capacity (Preferred approach):
- For VMs: Increase allocated disk size via hypervisor settings. Reference: System Requirements
- For physical servers: Add additional disks or partitions.
Tips
- Proactive monitoring:
- Retention policies:
- Define and enforce appropriate log retention periods based on compliance requirements.
- Capacity planning:
- Estimate storage needs based on daily ingestion rate, retention period, and future growth.
- Virtualization best practices (if applicable):
- Use thick provisioned, Eager Zeroed disks for VMware.
- Avoid thin provisioning and snapshots where possible.
- Regular maintenance windows:
- Schedule regular maintenance to clean up, archive, and optimize database and index storage.
- Documentation:
- Maintain detailed documentation of all storage-related configurations for future reference.
Related topics and articles
How to reach support
If the issue persists after following the above steps, you can contact the ManageEngine EventLog Analyzer Support Team.
Support Channels: