How to migrate live data (ES Data) from one location to another - Windows

How to migrate live data (ES Data) from one location to another - Windows

Objective  

This article provides a detailed step-by-step guide to migrate EventLog Analyzer live data / data stored in Elasticsearch to different location or server.

Prerequisites  

  • Access to EventLog Analyzer console as an administrator.
  • Server user access:
- If moved to a local path: 
        Account used in services.msc should have sufficient read and write permissions for product installation directory and new location.
- If moved to a remote location: 
  1. Ensure that the service account used by EventLog Analyzer has read and write access (Full permission) to the remote network path from the EventLog Analyzer machine.
  2. Account used in Services.msc should have sufficient read and write permissions for that service user account.
There should be no interruption in connectivity to the network share.
  • System requirements  
 
Network latency
Network speed
Disk IOPS
Local drive
<10ms
-
Remote storage
<10ms
> 20Mbps
- For optimal performance, 10ms or lower latency is recommended, and it should not exceed 100ms.
- 50% of the server's RAM should be kept free for off-heap utilization of Elasticsearch for optimal performance.
  • Share folder should be accessible from the Windows server where EventLog Analyzer is installed.

Steps to follow

Step 1: Stop EventLog Analyzer:
If EventLog Analyzer is running as service, press Windows + R to open Run and type services.msc and press OK. Stop ManageEngine EventLog Analyzer service
If EventLog Analyer is running as an application, Navigate to <EventLog Analyzer Home>\bin directory. Execute shutdown.bat.

Step 2: To ensure proper shutdown of all the residual process>> Open Command Prompt as Administrator and set the path to <EventLog Analyzer Home>/bin directory. Execute shutdown.bat, stopDB.bat, and stopSEC.bat to stop the product from installation directory.

Note: Ensure that the processes java.exe, postgres.exe, and SysEvtCol.exe are not running in the task manager. 
Step 3: Copy Elasticsearch data (<EventLog Analyzer home>/ES/data) and its archive (<EventLog Analyzer home>/ES/archive) folder to the desired location.
Note: Do not move or cut and paste the files directly. It is recommended that you copy the files instead. Once the instance is successfully up and running, you may delete the original files if needed.
Step 4: Open elasticsearch.yml as a text editor from <EventLog Analyzer Home>/ES/config and update the path.data and path.repo value accordingly.
Step 5: Verify that the local system or service account used to start EventLog Analyzer has full permissions for the new location. If the location is set to a remote path, it is recommended that you run the EventLog Analyzer service under a service account and grant it full permissions for the ES\data and ES\archive folder locations.
Step 6: Start the EventLog Analyzer service from services.msc to start the instance. Check if old data are getting populated.
The migration is now completed.
For more details refer to Data Migration in EventLog Analyzer.

Tips

  • Do not delete any files and folders from the current EventLog Analyzer/ES folder until the migration is confirmed to be successful.
  • It is always recommended to set Live logs in local path or directly attached storage as network connectivity issues in remote setup might disrupt the connection which leads to CachedRecord creation.
  • Set log retention based on your frequently searched duration in Settings > Admin settings > Retention settings to manage the storage effectively. We need to keep in mind that the higher the log retention set, the more the system resources are required for data processing and retrieval.

Related topics and articles  

  1. How to migrate live Data (ES data) from one location to another | Linux​

                  New to ADSelfService Plus?