How to Perform Scheduled Import Log Collection in EventLog Analyzer

How to Perform Scheduled Import Log Collection in EventLog Analyzer

Objective

EventLog Analyzer supports scheduled log imports from both remote paths and S3 buckets. You can enable scheduled log collection to have the application read data from the same file at regular intervals, or configure a file naming convention based on the created file name pattern by the respective device/application, so that new files are imported periodically.
This article offers you the steps to configure scheduled import log collection and to create a consistent file naming format for scheduled log imports in EventLog Analyzer.

Prerequisites 

  • Access to EventLog Analyzer
  • Administrative role or Manage Log Import role to create log import schedule on the EventLog Analyzer instance
  • Knowledge of the log source and the file names that you need to import.
  • Access to the location where the files are stored.

Steps to follow

  1. Login to EventLog Analyzer user interface.
  2. Navigate to Settings >> Log Source Configuration >> Import Logs
  3. Select the type of import as Remote or S3 Bucket. Refer How to import logs from Remote path or How to import logs from S3 bucket
    NOTE: Scheduled log import is not available for local path.
  4. Post selecting the files and associated device, Enable check box for Schedule Log import.
  5. Update the Schedule time interval using any of the condition as follows:
    • Hourly - Scheduled imports occur every hour at a specified minute mark. The minute value can range from 0 to 55 and can be set in multiples of 5.
    • Daily - Scheduled imports occur daily at a specified hour and minute. The hour value can range from 0 to 23, and the minute value can range from 0 to 55 in multiples of 5. The time is specified in 24-hour format.
    • Weekly - Scheduled imports occur weekly on a specified day, hour, and minute. Days are represented from Sunday to Saturday. The hour value can range from 0 to 23, and the minute value can range from 0 to 55 in multiples of 5. Time is specified in 24-hour format.
    • Monthly - Scheduled imports occur monthly on the specified date, hour, and minute. The date can range from 1 to 30, hour value can range from 0 to 23, and the minute value can range from 0 to 55 in multiples of 5. Time is specified in 24-hour format.
    • Every - Scheduled imports occur at the specified minute interval. The minimum value is 1, and it must be a whole number (no decimal values).
      Note: From the above screenshot, the schedule import is configured to import the logs daily at 10:30 AM (EventLog Analyzer server time)
  6. If the logs are written in a specifc log file the above configuration can be performed to collect the logs. If the application creates new log files with unique filenames based on date/file size/timestamp etc, you can enable Specify filename pattern to import them.
  7. Specifying the File name pattern allows the EventLog Analyzer to understand what will be the name of the file that will be created. Once a new file is created, EventLog Analyzer will understand the pattern of the new file name and will import them automatically.
  8. Define the Naming Convention:
    • Choose Advanced Options to create a name and pattern.
    • Enter the static values manually and dynamic values using any of the following pattern from the drop down available.
      • Day
      • Month
      • Year
      • Weekday
      • Week in year
      • Week in month
      • Hour
      • Minute
      • Second
      • Number Increments
    • Include the date and time in a consistent format (e.g., YYYYMMDD_HHMMSS).
    • Add the static string values as per file name
Example Naming Format:
Sample file name: 20231005_143000_MSSQL_ERROR_PROD
  • 2023 – Year, represented as a 4-digit string (YYYY)
  • 10 – Month, represented as a 2-digit string (MM)
  • 05 – Date, represented as a 2-digit string (DD)
  • _ – Static string value
  • 14 – Hour, represented as a 2-digit string (HH)
  • 30 – Minute, represented as a 2-digit string (MM)
  • 00 – Second, represented as a 2-digit string (SS)
  • _MSSQL_ERROR_PROD – Static string value

The pattern to be created for the above is as follows:
${yyyy}${MM}${dd}_${HH}${mm}${ss}_MSSQL_ERROR_PROD
  1. Once you create the pattern, Choose Apply to save the pattern.
  2. Choose import to initiate the import.

 Validation and confirmation

  • Verify that the naming convention is applied correctly by reviewing the imported log filenames under Advanced Options.
  • Check if the logs are imported in Search and check the import status in Import Logs settings tab.

 Tips

  • Configure log sources to generate multiple files using an incremental naming convention instead of writing all logs to a single file. This helps prevent large file sizes and improves manageability.
  • EventLog Analyzer can resume reading from the exact line where the last scheduled import ended. If multiple new files are created, the application continues log collection by reading to the end of the current file and then automatically switches to the next file based on the defined file name pattern.
  • Imported logs are parsed using the timestamps recorded in the log entries themselves.
  • Using a consistent naming convention simplifies both log management and analysis.

Related topics and articles


                  New to ADSelfService Plus?

                    • Related Articles

                    • Log import failure during remote log collection in EventLog Analyzer

                      Issue description EventLog Analyzer will display an error notification in the UI stating that the log import for selected files has failed. This issue will happen when EventLog Analyzer is unable to import a file during the scheduled log import ...
                    • How to perform offline log collection using the EventLog Analyzer agent

                      Objective When there is a intermittent connection or loss of communication between the agent and EventLog Analyzer server, the agent can perform offline log collection and store the logs to a data directory of a defined size. Once the connection is ...
                    • How to import logs in EventLog Analyzer from S3 bucket

                      Objective Some applications might record logs in the file system so that the user can import the logs in EventLog Analyzer. This can happen when the logs are located in Local file system of the EventLog Analyzer instance where the shared path is the ...
                    • Enabling historic log collection in EventLog Analyzer

                      EventLog Analyzer collects all the logs present in the Windows Event Viewer (i.e., Windows Logs > Application, Security, System) when the historic log collection option is enabled. To enable historic log collection, follow the steps below: Navigate ...
                    • Unable to start EventLog Analyzer

                      Issue description This issue occurs when the EventLog Analyzer service fails to start, or when users are unable to access the web client through the browser (typically on ports 8400 or 8445). Users may experience one or more of the following ...