OOM in ScanDataServlet for SDPJobsInfo table : Latest KB (Please refer this KB alone)

OOM in ScanDataServlet for SDPJobsInfo table : Latest KB (Please refer this KB alone)

Issue Cause :

This issue occurs only if the customer has not enabled the Schedule cleanup history under the Scheduled Scan settings. This schedule is responsible for clearing both audit history (scan diff data) and SDPJobsInfo entries (EC scan job_id posted entries). However, some customers choose not to enable this cleanup option because they need to retain audit history data for compliance or auditing purposes. As a result, a large volume of data accumulates in the SDPJobsInfo table. This behaviour is expected, as each completed EC scan logs job_ID entries into the SDPJobsInfo table. Over time if the cleanup is not explicitly enabled, this leads to significant data buildup ultimately causing performance issues in SDP.


To address this, we have created an issue ID to permanently resolve the issue, we plan to separate the cleanup schedule for the SDPJobsInfo table from the existing Schedule Cleanup History

Notes
 Previously the older KB article provided a direct DELETE query that had to be executed manually in the customer’s database. This query could take considerable time to complete, so we have now executing the same in SDPJobsCleanUpSchedule batch by batch. 

Although we initially provided a manual deletion query, customers often faced difficulties executing it due to the high number of records, which made it hard to proceed with subsequent queries. To resolve this, we are now providing the fjar to handle only SDPJobsInfo cleanup automatically during SDPJobsCleanUpSchedule.

 Steps to resolve this issue : 
 
               SDP
               AE
        Fjar version 
      14940 - 14980
         7630 - 7670
        SDPJobsInfo_14940_to_14980.fjar
     14900 - 14930
        7600 - 7620
       SDPJobsInfo_14900_to_14930.fjar
      14600-14820 
         7300 - 7520
        SDPJobsInfoFix_14600_to_14820.fjar
      15100-15130
         7700 - 7730
        SDPJobsInfo_15100_to_15130_new.fjar


  1. Provide the attached SDPJobsInfo.fjar file to the customer and place it under the <SDP Home Directory>/fixes folder for the relevant build versions. 
    Other older versions, please reach us for the relevant fjars.

  2. Restart the SDP server after placing the fjar file.

  3. After the server restarts, the schedule will run periodically and this will remove the excess entries from the SDPJobsInfo table older than 30 days and this will retain the data from the last 30 days so that no performance issue will occurs for SDPJobsInfo in future. 

We can ensure it by using the select query via DB connection : select count(*) from sdpjobsinfo; 



Alert
This RCA below is applicable for 15130 / 7730 :
Issue:
'getSDPJobIdForHistoryId' print is logged several times in serverout files & performance issue in SDP 15130 / AE 7730 alone due to SDPJobsCleanup Schedule.

RCA: In this SDPJobsCleanup schedule, we will get the first available auditid to clear out the data older than this auditid in SDPJobsInfo table. If there are no auditid (i.e auditid is null) in SDPJobsInfo table, until the auditid is found, every select query will be invoked. So there will be so many prints logged in serverout files. 

Note: This issue will be resolved in 15140 / 7740.



                  New to ADSelfService Plus?