APM Insight agent - Weblogic OOM

APM Insight agent - Weblogic OOM



We are facing issues on production weblogic server with APM Insight agent installed.
Weblogic stops responsing with error:

java.lang.OutOfMemoryError: nativeGetNewTLA
    at java.util.LinkedHashMap.createEntry(LinkedHashMap.java:424)
    at java.util.LinkedHashMap.addEntry(LinkedHashMap.java:406)
    at java.util.HashMap.put(HashMap.java:474)
    at sun.util.resources.OpenListResourceBundle.loadLookup(OpenListResourceBundle.java:118)
    at sun.util.resources.TimeZoneNamesBundle.handleGetObject(TimeZoneNamesBundle.java:59)
    at java.util.ResourceBundle.getObject(ResourceBundle.java:378)
    at java.util.TimeZone.getDisplayNames(TimeZone.java:399)
    at java.util.TimeZone.getDisplayName(TimeZone.java:351)
    at java.text.SimpleDateFormat.subFormat(SimpleDateFormat.java:1093)
    at java.text.SimpleDateFormat.format(SimpleDateFormat.java:882)
    at weblogic.utils.string.SimpleCachingDateFormat.format(SimpleCachingDateFormat.java:42)
    at weblogic.utils.string.CachingDateFormat.getDate(CachingDateFormat.java:94)
    at weblogic.logging.LogFileFormatter.format(LogFileFormatter.java:41)
    at com.bea.logging.RotatingFileStreamHandler.publish(RotatingFileStreamHandler.java:81)
    at java.util.logging.Logger.log(Logger.java:471)
    at weblogic.logging.WLLogger.log(WLLogger.java:38)
    at weblogic.servlet.internal.WebAppServletContext.log(WebAppServletContext.java:881)
    at com.ubisecure.util.log.AbstractLog.log(AbstractLog.java:61)
    at com.ubisecure.util.log.AbstractLog.debug(AbstractLog.java:67)
    at com.ubisecure.saml2.sp.servlet.ServiceProviderFilter.doFilter(ServiceProviderFilter.java:439)
    at weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.run(WebAppServletContext.java:3496)
    at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:321)
    at weblogic.servlet.internal.WebAppServletContext.execute(WebAppServletContext.java:2086)
    at weblogic.servlet.internal.ServletRequestImpl.run(ServletRequestImpl.java:1406)
    at weblogic.work.ExecuteThread.run(ExecuteThread.java:173)

Agent-log:

[23.9.2012 23:30:08][agentsc-1-thread-1][WARNING]:[MetricDataDispatcher] Exception
java.lang.OutOfMemoryError: allocLargeObjectOrArray - Object size: 33330560, Num elements: 16665270
    at java.util.Arrays.copyOf(Arrays.java:2882)
    at java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:100)
    at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:390)
    at java.lang.StringBuilder.append(StringBuilder.java:119)
    at com.manageengine.apminsight.agent.sequence.SequenceTrace$SequenceTraceItem.toString(SequenceTrace.java:245)
    at java.lang.String.valueOf(String.java:2827)
    at java.lang.StringBuilder.append(StringBuilder.java:115)
    at java.util.AbstractCollection.toString(AbstractCollection.java:422)
    at java.lang.String.valueOf(String.java:2827)
    at java.lang.StringBuilder.append(StringBuilder.java:115)
    at com.manageengine.apminsight.agent.sequence.SequenceTrace$SequenceTraceItem.toString(SequenceTrace.java:244)
    at java.lang.String.valueOf(String.java:2827)
    at java.lang.StringBuilder.append(StringBuilder.java:115)
    at java.util.AbstractCollection.toString(AbstractCollection.java:422)
    at java.lang.String.valueOf(String.java:2827)
    at java.lang.StringBuilder.append(StringBuilder.java:115)
    at com.manageengine.apminsight.agent.sequence.SequenceTrace$SequenceTraceItem.toString(SequenceTrace.java:244)
    at java.lang.String.valueOf(String.java:2827)
    at java.lang.StringBuilder.append(StringBuilder.java:115)
    at java.util.AbstractCollection.toString(AbstractCollection.java:422)
    at java.lang.String.valueOf(String.java:2827)
    at java.lang.StringBuilder.append(StringBuilder.java:115)
    at com.manageengine.apminsight.agent.sequence.SequenceTrace$SequenceTraceItem.toString(SequenceTrace.java:244)
    at com.manageengine.apminsight.agent.sequence.SequenceTrace.toString(SequenceTrace.java:196)
    at java.lang.String.valueOf(String.java:2827)
    at java.lang.StringBuilder.append(StringBuilder.java:115)
    at java.util.AbstractMap.toString(AbstractMap.java:490)
    at java.lang.String.valueOf(String.java:2827)
    at com.manageengine.apminsight.agent.metrics.data.transport.MetricDataDispatcher.dispatchSequenceTrace(MetricDataDispatcher.java:102)
    at com.manageengine.apminsight.agent.metrics.data.transport.MetricDataDispatcher.run(MetricDataDispatcher.java:56)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:442)
    at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
    at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:885)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.java:619)


Is the agent causing problems on the server? apminsight-javaagent.jar file is dated 28.8.2012 and was included in the newest service pack...

regards,

makeljoh

















































































                New to ADSelfService Plus?