Domino MTC Crashes with Java Error Insufficient Memory - Local Heap is Full

Mindwatering Incorporated

Author: Tripp W Black

Created: 08/29 at 09:45 PM

 

Category:
Domino Server Issues Troubleshooting
Server Crashes

Issue:
Message Tracking Center (MTC) task crashed Domino MTC task. However, this type of crash left Domino "up" but not servicing requests. In addition, the NSD/auto-restart of Domino services did not happen.


Error Message:
1321870:000002-00007F7AA9B05740] 08/26/2024 04:23:28 PM MT Collector: Error in creating entry for protocol lines: ,,,,,,,,,,,"1439602289.23311724680176772.JavaMail.wbbobr",,,,"emailaddress@corpdomain.com",,,,, ,,4,08/26/2024 13:49:41 GMT,"",,,,,,"OFF2701F2C:E6E58FA6 ONC1CB1A8E:FBD10328",,"1439602289.23311724680176772.JavaMail.wbbobr",,,,,,"Message Subject Here",, : Insufficient memory - local heap is full
...
1321870:000002-00007F7AA9B05740] 08/26/2024 04:23:35 PM MT Collector: Failed updating entry in message tracking store: [inmsgid]=1Oz1ue^abQ6g2TybaPvfDMvabc.^Q.1724670218554 & [inrcp]=sendingemailaddress@originatingdomain.com: Insufficient memory - local heap is full
...


Resolution:
The HTTP JVM has its own Java heap/pool of memory according to HCL as of 2024/08/28. See section at the bottom of the page.

The default Heap size has increased over the releases but the default maximum is 256MB.
The resolution is to increase the heap size above 256MB to a higher number.
e.g.
JavaMaxHeapSize = 1024


Monitoring code usage w/in the Java Application (e.g. Notes or Domino agent):
...
Runtime s_runtime = Runtime.getRuntime ();
System.out.println(String.valueOf(s_runtime.totalMemory() - s_runtime.freeMemory()))
...


Other Application Recommendations:
- Recycle (destroy) all Java objects via object.Recycle() when they are no longer being used/out-of-scope.
- When looping view and document collections kill the current object before setting it to the next on in a loop. Use a temporary variable.
- Garbage collection in a JVM is not consistent between JVM platforms and versions. Don't rely on it.


_________________


2024/08 MW and HCL JVM and Heap Interactions on Relations between HTTP JVM and this JavaHeap:
HTTP pool inside the JavaMaxHeapSizes?

JavaMaxHeapSize:
- A setting that controls the maximum heap size for the entire JVM.
- Sets the Java heap for all other Domino server processes that load a JVM (for example, amgr, periodic Java agents, or runjava).
Example:
JavaMaxHeapSize = 1024
- Default value w/no notes.ini variable: 256 MB

HTTPJVMMaxHeapSize:
- Sets the JVM heap only for the HTTP process.
- This is controlled via the HTTPJVM settings.
Example:
HTTPJVMMaxHeapSize=2048M
HTTPJVMMaxHeapSizeSet=1

- Default size w/no notes.ini variable: 1 GB

The HTTP pool is not inside the JavaMaxHeapSize. The JavaMaxHeapSize is a setting that determines the maximum heap size that the entire JVM can allocate. It controls the overall memory usage of the Java application.

The HTTP pool, on the other hand, is a collection of HTTP connections managed by an HTTP client. It is typically outside of the JVM's memory space. While it may use memory allocated from the operating system, it is not directly managed by the JavaMaxHeapSize.

HTTP pool: A collection of HTTP connections managed by an HTTP client. Therefore, the HTTP pool is not contained within the JavaMaxHeapSize.



previous page