Java memory and container

Properly limiting the JVM’s memory usage (Xmx isn’t enough)

This whole section taken from Properly limiting the JVM’s memory usage (Xmx isn’t enough), by Matt Rasband (3 Jun, 2017) medium1

The JVM is known to be greedy with memory and is infamously difficult to tune. This is pretty apparent when you set your -Xmx and find that the application is exceeding that value, and even more apparent when running a JVM based application in Docker — because the JVM can see the host’s memory in many cases. The error can manifest any number of ways such as higher latencies due to garbage collection or memory swapping, and in some cases (such as in Docker) getting OOMed.

Most solutions to this issue suggest just setting -Xmx256m and calling it a day. Unfortunately, that only limits the max heap size, not the total amount of memory the JVM will utilize as you need to account for metaspace, class space, stack size, and more. You can read a bit more in depth here. In short, the actual maximum utilized memory by your application is a function (credit to the link above):

Max memory = [-Xmx] + [-XX:MaxPermSize] + number_of_threads * [-Xss]

Of course the JVM itself needs some space to do its thing as well, so there is still a bit of overhead there too.

Long story short, just setting -Xmxis only going to defer when your application shows symptoms of using more memory than expected. Depending on usage volume the symptom can be deferred much longer, but eventually the symptoms of improper JVM tuning will be visible. In cases such as usage in Docker, we had seen cascading restarts due to service dependencies (only things like a central config and service discovery service really caused this).

#!/usr/bin/env bash

# You should make these values be dynamic based on your env, but I made
# them static to make the usage a little more obvious:
APPLICATION_TOTAL_MEMORY=512M  # how much memory is available, see /sys/fs/cgroup/memory/* if in Docker
APPLICATION_SIZE_ON_DISK_IN_MB=120  # how big on disk, e.g. `du -h target/*.jar`

# Calculations based on "rules of thumb" from
# https://github.com/dsyer/spring-boot-memory-blog/blob/master/cf.md
loaded_classes=$(($APPLICATION_SIZE_ON_DISK_IN_MB * 400))
stack_threads=$((15 + $APPLICATION_SIZE_ON_DISK_IN_MB * 6 / 10))

docker_opts=$(java-buildback-memory-calculator \
  -loadedClasses $loaded_classes \
  -poolType metaspace \
  -stackThreads $stack_threads \
  -totMemory ${APPLICATION_TOTAL_MEMORY})

java -jar ${application} ${docker_opts}

Sensible default for Spring boot applications

Using Cloud Foundry JVM Memory Calculator

The Java buildpack memory calculator determines values for JVM memory options with the goal of enabling an application to perform well while not exceeding the total memory available in a container (which results in the application being killed).2

The buildpack provides the following inputs to the memory calculator:2

the total memory available to the application,
an optional head room (a percentage of the total memory available, default 0) which should not be allocated,
an estimate of the number of threads that will be used by the application,
an estimate of the number of classes that will be loaded,
the type of JVM pool used in the calculation ('permgen' for Java 7 and 'metaspace' for Java 8 and later),
any JVM options specified by the user.

The java buildpack in Cloud Foundry calculates the memory settings for a java process. It has a hard job because it only has one input (the container memory limit) and it needs to come up with at least 5 numbers for the JVM. To do this it uses a standalone memory calculator program. We downloaded the memory calculator and used it to drive some tests on memory usage in a Spring Boot application. 3

Here are the command line options generated by the default settings with some typical container memory limits:3

Container

-Xmx (Heap)

-XX:MaxMetaspaceSize (Metaspace)

-Xss (Stack)

128m

54613K

64M

568K

256m

160M

64M

853K

512m

382293K

64M

995K

768M

104857K

$ java-buildpack-memory-calculator-linux -memorySizes='metaspace:64m..' -memoryWeights=heap:75,metaspace:10,native:10,stack:5 -memoryInitials=heap:100%,metaspace:100% -totMemory=128m
-Xmx54613K -XX:MaxMetaspaceSize=64M -Xss568K -Xms54613K -XX:MetaspaceSize=64M

Running a JVM in a Container Without Getting Killed

The JDK 8u131 has backported a nice feature in JDK 9, which is the ability of the JVM to detect how much memory is available when running inside a Docker container.4

$ docker run -m 1GB openjdk:8u131 java \
  -XX:+UnlockExperimentalVMOptions \
  -XX:+UseCGroupMemoryLimitForHeap \
  -XX:MaxRAMFraction=1 -XshowSettings:vm -version
VM settings:
    Max. Heap Size (Estimated): 910.50M
    Ergonomics Machine Class: server
    Using VM: OpenJDK 64-Bit Server VM

In Java 10 there is improved container integration. No need to add extra flags, the JVM will use 1/4 of the container memory for heap.5

$ docker run -m 1GB openjdk:10 java -XshowSettings:vm \
    -version
VM settings:
    Max. Heap Size (Estimated): 247.50M
    Using VM: OpenJDK 64-Bit Server VM

openjdk version "10.0.1" 2018-04-17
OpenJDK Runtime Environment (build 10.0.1+10-Debian-4)
OpenJDK 64-Bit Server VM (build 10.0.1+10-Debian-4, mixed mode)

Java 10 obsoletes the -XX:MaxRAM parameter, as the JVM will correctly detect the value.

You can still use the -XX:MaxRAMFraction=1 option to squeeze all the memory from the container.5

$ docker run -m 1GB openjdk:10 java -XshowSettings:vm \
    -XX:MaxRAMFraction=1 -version
OpenJDK 64-Bit Server VM warning: Option MaxRAMFraction was deprecated in version 10.0 and will likely be removed in a future release.
VM settings:
    Max. Heap Size (Estimated): 989.88M
    Using VM: OpenJDK 64-Bit Server VM

openjdk version "10.0.1" 2018-04-17
OpenJDK Runtime Environment (build 10.0.1+10-Debian-4)
OpenJDK 64-Bit Server VM (build 10.0.1+10-Debian-4, mixed mode)

But it can be risky if your container uses off heap memory, as almost all the container memory is allocated to heap. You would have to either set -XX:MaxRAMFraction=2 and use only 50% of the container memory for heap, or resort to Xmx.5

Java 10 and docker memory reporting

For java 10 to get the correct memory, it also needs a later version of docker:

Wrong: Max. Heap Size (Estimated): 15.69G

$ docker version
Client:
 Version:         1.12.6
 API version:     1.24
 Package version: docker-1.12.6-68.gitec8512b.el7.x86_64
 Go version:      go1.8.3
 Git commit:      ec8512b/1.12.6
 Built:           Thu Nov 16 15:19:17 2017
 OS/Arch:         linux/amd64

Server:
 Version:         1.12.6
 API version:     1.24
 Package version: docker-1.12.6-68.gitec8512b.el7.x86_64
 Go version:      go1.8.3
 Git commit:      ec8512b/1.12.6
 Built:           Thu Nov 16 15:19:17 2017
 OS/Arch:         linux/amd64
$ docker run -m 1GB openjdk:10 java -XshowSettings:vm -version
VM settings:
    Max. Heap Size (Estimated): 15.69G
    Using VM: OpenJDK 64-Bit Server VM

openjdk version "10.0.2" 2018-07-17
OpenJDK Runtime Environment (build 10.0.2+13-Debian-2)
OpenJDK 64-Bit Server VM (build 10.0.2+13-Debian-2, mixed mode)

Correct: Max. Heap Size (Estimated): 247.50M

$ docker version
Client: Docker Engine - Community
 Version:           18.09.0
 API version:       1.39
 Go version:        go1.10.4
 Git commit:        4d60db4
 Built:             Wed Nov  7 00:47:43 2018
 OS/Arch:           darwin/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          18.09.0
  API version:      1.39 (minimum version 1.12)
  Go version:       go1.10.4
  Git commit:       4d60db4
  Built:            Wed Nov  7 00:55:00 2018
  OS/Arch:          linux/amd64
  Experimental:     true

$docker run -m 1GB openjdk:10 java -XshowSettings:vm -version
VM settings:
    Max. Heap Size (Estimated): 247.50M
    Using VM: OpenJDK 64-Bit Server VM

openjdk version "10.0.2" 2018-07-17
OpenJDK Runtime Environment (build 10.0.2+13-Debian-2)
OpenJDK 64-Bit Server VM (build 10.0.2+13-Debian-2, mixed mode)

Allow more flexibility in selecting Heap % of available RAM

Three new flags based on percentages, from 0.0 to 100.0 6

-XX:MaxRAMPercentage
-XX:MinRAMPercentage
-XX:InitialRAMPercentage

MaxRAM vs mx

-Xmx specifies the precise upper limit for the heap. It is the preferred way to set the heap size.

-XX:MaxRAM does not define the heap size directly. Instead this parameter overrides the actual amount of physical RAM when calculating the heap limits basing on ergonomics.

If -Xmx is set, MaxRAM is never used. Otherwise the maximum heap size is estimated as7

MaxHeapSize = MaxRAM * MaxRAMPercentage / 100% (default MaxRAMPercentage=25)

null-pointer