2

I have a Java app that include a big file into its resource (650MB).

Somewhere in my code, I'm using this resource with :

this.getClass().getResourceAsStream("...")

When running the app in the dev environment (Gradle based), the app is running without any problem.

If I build a jlink image of this app (using jdk-21.0.2+13 from temurin21) the same line will throw

java.lang.OutOfMemoryError: Java heap space

After studying the memory with VisualVM, it seems that the memory is stable in my dev env while it is not when running on the JLink image.

I guess that the resource is not retrieved the same way depending the image is packaged with JLink or dynamically running with Gradle. However I don't understand if this is an internal issue or how I can bypass it.

I know about "Xmx" option, but this is not the solution : running with "Xmx512M" will be fine in dev but will cause OOM in jlink image.

Here is the full stack trace of the error :

java.lang.OutOfMemoryError: Java heap space at java.base/jdk.internal.jimage.BasicImageReader.getBufferBytes(Unknown Source) at java.base/jdk.internal.jimage.BasicImageReader.getResourceBuffer(Unknown Source) at java.base/jdk.internal.jimage.ImageReader.getResourceBuffer(Unknown Source) at java.base/jdk.internal.module.SystemModuleFinders$SystemModuleReader.read(Unknown Source) at java.base/jdk.internal.module.SystemModuleFinders$SystemModuleReader.open(Unknown Source) at java.base/jdk.internal.loader.BuiltinClassLoader.findResourceAsStream(Unknown Source) at java.base/java.lang.Class.getResourceAsStream(Unknown Source)

Thanks a lot !

Improve this question
6
  • You are indeed reading a picture ("image" could have meant more). Why not compress it more (jpg). Commented Feb 26 at 15:44
  • 1) Why not use -Xmx1G or more? Unpacking a 650 MB file into 512 MB of memory will not fit. 2) A .jar file is basically a .zip file. When a resource file is packed inside a .jar file, it has to be decompressed in-memory/ on the go. Perhaps the resource loader does not stream that file block-by-block, but unpacks everything into memory before granting access. So a) try to adapt the resource loader, or b) try to access this packed resource file in a different way (like manually finding the .jar file, then using an in-memory ZipInputStream or sth alike). Commented Feb 26 at 15:48
  • Thanks for your answers. JoopEggen I don't think you understand the question at all... @JayC667 I'm not try to unpack the file into memory. Basically the file is just copied to somewhere. To copy a file or to unzip a file, you don't have to load it fully in memory. I think like you: I guess the resource loader does not stream the file. Only option a) would be correct (as with compressed jlink image, you cannot access the jar directly as its compressed in the "modules" file). I just don't know how I could dynamically change the resource loader... Commented Feb 26 at 15:50
  • 1
    getResourceAsStream("xyz") is unpacking to memory, it is probably assigning 650MB as ByteArrayInputStream Commented Feb 26 at 15:52
  • 1
    Maybe Files.copy(Paths.get(getResource("xyz").toURI()), ...) would work? Commented Feb 26 at 15:57

1 Answer 1

Reset to default
3

Yup, it's kinda dumb but that's how it works. See the source which shows it just streams the entire thing into a byte array and there's no way to avoid this (a comment mentions sys properties jdk.image.map.all and jdk.image.use.jvm.map. These don't fix this).

jlink produces a whole bunch of files that you must then pack into an installer, zip up, or otherwise move to a system or run in place. You produce this during the build. The build is in your control.

Therefore, then, you can always just add the giant resource file directly into your jlink-produced sparse jvm build directly.

Your code can then just.. load the resource file straight from disk. This is ugly, error-prone, and requires writing multiple paths that are hard to test (one for 'I am running as a bootstrapped java module' and one for all other cases).

I need to know if .getResourceAsStream will OOM on me

Ask for yourself. Check out the result:

String v = TestClass.class.getResource("TestClass.class").toString();

This marvellous 'code trick' will tell you exactly where your own class is at. If that string starts with jrt: you're in trouble - in the sense that .getResourceAsStream will just load anything that is at the same place the resource TestClass.class was loaded from (i.e., jrt - the bootstrappy module stuff that jlink makes) by just blitting the whole resource into memory first.

Thus, you write an abstraction that provides an InputStream for a given resource, and the first thing it does, is this, to know which of 2 alternate paths should be taken. If it does not start with jrt:, just .getResourceAsStream it. If you load from a jar for example it will not load the whole thing in memory first, gRAS is fine there. But if it does...

How do I locate my resource then?

It's.. tricky.

Path.of(System.getProperty("java.home"), "myResource.bigFile");

should get you there. That sysproperty will be set to the 'root' of your jlink image, i.e. the path that, if you're there, would allow you to write bin/java -m your.module to start your jlinked app. Put your resource in next to lib, bin, release, include, legal, etc, and the above will find it. You can load it from there with e.g. Files.newInputStream(thatPath).

Improve this answer
Sign up to request clarification or add additional context in comments.

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.