Guide to Java Project Panama

1. Overview

In this tutorial, we’ll have a tour of the Project Panama components. We’ll first explore the Foreign Function and Memory API. Then, we’ll see how the JExtract tool facilitates its usage.

2. What Is Project Panama?

Project Panama aims to ease the interaction between Java and foreign (non-Java) APIs, i.e., native code written in C, C++, etc.

Until now, using Java Native Interface (JNI) was the solution to invoke foreign functions from Java. But JNI has some shortcomings that Project Panama solves by:

• removing the need to write intermediate native code wrappers in Java
• replacing the ByteBuffer API with a more future-proof Memory API
• introducing a platform-agnostic, secure, and memory-efficient way to invoke native code from Java

To achieve its goals, Panama includes a set of APIs and tools:

• Foreign-Function and Memory API: useful for allocating and accessing off-heap memory and calling foreign functions directly from Java code
• Vector API: enables advanced developers to express complex data parallel algorithms in Java
• JExtract: a tool to mechanically derive Java bindings from a set of native headers

3. Prerequisites

To use Foreign Function and Memory API, let’s download Project Panama Early-Access Build. At the time of this article, we used Build 19-panama+1-13 (2022/1/18). Next, we set the JAVA_HOME according to the system in use.

Due to the Foreign Function and Memory API being a preview API, we must compile and run our code with preview features enabled, i.e., by adding the –enable-preview flag to java and javac.

4. Foreign Function and Memory API

The Foreign function and Memory API helps Java programs interoperate with code and data outside of the Java runtime.

It does this by efficiently invoking foreign functions (i.e., code outside the JVM) and by safely accessing foreign memory (i.e., memory not managed by the JVM).

It combines two earlier incubating APIs: the Foreign-Memory Access API and the Foreign Linker API.

The API provides a set of classes and interfaces to do these:

• allocate foreign memory with MemorySegment, MemoryAddress, and SegmentAllocator
• control the allocation and deallocation of foreign memory through Arena (MemorySession was split into Arena and SegmentScope, starting with JDK20)
  
• manipulate structured foreign memory using MemoryLayout
• access structured foreign memory via VarHandles
• call foreign functions thanks to Linker, FunctionDescriptor, and SymbolLookup

4.1. Foreign Memory Allocation

First, let’s explore memory allocation. Here, the main abstraction is MemorySegment. It models a contiguous region of memory located either off-heap or on-heap. MemoryAddress is an offset within a segment. Simply put, a memory segment is made of memory addresses, and a memory segment can contain other memory segments.

Besides, memory segments are bound to their encapsulated Arena and are released when no longer required. The Arena manages the segments’ lifecycles and ensures they are properly freed when accessed by multiple threads.

Let’s create four bytes at consecutive offsets in a memory segment and then set a float value of 6:

try (Arena memorySession = Arena.ofConfined()) {
    int byteSize = 5;
    int index = 3;
    float value = 6;
    try(Arena arena = Arena.ofAuto()) {
        MemorySegment segment = arena.allocate(byteSize);
        segment.setAtIndex(JAVA_FLOAT, index, value);
        float result = segment.getAtIndex(JAVA_FLOAT, index);
        System.out.println("Float value is:" + result);
    }
}

In our code above, a confined memory session restricts access to the thread which created the session, whereas a shared memory session allows access from any thread.

Also, the JAVA_FLOAT ValueLayout specifies the properties of the dereference operation: correctness of type mapping and the number of bytes to be dereferenced.

The SegmentAllocator abstraction defines useful operations to allocate and initialize memory segments. It can be very useful when our code manages a large number of off-heap segments:

String[] greetingStrings = { "hello", "world", "panama", "baeldung" };
try(Arena arena = Arena.ofAuto()) {
    MemorySegment offHeapSegment = arena.allocateArray(ValueLayout.ADDRESS, greetingStrings.length);
    for (int i = 0; i < greetingStrings.length; i++) {
        // Allocate a string off-heap, then store a pointer to it
        MemorySegment cString = arena.allocateUtf8String(greetingStrings[i]);
        offHeapSegment.setAtIndex(ValueLayout.ADDRESS, i, cString);
    }
}

4.2. Foreign Memory Manipulation

Next, we dive into memory manipulation with memory layouts. A MemoryLayout describes a segment’s content. It’s useful to manipulate high-level data structures of native code such as structs, pointers, and pointers to structs.

Let’s use a GroupLayout to allocate off-heap a C struct representing a point with x and y coordinates:

GroupLayout pointLayout = structLayout(
    JAVA_DOUBLE.withName("x"),
    JAVA_DOUBLE.withName("y")
);

VarHandle xvarHandle = pointLayout.varHandle(PathElement.groupElement("x"));
VarHandle yvarHandle = pointLayout.varHandle(PathElement.groupElement("y"));

try (Arena memorySession = Arena.ofConfined()) {
    MemorySegment pointSegment = memorySession.allocate(pointLayout);
    xvarHandle.set(pointSegment, 3d);
    yvarHandle.set(pointSegment, 4d);
    System.out.println(pointSegment.toString());
}

It’s worth noting that offset computations aren’t needed since distinct VarHandles are used to initialize each point coordinate.

We can also construct arrays of data by using a  SequenceLayout. Here’s how to get a list of five points:

SequenceLayout ptsLayout = sequenceLayout(5, pointLayout);

4.3. Native Function Calls From Java

Foreign Function API allows Java developers to consume any native library without relying on a third-party wrapper. It heavily relies on Method Handles and provides three main classes: Linker, FunctionDescriptor, and SymbolLookup.

Let’s consider printing a “Hello world” message by calling the C printf() function:

#include <stdio.h>
int main() {
    printf("Hello World from Project Panama Baeldung Article");
    return 0;
}

First, we lookup the function in the class loader of the standard library:

Linker nativeLinker = Linker.nativeLinker();
SymbolLookup stdlibLookup = nativeLinker.defaultLookup();
SymbolLookup loaderLookup = SymbolLookup.loaderLookup();

A Linker is a bridge between two binary interfaces: the JVM and C/C++ native code, also known as C ABI.

Following, we need to describe the function prototype:

FunctionDescriptor printfDescriptor = FunctionDescriptor.of(JAVA_INT, ADDRESS);

Value layouts JAVA_INT and ADDRESS, respectively, correspond to the return type and input of the C printf() function:

int printf(const char * __restrict, ...)

Next, we get the method handle:

String symbolName = "printf";
String greeting = "Hello World from Project Panama Baeldung Article";
MethodHandle methodHandle = loaderLookup.lookup(symbolName)
  .or(() -> stdlibLookup.lookup(symbolName))
  .map(symbolSegment -> nativeLinker.downcallHandle(symbolSegment, printfDescriptor))
  .orElse(null);

The Linker interface enables both downcalls (calls from Java code to native code) and upcalls (calls from native code back to Java code). Finally, we invoke the native function:

try (Arena memorySession = Arena.ofConfined()) {
     MemorySegment greetingSegment = memorySession.allocateUtf8String(greeting);
     methodHandle.invoke(greetingSegment);
}

5. JExtract

With JExtract, there’s no need to directly use most of the Foreign Function & Memory API abstractions. Let’s recreate printing our “Hello World” example above.

First, we need to generate Java classes from the standard library headers:

jextract --source --output src/main -t foreign.c -I c:\mingw\include c:\mingw\include\stdio.h

The path to stdio must be updated to the one in the target OS. Next, we can simply import the native printf() function from Java:

import static foreign.c.stdio_h.printf;

public class Greetings {

    public static void main(String[] args) {
        String greeting = "Hello World from Project Panama Baeldung Article, using JExtract!";

        try (Arena memorySession = Arena.ofConfined()) {
            MemorySegment greetingSegment = memorySession.allocateUtf8String(greeting);
            // Generate JExtract bingings before uncommenting
            //  printf(greetingSegment);
        }
    }
}

Running the code prints the greeting to the console:

java --enable-preview --source 21 .\src\main\java\com\baeldung\java\panama\jextract\Greetings.java

6. Conclusion

In this article, we learned about the key features of Project Panama.

First, we explored native memory management with Foreign Function and Memory API. Then we called foreign functions using MethodHandles. Finally, we used to JExtract tool to hide the complexity of the Foreign Function and Memory API.

There’s a lot more to discover with Project Panama, particularly calling Java from native code, calling third-party libraries, and the Vector API.