Cold starts are a big drawback — Java and Spring Boot don’t Boot fast, and a typical full-fat Spring Boot converted lambda can take anywhere from 10 to 90 seconds, depending on the amount of memory and CPU you allocate. This may force you to overconfigure them to compensate for cold starts, but it’s a very expensive sledgehammer. There’s always preset concurrency, but it’s not much cheaper (and negates the response scalability of lambda, because you have to predict in advance how much you’ll need).
But what if I told you that same feature can be started from a cold boot in three seconds? It’s still a bit sluggish compared to other languages, but it’s pretty groundbreaking considering the comparable Boot times of the Sprint Boot JAR in containers or Lambda. This is probably because of GraalVM.
In this article, I’ll show you how to adapt a real-world example REST application (Spring Petclinic) to Spring-Cloud-Function and use GraalVM to significantly speed up cold startup times while reducing memory /CPU footprint.
I’ll finish the GitHub example I put together, feel free to follow along and borrow it for your own purposes.
Disclaimer – GraalVM is still in beta at the time of writing this article, and you may encounter issues other than those documented here. Caution is recommended if this approach is used for production workloads.
Migrating to GraalVM
First, I followed the Spring guide to get started with GraalVM.
The trick is to optimize build time as much as possible. You can push build time initialization as much as possible. By default, Spring Native initializes all classes at run time (which doesn’t offer much benefit over normal JVMS with JIT combinations), but you can explicitly declare classes that should be initialized at build time.
There is a good article here that discusses this default behavior and how to determine which classes are candidates for initialization at build time. Spring Native greatly simplifies this because it already knows all the Spring framework classes suitable for initialization at startup and configures the Native Image build accordingly.
GraalVM is fairly compatible with Spring and Spring Boot, however, there is a list of known issues between the two that, while hopefully fixed over time, are worth noting for now as they may trip you up. I’ve collected a list of the problems I’ve encountered along the way – there are ways to solve these problems, but they may not work for every application.
First, you need to add some dependencies and plug-ins to pom.xml to support GraalVM use.
This is based on my previous post that showed how to port a Spring Boot application to Lambda, so I won’t include those details here. You can view my full POM here, but specifically, this is what happens when you add the following to a lambda profile:
<properties> ... <repackage.classifier>exec</repackage.classifier> </properties> ... <dependency> <groupId>org.springframework.experimental</groupId> <artifactId>spring-native</artifactId> The < version > 0.10.3 < / version > < / dependency > < / dependencies >... <plugin> <groupId>org.springframework.experimental</groupId> <artifactId>spring-aot-maven-plugin</artifactId> <version>0.10.3</version> <executions> <id>test-generate</id> <goals> test-generate</goal> </goals> </execution> <execution> <id>generate</id> <goals> <goal>generate</goal> </goals> </execution> </executions> </plugin> <plugin> <groupId>org.hibernate.orm.tooling</groupId> <artifactId>hibernate-enhance-maven-plugin</artifactId> <version>5.4.30.Final</version> <executions> <configuration> <failOnError>true</failOnError> <enableLazyInitialization>true</enableLazyInitialization> <enableDirtyTracking>true</enableDirtyTracking> <enableAssociationManagement>true</enableAssociationManagement> <enableExtendedEnhancement>false</enableExtendedEnhancement> </configuration> <goals> <goal>enhance</goal> </goals> </execution> </executions> </plugin> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-deploy-plugin</artifactId> <configuration> <skip>true</skip> </configuration> </plugin> <plugin> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-maven-plugin</artifactId> <configuration> <classifier>${repackage.classifier}</classifier> </configuration> </plugin> <plugin> < the groupId > org. Graalvm. Buildtools < / groupId > < artifactId > native - maven - plugin < / artifactId > < version > 0.9.4 < / version > <executions> <execution> <goals> <goal>build</goal> </goals> <phase>package</phase> </execution> <execution> <id>test</id> <goals> <goal>test</goal> </goals> <phase>test</phase> </execution> </executions> <configuration> <buildArgs> --enable-url-protocols=http -H:+AddAllCharsets </buildArgs> </configuration> </plugin> <plugin> <artifactId>maven-assembly-plugin</artifactId> <executions> <execution> <id>native-zip</id> <phase>package</phase> <goals> <goal>single</goal> </goals> <inherited>false</inherited> </execution> </executions> <configuration> <descriptors> <descriptor>src/assembly/native.xml</descriptor> </descriptors> </configuration> </plugin> ``` ``` The above configuration has a few points worth mentioning: - 'hibernate-enhance-maven-plugin' - this allows Hibernate to optimize many of the things it does at build time to reduce startup time. It does not have to be used with Lambda or GraalVM - you can also use it on standard applications - 'spring-boot-maven-plugin' 'native-image' - the classifier property prevents Spring Boot from using incompatible Spring Boot Uber Jar overrides the Jar - 'native- Maven-plugin' used by the tool - this is where all the magic happens, as I'll cover in more detail later. One important part is' < Configuration > ', which allows you to control all aspects of the native image build process. - 'maven-assembly-plugin' - this is used to get the binaries we will create, Packaged in a ZIP archive along with the Boot script used by AWS Lambda is most of the configuration required to get spring-Cloud-function (or standard Spring Boot application) and generate native binaries from it. The next step is to run a Maven package command to start it. If you're like me, you'll want to run the build process in a Docker container that has been pre-configured with Java and GraalVM. Here are the images and commands I used to mount the application code and the '.m2 'directory into the container:Copy the codedocker run -v $(pwd):/petclinic -v ~/.m2:/root/.m2 -it –name petclinic-graalvm ghcr.io/graalvm/graalvm-ce:latest bash
Copy the codeWhile in this container, you can run the following commands to trigger builds (skipTests are purely for speed and are not recommended for your application!) :
./mvnw clean package -D skipTests -P lambda
Copy the codeThe first problem I encountered (and more was documented in the end) was that Devtools didn’t support it yet:
<! -- <dependency>--> <! -- <groupId>org.springframework.boot</groupId>--> <! -- <artifactId>spring-boot-devtools</artifactId>--> <! -- <optional>true</optional>--> <! -- </dependency>-->Copy the codeRun the Maven command again with a cuppa and the build completes successfully:
Although we have a binary at this point, we cannot run it in AWS Lambda. It’s not a JAR file, so we can’t just upload it and tell Lambda to execute it in the Java runtime.
Use custom runtimes
The next thing I need to know is how to get GraalVM native images to run in AWS Lambda. I knew there was the ability to build custom runtimes in AWS Lambda, but I had never tried this before, so this was new territory for me. I’m curious about how AWS Lambda takes the JAR and handler classes and boots them into the JVM. I think I need to know this to understand how to build the equivalent custom runtime for our native binaries.
It turns out AWS Lambda treats your JAR file as a ZIP. Not a jar. So metadata like Jar Manifest and main-class configurations are irrelevant. This article gives you a good look at what goes on behind the scenes and how to build your artifacts directly into ZIP files if you wish. This is TL; DR: Lambda adds the expanded JAR contents (including your handler classes) as a custom classpath, rather than running your JAR directly (using java-jar myjar.jar.
In effect, AWS Lambda runs your handler by including this class and all the other classes in the bundled ZIP into the classpath, and then executes its own Lambda Java runtime, which handles requests and responses from incoming and outgoing handler classes. If you are using the latest version of Spring-Cloud-Function (3.2.0-M1 at the time of this writing), you can see that the FunctionInvoker class is configured for the handler to initialize the Spring Boot application context as part of its constructor.
Great, but how do I write a custom runtime that interoperates between Lambda and my binary? Well, by reading more, I’ve learned that the Lambda API is RESTful, and interacting with it depends on the runtime. In addition, this endpoint is provided to all runtimes through the AWS_LAMBDA_RUNTIME_API environment variable. I started thinking about how to write a bash script to poll this endpoint and call my binaries, passing the event payload, but this felt cumbersome and meant that the application would have to regenerate on every request that felt wrong.
After some groping, I finally understand! I wonder if the Spring-Cloud-Function team has raised this issue? Of course, it turns out that by doing a quick search in my code for the code, AWS_LAMBDA_RUNTIME_API I found the CustomRuntimeEventLoop and CustomRuntimeInitializer classes, perfect!
Custom run-time event loops
There is already an example of how to run Spring Cloud functionality using GraalVM:
Be sure to set the following to trigger the Spring-Cloud function to run the CustomRuntimeEventLoop
spring.cloud.function.web.export.enabled=true
spring.cloud.function.web.export.debug=true
spring.main.web-application-type=none
debug=true
Copy the codeIn fact, I notice that you should not be enabled when debugging spring. Cloud. Function. Web. Export. Enabled, because it can lead to CustomRuntimeInitializer stop CustomRuntimeEventLoop.
AWS Lambda allows you to provide a custom runtime that runs on Amazon Linux by providing bootstrapshell scripts. You can use it to bootstrap applications written in multiple languages. But for us, all we need to do is execute our binary:
#! /bin/sh cd ${LAMBDA_TASK_ROOT:-.} ./spring-petclinic-restCopy the codeFinally, we just need to bundle the bootstrap script and binary into a ZIP file that we can upload to the AWS Lambda. This is what the Maven-assembly-plugin does, using the following configuration/SRC /assembly/native.xml
< the assembly XMLNS = "http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.2" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" Xsi: schemaLocation = "HTTP: / / http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.2 https://maven.apache.org/xsd/assembly-1.1.2.xsd "> < id > native - zip < / id > < formats > < format > zip < / format > < / formats > <baseDirectory></baseDirectory> <fileSets> <fileSet> <directory>src/shell</directory> <outputDirectory>/</outputDirectory> <useDefaultExcludes>true</useDefaultExcludes> <fileMode>0775</fileMode> <includes> <include>bootstrap</include> </includes> </fileSet> <fileSet> <directory>target</directory> <outputDirectory>/</outputDirectory> <useDefaultExcludes>true</useDefaultExcludes> <fileMode>0775</fileMode> <includes> <include>spring-petclinic-rest</include> </includes> </fileSet> </fileSets> </assembly>Copy the codeAt this point, we have a bundled ZIP file that contains everything we need to run the GraalVM binaries on our custom runtime on AWS, Huzzah!
Configure in CDK
In my CDK code, I have a lambda stack that contains all the code needed to build and deploy GraalVM lambda.
The GraalvM-CE: LatestDocker image I used earlier to build binaries can also be used in the CDK process. The main difference is that when using our code /asset-input in the CDK framework, we have to put the final.zip file in the /asset-output folder so that CDK can extract and upload it to AWS Lambda:
Const bundlingOptions = {bundlingOptions = {bunkerimage.fromregistry ("ghcr. IO/GraalVM/Graalvm-CE :21.2.0"), command: [ "/bin/sh", "-c", ["cd /asset-input/ ", "./mvnw clean package -P lambda -D skipTests ", "Cp /asset-input/target/spring-petclinic-rest-2.4.2-native zip. Zip /"]. Join (" && ")], outputType: BundlingOutput.ARCHIVED, user: 'root', volumes: [{hostPath: `${homedir()}/.m2`, containerPath: '/root/.m2/'}] } };Copy the codeTo run the GraalVM lambda function, it must run in the Amazon Linux 2 runtime. I’ve extracted the basic configuration of a function into the following, so I can reuse it in my two sample lambdas:
const baseProps = { vpc: props? .vpc, runtime: Runtime.PROVIDED_AL2, code: Code.fromAsset(path.join(__dirname, '.. /.. /'), bundlingOptions), handler: 'duff.Class', vpcSubnets: { subnetType: ec2.SubnetType.PRIVATE }, memorySize: 256, timeout: Duration.minutes(1), securityGroups: [lambdaSecurityGroup] }Copy the codeIf you want to see the difference between Java Lambda deployments, you can compare this file between my Java and GraalVM branches. One major improvement is the significant reduction in memory required – while Java Lambda doesn’t necessarily need 3GB to work, it needs to reduce the cold startup time to around 20 seconds, which is still far from ideal.
Some of you may be looking at this and thinking “What is duff.class?” . I’m not sure if this was an oversight on my part or a potential misconfiguration, But if you use your org. Springframework. Cloud. The function. The adapter. Aws. FunctionInvoker spring – cloud – function CustomEventRuntimeLoop won’t start. A specific check is made on the use of this handler, which appears to assume that it is running the Java runtime on AWS Lambda in the standard (if it is in use).
Changing the handler to anything else (it doesn’t even have to be a real class) triggers the CustomEventRuntimeLoop to effectively act as a handler for the entry point in the custom runtime, rather than the FunctionInvoker used in the Java runtime.
Learn more JAVA knowledge and tips and follow private bloggers
The deployment of Lambda
The last thing to do is deploy the Lambda and supporting resources (VPCS, RDS MySQL instances, and so on). If you’re looking at my GitHub repository, you can do the following and have a full working setup from scratch in 30 minutes:
cdk deploy --require-approval=never --all
Copy the codeFrom there, you’ll deploy a load balancer that routes traffic to the newly created GraalVM Lambda with impressive (for Java) cold start times:
conclusion
This is a real example of taking a “full-fat” Spring Boot application and converting it to a responsive Lambda using GraalVM. You can also optimize Spring Boot and GraalVM to further improve cold startup times, but with minimal configuration this still results in impressive startup times.
It was not an easy journey and I spent a long time encountering various rabbit holes. To help those who want to try this feature out on their own apps, I’ve compiled a list of the problems I’ve encountered below.
Q&A
-
Build-time issues
Out of memory
For me, GraalVM at its peak used 10GB of memory to build native binaries. When I ran the build in the Docker container, the Docker VM runtime on my Mac was a measly 2GB. Frustratingly, all you have to do is this mysterious error code 137:
Classes that are unintentionally initialized at build time
Error: Classes that should be initialized at run time got initialized during image building:
jdk.xml.internal.SecuritySupport was unintentionally initialized at build time. To see why jdk.xml.internal.SecuritySupport got initialized use --trace-class-initialization=jdk.xml.internal.SecuritySupport
javax.xml.parsers.FactoryFinder was unintentionally initialized at build time. To see why javax.xml.parsers.FactoryFinder got initialized use --trace-class-initialization=javax.xml.parsers.FactoryFinder
Copy the codeBy default, Spring Native sets classes to be initialized at run time unless explicitly stated in configuration at build time. Many of the benefits of GraalVM are optimized load times by initializing appropriate classes at build time rather than run time. Spring does a lot of this OOTB, identifying classes that can be initialized at build time and setting this configuration up for use by native images.
Spring-boot-aot-plugin does a lot of introspection and determines which classes are candidates for build-time initialization, and then generates a “native image” property file used by the GraalVM compiler to know which classes are initialized at build time. If you receive a “class that should be initialized at run time was initialized during image build” error, it is likely that the class that was marked initialized at build time inadvertently initialized another class that was not explicitly marked at build time.
If this happens, you can use the flag native-Maven-plugin in the POM configuration with –trace-class-initialization, and then re-run the build:
<configuration>
<buildArgs>
--trace-class-initialization=jdk.xml.internal.SecuritySupport
</buildArgs>
</configuration>
</plugin>
Copy the codeThis then outputs the call stack that caused the class to be initialized.
native-image.propertiesFile,resources/META-INF/native-imageThis contains a comma-separated list of classes that you want to initialize:
Args = --initialize-at-build-time=jdk.xml.internal.SecuritySupport
Copy the codeUnfortunately, in this case, once you start finding more and more classes that need to be initialized at build time, it becomes a bit of a rabbit hole. You end up with a thread that tries to generate a thread that cannot happen at build time.
Anything launched at build time that depends on being active at run time is not going to be as good. GraalVM has some checks built in to detect various situations and fail quickly. One example is threads — if a class initialization produces a thread for some reason, GraalVM selects these and notifies you:
logback.xmlBased on configuration. Deleting this file, which would require moving to a different way to configure logging, solves this problem.
At this point, we’ve built a working binary. Great! Unfortunately, it seems that many problems tend to spread at run time.
-
Runtime problems
This is where the feedback loop becomes so long, because you have to build the image before the problem occurs (which took eight minutes to complete on my laptop).
In this experiment, my test loop involved building a new binary and then uploading it to AWS, repeating as I worked through the problem. Ideally, we would test this locally to speed up the feedback loop significantly. I suspect this is easy to do, but without time to explore further, I’ll probably write a follow-up article showing how to do it.
A MalformedURLException occurred at startup
Caused by: java.net.MalformedURLException: Accessing an URL protocol that was not enabled. The URL protocol http is supported but not enabled by default. It must be enabled by adding the --enable-url-protocols=http option to the native-image command.
How to solve this problem? Enable HTTP support in Native Image Builder:
< plugin > < groupId > org. Graalvm. Buildtools < / groupId > < artifactId > native - maven - plugin < / artifactId > < version > 0.9.4 < / version > . <configuration> <buildArgs> --enable-url-protocols=http </buildArgs> </configuration> </plugin>Copy the codeMissing reflection configuration
Whenever this message occurs, it is because the class is referenced through the reflection API, and you need to add reflection configuration for the mentioned class.
You can do this in code (such as your base @SpringBootApplication class) using the Spring Aot@nativeHint annotation at the top of the configuration class, Alternatively, you can create a reflect-config.json file read by the native image tool.
Sometimes it’s not obvious — Spring tries to advise where it can, but it can only advise on what it knows. There are many errors that do not clearly indicate the problem.
I’ve come across some variations of this error, but the solution is always the same – add reflection configuration meta-inf /native-image/reflect-config.json to:
[
{
"name": "org.springframework.context.annotation.ProfileCondition",
"allDeclaredConstructors": true,
"allPublicConstructors": true,
"allDeclaredMethods": true,
"allPublicMethods": true,
"allDeclaredClasses": true,
"allPublicClasses": true
}
...
]
Copy the codeThis is a fairly rough reflection configuration – you can be more specific about what you want to enable reflection. However, doing the above will yield quick results.
Below are all the errors I encountered during migration to GraalVM, all of which required adding the above configuration for the affected classes. These are also in the order in which they occur:
Caused by: org.springframework.beans.BeanInstantiationException: Failed to instantiate [com.zaxxer.hikari.HikariDataSource]: Factory method 'dataSource' threw exception; nested exception is java.lang.IllegalStateException: Cannot load driver class: com.mysql.jdbc.Driver
Copy the codeCaused by: java.io.UncheckedIOException: com.amazonaws.lambda.thirdparty.com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Cannot construct instance of com.amazonaws.services.lambda.runtime.events.APIGatewayProxyRequestEvent (no Creators, like default construct, exist): cannot deserialize from Object value (no delegate- or property-based Creator)
Copy the codeCaused by: java.lang.ClassCastException: com.amazonaws.services.lambda.runtime.events.APIGatewayProxyResponseEvent cannot be cast to byte[]
at org.springframework.cloud.function.adapter.aws.AWSLambdaUtils.generateOutput(AWSLambdaUtils.java:173) ~[na:na]
Copy the codeLearn more JAVA knowledge and tips from private bloggers at docs.qq.com/doc/DQ2Z0eE…
Free learning to receive JAVA courseware, source code, installation package and other information
This made me look at it for a while because it didn’t seem relevant. I don’t remember whether I understand it, but I believe that with the use of generics, the lack of reflection information APIGatewayProxyResponseEvent and type erasure, leading byte [] in the early days of this process has not been Jackson should convert. Add reflection information to APIGatewayProxyResponseEvent to solve the problem.
Unsupported character encoding
Caused by: org.springframework.jdbc.CannotGetJdbcConnectionException: Failed to obtain JDBC Connection; nested exception is java.sql.SQLException: Unsupported character encoding 'CP1252'
Copy the codeAs a result, only part of the character encoding is embedded in the binary when the native image is built. If you want them, you must ask for them:
< plugin > < groupId > org. Graalvm. Buildtools < / groupId > < artifactId > native - maven - plugin < / artifactId > < version > 0.9.4 < / version > . <configuration> <buildArgs> --enable-url-protocols=http -H:+AddAllCharsets </buildArgs> </configuration> </plugin> ``` # # # `Copy the code