Maven: Create Runnable Jar with Dependency Jar Files (Fat Jar)

[Ron McLeod]

This is my first attempting at building with Maven (previously I have always used Ant).

I am trying to assemble a runnable jar file which includes the dependent jar files for netty, kafka, etc.  I am able to build without any errors, but when I try and run by jar, I find that the jars for the dependencies were not bundled in the jar which was built.  

Any hints to help me sort this out?  I'm sure hoping that it is something simple.
$ java -jar ~/dev/StmpRelayKafkaProducer/target/jars/SmtpRelayKafkaProducer.jar Error: A JNI error has occurred, please check your installation and try again Exception in thread "main" java.lang.NoClassDefFoundError: io/netty/channel/EventLoopGroup        at java.lang.Class.getDeclaredMethods0(Native Method)        at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)        at java.lang.Class.privateGetMethodRecursive(Class.java:3048)        at java.lang.Class.getMethod0(Class.java:3018)        at java.lang.Class.getMethod(Class.java:1784)        at sun.launcher.LauncherHelper.validateMainClass(LauncherHelper.java:544)        at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:526) Caused by: java.lang.ClassNotFoundException: io.netty.channel.EventLoopGroup        at java.net.URLClassLoader.findClass(URLClassLoader.java:382)        at java.lang.ClassLoader.loadClass(ClassLoader.java:424)        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)        at java.lang.ClassLoader.loadClass(ClassLoader.java:357)        ... 7 more $ jar tf ~/dev/StmpRelayKafkaProducer/target/jars/SmtpRelayKafkaProducer.jar META-INF/ META-INF/MANIFEST.MF com/ com/mcleodnet/ com/mcleodnet/netty/ com/mcleodnet/netty/syslog/ com/mcleodnet/netty/syslog/SimpleServer.class com/mcleodnet/netty/syslog/SyslogMessageParser.class com/mcleodnet/netty/syslog/SendmailEventPattern.class com/mcleodnet/netty/syslog/SyslogMessage.class com/mcleodnet/netty/syslog/SendmailEvent.class com/mcleodnet/netty/syslog/SendmailEventKafkaProducer.class com/mcleodnet/netty/syslog/SimpleServer$1.class com/mcleodnet/netty/syslog/SendmailEventMonitor.class com/mcleodnet/netty/syslog/SendmailEventExtractor.class com/mcleodnet/netty/syslog/MatchedEvent.class com/mcleodnet/netty/syslog/SimpleServer$1$1.class META-INF/maven/ META-INF/maven/com.mcleodnet/ META-INF/maven/com.mcleodnet/smtp-relay-kafka-producer/ META-INF/maven/com.mcleodnet/smtp-relay-kafka-producer/pom.xml META-INF/maven/com.mcleodnet/smtp-relay-kafka-producer/pom.properties
My pom.xml:<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> <modelVersion>4.0.0</modelVersion> <groupId>com.mcleodnet</groupId> <artifactId>smtp-relay-kafka-producer</artifactId> <version>0.0.1-SNAPSHOT</version> <packaging>jar</packaging> <build> <pluginManagement> <plugins> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-compiler-plugin</artifactId> <configuration> <source>1.8</source> <target>1.8</target> </configuration> </plugin> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-jar-plugin</artifactId> <configuration> <outputDirectory>${project.build.directory}/jars</outputDirectory> <finalName>SmtpRelayKafkaProducer</finalName> <archive> <manifest> <addClasspath>true</addClasspath> <mainClass>com.mcleodnet.netty.syslog.SimpleServer</mainClass> </manifest> </archive> </configuration> </plugin> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-dependency-plugin</artifactId> <version>2.5.1</version> <executions> <execution> <id>copy-dependencies</id> <phase>package</phase> <goals> <goal>copy-dependencies</goal> </goals> <configuration> <includeScope>runtime</includeScope> <outputDirectory>${project.build.directory}/jars</outputDirectory> </configuration> </execution> </executions> </plugin> </plugins> </pluginManagement> </build> <dependencies> <!-- https://mvnrepository.com/artifact/org.apache.kafka/kafka-clients --> <dependency> <groupId>org.apache.kafka</groupId> <artifactId>kafka-clients</artifactId> <version>2.1.0</version> </dependency> <!-- https://mvnrepository.com/artifact/org.slf4j/slf4j-simple --> <dependency> <groupId>org.slf4j</groupId> <artifactId>slf4j-simple</artifactId> <version>1.7.25</version> </dependency> <!-- https://mvnrepository.com/artifact/io.netty/netty-all --> <dependency> <groupId>io.netty</groupId> <artifactId>netty-all</artifactId> <version>4.1.32.Final</version> </dependency> <!-- https://mvnrepository.com/artifact/com.google.code.gson/gson --> <dependency> <groupId>com.google.code.gson</groupId> <artifactId>gson</artifactId> <version>2.8.5</version> </dependency> </dependencies> </project>
Console output from build:$ mvn clean install [INFO] Scanning for projects... [INFO] [INFO] ------------------------------------------------------------------------ [INFO] Building smtp-relay-kafka-producer 0.0.1-SNAPSHOT [INFO] ------------------------------------------------------------------------ [INFO] [INFO] --- maven-clean-plugin:2.4.1:clean (default-clean) @ smtp-relay-kafka-producer --- [INFO] Deleting /home/mqtt/dev/StmpRelayKafkaProducer/target [INFO] [INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ smtp-relay-kafka-producer --- [debug] execute contextualize [WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent! [INFO] Copying 0 resource [INFO] [INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ smtp-relay-kafka-producer --- [WARNING] File encoding has not been set, using platform encoding UTF-8, i.e. build is platform dependent! [INFO] Compiling 9 source files to /home/mqtt/dev/StmpRelayKafkaProducer/target/classes [INFO] [INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @ smtp-relay-kafka-producer --- [debug] execute contextualize [WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent! [INFO] Copying 0 resource [INFO] [INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ smtp-relay-kafka-producer --- [INFO] Nothing to compile - all classes are up to date [INFO] [INFO] --- maven-surefire-plugin:2.10:test (default-test) @ smtp-relay-kafka-producer --- [INFO] Surefire report directory: /home/mqtt/dev/StmpRelayKafkaProducer/target/surefire-reports ------------------------------------------------------- T E S T S ------------------------------------------------------- Results : Tests run: 0, Failures: 0, Errors: 0, Skipped: 0 [INFO] [INFO] --- maven-jar-plugin:2.3.2:jar (default-jar) @ smtp-relay-kafka-producer --- [INFO] Building jar: /home/mqtt/dev/StmpRelayKafkaProducer/target/jars/SmtpRelayKafkaProducer.jar [INFO] [INFO] --- maven-install-plugin:2.3.1:install (default-install) @ smtp-relay-kafka-producer --- [INFO] Installing /home/mqtt/dev/StmpRelayKafkaProducer/target/jars/SmtpRelayKafkaProducer.jar to /home/mqtt/.m2/repository/com/mcleodnet/smtp-relay-kafka-producer/0.0.1-SNAPSHOT/smtp-relay-kafka-producer-0.0.1-SNAPSHOT.jar [INFO] Installing /home/mqtt/dev/StmpRelayKafkaProducer/pom.xml to /home/mqtt/.m2/repository/com/mcleodnet/smtp-relay-kafka-producer/0.0.1-SNAPSHOT/smtp-relay-kafka-producer-0.0.1-SNAPSHOT.pom [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 2.015s [INFO] Finished at: Wed Dec 19 17:13:33 PST 2018 [INFO] Final Memory: 14M/131M [INFO] ------------------------------------------------------------------------

[salvin francis]

I am not much sure about this since I haven't done it, but I think this might help : https://www.mkyong.com/maven/create-a-fat-jar-file-maven-assembly-plugin/
Most of my maven builds are thin jars with dependencies deployed separately and not a far jar as your problem is stated.

[Ron McLeod]

Following the advice from this StackOverflow thread and this other StackOverflow thread, I was able to create the runnable fat jar I wanted using the Apache Maven Assembly Plugin

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> <modelVersion>4.0.0</modelVersion> <groupId>com.mcleodnet</groupId> <artifactId>smtp-relay-kafka-producer</artifactId> <version>0.0.1-SNAPSHOT</version> <packaging>jar</packaging> <build> <pluginManagement> <plugins> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-compiler-plugin</artifactId> <configuration> <source>1.8</source> <target>1.8</target> </configuration> </plugin> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-assembly-plugin</artifactId> <configuration> <outputDirectory>${project.build.directory}/jars</outputDirectory> <finalName>SmtpRelayKafkaProducer</finalName> <archive> <manifest> <mainClass>com.mcleodnet.netty.syslog.SimpleServer</mainClass> </manifest> </archive> <descriptorRefs> <descriptorRef>jar-with-dependencies</descriptorRef> </descriptorRefs> <appendAssemblyId>false</appendAssemblyId> </configuration> <executions> <execution> <id>make-assembly</id> <phase>package</phase> <goals> <goal>single</goal> </goals> </execution> </executions> </plugin> </plugins> </pluginManagement> </build> <dependencies> <!-- https://mvnrepository.com/artifact/org.apache.kafka/kafka-clients --> <dependency> <groupId>org.apache.kafka</groupId> <artifactId>kafka-clients</artifactId> <version>2.1.0</version> </dependency> <!-- https://mvnrepository.com/artifact/org.slf4j/slf4j-simple --> <dependency> <groupId>org.slf4j</groupId> <artifactId>slf4j-simple</artifactId> <version>1.7.25</version> </dependency> <!-- https://mvnrepository.com/artifact/io.netty/netty-all --> <dependency> <groupId>io.netty</groupId> <artifactId>netty-all</artifactId> <version>4.1.32.Final</version> </dependency> <!-- https://mvnrepository.com/artifact/com.google.code.gson/gson --> <dependency> <groupId>com.google.code.gson</groupId> <artifactId>gson</artifactId> <version>2.8.5</version> </dependency> </dependencies> </project>
$ mvn clean compile assembly:single [INFO] Scanning for projects... [INFO] [INFO] ------------------------------------------------------------------------ [INFO] Building smtp-relay-kafka-producer 0.0.1-SNAPSHOT [INFO] ------------------------------------------------------------------------ [INFO] [INFO] --- maven-clean-plugin:2.4.1:clean (default-clean) @ smtp-relay-kafka-producer --- [INFO] [INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ smtp-relay-kafka-producer --- [debug] execute contextualize [WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent! [INFO] Copying 0 resource [INFO] [INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ smtp-relay-kafka-producer --- [WARNING] File encoding has not been set, using platform encoding UTF-8, i.e. build is platform dependent! [INFO] Compiling 9 source files to /home/mqtt/dev/StmpRelayKafkaProducer/target/classes [INFO] [INFO] --- maven-assembly-plugin:2.2-beta-5:single (default-cli) @ smtp-relay-kafka-producer --- [INFO] META-INF/ already added, skipping [INFO] META-INF/MANIFEST.MF already added, skipping [INFO] META-INF/MANIFEST.MF already added, skipping [INFO] META-INF/MANIFEST.MF already added, skipping [INFO] org/ already added, skipping [INFO] META-INF/ already added, skipping [INFO] META-INF/MANIFEST.MF already added, skipping [INFO] org/ already added, skipping [INFO] META-INF/ already added, skipping [INFO] META-INF/MANIFEST.MF already added, skipping [INFO] org/ already added, skipping [INFO] org/slf4j/ already added, skipping [INFO] META-INF/maven/ already added, skipping [INFO] META-INF/maven/org.slf4j/ already added, skipping [INFO] META-INF/MANIFEST.MF already added, skipping [INFO] META-INF/ already added, skipping [INFO] META-INF/maven/ already added, skipping [INFO] META-INF/ already added, skipping [INFO] META-INF/MANIFEST.MF already added, skipping [INFO] com/ already added, skipping [INFO] META-INF/maven/ already added, skipping [INFO] Building jar: /home/mqtt/dev/StmpRelayKafkaProducer/target/jars/SmtpRelayKafkaProducer.jar [INFO] META-INF/ already added, skipping [INFO] META-INF/MANIFEST.MF already added, skipping [INFO] META-INF/MANIFEST.MF already added, skipping [INFO] META-INF/MANIFEST.MF already added, skipping [INFO] org/ already added, skipping [INFO] META-INF/ already added, skipping [INFO] META-INF/MANIFEST.MF already added, skipping [INFO] org/ already added, skipping [INFO] META-INF/ already added, skipping [INFO] META-INF/MANIFEST.MF already added, skipping [INFO] org/ already added, skipping [INFO] org/slf4j/ already added, skipping [INFO] META-INF/maven/ already added, skipping [INFO] META-INF/maven/org.slf4j/ already added, skipping [INFO] META-INF/MANIFEST.MF already added, skipping [INFO] META-INF/ already added, skipping [INFO] META-INF/maven/ already added, skipping [INFO] META-INF/ already added, skipping [INFO] META-INF/MANIFEST.MF already added, skipping [INFO] com/ already added, skipping [INFO] META-INF/maven/ already added, skipping [WARNING] Configuration options: 'appendAssemblyId' is set to false, and 'classifier' is missing. Instead of attaching the assembly file: /home/mqtt/dev/StmpRelayKafkaProducer/target/jars/SmtpRelayKafkaProducer.jar, it will become the file for main project artifact. NOTE: If multiple descriptors or descriptor-formats are provided for this project, the value of this file will be non-deterministic! [WARNING] Replacing pre-existing project main-artifact file: /home/mqtt/dev/StmpRelayKafkaProducer/target/classes with assembly file: /home/mqtt/dev/StmpRelayKafkaProducer/target/jars/SmtpRelayKafkaProducer.jar [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 4.820s [INFO] Finished at: Wed Dec 19 17:59:16 PST 2018 [INFO] Final Memory: 20M/171M [INFO] ------------------------------------------------------------------------

[Ron McLeod]

salvin francis wrote:I am not much sure about this since I haven't done it, but I think this might help : https://www.mkyong.com/maven/create-a-fat-jar-file-maven-assembly-plugin/

Thanks for the link It looks like ti is also using the assembly plug-in.

[salvin francis]

Happy to help . The reason why you found it difficult was that it is not a common scenario to build a fat jar in maven. Even if you want to run it as a standalone jar, maybe you can consider including dependencies in classpath instead of bundling them ?

[Rob Spoor]

I prefer the shade plugin over the assembly plugin for fat JARs. I've also used the One Jar maven plugin but that creates a fat JAR with the dependencies included as JAR files with a custom class loader.