Warm tip: This article is reproduced from serverfault.com, please click

java.lang.NoClassDefFoundError in Talend Studio

发布于 2020-12-19 13:05:39

I want install some Google Cloud Java Client Libraries to use it in a tJava component (I don't want use the talend's pre-build component).

I'm trying some test with a job with a tJava component. This is the code that I want to run:

BigQuery bigquery = BigQueryOptions.getDefaultInstance().getService(); 

in the advance setting I import the following libraries:

import com.google.cloud.bigquery.BigQuery;
import com.google.cloud.bigquery.BigQueryOptions;

To setting the following library I did the following step:

  1. I download the last version of the artifact google-cloud-bigquery from https://search.maven.org/search?q=g:com.google.cloud%20AND%20a:google-cloud-bigquery&core=gav

  2. I download the last version of the artifact google-cloud-core from https://mvnrepository.com/artifact/com.google.cloud/google-cloud-core/1.94.0

  3. I put this two jar in the following folder:

    -...\TALEND STUDIO\configuration.m2\repository\com\google\cloud\google-cloud-bigquery\1.126.3\google-cloud-bigquery-1.126.3.jar

    -...\TALEND STUDIO\configuration.m2\repository\com\google\cloud\google-cloud-core\1.94.0\google-cloud-core-1.94.0

  4. From the navigator tab I open: NAME_PROJECT/poms/pom.xml and I add the dependencies as following:

enter image description here

using the the bottom "Add" this change the pom.xml as following:

...
  <dependencies>
    <dependency>
        <groupId>com.google.cloud</groupId>
        <artifactId>google-cloud-bigquery</artifactId>
        <version>1.126.3</version>
    </dependency>
    <dependency>
        <groupId>com.google.cloud</groupId>
        <artifactId>google-cloud-core</artifactId>
        <version>1.94.0</version>
    </dependency>
  </dependencies>
</project>
  1. Run the job, not compile errors occur but a runtime error:

Exception in thread "main" java.lang.NoClassDefFoundError: com/google/cloud/bigquery/BigQueryOptions at bce_datahub.bigquery_template_0_1.bigquery_template.tJava_1Process(bigquery_template.java:331) at bce_datahub.bigquery_template_0_1.bigquery_template.runJobInTOS(bigquery_template.java:638) at bce_datahub.bigquery_template_0_1.bigquery_template.main(bigquery_template.java:464) Caused by: java.lang.ClassNotFoundException: com.google.cloud.bigquery.BigQueryOptions at java.net.URLClassLoader.findClass(URLClassLoader.java:382) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ... 3 more

I saw from this thread Why am I getting a NoClassDefFoundError in Java? that the problem is due to the fact that the class code is not found at runtime. I don't know how to put the class (jar) in the runtime path.

Thank you for your help.

EDIT:

I don't know if it is useful but I put all the pom.xml. I hide (with xxx) some part of it for privacy. I think that the list of modules (talend jobs) isn't useful so I delete a part of that list:

<?xml version="1.0"?>
<project xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd" xmlns="http://maven.apache.org/POM/4.0.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <modelVersion>4.0.0</modelVersion>
  <groupId>org.example.xxx</groupId>
  <artifactId>code.Master</artifactId>
  <version>7.0.1</version>
  <packaging>pom</packaging>
  <name>xxx Codes Master</name>
  <url>http://www.talend.org/</url>
  <modules>
    <module>code/routines</module>
    <module>jobs/process/xxx</module>
    <module>jobs/process/xxx/xxx/xxx</module>
    <module>...</module>
    <module>jobs/process/xxx/xxx/xxx</module>
    <module>../../REFERENCE/poms</module>
    <module>jobs/process/xxx</module>
  </modules>
  <properties>
    <talend.project.name>xxx</talend.project.name>
    <encoding>UTF-8</encoding>
  </properties>
  <build>
    <pluginManagement>
      <plugins>
        <plugin>
          <artifactId>maven-clean-plugin</artifactId>
          <version>3.0.0</version>
        </plugin>
        <plugin>
          <artifactId>maven-compiler-plugin</artifactId>
          <version>3.6.1</version>
          <dependencies>
            <dependency>
              <groupId>org.eclipse.tycho</groupId>
              <artifactId>tycho-compiler-jdt</artifactId>
              <version>1.0.0</version>
            </dependency>
          </dependencies>
          <configuration>
            <source>1.8</source>
            <target>1.8</target>
            <compilerId>jdt</compilerId>
          </configuration>
        </plugin>
        <plugin>
          <artifactId>maven-resources-plugin</artifactId>
          <version>3.0.2</version>
        </plugin>
        <plugin>
          <artifactId>maven-jar-plugin</artifactId>
          <version>3.0.2</version>
          <configuration>
            <archive>
              <addMavenDescriptor>false</addMavenDescriptor>
            </archive>
          </configuration>
        </plugin>
        <plugin>
          <artifactId>maven-assembly-plugin</artifactId>
          <version>3.0.0</version>
        </plugin>
        <plugin>
          <artifactId>maven-surefire-plugin</artifactId>
          <version>2.20</version>
        </plugin>
      </plugins>
    </pluginManagement>
  </build>
  <profiles>
    <profile>
      <id>ci-builder</id>
      <activation>
        <activeByDefault>true</activeByDefault>
      </activation>
      <build>
        <plugins>
          <plugin>
            <groupId>org.talend</groupId>
            <artifactId>ci.builder</artifactId>
            <version>7.0.1</version>
            <executions>
              <execution>
                <phase>generate-sources</phase>
                <goals>
                  <goal>generate</goal>
                </goals>
              </execution>
            </executions>
          </plugin>
        </plugins>
      </build>
    </profile>
  </profiles>
  <dependencies>
    <dependency>
        <groupId>com.google.cloud</groupId>
        <artifactId>google-cloud-bigquery</artifactId>
        <version>1.126.3</version>
    </dependency>
    <dependency>
        <groupId>com.google.cloud</groupId>
        <artifactId>google-cloud-core</artifactId>
        <version>1.94.0</version>
    </dependency>
  </dependencies>
</project>
Questioner
Simone Giusso
Viewed
0
Balazs Gunics 2020-12-28 20:20:01

The POM files of the jobs are generated by studio actions, so I'd avoid editing them manually.

There are other ways to achieve this, first I'd suggest to take a look at tLibraryLoad component to add dependency jars: https://help.talend.com/r/lw17672ujtsKAM3zY5H2oQ/wcAnfUMxgsm90CCS6~jiZw

If you have more than 2-3 jar files you could also handle the dependencies via Routines, and then set up the routine as a dependency for the job(s). Keep in mind that if your SDK has additional dependency jars then those should be added as well.