1

Developing maven spring boot application with Google Cloud Platform services including BigQuery, PubSub and Dataflow/Beam in the same application.

I added following dependencies through maven:

 <dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-gcp-starter-pubsub</artifactId> </dependency> <dependency> <groupId>org.apache.beam</groupId> <artifactId>beam-sdks-java-core</artifactId> <version>2.2.0</version> </dependency> <dependency> <groupId>com.google.cloud</groupId> <artifactId>google-cloud-bigquery</artifactId> <version>0.33.0-beta</version> </dependency> <dependency> <groupId>org.apache.beam</groupId> <artifactId>beam-runners-google-cloud-dataflow-java</artifactId> <version>2.2.0</version> </dependency>

The problem is when the application is run, following exception is thrown:

Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'pubSubAdmin' defined in class path resource [org/springframework/cloud/gcp/pubsub/autoconfig/GcpPubSubAutoConfiguration.class]: Unsatisfied dependency expressed through method 'pubSubAdmin' parameter 0; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'topicAdminClient' defined in class path resource [org/springframework/cloud/gcp/pubsub/autoconfig/GcpPubSubAutoConfiguration.class]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [com.google.cloud.pubsub.v1.TopicAdminClient]: Factory method 'topicAdminClient' threw exception; nested exception is java.lang.NoClassDefFoundError: com/google/api/gax/grpc/GrpcStatusCode

My POM file content

<parent> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-parent</artifactId> <version>2.0.0.RC1</version> <relativePath/> <!-- lookup parent from repository --> </parent> <properties> <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding> <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding> <java.version>1.8</java.version> </properties> <dependencies> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-webflux</artifactId> </dependency> <dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-gcp-starter-pubsub</artifactId> </dependency> <dependency> <groupId>org.apache.beam</groupId> <artifactId>beam-sdks-java-core</artifactId> <version>2.2.0</version> </dependency> <dependency> <groupId>com.google.cloud</groupId> <artifactId>google-cloud-bigquery</artifactId> <version>0.33.0-beta</version> </dependency> <dependency> <groupId>org.apache.beam</groupId> <artifactId>beam-runners-google-cloud-dataflow-java</artifactId> <version>2.2.0</version> </dependency> <dependency> <groupId>org.projectlombok</groupId> <artifactId>lombok</artifactId> <optional>true</optional> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-test</artifactId> <scope>test</scope> </dependency> <dependency> <groupId>io.projectreactor</groupId> <artifactId>reactor-test</artifactId> <scope>test</scope> </dependency> </dependencies> <build> <plugins> <plugin> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-maven-plugin</artifactId> <executions> <execution> <goals> <goal>repackage</goal> </goals> </execution> </executions> </plugin> <plugin> <groupId>com.google.cloud.tools</groupId> <artifactId>appengine-maven-plugin</artifactId> <version>1.3.2</version> </plugin> </plugins> </build> <repositories> <repository> <id>spring-milestones</id> <name>Spring Milestones</name> <url>https://repo.spring.io/milestone</url> </repository> </repositories> <pluginRepositories> <pluginRepository> <id>spring-snapshots</id> <name>Spring Snapshots</name> <url>https://repo.spring.io/snapshot</url> <snapshots> <enabled>true</enabled> </snapshots> </pluginRepository> <pluginRepository> <id>spring-milestones</id> <name>Spring Milestones</name> <url>https://repo.spring.io/milestone</url> <snapshots> <enabled>false</enabled> </snapshots> </pluginRepository> </pluginRepositories> <dependencyManagement> <dependencies> <dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-gcp-dependencies</artifactId> <version>1.0.0.M2</version> <type>pom</type> <scope>import</scope> </dependency> </dependencies> </dependencyManagement>

With changes suggested by João André Martins another exception came up: 

Exception in thread "main" java.lang.RuntimeException: Failed to construct instance from factory method DataflowRunner#fromOptions(interface org.apache.beam.sdk.options.PipelineOptions) at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod(InstanceBuilder.java:233) at org.apache.beam.sdk.util.InstanceBuilder.build(InstanceBuilder.java:162) at org.apache.beam.sdk.PipelineRunner.fromOptions(PipelineRunner.java:52) at org.apache.beam.sdk.Pipeline.create(Pipeline.java:142) Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod(InstanceBuilder.java:222) ... 4 more Caused by: java.lang.NoClassDefFoundError: com/google/api/services/cloudresourcemanager/CloudResourceManager$Builder at org.apache.beam.sdk.extensions.gcp.options.GcpOptions$GcpTempLocationFactory.newCloudResourceManagerClient(GcpOptions.java:369) at org.apache.beam.sdk.extensions.gcp.options.GcpOptions$GcpTempLocationFactory.create(GcpOptions.java:240) at org.apache.beam.sdk.extensions.gcp.options.GcpOptions$GcpTempLocationFactory.create(GcpOptions.java:228) at org.apache.beam.sdk.options.ProxyInvocationHandler.returnDefaultHelper(ProxyInvocationHandler.java:592) at org.apache.beam.sdk.options.ProxyInvocationHandler.getDefault(ProxyInvocationHandler.java:533) at org.apache.beam.sdk.options.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:156) at com.sun.proxy.$Proxy86.getGcpTempLocation(Unknown Source) at org.apache.beam.runners.dataflow.DataflowRunner.fromOptions(DataflowRunner.java:223) ... 9 more Caused by: java.lang.ClassNotFoundException: com.google.api.services.cloudresourcemanager.CloudResourceManager$Builder at java.net.URLClassLoader.findClass(URLClassLoader.java:381) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ... 17 more
Improve this question

1 Answer 1

Reset to default
2

I was able to make a similar project compile using your pom.xml, spring-cloud-gcp version 1.0.0.M2 and the following changes to your dependencies.

Hope this helps!

 <dependency> <groupId>io.grpc</groupId> <artifactId>grpc-core</artifactId> <version>1.9.0</version> </dependency> <dependency> <groupId>io.grpc</groupId> <artifactId>grpc-netty</artifactId> <version>1.9.0</version> </dependency> <dependency> <groupId>org.apache.beam</groupId> <artifactId>beam-runners-google-cloud-dataflow-java</artifactId> <version>2.2.0</version> <exclusions> <exclusion> <groupId>io.grpc</groupId> <artifactId>grpc-core</artifactId> </exclusion> <exclusion> <groupId>io.grpc</groupId> <artifactId>grpc-netty</artifactId> </exclusion> </exclusions> </dependency>
Improve this answer
Sign up to request clarification or add additional context in comments.

6 Comments

With additional dependencies you provided, i still get this: Failed to read artifact descriptor for io.netty:netty-codec-http2:jar:4.1.20.Final Failed to read artifact descriptor for org.apache.httpcomponents:httpclient:jar:4.5.5
It could be useful to send your whole pom.xml file. Using the dependencies I mentioned, along with the remaining ones you mentioned, using our BOM, worked for me.
i have include your suggestions and got a different exception java.lang.NoClassDefFoundError: com/google/api/services/cloudresourcemanager/CloudResourceManager$Builder. Please check updated post
Using your pom.xml with the changes I recommended, I'm able to do mvn clean spring-boot:run without seeing that issue. I'd recommend running mvn clean on your project and if that doesn't resolve the issue, possibly also clear your local Maven repo (e.g., delete ~/.m2).
A workaround was suggested in github.com/spring-cloud/spring-cloud-gcp/issues/442. Let's follow up there.
|

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.