Apache Camel used on a Kubernetes Cassandra cluster

Introduction

Eclipse JKube 1.0.0 GA will be released on September 9, 2020. As I mentioned in previous posts, JKube is the successor of the deprecated Fabric8 Maven Plugin (FMP). As such, our main goal right now is to migrate the current user-base to the new project. Aligned with this strategy, we started to create Pull Requests on those repositories that are currently using FMP in order to replace the deprecated dependency.

In this article, I’ll describe how to run the updated Apache Camel example (now using JKube) for Cassandra running on Kubernetes, and its expected behavior. This blog post is strongly based on the original article written by the great Andrea Cosentino back in 2016.

Spinning up the Apache Cassandra cluster on Kubernetes

The first step is to clone the Apache Camel examples project and navigate to the example folder:

1git clone git@github.com:apache/camel-examples.git
2cd camel-examples/examples/camel-example-cassandra-kubernetes

For the purpose of this blog, I’m using Minikube, but you could run the example on any other Kubernetes cluster.

In order to spin up the Apache Cassandra cluster we need to apply the two YAML files provided in the src/main/resources/jkube directory:

1kubectl create -f src/main/resources/jkube/cassandra-service.yaml
2kubectl create -f src/main/resources/jkube/cassandra-statefulset.yaml

After a few minutes, the Pods defined in the cluster StatefulSet should have started, we can check their status by running:

1$ kubectl get pods
2NAME          READY   STATUS    RESTARTS   AGE
3cassandra-0   1/1     Running   0          3m35s
4cassandra-1   1/1     Running   0          98s

When both Pods are with running status, the Apache Cassandra cluster will be ready to use.

Apache Camel example

The example is a very simple Apache Camel application that contains a single route that will log some information every 5 seconds.

Java classes

The project contains 2 Java classes. CqlPopulateBean will create a Cassandra Keyspace and Table and add 2 entries to that table in the Cassandra cluster we deployed in the previous step whenever the populate() method is invoked.

The RowProcessor is an Apache Camel Processor implementation that will transform the List of Rows retrieved from the CQL query into a more friendly String that we can log.

Camel Context

Following you can see an excerpt of the camel-context.xml file:

1<bean id="populate" class="org.apache.camel.example.kubernetes.jkube.CqlPopulateBean" init-method="populate"/>
2<bean id="rowProcessor" class="org.apache.camel.example.kubernetes.jkube.RowProcessor"/>
3<camelContext xmlns="http://camel.apache.org/schema/spring" depends-on="populate">
4<route id="cassandra-route">
5  <from uri="timer:foo?period=5000"/>
6  <to uri="cql://cassandra/test?datacenter=DC1-K8Demo&amp;cql=SELECT * FROM users;&amp;consistencyLevel=quorum" />
7  <process ref="rowProcessor"/>
8  <log message="Query result set [${body}]"/>
9</route>
10</camelContext>

The file provides a definition for 2 beans and a single route.

The first bean (populate) will be instantiated during the application startup and the populate method will be called. As I explained before, this will add 2 rows to the users table.

The second bean (rowProcessor) is an instance of the RowProcessor class defined earlier.

Finally, we can see the route definition (cassandra-route). The route starts from a timer that runs every 5 seconds. It then performs a CQL query that will fetch all users from the test Keyspace in the DC1-K8Demo datacenter. The resulting list of rows will be processed by the processor bean and the resulting string will be logged.

Running the example

Now that we’ve seen all of the components for our example we can run it and take it for a spin.

If you are running this on Minikube, you will first need to share your Docker Daemon with your host system by running: eval $(minikube docker-env)

Note: In case you are running the example in an external cluster you will need to push the built Docker image to a registry accessible to the K8s cluster so that the image can be pulled.

We are now ready to deploy the application, we can do so by running:

1mvn clean -Pkubernetes-install k8s:deploy

The previous command will package the application, build a Docker Image, create cluster resource manifests, and apply them into the Minikube cluster. As you can see, the use of Eclipse JKube really simplifies the development effort by leveraging these tasks from the developer. Following is an excerpt of the pom.xml containing the JKube’s Kubernetes Maven Plugin configuration:

1<plugin>
2  <groupId>org.eclipse.jkube</groupId>
3  <artifactId>kubernetes-maven-plugin</artifactId>
4  <version>${jkube-version}</version>
5  <executions>
6    <execution>
7      <goals>
8        <goal>resource</goal>
9        <goal>build</goal>
10      </goals>
11    </execution>
12  </executions>
13</plugin>

As you can see, the plugin doesn’t require much configuration. The only specific configuration is the one for the execution that will simply bind the k8s:resource and k8s:build goals to the Maven install phase.

After a few moments, the Pod defined in the Deployments will start.

1$ kubectl get pods
2NAME                                                  READY   STATUS    RESTARTS   AGE
3camel-example-cassandra-kubernetes-644d4959b6-574nk   1/1     Running   0          78s
4cassandra-0                                           1/1     Running   0          32m
5cassandra-1                                           1/1     Running   0          30m

We can now retrieve the logs by running the following Maven command:

1mvn -Pkubernetes-install k8s:log

If everything went well you should see an output similar to this:

1GuavaCompatibility             - Detected Guava >= 19 in the classpath, using modern compatibility layer
2ClockFactory                   - Using native clock to generate timestamps.
3NettyUtil                      - Did not find Netty's native epoll transport in the classpath, defaulting to NIO.
4DCAwareRoundRobinPolicy        - Using data-center name 'DC1-K8Demo' for DCAwareRoundRobinPolicy (if this is incorrect, please provide the correct datacenter name with DCAwareRoundRobinPolicy constructor)
5Cluster                        - New Cassandra host cassandra/172.17.0.7:9042 added
6Cluster                        - New Cassandra host cassandra/172.17.0.6:9042 added
7CqlPopulateBean                - Cassandra was populated with sample values for test.users table
8LRUCacheFactory                - Detected and using LRUCacheFactory: camel-caffeine-lrucache
9AbstractCamelContext           - Apache Camel 3.5.0-SNAPSHOT (camel-1) is starting
10AbstractCamelContext           - StreamCaching is not in use. If using streams then its recommended to enable stream caching. See more details at http://camel.apache.org/stream-caching.html
11AbstractCamelContext           - Using HealthCheck: camel-health
12DefaultMavenCoordinates        - DataStax Java driver for Apache Cassandra(R) (com.datastax.oss:java-driver-core) version 4.8.0
13Clock                          - Using native clock for microsecond precision
14InternalRouteStartupManager    - Route: cassandra-route started and consuming from: timer://foo
15AbstractCamelContext           - Total 1 routes, of which 1 are started
16AbstractCamelContext           - Apache Camel 3.5.0-SNAPSHOT (camel-1) started in 4.435 seconds
17BaseMainSupport                - Using properties from: classpath:application.properties;optional=true
18DefaultRoutesCollector         - No additional Camel XML routes discovered from: classpath:camel/*.xml
19DefaultRoutesCollector         - No additional Camel XML route templates discovered from: classpath:camel-template/*.xml
20DefaultRoutesCollector         - No additional Camel XML rests discovered from: classpath:camel-rest/*.xml
21cassandra-route                - Query result set [1-oscerd,2-not-a-bot]
22cassandra-route                - Query result set [1-oscerd,2-not-a-bot]

As you can see in the last entries of the log, the rows inserted with the CqlPopulateBean class are now being retrieved using CQL and transformed using the RowProcessor.

Conclusion

In this article, I described to you the behavior of the Apache Camel Cassandra Kubernetes example. This is a really simple example but highlights the power of Apache Camel, Apache Cassandra, and Eclipse JKube used together. More complex architectures to solve real-world scenarios can be easily built on top of this example.