README.md

Java SDK for Numaflow

This SDK provides the interface for writing UDSources, UDTransformer, UDFs and UDSinks in Java.

Getting Started

Requirements

• java 11+
• maven 3.6+

Importing Numaflow Java SDK

Maven users

Add this dependency to your project's POM:

<dependency>
  <groupId>io.numaproj.numaflow</groupId>
  <artifactId>numaflow-java</artifactId>
  <version>0.9.0</version>
</dependency>

Gradle users

Add this dependency to your project's build file:

compile "io.numaproj.numaflow:numaflow-java:0.9.0"

Build

mvn clean install

Examples on how to write UDSources, UDTransformers, UDFs, UDSinks and SideInputs in Java

• User Defined Source(UDSource)

  • Source

• User Defined Source Transformer(UDTransformer)

  • Source Transformer

• User Defined Function(UDF)

  • MapStream
  • Map
  • Reduce
  • ReduceStream
  • SessionReduce

• User Defined Sink(UDSink)

  • Sink

• User Defined SideInput(SideInput)

  • SideInput

You will see a warning in the log on startup, which you can safely ignore:

Oct 25, 2022 12:26:30 PM io.netty.bootstrap.AbstractBootstrap setChannelOption
WARNING: Unknown channel option 'SO_KEEPALIVE' for channel '[id: 0x6e9c19c7]'

This is due to grpc-netty trying to set SO_KEEPALIVE when it shouldn't (https://github.com/grpc/grpc-java/blob/47ddfa4f205d4672e035c37349dfd3036c74efb6/netty/src/main/java/io/grpc/netty/NettyClientTransport.java#L237)

It is suggested to use amazoncorretto:11 as base for your docker image. We observed that without a base image, the java user-defined container can fail the Numaflow health check.

API Documentation

Please, refer to our Javadoc website.

Development

Updating proto definition files

To keep up-to-date, do the following before building(using udf as an example):

• copy the *.proto files from numaflow-go into /src/main/proto
• replace the go_package lines with the following java_package:

option java_package = "io.numaproj.numaflow.function.v1";

Code Style

Use Editor Config.