Flink Project Template

The Quickstart Archetype serves as a template for an Apache Flink streaming application. You can use the Archetype to add source, sink and computation to the template. Like this you can practice the development of a simple Flink application, or use the Archetype as the starting point for a more complex application including state,watermark and checkpoint.

The Flink quickstart archetype can be used to quickly build a basic Flink streaming project.

Perform the following commands to create the archetype locally:

git clone https://github.com/cloudera/flink-tutorials
cd flink-tutorials
cd flink-quickstart-archetype
mvn clean install
cd ..

The following entry should be seen in your local catalog:

cat ~/.m2/repository/archetype-catalog.xml
 ...
 <archetypes>
    <archetype>
      <groupId>com.cloudera.flink</groupId>
      <artifactId>flink-quickstart-archetype</artifactId>
      <version>1.14.0-csa1.6.0.0</version>
      <description>flink-quickstart-archetype</description>
    </archetype>
  </archetypes>

Generate the project skeleton with the following commands:

mvn archetype:generate \
      -DarchetypeGroupId=com.cloudera.flink \
      -DarchetypeArtifactId=flink-quickstart-archetype \
      -DarchetypeVersion=1.14.0-csa1.6.0.0

Provide basic information about the streaming application.

You can choose to customize configurations yourself or use automatically generated information.

To use automatically generated information, press Enter.

If you choose to customize configuration, set the following properties:

Define value for property 'groupId': com.cloudera.flink
                                Define value for property 'artifactId': sample-project
                                Define value for property 'version' 1.0-SNAPSHOT: :
                                Define value for property 'package' com.cloudera.flink: :
                                Confirm properties configuration:
                                groupId: com.cloudera.flink
                                artifactId: sample-project
                                version: 1.0-SNAPSHOT
                                package: com.cloudera.flink
                                Y: :

The generated project will look like this:

sample-project
├── pom.xml
└── src
    └── main
        ├── java
        │   └── com
        │       └── cloudera
        │           └── flink
        │               └── StreamingJob.java
        └── resources
            └── log4j.properties

Open StreamingJob.java file.

The archetype application will look like this:

public class StreamingJob {
	public static void main(String[] args) throws Exception {
		final StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
		DataStream<Integer> ds = env.fromElements(1,2,3,4);
		ds.printToErr();
		env.execute("Flink Streaming Java API Skeleton");
	}
}

Customize the archetype application by adding source, stream transformation and sink to the Datastream class.
Run the application with env.execute command.

You have built your Flink streaming project.

You can further develop your application, or you can run the Flink application archetype.