Data Governance
Also available as:
PDF
loading table of contents...

Deploying Data Pipelines

After you create your data pipeline with Falcon, you can deploy it with the Falcon CLI.

To deploy the data pipeline:

  1. Submit your entities to Falcon. Be sure to specify the correct entity type.

    1. Submit your cluster entity.

      For example, to submit $sampleClusterFile.xml:

      falcon entity -type cluster -submit -file $sampleClusterFile.xml
    2. Submit your dataset or feed entity.

      For example, to submit $sampleFeedFile.xml:

      falcon entity -type feed -submit -file $sampleFeedFile.xml
    3. Submit your process entity.

      For example, to submit $sampleProcessFile.xml:

      falcon entity -type process -submit -file $sampleProcessFile.xml
  2. Schedule your feed and process entities.

    1. Schedule your feed.

      For example, to schedule $feedName:

      falcon entity -type feed -schedule -name $feedName
    2. Schedule your process.

      For example, to schedule $processName:

      falcon entity -type process -schedule -name $processName

Your data pipeline is now deployed with basic necessary information to run Oozie jobs, Pig scripts, and Hive queries. You can now explore other sections such as Late Data Handling or Retry Policy.