StartSnowflakeIngest 2.3.0.4.10.0.0-147

Bundle
org.apache.nifi | nifi-snowflake-processors-nar
Description
Ingests files from a Snowflake internal or external stage into a Snowflake table. The stage must be created in the Snowflake account beforehand. The result of the ingestion is not available immediately, so this processor can be connected to an GetSnowflakeIngestStatus processor to wait for the results
Tags
ingest, snowflake, snowpipe
Input Requirement
REQUIRED
Supports Sensitive Dynamic Properties
false
  • Additional Details for StartSnowflakeIngest 2.3.0.4.10.0.0-147

    StartSnowflakeIngest

    Description

    The StartSnowflakeIngest processor triggers a Snowflake pipe ingestion for a staged file. Please note, that the pipe has to be created in your Snowflake account manually. The processor requires an upstream connection that provides the path of the file to be ingested in the stage through the “snowflake.staged.file.path” attribute. This attribute is automatically filled in by the PutSnowflakeInternalStage processor when using an internal stage. In case a pipe copies data from an external stage, the attribute shall be manually provided (e.g. with an UpdateAttribute processor). NOTE: Since Snowflake pipes ingest files asynchronously, this processor transfers FlowFiles to the “success” relationship when they’re marked for ingestion. In order to wait for the actual result of the ingestion, the processor may be connected to a downstream GetSnowflakeIngestStatus processor.

    Example flow for internal stage

    GetFile -> PutSnowflakeInternalStage -> StartSnowflakeIngest -> GetSnowflakeIngestStatus

    Example flow for external stage

    ListS3 -> UpdateAttribute (add the “snowflake.staged.file.path” attribute) -> StartSnowflakeIngest -> GetSnowflakeIngestStatus

Properties
Relationships
Name Description
success For FlowFiles of successful ingest request
failure For FlowFiles of failed ingest request
Reads Attributes
Name Description
snowflake.staged.file.path Staged file path
See Also