Extracting S3 Metadata using AtlasPDF version

Extraction Command

The extractor script is located in the extractors/bin directory in $ATLAS_HOME.

By default, the location is /opt/cloudera/parcels/CDH/lib/atlas/extractors/bin/aws-s3-extractor.sh

Using the following command to start S3 extraction:

$ aws-s3-extractor.sh [--config <config file>]
                      [--extraction <mode>] 
                      [--failOnError]
                      [--logdir <log directory>]
Command Line Option Description
 -c <config file>
--config <config file>

The path of the configuration file containing extraction configuration properties. See Extractor configuration properties.

Defaults to $ATLAS_HOME/extractors/conf/aws-s3.conf

 -e <mode>
--extraction <mode> 
The type of extraction to be done, “Incremental” or “Bulk”. If not specified, the extraction mode is "Bulk".

-f

--failOnError

Flag to set the extraction to stop when it encounters an exception. Without the flag, the extraction continues after exceptions.

We want your opinion

How can we improve this page?

What kind of feedback do you have?