Extractor configuration properties

Some of the extractor configuration properties.


Configuration Parameter (prefixed with `atlas.s3.extraction`)	Purpose
`.aws.region`	Required. AWS S3 region for extraction. CDP Public Cloud: provided by IDBroker integration. CDP Data Center: the region must be provided.
`.access.key` `.secret.key` `.session.token`	AWS S3 credentials. CDP Public Cloud: provided by IDBroker integration. CDP Data Center: For basic authentication, provide access and secret keys. For session based authentication, provide all three credentials. Configure credentials for Atlas extraction.
`.whitelist.paths`	Set of data assets to include in metadata extraction. If no paths are included, the extractor extracts metadata for all buckets. Buckets inside the region and accessible by the credential used to run the extractor. For public cloud, this is defined in `aws-cdp-bucket-access-policy`. The list can include buckets or paths; separate multiple values with commas. See Defining what assets to extract metadata for.
`.blacklist.paths`	Set of data assets to exclude from metadata extraction. The list can include buckets or paths; separate multiple values with commas. See Defining what assets to extract metadata for.
`.max.object.per.call`	Number of objects returned in one call to AWS S3 in bulk extraction. Defaults to 1000.
`.resume.from.progress.file`	Enable bulk extraction resume. Disabled by default.
`.progress.file`	Name of the progress file that tracks bulk extraction that will be used if the extraction has to be resumed mid-way after a failure. This file is created in the Atlas log directory. Defaults to `extractor_progress_file.log`
`.max.reconnect.count`	Number of times the extractor will try to reconnect to AWS in case of credentials expiring or `retryable` exception from AWS S3. Defaults to 2.
`.fs.scheme`	File system scheme used in case of AWS S3. At this time, `S3a` is the only option.
`.incremental.queueName`	Required for incremental mode. AWS SQS Queue name configured to get the AWS S3 bucket events.
`.incremental.messagesPerRequest`	The number of messages the extractor will try to get from the SQS queue in one call. Defaults to 10.
`.incremental.requestWaitTime`	The interval that the extractor will wait to collect messages so long as the limit set in .incremental.messagesPerRequest hasn't been met. Defaults to 20 seconds.
`.incremental.max.retry`	Maximum retry count if the SQS queue doesn't respond during an extractor request. Defaults to 20.