QueryElasticsearchHttp

Deprecation notice:

This processor is deprecated and may be removed in future releases.

Please consider using one the following alternatives: PaginatedJsonQueryElasticsearch

Description:

Queries Elasticsearch using the specified connection properties. Note that the full body of each page of documents will be read into memory before being written to Flow Files for transfer. Also note that the Elasticsearch max_result_window index setting is the upper bound on the number of records that can be retrieved using this query. To retrieve more records, use the ScrollElasticsearchHttp processor.

Tags:

elasticsearch, query, read, get, http

Properties:

In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.

Display Name	API Name	Default Value	Allowable Values	Description
Elasticsearch URL	elasticsearch-http-url			Elasticsearch URL which will be connected to, including scheme (http, e.g.), host, and port. The default port for the REST API is 9200. Supports Expression Language: true (will be evaluated using variable registry only)
SSL Context Service	SSL Context Service		Controller Service API: SSLContextService Implementations: StandardRestrictedSSLContextService StandardSSLContextService	The SSL Context Service used to provide client certificate information for TLS/SSL connections. This service only applies if the Elasticsearch endpoint(s) have been secured with TLS/SSL.
Character Set	Character Set	UTF-8		Specifies the character set of the document data. Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)
Username	Username			Username to access the Elasticsearch cluster Supports Expression Language: true (will be evaluated using variable registry only)
Password	Password			Password to access the Elasticsearch cluster Sensitive Property: true Supports Expression Language: true (will be evaluated using variable registry only)
Connection Timeout	elasticsearch-http-connect-timeout	5 secs		Max wait time for the connection to the Elasticsearch REST API. Supports Expression Language: true (will be evaluated using variable registry only)
Response Timeout	elasticsearch-http-response-timeout	15 secs		Max wait time for a response from the Elasticsearch REST API. Supports Expression Language: true (will be evaluated using variable registry only)
Proxy Configuration Service	proxy-configuration-service		Controller Service API: ProxyConfigurationService Implementation: StandardProxyConfigurationService	Specifies the Proxy Configuration Controller Service to proxy network requests. If set, it supersedes proxy settings configured per component. Supported proxies: HTTP + AuthN, SOCKS In case of SOCKS, it is not guaranteed that the selected SOCKS Version will be used by the processor.
Proxy Host	elasticsearch-http-proxy-host			The fully qualified hostname or IP address of the proxy server Supports Expression Language: true (will be evaluated using variable registry only)
Proxy Port	elasticsearch-http-proxy-port			The port of the proxy server Supports Expression Language: true (will be evaluated using variable registry only)
Proxy Username	proxy-username			Proxy Username Supports Expression Language: true (will be evaluated using variable registry only)
Proxy Password	proxy-password			Proxy Password Sensitive Property: true Supports Expression Language: true (will be evaluated using variable registry only)
Query	query-es-query			The Lucene-style query to run against ElasticSearch (e.g., genre:blues AND -artist:muddy) Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)
Page Size	query-es-size	20		Determines how many documents to return per page during scrolling. Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)
Index	query-es-index			The name of the index to read from. If the property is unset or set to _all, the query will match across all indexes. Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)
Type	query-es-type			The type of document (if unset, the query will be against all types in the _index). This should be unset or '_doc' for Elasticsearch 7.0+. Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)
Fields	query-es-fields			A comma-separated list of fields to retrieve from the document. If the Fields property is left blank, then the entire document's source will be retrieved. Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)
Sort	query-es-sort			A sort parameter (e.g., timestamp:asc). If the Sort property is left blank, then the results will be retrieved in document order. Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)
Limit	query-es-limit			If set, limits the number of results that will be returned. Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)
Target	query-es-target	Flow file content	Flow file content Flow file attributes	Indicates where the results should be placed. In the case of 'Flow file content', the JSON response will be written as the content of the flow file. In the case of 'Flow file attributes', the original flow file (if applicable) will be cloned for each result, and all return fields will be placed in a flow file attribute of the same name, but prefixed by 'es.result.'
Routing Strategy for Query Info	routing-query-info-strategy	Never	Always Never No Hits Append as Attributes	Specifies when to generate and route Query Info after a successful query

Dynamic Properties:

Supports Sensitive Dynamic Properties: No

Dynamic Properties allow the user to specify both the name and value of a property.

Name	Value	Description
A URL query parameter	The value to set it to	Adds the specified property name/value as a query parameter in the Elasticsearch URL used for processing Supports Expression Language: true (will be evaluated using variable registry only)

Relationships:

Name	Description
retry	A FlowFile is routed to this relationship if the document cannot be fetched but attempting the operation again may succeed. Note that if the processor has no incoming connections, flow files may still be sent to this relationship based on the processor properties and the results of the fetch operation.
success	All FlowFiles that are read from Elasticsearch are routed to this relationship.
failure	All FlowFiles that cannot be read from Elasticsearch are routed to this relationship. Note that only incoming flow files will be routed to failure.

Reads Attributes:

None specified.

Writes Attributes:

Name	Description
filename	The filename attribute is set to the document identifier
es.query.hitcount	The number of hits for a query
es.id	The Elasticsearch document identifier
es.index	The Elasticsearch index containing the document
es.query.url	The Elasticsearch query that was built
es.type	The Elasticsearch document type
es.result.*	If Target is 'Flow file attributes', the JSON attributes of each result will be placed into corresponding attributes with this prefix.

State management:

This component does not store state.

Restricted:

This component is not restricted.

Input requirement:

This component allows an incoming relationship.

System Resource Considerations:

None specified.