ReadyFlow: S3 to OpenSearch [Technical Preview]
You can use the S3 to OpenSearch [Technical Preview] Readyflow to consume PDF documents from S3, vectorize them using an OpenAI model and write the results to OpenSearch.
This ReadyFlow consumes PDF documents from a source S3 location, partitions the PDFs, chunks the data, vectorizes the data using an OpenAI embedding model, and stores the results in OpenSearch. The default OpenAI model is 'text-embedding-ada-002'. An OpenAI API key and an OpenSearch password are required to run this flow. Define a KPI on the failure_WriteToOpenSearch connection to monitor failed write operations.
| S3 to OpenSearch [Technical Preview] ReadyFlow details | |
|---|---|
| Source | Cloudera Managed S3 |
| Source Format | |
| Destination | OpenSearch |
| Destination Format | Vector DB |
