ReadyFlow overview: S3 to Milvus [Technical Preview]
You can use the S3 to Milvus [Technical Preview] ReadyFlow to consume PDF documents from S3, vectorize them using a HuggingFace model and write the results to Milvus.
This ReadyFlow consumes PDF documents from a source S3 location, partitions the PDFs, chunks the data, vectorizes the data using a HuggingFace embedding model, and stores the results in Milvus vector DB. The default HuggingFace model is 'all-MiniLM-L12-v2'. A Milvus access token is required to run this flow. Define a KPI on the failure_WriteToMilvus connection to monitor failed write operations.
| S3 to Milvus [Technical Preview] ReadyFlow details | |
|---|---|
| Source | Cloudera Public Cloud managed Amazon S3 |
| Source Format | |
| Destination | Milvus |
| Destination Format | Vector DB |
