ReadyFlow overview: ADLS to ADLS Avro
You can use the ADLS to ADLS Avro ReadyFlow to move data between source and destination ADLS locations while converting the files into Avro format.
This ReadyFlow consumes JSON, CSV or Avro files from a source Azure Data Lake Service (ADLS) location, converts the files into Avro and writes them to the destination ADLS location. You can specify the source format, the source and target location as well as the schema to use for reading the source data. The ReadyFlow polls the source container for new files (it performs a listing periodically).
| ADLS to ADLS Avro ReadyFlow details | |
|---|---|
| Source | ADLS |
| Source Format | JSON, CSV, Avro |
| Destination | ADLS |
| Destination Format | Avro |
Moving data with an ADLS to ADLS Avro flow
You can use an ADLS to ADLS Avro data flow when you want to move data from a location in ADLS to another ADLS location, and at the same time convert the data to Avro format. You need to specify the source format, the source and target location as well as the schema to use for handling the data. Your flow can consume JSON, CSV or Avro files from the source ADLS location. It converts the files into Avro format and writes them to the destination ADLS location. You define and store the data processing schema in the Schema Registry on a Streams Messaging Data Hub cluster. The data flow parses the schema by looking up the schema name in the Schema Registry.
