You need to schedule the population of the medallion layers to meet the technical requirements.
What should you do?
The technical requirements specify that:
Why Use a Data Pipeline That Calls Other Data Pipelines?
- Sequential execution of child pipelines.
- Error handling to send email notifications upon failures.
- Parallel execution of tasks where possible (e.g., simultaneous imports into the bronze layer).
Davida
4 days agoGerald
4 days ago