A data engineer needs to capture pipeline settings from an existing in the workspace, and use them to create and version a JSON file to create a new pipeline.
Which command should the data engineer enter in a web terminal configured with the Databricks CLI?
The Databricks CLI provides a way to automate interactions with Databricks services. When dealing with pipelines, you can use the databricks pipelines get --pipeline-id command to capture the settings of an existing pipeline in JSON format. This JSON can then be modified by removing the pipeline_id to prevent conflicts and renaming the pipeline to create a new pipeline. The modified JSON file can then be used with the databricks pipelines create command to create a new pipeline with those settings.
Databricks Documentation on CLI for Pipelines: Databricks CLI - Pipelines
Matilda
4 days agoMargot
4 days agoCarmela
5 days agoJosephine
7 days agoMarcos
7 days agoMeghan
18 days agoLynette
25 days agoReta
4 hours agoLamar
9 days agoCoral
12 days ago