A data engineer wants to schedule their Databricks SQL dashboard to refresh every hour, but they only want the associated SQL endpoint to be running when It is necessary. The dashboard has multiple queries on multiple datasets associated with it. The data that feeds the dashboard is automatically processed using a Databricks Job.
Which approach can the data engineer use to minimize the total running time of the SQL endpoint used in the refresh schedule of their dashboard?
To minimize the total running time of the SQL endpoint used in the refresh schedule of a dashboard in Databricks, the most effective approach is to utilize the Auto Stop feature. This feature allows the SQL endpoint to automatically stop after a period of inactivity, ensuring that it only runs when necessary, such as during the dashboard refresh or when actively queried. This minimizes resource usage and associated costs by ensuring the SQL endpoint is not running idle outside of these operations.
Reference: Databricks documentation on SQL endpoints: SQL Endpoints in Databricks
Florinda
19 days agoValentine
21 days agoJerry
7 days agoGladys
12 days agoBarney
24 days agoJames
27 days agoHoney
1 months agoHubert
8 days agoJulene
9 days agoAlex
13 days agoAsuncion
2 months agoLuisa
2 months agoGlenn
2 months agoGiovanna
2 months agoJoanna
2 months agoDesmond
17 days agoJamie
21 days agoMonte
23 days agoFlorinda
1 months agoLuisa
2 months ago