A data engineer wants to schedule their Databricks SQL dashboard to refresh every hour, but they only want the associated SQL endpoint to be running when It is necessary. The dashboard has multiple queries on multiple datasets associated with it. The data that feeds the dashboard is automatically processed using a Databricks Job.
Which approach can the data engineer use to minimize the total running time of the SQL endpoint used in the refresh schedule of their dashboard?
To minimize the total running time of the SQL endpoint used in the refresh schedule of a dashboard in Databricks, the most effective approach is to utilize the Auto Stop feature. This feature allows the SQL endpoint to automatically stop after a period of inactivity, ensuring that it only runs when necessary, such as during the dashboard refresh or when actively queried. This minimizes resource usage and associated costs by ensuring the SQL endpoint is not running idle outside of these operations.
Reference: Databricks documentation on SQL endpoints: SQL Endpoints in Databricks
Florinda
2 months agoValentine
2 months agoCarman
1 months agoJerry
1 months agoGladys
2 months agoBarney
2 months agoJames
2 months agoTimothy
30 days agoGladys
1 months agoJosefa
1 months agoHoney
2 months agoHubert
1 months agoJulene
1 months agoAlex
2 months agoAsuncion
3 months agoLuisa
3 months agoGlenn
3 months agoGiovanna
3 months agoJoanna
3 months agoDesmond
2 months agoJamie
2 months agoMonte
2 months agoFlorinda
2 months agoLuisa
3 months ago