A data engineer wants to schedule their Databricks SQL dashboard to refresh every hour, but they only want the associated SQL endpoint to be running when It is necessary. The dashboard has multiple queries on multiple datasets associated with it. The data that feeds the dashboard is automatically processed using a Databricks Job.
Which approach can the data engineer use to minimize the total running time of the SQL endpoint used in the refresh schedule of their dashboard?
To minimize the total running time of the SQL endpoint used in the refresh schedule of a dashboard in Databricks, the most effective approach is to utilize the Auto Stop feature. This feature allows the SQL endpoint to automatically stop after a period of inactivity, ensuring that it only runs when necessary, such as during the dashboard refresh or when actively queried. This minimizes resource usage and associated costs by ensuring the SQL endpoint is not running idle outside of these operations.
Reference: Databricks documentation on SQL endpoints: SQL Endpoints in Databricks
Florinda
8 months agoValentine
8 months agoCarman
8 months agoJerry
8 months agoGladys
8 months agoBarney
9 months agoJames
9 months agoTimothy
8 months agoGladys
8 months agoJosefa
8 months agoHoney
9 months agoHubert
8 months agoJulene
8 months agoAlex
8 months agoAsuncion
10 months agoLuisa
10 months agoGlenn
10 months agoGiovanna
10 months agoJoanna
10 months agoDesmond
8 months agoJamie
8 months agoMonte
9 months agoFlorinda
9 months agoLuisa
10 months ago