After reading about the new SQL Pools feature for Warehouses in Fabric, I had an idea, if I could change the SQL Pool configuration based on the expected query load, I could then consume less capacity and have better performance.

https://learn.microsoft.com/en-us/fabric/data-warehouse/custom-sql-pools

Here is an Example I thought of below.

  • When the ETL load is running optimize the SQL pool for writing as typically data is being inserted.
  • After the ETL load and for the rest of the day, almost all queries are read by the Warehouse, so change the SQL pool to be read optimized.

Below is a Python Notebook that I have created which is used in combination with a Fabric pipeline.

Once again, I am using the Python notebook, it starts quickly and, in my testing, this ran for 5 seconds, so it would consume 5 CUs.

ETL SQL Pool Configuration

  • Here is the Parameters Cell where right now it is configured for the ETL SQL Pool
  • I then got the following ETL SQL Pool configuration as shown below.
    • NOTE: The configuration above could also have parameters if you want to make it all dynamic and pass it from your Pipeline.
  • I then let the rest of the Notebook run and complete successfully.
  • When I go into the Workspace Settings, I can then see the configuration has been applied.

Reporting SQL Pool Configuration

  • Here is the Parameters Cell where right now it is configured for the Reporting SQL Pool
  • I then got the following Reporting SQL Pool configuration as shown below.
    • NOTE: The configuration above could also have parameters if you want to make it all dynamic and pass it from your Pipeline.
  • I then let the rest of the Notebook run and complete successfully.
  • When I go into the Workspace Settings, I can then see the configuration has been applied.

Summary

In this blog post I have shown you how to dynamically change your SQL Pool configuration based on the query workload.

Here is a copy of the Notebook if you want to use this in your environment.

Custom SQL Pools Notebook

Thanks for reading I hope you found this something you can use in your environment!

Comments or questions are always welcome!