Reusable AI Skills for Writing to Microsoft Fabric Lakehouse Tables from Python Notebooks and to reduce Capacity Units consumption
In this blog post below, I explain what the two skills do, why I use them in Fabric Python notebooks, how they can reduce unnecessary capacity consumption, and how to install them for use in Visual Studio Code or Claude Code.
I must give big thanks to Mim who first introduced me to DuckDB. By using the combination of a Python notebook and DuckDB, I can. load a significant amount of data into Microsoft Fabric with relatively low-capacity consumption. This is my go-to way when initially working with data ingestion.
Why I created these skills
Writing a Pandas Data Frame to a Microsoft Fabric Lakehouse table should not require rebuilding the same connection, path, and Delta-writing logic in every notebook. I created two reusable skills that guide an AI coding assistant to generate a consistent implementation for common Lakehouse write patterns: overwrite or append, and merge (upsert).
The skills are intended for Fabric Python notebooks and DuckDB as part of a lightweight data-processing pattern. They are especially useful when I want to work without attaching a Lakehouse directly to the notebook, which makes the notebook easier to move between development, test, and production environments.
Fabric-python-delta-write Skill
This skill will write using a Python notebook to a Lakehouse table to overwrite or append
It has been designed so that you do not need to attach the Lakehouse to the notebook.
I personally prefer this option because it allows me to move this notebook to different environments.
It is also really fast when writing to the Lakehouse table.
Fabric-python-delta-merge Skill
This skill will write using a Python notebook to a Lakehouse table to merge data.
I am using this more often to allow for easy changes to my code. This is because if no data is found it will be inserted which is similar to the above skill using append.
It has been designed so that you do not need to attach the Lakehouse to the notebook.
I personally prefer this option because it allows me to move this notebook to different environments.
It is also really fast when writing to the Lakehouse table.
Where to put the skill when using Visual Studio Code
I use Visual Studio Code for my development, and this is where you can put the skill if downloading the skill manually.
In the example below I will be using my Windows PC, the screenshots below show the Windows user profile folder. The exact username in your path will be different.
- Go into the users folder on your Windows PC and then find the .copilot folder.
- In the .copilot folder click on Skills
- Create the folder “fabric-key-vault-pure-python”
- Then paste in the Skill.MD
- Once that is done, close and re-open VS Code to have the skills available.
Using the Skill
I thought it would be good to show you how I used this skill in a prompt using Visual Studio Code.
- And here is the details I got for the notebook creation and how many credits were used.
Summary
I hope that you use this skill with Python notebooks.
Here is a link to the Skills in GitHub.
https://github.com/GilbertQue/Fabric/blob/main/fabric-python-delta-write/SKILL.md
https://github.com/GilbertQue/Fabric/blob/main/fabric-python-delta-merge/SKILL.md
NOTE: Also in this repository is a link to download a PowerShell script to download and have the skills put in the correct location automatically.
These skills make it easier to generate consistent Python-notebook code for writing to Microsoft Fabric Lakehouse Delta tables. The overwrite/append skill supports straightforward batch writes, while the merge skill supports incremental upserts. For suitable workloads, this lightweight pattern can also reduce notebook compute consumption compared with heavier alternatives.
The main benefit is repeatability: install the skills once, provide the environment-specific values in your prompt, review the generated code, and reuse a consistent Lakehouse write pattern across projects.