In this blog post I’m going to show you how you can use variable libraries with connection strings as well as items.

This approach allows you to manage environment-specific configuration when deploying Fabric items across development, test, and production workspaces

If you’re looking to move your fabric capacities across workspaces or regions, you can do this quickly and easily without having to re-Plumb a lot of the work.

Working Example

In this example, I’ll create a solution that retrieves public holiday data from the Nager.Date API and writes it to a Fabric Lakehouse table.

The solution contains:

The objective is to avoid hardcoding workspace and item IDs in the pipeline and notebook. When the solution is deployed to another workspace, only the Variable Library values should need to be configured.

NOTE: The only item that would have to be created would be a Lakehouse.

Creating Fabric Items (Lakehouse, Notebook, Pipeline, Workspace ID)

The first step is to create the Fabric items.

The reason to create the fabric items first is you need the GUIDs or itemid’s to populate the variable libraries later in subsequent steps.

Creating the Lakehouse

The first step is to create a Lakehouse which will store the data.

  • I created my Lakehouse and once completed I then got the Lakehouse GUID which I got from the URL as shown below.
  • NOTE: It is always the GUID after the /lakehouse/ in the URL
  • I then make a note of the item id for Lakehouse which is: 260cc545-9a59-45a7-9897-fb705d4bc4ae

Creating the Notebook

Next, I created the Notebook initially it only has got the code for the API.

This is because once we have created the Variable Library later, I will then update the cells to get the required information from the Variable Library.

  • I created the notebook as shown below.
  • What you will also see is that I have not added any “data items”, this is because I am going to use the Variable Library with the ABFSS path to write to my Lakehouse table.
    • This ensures that my notebook does not have a Lakehouse data item connection, which needs to be updated with CI/CD or when moving to a different workspace.
  • To get the Workspace ID, in my workspace where I am creating my Fabric Items I can get it from the URL as shown below.
  • NOTE: It is always the GUID after the /groups/ in the URL

Getting the Workspace ID

All items are stored within a workspace, so the workspace ID is also required.

  • To get the Workspace ID, in my workspace where I am creating my Fabric Items I can get it from the URL as shown below.
  • NOTE: It is always the GUID after the /groups/ in the URL

I then made a note of the item id for Lakehouse which is: cb2af739-998e-45c8-8c41-f78d2e8fc1aa

Creating the Variable Library

Now that I have got the details I need for my Variable Library, I can now create my Variable Library.

NOTE: As you will see below, I am creating the variables without the Connection Reference or Item Reference because I want it to be dynamic when moving across workspaces.

You can find more details on the Variable Library here: Notebook Variables Link

  • I went to create a new Item, selected Variable Library.
  • I then put in the name as shown below.
  • I then created the following variables.
    • For the WorkspaceId I used the workspace GUID I got above.
    • For the LH_ItemId, this is the Lakehouse GUID
    • For the NB_PublicHolidaysItemId, this is the Notebook GUID
  • I only created a single Value set because currently Variable Libraries cannot be used in a different workspace.
  • I then clicked Save.

Creating the Pipeline

Below are the steps where I created the pipeline to use the Variable Library

NOTE: The code below was created initially with GitHub Copilot

  • I added the Notebook activity and gave it a name.
  • I clicked on the Notebook activity, then settings.
  • Where it says Workspace, I clicked on the dropdown and selected Add Dynamic Content
  • This then opens the Add dynamic Content window.
  • I then needed to click on the three dots to find the Library Variables
  • Then click on the plus sign “+” to open the Select variable window
  • I then selected the WorkspaceId variable
  • I could now see the variable, I clicked on it again to add it into the Expression
  • I could see the expression
    • I clicked Ok
  • Now the Workspace ID is now using the variable
  • I completed the same steps for the Notebook ID
  • Now the Notebook ID is using the variable

Updating the Notebook to use the Variable Library

Next, I had to update the notebook to use the variable library.

Code note: GitHub Copilot was used to help create the initial code shown in this example. I reviewed and tested the code before using it.

  • In the first cell is where I am connecting to the variable library and getting the variables.
  • As you can see above, I am successfully getting the variables and also creating the ABFSS path in order to write to the Lakehouse without attaching the Lakehouse to the notebook.
  • I then ran the notebook, and I can see it successfully wrote the data to the Lakehouse table.

I then tested running the Pipeline to ensure that the variables are being used successfully.

This is now showing how it is using variables where nothing is hardcoded in the pipeline or in the notebook.

Moving Items to another workspace

Now the test was to move my pipeline and notebook into another workspace which had an existing Lakehouse.

Creating the Pipeline

  • I then copied/downloaded or used CI/CD to move the pipeline to my prod workspace
  • What I did was I took my Pipeline, got the JSON code and pasted it into a new Pipeline.
    • Make sure to only copy the details after the objectId
  • I then saved my Pipeline

Creating the Notebook

  • I then copied/downloaded or used CI/CD to move the notebook to my prod workspace.

Creating the Variable Library in my new workspace

I had to create my variable library and give it the new GUIDs because these items are in my new workspace.

NOTE: I created the Variable Library in the production workspace using the same name as the library in the development workspace. Because the pipeline and notebook refer to the library by name, keeping the name consistent means, I don’t need to modify their code after deployment.

Although the library name remains the same, its variable values are updated to contain the workspace and item IDs for the production environment.

  • Here is the Variable Library after I had put in the new GUIDS
    • NOTE: You can refer to the steps I did previously to get the associated GUIDs.
  • As you can see I gave it a new name of VL_PublicHolidays

Testing the Pipeline, Notebook and Variables

The final step in my new/prod workspace was to test the pipeline to see if it would work without me having to modify any of the code.

This ran successfully

Summary

Microsoft Fabric Variable Libraries provide a central location for managing configuration values that differ between environments, such as workspace IDs, Lakehouse item IDs, and notebook item IDs.

In this example, I created a pipeline that runs a notebook to retrieve public holiday data and write it to a Lakehouse table. Instead of hardcoding environment-specific IDs in the pipeline and notebook, I stored them in a Variable Library.

I then deployed the pipeline and notebook to another workspace and created the corresponding Variable Library using the destination workspace’s values. Because the library and variable references remained consistent, the solution ran successfully without requiring changes to the pipeline definition or notebook code.

This approach can reduce manual configuration, improve consistency, and make Fabric solutions easier to deploy and maintain across development, test, and production environments

Thanks for reading!