In this blog post I show how it is possible to access data between a Private Link Workspace, where I want to read the data from my Public Workspace.

An example of this is where I wanted to use a DirectLake Semantic Model sitting in my Public Workspace where the data is from data in my Private Link Workspace.

Get the Resource Identified from Azure Portal

To create a Private Endpoint, you first need to get the Resource identifier which you can get from the Private Link Service you created earlier.

  • Log into the Azure Portal
  • Search for the Private Link Services
  • To get to the resource identifier follow the steps below
    • 1. Click on Private endpoints
    • 2. Make sure to find the private endpoint created earlier
      • In your example this is “pe-fabric-workspace”
    • 3. Click on the Resource.
  • Once in the template, you need to copy the Resource id as shown below.
  • Click on the Copy to Clipboard
  • Copy this to notepad.

Create the Private Endpoint on Destination Workspace

You will be creating a private endpoint in the workspace to allow it to connect to your restricted workspace using the Private Endpoint which will then go via the Private Link to the restricted workspace.

Go to the workspace where you want to create a shortcut to.

EG: My workspace where I want to create a Lakehouse Shortcut into is called “Dev Workspace”, I will then go to my workspace called “Dev Workspace”

  • Go to the workspace and follow the steps below
    • 1. Click on Outbound networking
    • 2. Click on create to create a new Managed private endpoint
  • On the Create managed private endpoint complete the details below.
    • 1. Give your managed private endpoint a meaningful name
    • 2. From notepad copy the resource identifier we created in the earlier step
    • 3. Select “Workspace” from the “Target sub-resource”
    • 4. Put in a request message for the person who is going to approve this in the Azure Portal in the next steps.
    • 5. Click Create
  • You will then see the start provisioning
  • It will then take a few minutes to activate. Once activated it will show in the status as shown below.

Approve Private Endpoint connection in Private Link Centre

Before the private endpoint can work, it needs to be approved from the Network foundation.

  • Log into the Azure Portal
  • Search for the Private Link Services
  • Complete the steps below to approve the request
    • 1. Click on Pending connections
    • 2. You can then select the connection you want to approve
      • NOTE: You can confirm it by looking at the Resource name
    • 3. Click on Approve.
  • You will then get a confirmation screen as shown below.
    • NOTE: You can confirm in the box highlighted above which will be the Private Endpoint name
    • Click Yes
  • You will then get confirmation of the approval.

Validate Managed Private Endpoint

The next step is to validate it has been approved and is ready to use.

  • Go back to the Workspace
  • Click on Workspace Settings, then Outbound Connections.
  • You should then see the Managed Private Endpoint has the Approval as Approved

Testing the Managed Private Endpoint can read from the Restricted Workspace

The final step is to ensure that it is working via the Managed Private Endpoint using the API Shortcut.

  • Using my Virtual Machine, which is in the Virtual Network, I have a Lakehouse with a table called “restricted-publicholidays” as shown below.
  • You then need to get the following details from the above workspace (source workspace)
    • Workspace ID
      • This can be obtained from the URL directly after groups/
      • https://app.fabric.microsoft.com/groups/5d7b4573-f0dc-45d8-9f85-d3f03cbb1c1c
      • EG: 5d7b4573-f0dc-45d8-9f85-d3f03cbb1c1c
    • Lakehouse ID
      • Open the Lakehouse
      • This can be obtained from the URL directly after lakehouse/
      • https://app.fabric.microsoft.com/groups/5d7b4573-f0dc-45d8-9f85-d3f03cbb1c1c/ lakehouses/1e227d34-22b0-4648-bb66-676bd655aacb
      • EG: 1e227d34-22b0-4648-bb66-676bd655aacb
    • Workspace FQDN
  • Next, create a notebook in the workspace where we want to query the data.
  • Copy the code below and modify it to your details captured above.
# Welcome to your new notebook
# Type here in the cell editor to add code!

Source_WorkspaceId = "5d7b4573-f0dc-45d8-9f85-d3f03cbb1c1c"
Source_WorkspaceFQDN = " 5d7b4573f0dc45d89f85d3f03cbb1c1c.z5d.w.api.fabric.microsoft.com"
Source_LakehouseID = "1e227d34-22b0-4648-bb66-676bd655aacb"

# Read Delta Lake table from the restricted lakehouse by using the workspace DNS-based ABFSS URI

from pyspark.sql import SparkSession

path = f"abfss://{Source_WorkspaceId}@{Source_WorkspaceFQDN}/{Source_LakehouseID}/Tables/dbo/restricted-publicholidays"

df = spark.read.format("delta").load(path)

display(df.limit(10))  # preview some rows to validate the read

 

  • Once the notebook has run I can see the data in the dataframe below.

Summary

In this blog post I have shown you how I had to set up and configure the various settings between my Private and Public workspaces to get it to work.

Thanks for reading, any comments or suggestions are always welcome!