Find cross-tenant private endpoint connections

Posted on: June 17, 2024

---

In a hurry? Skip the introduction and jump directly to the solution:
Finding cross-tenant private endpoints and their connections

Private endpoints is a capability which allows you to access various services over a private connection. Private endpoints are commonly used with services like Azure Storage, Azure SQL Database, and Azure Key Vault. If you’re developing your own service, then you can use Private Link Service. There is excellent workflow diagram that explains the whole process in a single picture. Please check it out.

Private endpoints can be used to connect services between different tenants. This is useful when you have a service that you want to share with another tenant. It can be a storage account that you want the other party to connect and push data to.

Let’s look at this example from the eyes of our dear fictitious companies: Contoso and Litware. Contoso is a service provider and Litware is a consumer of that service. To share data between these companies, Contoso has created storage account and Litware will connect to that storage account using private endpoint.

Note:
I’m using Storage account as an example here, but this can be any service that supports private endpoints.

Here is high-level overview of our scenario:

Contoso has created stprovider storage account into resource group rg-provider:

Contoso shares the ResourceID of the storage account with Litware:

/subscriptions/<contoso>/resourceGroups/rg-provider/providers/Microsoft.Storage/storageAccounts/stprovider

And since they’re going to use blob for data sharing, they’ll share sub-resource blob with Litware as well.

---

Hint:
You can use Get-AzPrivateLinkResource cmdlet with resource ID to get all the supported sub-resource types for any service.

Get-AzPrivateLinkResource `
  -PrivateLinkResourceId "/subscriptions/<contoso>/resourceGroups/rg-provider/providers/Microsoft.Storage/storageAccounts/stprovider" | `
  Format-Table

You can read more about it at the Manage Azure private endpoints documentation.

---

Litware prepares their environment by creating resource group rg-consumer and virtual network vnet-consumer:

Now they’re ready to create private endpoint to the storage account:

They input the ResourceID of the storage account and the sub-resource blob as shared by Contoso. They also provide message that can be then verified by Contoso later when they’re approving the connection:

After they have finished creating the private endpoint, they can see that the connection is in Pending state:

If they now try to click the Private link resource link in the above screenshot, they’ll get this error:

The above is of course expected, since it tries to navigate to the storage account in Contoso’s subscription and Litware does not have access to that subscription.

Now, Litware must wait for Contoso to approve the connection.

Contoso now sees the pending request in their portal:

In storage account:

In Private Link Center:

They can now approve the connection and add their own message to the connection:

Litware can now see this approved connection and both messages are stored in the private endpoint as well:

Similarly, Contoso sees the connection:

Note:
In the above screenshot, you can see that the original requestMessage is not stored in the connection.
Therefore, it’s good idea to just append your own note to the end of the original message.

If Contoso tries to access now the private endpoint, they’ll see same error as Litware did before:

Now the handshake is complete and Litware can start using the private endpoint to access the storage account.

Finding cross-tenant private endpoints and their connections

In the introduction, we saw how you can share a service between different tenants. This might not be relevant to every company, so maybe you want to limit cross-tenant private endpoint connections. This Azure Policy based approach helps you to prevent them from being created in the first place and you can always make exemptions if needed. It’s good to understand that this same sharing technique is used with managed private endpoints as well.

Next, I’ll show you how to find all the cross-tenant private endpoints and their connections in your environment.

Let’s study Storage Account we created in the above example:

We can see that we have resource IDs to the other party’s target resource. This is the key to finding cross-tenant private endpoint connections.

In the producer side, we can see this information stored in the privateEndpointConnections of the ARM object. I want to use Resource Graph to find this information, so I looked prior art on this topic online. I found this post which has good starting point for my Resource Graph query:

An overview of Azure Managed Virtual Networks (Managed VNets)

I wanted to collect additional information about the connection, so I decided to go full PowerShell way. My implementation has these steps:

1. Get all subscriptions in the tenant
2. Get all resources with privateEndpointConnections
3. Collect all the possible information about the connection
   • Subscription and tenant information from both the source and target
4. Output the information to CSV file

Step 1: Get all subscriptions in the tenant

resourcecontainers
| where type == 'microsoft.resources/subscriptions'
| project  subscriptionId, name, tenantId

In the above query, we’re also getting all the tenant IDs used by the subscriptions.

Step 2: Get all resources with privateEndpointConnections

resources
| where isnotnull(properties) and properties contains "privateEndpointConnections"
| where array_length(properties.privateEndpointConnections) > 0
| mv-expand properties.privateEndpointConnections
| extend status = properties_privateEndpointConnections.properties.privateLinkServiceConnectionState.status
| extend description = coalesce(properties_privateEndpointConnections.properties.privateLinkServiceConnectionState.description, "")
| extend privateEndpointResourceId = properties_privateEndpointConnections.properties.privateEndpoint.id
| extend privateEndpointSubscriptionId = tostring(split(privateEndpointResourceId, "/")[2])
| project id, name, location, type, resourceGroup, subscriptionId, tenantId, privateEndpointResourceId, privateEndpointSubscriptionId, status, description

The above query heavily borrows the query structure presented in this post .

Step 3: Collect all the possible information about the connection

I now use the previous information about the subscriptions and tenants to get more information about the connection.

If I have subscription and I don’t have tenant id of that subscription, then I’ll make a simple call to get this information in the error message (this is exactly what Azure Portal does and what we saw in the above screenshots):

$subscriptionResponse = Invoke-AzRestMethod -Path "/subscriptions/$($SubscriptionID)?api-version=2022-12-01"
$startIndex = $subscriptionResponse.Headers.WwwAuthenticate.Parameter.IndexOf("https://login.windows.net/")
$tenantID = $subscriptionResponse.Headers.WwwAuthenticate.Parameter.Substring($startIndex + "https://login.windows.net/".Length, 36)

WWW-Authenticate header contains the following error message:

Bearer 
authorization_uri="https://login.windows.net/33e01921-4d64-4f8c-a055-5bdaffd5e33d",
error="invalid_token",
error_description="The access token is from the wrong issuer. 
It must match the tenant associated with this subscription. Please use correct authority to get the token."

From the above output, you can parse the tenant id e.g.,

33e01921-4d64-4f8c-a055-5bdaffd5e33d

Each of the tenant ids are then used to get more information about the tenant using Graph API:

$tenantResponse = Invoke-AzRestMethod `
  -Uri "https://graph.microsoft.com/v1.0/tenantRelationships/findTenantInformationByTenantId(tenantId='$TenantID')"
$tenantInformation = ($tenantResponse.Content | ConvertFrom-Json)
$tenantInformation

Here is the output of the above call:

@odata.context      : https://graph.microsoft.com/v1.0/$metadata#microsoft.graph.tenantInformation
tenantId            : 33e01921-4d64-4f8c-a055-5bdaffd5e33d
federationBrandName : 
displayName         : MS Azure Cloud
defaultDomainName   : MSAzureCloud.onmicrosoft.com

This helps us identify the tenants and see which ones are Microsoft managed (like the above) and which ones are not.

Step 4: Output the information to CSV file

After we have collected all the information, we can output it to CSV file. Data in here is transposed so that it’s easier to read:

External column indicates if the connection is cross-tenant connection or not with Yes or No value. If the value is Managed by Microsoft, then it’s Microsoft managed tenant.

In short:

• External = No means that the connection is within the same tenant.
  • You should have many of these in your environment and these are mainly for information purposes. You can filter these out.
• External = Managed by Microsoft means that the other party is Microsoft.
• External = Yes means that the connection is a cross-tenant connection.
  • This is the value we’re interested in. You can then filter the output based on this value.

Here is similar output but from different environment and some columns are removed:

From Consumer (Litware) point of view

As shown in the above screenshots, Litware can see the connection in their portal like this:

We can do a very similar process to manualPrivateLinkServiceConnections values in Private Endpoints.

Here is example output from that:

Try it yourself

Here is the script to scan all the private endpoint connections in your environment:

JanneMattila/some-questions-and-some-answers/scan-private-endpoint-connections.ps1

.\scan-private-endpoint-connections.ps1

It will get a list of all the private endpoint connections in your environment in CSV format. Here is abbreviated example of the output:

SubscriptionName        : development
SubscriptionID          : <contoso>
ResourceGroupName       : rg-provider
Name                    : stprovider
Type                    : Microsoft.Storage/storageAccounts
TargetResourceId        : /subscriptions/<litware>
                          /resourceGroups/rg-consumer
                          /providers/Microsoft.Network/privateEndpoints/pepstoragesvc
TargetSubscription      : <litware>
TargetTenantID          : <litware-tenant-id>
TargetTenantDisplayName : Litware
TargetTenantDomainName  : litware.onmicrosoft.com
Description             : Litware connecting to the shared storage account CR#12345 - APR#345
Status                  : Approved
External                : Yes

Similarly, you can scan the manualPrivateLinkServiceConnections values from Private Endpoints in the consumer side with this script:

JanneMattila/some-questions-and-some-answers/scan-private-endpoints-with-manual-connections.ps1

.\scan-private-endpoints-with-manual-connections.ps1

Conclusion

I tried to explain how cross-tenant private endpoint connections work and how you can find them in your environment. This is a complex topic and I hope I was able to explain it in a way that makes sense.

And I want to still highlight that this is about connectivity, and you still have authentication and authorization on top of this. This is just one, but important piece of the puzzle.

I hope you find this useful!