Monitoring the Power Platform: Canvas Apps - Auditing and Activity Logs Part 2 (2023)

Author Note

All content, including samples, images and writings can be found in my GitHub repo:aliyoussefi/MonitoringPowerPlatform: This repo is designed to document and provide guidance on Monitoring with the Power Platform (github.com)

Revisions

02/12/2023 - Fixed images, updated Office 365 Security and Compliance Center to Microsoft Purview Compliance Center. Updated Titles and workshop direct link.

Summary

Microsoft Power Apps Canvas Apps provide users the ability to quickly create and publish applications to their enterprise. With this ability comes concerns with governance to ensure users are allowed the appropriate permissions as well as identifying and highlighting specific apps across the organization.

This article will build off of Canvas Apps - Auditing and Activity Part 1to examine how to search and store audit and activity logs for Canvas Apps. This includes how to turn on auditing, how to use the Purview CompliancePortal, the Unified Audit Log PowerShell command and the Office 365 Management Activity API. We will conclude with thoughts on monitoring tools such as Azure Sentinel, and storage tools such as Cosmos DB and Azure Blob Storage.

Searching the Unified Audit Log

Searching the Unified Audit Log can be performed both manually and with automation. Manually there is a portal called the Microsoft Purview CompliancePortal that provides a centralized area for auditing all Office 365 services including the Power Platform. If automation is desired, the Unified Audit Log offers both a PowerShell module and an API which can be subscribed to. This section covers all three of these in detail.

TheMicrosoft Purview Compliance Portal

Accessing the Office 365 Audit Reports

To begin using the Office 365 Unified Audit Log, your organization will need to have at minimum a specific Office 365 or Microsoft 365 license. Currently the minimum license is the E3 license. The type of license will impact the retention of the logs, E3 is for 90 days while the E5 license retains for a year by default. To turn on, click the "Turn on auditing" button. If an error message occurs, most likely its due to licensing.

Users will need to be an Office 365 Global Administrator or be a member of one or morePurview Compliance Center role groups. To provide a user access to the Security and Compliance Center, use the Office 365 Admin Center and open the Admin center for Compliance. Alternatively, you can directly access the permissions page from here. The permission needed is "View-Only Audit Logs" or "Audit Logs" which allows for viewing and exporting of audit reports.

Navigating theMicrosoft Purview Compliance Center Portal

The PurviewCompliance Center, found at https://protection.office.com, can be used by security analysts to manually search for specific events using a portal user interface.

As shown in the image above, a single or multiple activities performed on aCanvas App can be chosen for analysis. To begin our search, run an audit log search by specifying the activities, the time range and optional search parameters such as users or site. Once the search is complete, review the results in the portal. The results are maxed out at the most 5000 newest events and are incremented 150 records at a time.

Below is an image showing a drill down into the Launched app audit record.

Here is an image showing where I edited the App permissions to allow Everyone in my organization access to view:

The Object Id within Azure Active Directory:

PowerShell

InPowerShellthe Search-UnifiedAuditLog, part of the Exchange Online PowerShell V2 module, is used for Office 365 services including Power Apps Canvas and Model Driven Apps. For Canvas Apps there exist a record type called "PowerAppsApp" that will scope to only logs from Canvas Apps. Using the events listed in the Power Apps logging documentation, here are the PowerShell Operation equivalents to each captured event:

To use the Search-UnifiedAuditLog, connect using the Connect-ExchangeOnline cmdlet. This is a replacement for using the New-PSSession, however currently creating this session will also work.

Import-Module ExchangeOnlineManagement $UserCredential = Get-CredentialConnect-ExchangeOnline -Credential $UserCredential

NOTE: Please refer to the section "Important Notes on the Exchange Online Module" for important authentication considerations.

The below PowerShell command uses the Search-UnifiedAuditLog to search for any activities related to Canvas Apps. To specify this, use the -RecordType argument with the value "PowerAppsApp".

$endDate = Get-Date$startDate = $endDate.AddDays(-7) #Search last 7 daysSearch-UnifiedAuditLog -StartDate $startDate -EndDate $endDate -RecordType PowerAppsApp

As the image above shows, this will bring back all activities related to Canvas Apps. Depending on what type of audit record you're looking for, the search can be filtered using the Operation argument.

Example of searching for Canvas App creation events:

Search-UnifiedAuditLog -StartDate $startDate -EndDate $endDate -RecordType PowerAppsApp -Operation "CreatePowerApp"

Example of searching for Canvas App launch events:

Search-UnifiedAuditLog -StartDate $startDate -EndDate $endDate -RecordType PowerAppsApp -Operation "LaunchPowerApp"

Each operation can also be filtered down to the specific user using the -UserIds argument. Consider the below script for filtering launch events for a specific user:

Search-UnifiedAuditLog -UserIds "user@tenant.onmicrosoft.com" -StartDate $startDate -EndDate $endDate -RecordType "PowerAppsApp" -Operations "LaunchPowerApp"

Each record returned includes the data of the audit, the user and the data collected within the AuditData property. The app and environment properties are available within this property which can be extracted and stored for contextual information to a log data store. For help working with the AuditData property, consider using with the "ConvertFrom-Json" cmdlet.

$AuditData = ConvertFrom-Json -InputObject $_.AuditData

Azure Automation allows for the use of running PowerShell runbooks in Azure. Here is an example image of the output from running the Search-UnifiedAuditLog in Azure Automation:

One of the benefits of Azure Automation is the ability to schedule the execution of our script. The image below shows the recurrence based on a schedule created.

Another benefit is thenative integration with Azure Log Analytics.This integration will come in handy further in this article.

Kusto Query:

AzureDiagnostics | where StreamType_s == "Output" | project TimeGenerated, ResultDescription

Important Notes on the Exchange Online Module

NOTE: I want to point out verbiage that exists on the Search-UnifiedAuditLog page referring to programmatically downloading data from the audit log:

If you want to programmatically download data from the Office 365 Audit Log, we recommend that you use the Office 365 Management Activity API instead of using the Search-UnifiedAuditLog cmdlet in a PowerShell script. The Office 365 Management Activity API is a REST web service that you can use to develop operations, security, and compliance monitoring solutions for your organization. For more information, see Office 365 Management Activity API reference.

The above is important to point out that the current Exchange Online modules rely on Basic authentication. This is a major concern as it requires either a user to interactively login or to supply credentials as a username and password stored somewhere like Azure Key Vault.

Office 365 Management Activity API

Another way to automate delivery of audit data is by use of the Office 365 Management Activity API. To begin using the API, an App Registration needs to be created in Azure Active Directory. The App Registration will need to have permissions to the Office 365 Management APIs, scoped to the ActivityFeed.Read permission. Once created, the App Registration can be used to get an access token for working with the API subscriptions and blobs. The below image shows the authentication flow and a single request to the API.

Now that the authentication mechanism is in place, we will have to tell the Office 365 Management API which activities we are interested in. This, in my opinion, is where I hope to see the API achieve filter parity with the PowerShell or Portal for the Unified Audit Log. For Power Apps Canvas Apps, there is no way to filter to the specific workload like what is available in the Portal or through PowerShell. This means we have to subscribe to a general bucket of events called Audit.General.

To get the authorization token and create the subscription I used Postman. I've included my sample Postman collection here which shows how to get an authorization token, to start a subscription, as well as poll for notifications.

As stated in the documentation, a subscription needs to be created for Audit.General. Each tenant you intend to monitor will need its own subscription. Once created, the subscription can be used to poll for events or as a webhook to have notification delivered when events are ready. NOTE: Subscriptions can take up to 12 hours before the content blobs are available.

For the webhook I followed a really great article by Amreek Singh that covers how to setup a subscription and webhook to deliver to Cosmos DB for storing and analysis. Included in his article are Power Automate flow examples that can be used to learn more. The example does not include the authorization token and I'm including this as an addition to his flows.

When receiving a notification, an array of content blobs will be delivered. Here's an example of a content blob notification:

{
 "contentUri": "https://manage.office.com/api/v1.0/1557f771-4c8e-4dbd-8b80-dd00a88e833e/activity/feed/audit/$audit_general$Audit_General$na0045",
 "contentId": "$audit_general$Audit_General$na0045",
 "contentType": "Audit.General",
 "contentCreated": "2020-04-20T21:43:05.262Z",
 "contentExpiration": "2020-04-27T21:40:03.973Z"
}

From what I've seen, each blob contains around 10-15 minutes worth of audits but this could vary depending on the activity captured. Below is an image of the frequency of delivery to the webhook to my flow.

Managing, Archiving and Retaining Events

Utilizing Azure Log Analytics or Azure Application Insights offers native integration with additional monitoring tools to help detect anomalies. Tools such as Azure Sentinel, offering a Security Information Event Management (SIEM) and Security Orchestration Automated Response (SOAR), or Azure Monitoring, offering analysis detection solutions, are perfect for this.

As stated above, based on the license type, audit logs are retained from 90 days to one year. However, business requirements may dictate a long term solution that could cover multiple years of audit activity. Regardless which mechanism to extract data, be it from the Portal or through PowerShell or the API, archiving these audits is a key requirement for most organizations. Each of these tools provide us capabilities to integrate these logs into a centralized data store such as Azure Blob Storage or Cosmos DB for long term retention.

This topic has a considerable amount of impact in your organization's strategy. A future article will cover this topic in more detail, and when published, I'll link from here.

Advanced Audit in Microsoft 365

A recent addition to the compliance tooling within Office 365 is the Advanced Audit feature. Currently I do not see anything specific to the Power Platform but this may change in the future. The key call out I see from the documentation is the ability to retain audit logs from one year and the high bandwidth accessibility for the Office 365 Management Activity API.

The other call out, is the throttling which is capped at 2,000 requests per minute.

Next Steps

In this article we have covered the Unified Audit Logs and what activities are currently captured for Power Apps Canvas Apps. Discussed were techniques to view the audit logs within the Microsoft PurviewCompliance Portal as well as automated techniques using subscriptions and webhooks as well as automating PowerShell using Azure Automation. Consider the combination of using the Unified Audit Log to notify of events happening and the Power App Administration cmdlets to apply security and enrich audits from the audit log.

This article is designed to supplement the article on Power App Analytics, which provides more of an all up view of Canvas App usage. Combining these two documents, an administrator can now track analytic metrics as well as the events that define those metrics.

If you are interested in learning more about specialized guidance and training for monitoring or other areas of the Power Platform, which includes a monitoring workshop, please contact your Customer SuccessAccount Manager or Microsoft representative for further details.

Your feedback is extremely valuable so please leave a comment below and I'll be happy to help where I can! Also, if you find any inconsistencies, omissions or have suggestions, please go here to submit a new issue.

Index

Monitoring the Power Platform: Introduction and Index