Monitor your Logic Apps with Log Analytics (Preview)

This is a continuation of the blog series that fellow MVP Billy York and I are putting on around the journey from on-prem SCORCH to Automation in Azure. In this post we will look at a new preview Log Analytics solution that will help you monitor and respond to issues that will invariably happen with Logic Apps.

Logic App Management allows heavy users of Logic Apps to get both ‘at-a-glance’ updates on the success or failure of their apps, or drill down deeply into app runs and get details such as start/stop time, execution duration, action durations, etc… It is especially useful when you have a very large set of Logic Apps. Trying to get a single pane of glass when you have dozens or hundreds of Logic Apps is difficult with this exciting preview feature.

To add this solution to Log Analytics, you will need…..wait for it…..a Log Analytics workspace! Your existing Log Analytics workspace will do just fine – no need to create a new one. In fact, there are a lot of reasons why you wouldn’t want to! Once you have an eligible workspace, it’s time to add the solution. You can get to this either by adding a resource to the resource group, or adding the solution directly from the workspace – both will start same wizard

One option to add the solution is to do it directly from your Log Analytics workspace.
Searching for the solution…
The actual solution is still in preview, but is very stable.

The actual wizard is very simple – just pick your workspace and click create. Once you have your solution provisioned, you can turn it on or off per Logic App by selecting the “Diagnostic Settings” on the LA and editing the setting below – if you want to turn it off, just uncheck the “Send to Log Analytics” setting.

For this article, I setup a couple of simple Logic Apps – both of them query RSS feeds, but one has a good feed URL, and the other doesn’t. I varied their start times a bit just to show some contrasting data.

The good logic app…..
And the bad one – this URL doesn’t actually work.

I let these apps run for a couple of days. When I look at my Log Analytics workspace, I am greeted with this new tile, and can drill down into to get a nice Logic App run overview:

New tile!
And some awesome graphs!!!

This is a great overview screen for your apps. Notice that the central graph is slightly off – I have tried it in multiple browsers on multiple themes, but it shows the same in each. Hence the ‘preview’ moniker on the solution, I guess. Regardless, at a simple glance I can see the LAs that have succeeded or failed, get a list of the errors, and see a visual comparison of the status of my apps. Clicking on the ‘runs’ tab brings me to a bit more details, and if I click on the “See All” link in the bottom left of each tile, I get a familiar looking interface 🙂

A bit more detail……
Our old friend – Kusto!

For anyone wanting to know the actual query, here it is:

AzureDiagnostics
| where Category == "WorkflowRuntime"
| where OperationName == "Microsoft.Logic/workflows/workflowRunCompleted"
| join kind = rightouter
(
    AzureDiagnostics
    | where Category == "WorkflowRuntime"
    | where OperationName == "Microsoft.Logic/workflows/workflowRunStarted"
    | where resource_runId_s in (( AzureDiagnostics
    | where Category == "WorkflowRuntime"
    | where OperationName == "Microsoft.Logic/workflows/workflowTriggerCompleted"
    | project resource_runId_s ))
    | project WorkflowStartStatus=status_s, WorkflowNameFromInnerQuery=resource_workflowName_s, WorkflowIdFromInnerQuery=workflowId_s, resource_runId_s
)
on resource_runId_s
| extend WorkflowStatus=iff(isnotempty(status_s), status_s, WorkflowStartStatus)
| extend WorkflowName=iff(isnotempty(resource_workflowName_s), resource_workflowName_s, WorkflowNameFromInnerQuery)
| extend WorkflowId=iff(isnotempty(workflowId_s), workflowId_s, WorkflowIdFromInnerQuery)
| summarize Count=count() by WorkflowId, WorkflowName, WorkflowStatus

The benefits of having this data in the “Gateway Drug” (trademark pending) of Azure – Log Analytics – should be obvious. Want to export your Logic App data to PowerBI, or even better setup alerts when a LA fails? Combine your LA data with feeds from AAD, Graph, Office365, etc…. Having all of this data in an easily queried repository is priceless.

Azure ARM Fragments

SCOM Trick adopted for Azure ARM

I am working on a project for a very popular conference – working on backend automation that controls everything from speaker coordination to session scheduling.  The current task at hand involves deploying the entire suite of Logic Apps, Azure Automation accounts, Azure Functions, etc…  These would all be deployed to a new Resource Group.  When deploying these resources for a new conference a few things change – changing the conference name from ‘Jazz’ to ‘Midway’ for example, or changing the backend data sources.  

Normally you would use Azure ARM Template Parameters to pass these values when you deploy the resources, and you would be absolutely right!  They are powerful assets to have in your pocket.  It does get a bit dodgy, however, when you start to deploy Logic Apps in the ARM template and those Logic Apps have parameters of their own.  

Logic Apps have parameters and variables all their own, and they are defined just like parameters and variables in ARM templates.  When you want to deploy a Logic App that has parameters you can put them in the ARM Template and reference them in the Logic App, or you can use an ARM template expression in the Logic App.  The latter is not a popular idea, and the former doesn’t evaluate the parameters until the execution of the Logic App.  That obviously makes it difficult to work on the Logic App after it’s deployed.

That’s when I got the idea of using a popular SCOM tool – SCOM Fragments by Kevin Holman – in order to get the best of both worlds.  I wanted a quick way to deploy, and a quick way to edit after deployment.  Cake and all that….

The idea is simplistic – Find/Replace what you want to change before you deploy.  Sounds simple, and it is!  That is essentially how the SCOM Fragments work, and the same idea can be utilized here.  Say, for example, you have a simple Conference Name parameter you want to change.  The first thing you would do is build your Logic App like normal, keeping in place a static conference name.  Export that template.  Now, at the top of the template, add a new parameter to the parameters section in the template, like this:

{
    "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {
        "Conference_Name": {
            "metadata": {
                "defaultValue": "###Awesome_Conference_Name###",
                "description": "Name of the conference - i.e. Management Conference 2019"
            }
        },

Now, find your static conference name down in the code for the Logic App itself, and replace the name with ###Awesome_Conference_Name###.   That’s it!  That is all that is needed to prepare your template for a rapid deployment.  When you need to deploy this template, simply Find/Replace ###Awesome_Conference_Name### with whatever text you want – i.e. “My Super Conference 2019”.  It will update it both in the parameters section, and in the code itself.  Do we really need the Parameter at the top of ARM template, especially if we are just going to replace the text ourselves before we deploy?  That answer is no, but it does help immensely when keeping track when you have a ton of parameters:

{
    "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {
        "Conference_Name": {
            "metadata": {
                "defaultValue": "###Awesome_Conference_Name###",
                "description": "Name of the conference - i.e. Management Conference 2019"
            }
        },
        "Sharepoint_Base_URL": {
            "defaultValue": "https://www.sharepoint.com/sites/Communications"
        },
        "Sharepoint_Speaker_ListName": {"defaultValue": "###Sharepoint_Speaker_ListName###"},
        "Sharepoint_Session_ListName": {"defaultValue": "###Sharepoint_Session_ListName###"},
        "Sharepoint_Session_Selection_Team_ListName": {"defaultValue": "###Sharepoint_Session_Selection_Team_ListName###"},
        "Sharepoint_Approved_Sessions_ListName": {"defaultValue": "###Sharepoint_Approved_Sessions_ListName###"},
        "Speaker_Agreement_URL": {"defaultValue": "###Speaker_Agreement_URL###"},
        "Session_Submission_FormName": {"defaultValue": "###Session_Submission_FormName###"},
        "Sched_Speakers_API": {"defaultValue": "###Sched_Speakers_API###"},
        "Sched_Sessions_API": {"defaultValue": "###Sched_Sessions_API###"},
        "Sched_Base_URL": {"defaultValue": "###Sched_Base_URL###"},
        "Sched_API_Key": {"defaultValue": "###Sched_API_Key###"},
        "MediaPack_URL": {"defaultValue": "###MediaPack_URL###"},

By putting the parameters in your code, even though you aren’t going to use them in the way they are intended, you can easily see which ones you need to replace in one place.  

Custom Log Analytics logging with Logic Apps!

Here is a quick demo on sending a simple API to Log Analytics using Logic Apps. It used to be quite a pain to get data to Log Analytics – using an API, sending via something like an Azure Function, Azure Automation Runbook, PowerShell scripts, etc… Now, you can do it in about 3 minutes with no code!

First – some assumptions:
1: You have a Log Analytics workspace already set in Azure. See this article if you need help with that.
2: Actually, there is no 2. Just make sure you have a Log Analytics workspace.

Create a Logic Apps….App. That naming convention seems wrong.

It will take a few seconds for the app to be created. When it is, enter the designer. In this example, we are going to retrieve a simple piece of data via a web API – we are going to get a current stock quote from IEXTrading.com. For those that don’t know, IEXTrading has an AMAZING web API for pulling stock data. Seriously – check it out: https://iextrading.com/developer/docs/

In this example we will setup a simple 15 minute timer, pull the data from IEXTrading, take the JSON payload from the API call, and send that to Log Analytics. It’s actually really easy.

If you haven’t setup a Log Analytics connection in Logic Apps, then there are a couple of pieces of information from Log Analytics you are going to need. Go into your Log Analytics workspace, click on the ‘Advanced Settings’ section and copy down the “Workspace ID” and either the “Primary” or “Secondary” key. Enter those into the connection information for the Logic Apps Connector. I’ve shown you mine here:

Now – just let it run! It might fail for the first couple of runs – I believe this has something to do with the creation of the custom log in Log Analytics. After a (short) period of time, you can query for your custom log in Log Analytics. The one thing you should know is that the log name you specified in Logic Apps will be appended with “_CL”. That stands for Custom Log, and it will show up if you want it to or not. You can search for you log like this:

search *
| where Type == "Stock_Prices_CL"