Tag: Azure Automation

Automation Tools in Azure – Q1 2020 Edition

Whether you are well into your automation journey or just starting out, it’s important to know what options are available. Moving a manual workload to the wrong automation engine can be just as disruptive as automating a bad workload. Luckily Microsoft has a plethora of tools, so you can be sure to pick the right tool for the right job.

Azure Automation – Process Automation

It’s tough to start an article about automation tools in Azure without starting with Azure Automation – so I won’t try. Azure Automation is going to be the first place you look if you are migrating things like:

  • PowerShell scripts
  • Python scripts
  • System Center Orchestrator runbooks
  • Simple commands called repeatedly (restarting services, for example)

Azure Automation uses RunBooks and Jobs, which will immediately be familiar to Orchestrator admins. Azure Automation supports PowerShell, and Python 2 scripts, along with PowerShell workflows. The automation jobs can be run either on-prem via Hybrid Workers, or in the cloud. A little known secret about Azure Automation – it runs a lot of the backend process that power Azure!

There is another piece to Azure Automation worth calling out – it’s CHEAP. Azure gives you 500 run-time minutes for free each month, with each additional minute costing only $0.002. Watcher tasks are even cheaper – I will go over those in another blog post.

Azure Functions

The server-less powerhouse of the automation options in Azure – Functions are designed for scale, speed, and complete extensibility. Deploy code or docker containers for your function and then build your functions with .Net Core, Node.js, Python, Java, and even PowerShell Core.

With the language options available, moving on-prem workloads should be a breeze. Access your functions from anywhere via API or schedule them to run automatically. Customize your compute stack, secure the functions with multiple keys, and monitor your runs with Log Analytics and App Insights.

You can build your functions in VSCode, any other code editor you choose, or edit and test your function right in the Azure portal. Each Function App can have multiple functions, and scaling can occur manually or automatically. There are so many options available for Azure Functions, it deserves it’s own blog series.

As with Azure Process Automation, Functions are priced really competitively. Check out the pricing list here.

Azure Logic Apps

Anyone coming from a tool like System Center Orchestrator, or other automation tools like MicroFocus Operations Orchestration will tell you one thing those tools have that the tools I have previously mentioned dont – a UI that shows logic flow. Microsoft’s answer to that – Logic Apps. Logic Apps are a personal favorite of mine, and I use them extensively

Building a Logic App couldn’t be simpler. You can start with a blank app, or choose from a LARGE selection of templates that are pre-built. Once in the Logic App Editor, it’s practically drag and drop automation creation. Logic Apps are started with ‘Triggers’, which lead to ‘Actions’. The apps can access services via ‘connections’, of which there are hundreds. If you do happen to find a 3rd party service that doesn’t have a built-in connector, build a custom one!

Logic Apps makes it easy to build complex automations by helping you with things like automatically creating loops when arrays are detected. Allowing you to control parallelism, offering you hundreds of ways to call your app, and more. You can add conditions, switches, do-until loops, etc… There isn’t much they can’t do.

Of course you get the enterprise controls you would expect – version controls, IP access restrictions, full metrics and logging, diagnostics, etc. We run a massive Systems Management and Monitoring conference almost entirely with Logic Apps.

If you are considering migrating from Orchestrator (or other 3rd party automation tool), then look no further. This should be one of the first Azure tools you do a proof of concept with.

Power Apps/Power BI/Power Automate

The tools I have talked about so far are focused on you – the enterprise system admin. But PowerApps gives your organization an exciting and probably under-utilized automation opportunity – your Business users! Even the biggest automation organizations don’t have the resources to automate everything their users want, so why not let them handle some of that automation on their own.

Power Apps let you or your users create new desktop or mobile business applications in a matter of minutes or hours. These can be self contained, or reach out to tools like Azure Functions to extend these simple to make apps into something truly enterprise worthy.

Power BI gives world class data visualizations and business intelligence to the average business user. Using Power BI you can allow your users to create their own dashboards or become their own data scientists directly from their desktop.

Power Automate is the tool formerly known as Flow. If you are familiar with Logic Apps, then Power Automate will look almost identical – and for good reason! Flow was originally built from Logic App code. There are a couple of big differences, though:

  • Power Automate has an amazing mobile app. Start a flow, or even create one from your phone.
  • Power Automate can no simulate screen clicks – Remember AutoIt?

Configuration and Update Management

I am going to lump these two into one description, mainly because each is slightly meta. Configuration management is like PowerShell DSC for your Azure and on-prem resources. Describe what your environment should look like, and determine if you want auto-remediation or not. Expect more information on this in future blog posts.

Update management is patching for all of your resources – on-prem or in Azure. Group your servers logically and schedule OS and app updates, or trigger update management from Log Analytics, runbooks, or functions.

The great thing about Configuration and Update management? The cost. Update and configuration management is practically free – only pay for the data ingestion used by Log Analytics. Update management is even ‘free’ for on-prem resources, including Linux! Configuration management does have a cost for on-prem resources, but the cost is still low.

Event Grid and Hub

Although not automation in the strictest sense of the term, Event Grids and Hubs are prime examples of triggers for automation. For most use cases, Event Grid is going to be the best trigger – Event Hub and even Service Bus are more for telemetry and high-value data, but Event Grid is designed to handle reactionary data. Filter events as they come into Grid, and create action groups based off filtered events. Action groups can have actions for starting Azure Functions, generic web-hooks, automation rubooks, Logic Apps, and more! Send your events to a endpoint API, and you are set to start your automation flows automatically!

Meta – ARM

What’s the first thing you need to automate if you are moving to Azure? The automation workflows themselves, of course! Whether it’s configuration or full deployments, ARM is your best friend.

Azure ARM Fragments

SCOM Trick adopted for Azure ARM

I am working on a project for a very popular conference – working on backend automation that controls everything from speaker coordination to session scheduling.  The current task at hand involves deploying the entire suite of Logic Apps, Azure Automation accounts, Azure Functions, etc…  These would all be deployed to a new Resource Group.  When deploying these resources for a new conference a few things change – changing the conference name from ‘Jazz’ to ‘Midway’ for example, or changing the backend data sources.  

Normally you would use Azure ARM Template Parameters to pass these values when you deploy the resources, and you would be absolutely right!  They are powerful assets to have in your pocket.  It does get a bit dodgy, however, when you start to deploy Logic Apps in the ARM template and those Logic Apps have parameters of their own.  

Logic Apps have parameters and variables all their own, and they are defined just like parameters and variables in ARM templates.  When you want to deploy a Logic App that has parameters you can put them in the ARM Template and reference them in the Logic App, or you can use an ARM template expression in the Logic App.  The latter is not a popular idea, and the former doesn’t evaluate the parameters until the execution of the Logic App.  That obviously makes it difficult to work on the Logic App after it’s deployed.

That’s when I got the idea of using a popular SCOM tool – SCOM Fragments by Kevin Holman – in order to get the best of both worlds.  I wanted a quick way to deploy, and a quick way to edit after deployment.  Cake and all that….

The idea is simplistic – Find/Replace what you want to change before you deploy.  Sounds simple, and it is!  That is essentially how the SCOM Fragments work, and the same idea can be utilized here.  Say, for example, you have a simple Conference Name parameter you want to change.  The first thing you would do is build your Logic App like normal, keeping in place a static conference name.  Export that template.  Now, at the top of the template, add a new parameter to the parameters section in the template, like this:

{
    "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {
        "Conference_Name": {
            "metadata": {
                "defaultValue": "###Awesome_Conference_Name###",
                "description": "Name of the conference - i.e. Management Conference 2019"
            }
        },

Now, find your static conference name down in the code for the Logic App itself, and replace the name with ###Awesome_Conference_Name###.   That’s it!  That is all that is needed to prepare your template for a rapid deployment.  When you need to deploy this template, simply Find/Replace ###Awesome_Conference_Name### with whatever text you want – i.e. “My Super Conference 2019”.  It will update it both in the parameters section, and in the code itself.  Do we really need the Parameter at the top of ARM template, especially if we are just going to replace the text ourselves before we deploy?  That answer is no, but it does help immensely when keeping track when you have a ton of parameters:

{
    "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {
        "Conference_Name": {
            "metadata": {
                "defaultValue": "###Awesome_Conference_Name###",
                "description": "Name of the conference - i.e. Management Conference 2019"
            }
        },
        "Sharepoint_Base_URL": {
            "defaultValue": "https://www.sharepoint.com/sites/Communications"
        },
        "Sharepoint_Speaker_ListName": {"defaultValue": "###Sharepoint_Speaker_ListName###"},
        "Sharepoint_Session_ListName": {"defaultValue": "###Sharepoint_Session_ListName###"},
        "Sharepoint_Session_Selection_Team_ListName": {"defaultValue": "###Sharepoint_Session_Selection_Team_ListName###"},
        "Sharepoint_Approved_Sessions_ListName": {"defaultValue": "###Sharepoint_Approved_Sessions_ListName###"},
        "Speaker_Agreement_URL": {"defaultValue": "###Speaker_Agreement_URL###"},
        "Session_Submission_FormName": {"defaultValue": "###Session_Submission_FormName###"},
        "Sched_Speakers_API": {"defaultValue": "###Sched_Speakers_API###"},
        "Sched_Sessions_API": {"defaultValue": "###Sched_Sessions_API###"},
        "Sched_Base_URL": {"defaultValue": "###Sched_Base_URL###"},
        "Sched_API_Key": {"defaultValue": "###Sched_API_Key###"},
        "MediaPack_URL": {"defaultValue": "###MediaPack_URL###"},

By putting the parameters in your code, even though you aren’t going to use them in the way they are intended, you can easily see which ones you need to replace in one place.  

Azure Runbook for Posting to the OMS API

For MMSMOA 2017, I created an Azure Runbook that could post to the OMS API. Well, it’s more than a month later, but I finally got around to making a post around it. I’m going to skip the basics of creating a runbook, but if you need a primer, I suggest starting here.

Let’s start with the runbook itself. Here is a decent template that I modified from the OMS API documentation. This template takes an input string, parses the string into 3 different fields, and sends those fields over to OMS. Here’s the runbook:

Param
(
    [Parameter (Mandatory= $true)]
    [string] $InputString
)



$CustomerID = Get-AutomationVariable -Name "CustomerID"
$SharedKey = Get-AutomationVariable -Name "SharedKey"
write-output $customerId
write-output $SharedKey
$date = (get-date).AddHours(-1)

# Specify the name of the record type that you'll be creating
$LogType = "MyRecordType"

# Specify a field with the created time for the records
$TimeStampField = "DateValue"

# Create the function to create the authorization signature
Function Build-Signature ($customerId, $sharedKey, $date, $contentLength, $method, $contentType, $resource)
{
    $xHeaders = "x-ms-date:" + $date
    $stringToHash = $method + "`n" + $contentLength + "`n" + $contentType + "`n" + $xHeaders + "`n" + $resource

    $bytesToHash = [Text.Encoding]::UTF8.GetBytes($stringToHash)
    $keyBytes = [Convert]::FromBase64String($sharedKey)

    $sha256 = New-Object System.Security.Cryptography.HMACSHA256
    $sha256.Key = $keyBytes
    $calculatedHash = $sha256.ComputeHash($bytesToHash)
    $encodedHash = [Convert]::ToBase64String($calculatedHash)
    $authorization = 'SharedKey {0}:{1}' -f $customerId,$encodedHash
    return $authorization
}


# Create the function to create and post the request
Function Post-OMSData($customerId, $sharedKey, $body, $logType)
{
    $method = "POST"
    $contentType = "application/json"
    $resource = "/api/logs"
    $rfc1123date = [DateTime]::UtcNow.ToString("r")
    $contentLength = $body.Length
    $signature = Build-Signature `
        -customerId $customerId `
        -sharedKey $sharedKey `
        -date $rfc1123date `
        -contentLength $contentLength `
        -fileName $fileName `
        -method $method `
        -contentType $contentType `
        -resource $resource
    $uri = "https://" + $customerId + ".ods.opinsights.azure.com" + $resource + "?api-version=2016-04-01"

    $headers = @{
        "Authorization" = $signature;
        "Log-Type" = $logType;
        "x-ms-date" = $rfc1123date;
        "time-generated-field" = $TimeStampField;
    }

    $response = Invoke-WebRequest -Uri $uri -Method $method -ContentType $contentType -Headers $headers -Body $body -UseBasicParsing
    $WhatISent = "Invoke-WebRequest -Uri $uri -Method $method -ContentType $contentType -Headers $headers -Body $body -UseBasicParsing"
    write-output $WhatISent
    return $response.StatusCode
}

# Submit the data to the API endpoint

$ComputerName = $InputString.split(';')[0]
$AlertName = $InputString.split(';')[1]
$AlertValue = $InputString.split(';')[2]

# Craft JSON
$json = @"
[{  "StringValue": "$AlertName",
    "Computer": "$computername",
    "NumberValue": "$AlertValue",
    "BooleanValue": true,
    "DateValue": "$date",
    "GUIDValue": "9909ED01-A74C-4874-8ABF-D2678E3AE23D"
}]
"@

Post-OMSData -customerId $customerId -sharedKey $sharedKey -body ([System.Text.Encoding]::UTF8.GetBytes($json)) -logType $logType

There are a couple of things to note with this runbook – first, the date it will post into OMS will be Central Standard Time. If you want to change to another timezone, change the $date = (get-date).AddHours(-1) line (aligning to EST). Second, this script has output which you can remove. The output will only show up in the output section in Azure Automation, which makes it handy for troubleshooting. The third thing you might want to change is the $LogType = “MyRecordType” line. This is the name that OMS will give the log (with one caveat mentioned below).

So, create your runbook in Azure Automation, and give it a test. You will be prompted for the InputString. In my example here, I will use the input string of “Blog Test;Critical;This is a test of an Azure Runbook that calls the OMS HTTP API”

Give it a minute or so, and you are rewarded with this:

Notice the “_CL” at the end of my log name? Notice the “_S” at the end of the fields? OMS does that automatically – CL for custom log, S for string (or whatever data type you happen to pass).

There you have it – runbooks that post to OMS. Add a webbook to the Runbook, and call it from Flow. Send an email to an inbox, have Flow trigger the Runbook with some of the email data, and suddenly you have the ability to send emails and have that data appear in OMS.

Flow to the Rescue! Overcoming Azure Automation Scheduling Limitations

Azure Automation – the 800lb gorilla in the room. If you can think of a way to accomplish a task, more than likely Azure Automation can do it. Combining the ease of use of PowerShell, the sheer power of Azure, and the multitude of integrations available, you can build enterprise worthy automation runbooks quickly and easily.

It’s a no-brainer.
Until it isn’t.

One of the most frustrating limitations of Azure Automation is the scheduling tool. Sure – you can setup one-time and reoccurring schedules with ease, but what if you need something to run often – say every 10 or 15 minutes? Unfortunately, when you go to set a schedule like that, you are greeted with this:

That means you can’t set a single schedule shorter than an hour. Sure, you can set multiple schedules – you would need 4 if you want to run every 15 minutes. What if you want something to run every 5? Are you going to create 20 schedules? Of course not! This is where Microsoft Flow comes to the rescue.

There are 2 ways to trigger an Azure Automation job from Flow (probably more, if you dig deep), so let’s start with the simplest one. If you have a connection from Flow to Azure Automation already, then this simple Flow will start an Azure Automation Runbook:

Boom – no need for 20 schedules here! This simple Flow probably just saved you an hour of clicking and checking. But what if you don’t have a connection to Azure Automation already established? Perhaps you want to run a Runbook that isn’t in your subscription? It’s still pretty easy. The first step is to obtain the webhook URL for the Runbook. Start by access your Azure Automation Runbook – make sure it has focus. In the left navigation pane you should see a link for ‘Webhooks’. Click that to shift focus to the Webhook listing page. Click the ‘Add Webhook’ button at the top of the main pane. From here on out, it’s pretty straight forward. Give your webhook a name, set the expiration date for the webhook, and it your Runbook has any parameters you want the webhook to pass, specify those name.

IMPORTANT – Make sure you copy the URL before clicking OK. Finding that URL later is like pulling teeth – you might be able to do it, but it will be painful.

When you are satisfied with the settings, and have copied the URL, click ‘OK’.

No that you have your webhook created in Azure Automation, it time to setup flow. Luckily for you, it couldn’t be easier!

That’s it! No more Azure Automation Scheduling limitations! Run those runbooks as often as you like!