Creating connectors for your Logic Apps/Flow (Part 3 – More triggers)

So, let's recap: we had regular connectors, we had poll triggers. So, we're still missing the push triggers. A poll trigger needs to be polled regularly by the logic app to see if it can continue. A push trigger does not have to be polled, it will tell the logic app when it needs to continue, passing the necessary information. At this moment a push trigger needs to be called by a webhook-action. Your trigger needs two function : Subscribe and Unsubscribe. In the Webhook-action these functions will automatically be called when the Logic App starts or stops. The WebHook action needs to pass the callback-url to your Subscribe/Unsubscribe-action. This callback-url is the url your API app will call for starting the Logic App. Your functions should look like this : public static List subscriptions = new List();[HttpPost, Route("api/webhooktrigger/subscribe")]public HttpResponseMessage Subscribe([FromBody] string callbackUrl){ subscriptions.Add(callbackUrl); return Request.CreateResponse();}[HttpPost, Route("api/webhooktrigger/unsubscribe")]public HttpResponseMessage Unsubscribe([FromBody] string callbackUrl){ subscriptions.Remove(callbackUrl); return Request.CreateResponse();} .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } The idea is that you call back the logic apps using the list of subscriptions as this: using (HttpClient client = new HttpClient()){ foreach (string callbackUrl in subscriptions) await client.PostAsync(callbackUrl, @"{""trigger"":""fired""}", new JsonMediaTypeFormatter(), "application/json");} After this your Logic app continues. For using the pushtrigger you need a webhook-connector in your logic app. This needs to call the subscribe and unsubscribe-methods of your API App. For the body you need to pass the callback url.     But still it won't work. When going to the Trigger History you can find that you get a failure because of an error with the content-Type. It is simple to solve: Show the advanced options of your webhook-connector and add the content-type to your subscribe- and unsubscribe-headers as this: Problem solved !

Creating connectors for your Logic Apps/Flow (Part 2 – Triggers)

  In part 1, we saw how to add regular connectors for using in Azure Logic Apps. Remember: the purpose of these blog posts is just for pointing out some gaps in the documentation you'll find on the internet. Next to regular connectors, we can also create connectors to be used as connectors, i.e. connectors that can initiate the logic apps. There are two types of connectors : poll and push connectors. The difference in short is : Poll connectors will be regularly called to see if the logic app should start, using a polling mechanism. Push connectors will call the Logic app to start, using a webhook mechanism.   Poll triggers As mentioned before: poll triggers will be polled on a regular interval to see if something is changed, starting the logic app when this is the case. So how is this polling working ? If you add your API app as first connector in your logic app, he'll show you your triggerfunctions. Just select your trigger, and specify a polling-interval.The triggerstate can be left blank, but will be used for keeping track of the last poll-time. What will happen now: The Logic App will poll your API app, by calling the Get-method. The triggerstate will initially be empty. The Trigger should return HTTP 202 Accepted. The triggerstate should also be returned, containing the time of calling. next polls will contain a value for the triggerstate. Here we can have two options: if we do not need to trigger (i.e. no need to start the Logic App), the App just needs to return HTTP 202 again, with no changes. If the Logic app now has to start, the app needs to return HTTP 200 OK, with a modified triggerstate, and with, of course, the information you would like to pass to the Logic App In my case, I want to trigger the logic app, when the weather changed. The information to pass looks like this : public class WeatherInfo { public int Temperature { get; set; } = 10; public string WeatherCondition { get; set; } = "Stormy"; public int ChanceOfPrecipitation { get; set; } = 80; } .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; } Again : on the get-function, make sure to specify what will be the output. Otherwise your output will not be usable/visible in the logic app. [SwaggerResponse(HttpStatusCode.Accepted)] [SwaggerResponse(HttpStatusCode.OK, "triggered", typeof(WeatherInfo))] [SwaggerOperation("CheckWeatherReport")] For being able to send back the information, ánd the triggerstate, I created a simple helperfunction. private HttpResponseMessage GenerateResponse( HttpStatusCode code, string triggerState, string retryAfter, WeatherInfo data = null) { HttpResponseMessage responseMessage = Request.CreateResponse(code, data); responseMessage.Headers.Add("location", $"{Request.RequestUri.Scheme}://{Request.RequestUri.Host}/api/weather?" + $"triggerState={HttpUtility.UrlEncode(triggerState)}"); responseMessage.Headers.Add("retry-after", retryAfter); return responseMessage; }   After deploying the API app to AZure (don't forget the metadata and the CORS-settings!), I add my API app to my logic app by first selecting "App Services", and then your API App.After that, start using your output, e.g. in a mail. For testing it out I created some view, that allows me to "change" the weather. If that happens, the logic apps starts working (and sends me a mail). For completeness I'll give the full code for my API polltrigger underneath. Next time : Push triggers !! public class WeatherController : ApiController { [SwaggerResponse(HttpStatusCode.Accepted)] [SwaggerResponse(HttpStatusCode.OK, "triggered", typeof(WeatherInfo))] [SwaggerOperation("CheckWeatherReport")] public HttpResponseMessage Get(string triggerState = "") { if (string.IsNullOrEmpty(triggerState)) { triggerState = DateTime.UtcNow.ToString(); return GenerateResponse(HttpStatusCode.Accepted, triggerState, "15"); } else { if (DateTime.Parse(triggerState) < DateTime.UtcNow && WeatherReport.Changed) { WeatherReport.Changed = false; triggerState = DateTime.UtcNow.AddMinutes(2).ToString(); WeatherInfo info = new WeatherInfo() { ChanceOfPrecipitation = 80, Temperature = 8, WeatherCondition = "Storm" }; return GenerateResponse(HttpStatusCode.OK, triggerState, "15", info); } else { return GenerateResponse(HttpStatusCode.Accepted, triggerState, "15"); } } } private HttpResponseMessage GenerateResponse( HttpStatusCode code, string triggerState, string retryAfter, WeatherInfo data = null) { HttpResponseMessage responseMessage = Request.CreateResponse(code, data); responseMessage.Headers.Add("location", $"{Request.RequestUri.Scheme}://{Request.RequestUri.Host}/api/weather?" + $"triggerState={HttpUtility.UrlEncode(triggerState)}"); responseMessage.Headers.Add("retry-after", retryAfter); return responseMessage; } }

Creating connectors for your Logic Apps/Flow (Part 1–regular Connectors)

  Azure logic apps have been around for a while now, most of the time as a preview. These apps allow you to create workflow-like apps, by connecting so called Connectors together. These connectors are actually just REST-services, with some extra's, like the exposure of metadata by using Swagger. These Logic apps have also been made available through Office 365, where it is known as "Flow". Office 365 Flow, or Azure Logic apps have access to a library of Connectors, like connectors for Twitter, Facebook, Outlook Mail and Calendar, SharePoint and many more. At the moment I write this there are 84 connectors available. But of course we can add our own connectors. You can find information on how to do this on the internet, however I found myself struggling with Azure because of incomplete information, even in the official Azure documentation. So, I decided to complete the information. From a developer perspective, there are two types of connectors we can create: "regular" connectors and triggers. A regular connector can just be used within your flow, but not at the start. It can take input from connectors preceding it in the flow, and it can return output to the flow.  A trigger can be used in the beginning of a flow and is responsible for starting the flow. Creating a regular Connector Connectors are REST services. In my example I will do this with Web API, but it can be done in any other web-technology. While creating a web application just choose Azure API App. I will create a simple example in which I just reverse an incoming string. The code is pretty simple : [SwaggerResponse(HttpStatusCode.OK,Type =typeof(StringManipulator))] [SwaggerOperation("GetReversedString")] public IHttpActionResult GetReversedString(string id) { var temp = ""; foreach (var item in id) { temp = item + temp; } return Ok(new StringManipulator { Original = id, Reversed = temp }); } .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; } However, the second argument in your SwaggerResponse-attribute, the Type, is super important. Without this argument Logic apps, using your API, will just assume your function returns nothing ! It took me some time to find this out, since it used to work fine without Type in an earlier version of Logic apps, and it's nor really clear from the documentation. So, beware.  After deploying your API app to Azure, there are two more things to do. Locate your API app in the Azure portal, and go to the API settings. The API definition should point to your swagger metadata. Make sure you refer to it using https !!!.Also make sure that CORS is enabled for your logic app, or just use * here. After that you need to restart your app. Otherwise your logic app will not pick it up. There you go. If you want to use it in a Logic app just add the Action App Services, select you API app, and select the action you want to use. And there you go, the following connectors have access to the output of your API app. Pretty straightforward, if you don't forget the Type in your SwaggerResponse-attribute. Using API apps as triggers is somewhat more work. We'll discuss that in part 2.

Loading data in Azure Machine Learning

In July 2014 Microsoft made their cloud-based data mining environment (known as Azure Machine Learning, or AzureML) available to the public. With this platform users can analyze large amounts of data without the need to install and configure special software: A browser and a credit card is all you need . With the increasing number of people in a number-crunching job (data scientists) it is nice to see Microsoft focusing on this. In a previous blog post (see I show how to get started with setting up an AzureML environment. In this blog post we take a look at loading data in AzureML. Supported data formats Currently AzureML is focusing on the most common formats used in the world of machine learning: Text files containing comma separated values (CSV), tab-separated values (TSV), the Attribute-Relation File Format (ARFF) which was introduced by the open-source Weka machine learning framework, RData files or the SVMLight format. Database tables: Hive tables (Hadoop), Azure Tables and SQL Databases in Azure Since AzureML runs in the Azure cloud, all your data must be in the cloud as well. Either you already uploaded your data to Azure (e.g. your data is stored in an Azure SQL Database) or you will upload it explicitly for this project. In both cases be careful to store your data in the same region as where you’re running your AzureML, since in the preview period, AzureML only runs from the South Central US data center. If you store your data in another data center it will be slower and more expensive to run your experiments. Let’s first consider the scenario where you upload your data from a local file directly into AzureML (Uploading a DataSet), then we cover the scenario where your data is already somewhere in Azure (Reading data). Uploading a DataSet A lot of sample machine learning data sets are already available out-of-the-box in Azure ML. But after some experimenting with public data, you probably want to play with your own data. If you didn’t have your data anywhere on Azure yet, you can upload it as a new dataset in AzureML directly. But before we start adding data sets, first a warning: In the current preview we cannot delete uploaded datatsets. We can override an existing data set with new data, but if you create 1001 data sets, they will be in the list forever (that is: until Microsoft fixes this limitation). Because of this, if your dataset is not yet fixed, consider uploading the data file(s) into a custom Azure blob store and then load them with the reader from within your experiment. To add a new dataset, click the +New button at the bottom left of the ML Studio screen, and select DataSet –> From local file. In the next dialog box, we can pick the file to upload, provide a name (choose well, it cannot be altered later on), select the type of data in the file and provide an optional description: If you select the checkbox you select an existing dataset, who’s content will be overwritten by the file you select. It is impossible to delete or rename a datset, but you can always upload an empty file ‘as a new version’ of a large data set to truncate it. If we now want to use this data, we create a new experiment by clicking the +New button. In this new experiment under the Saved Datasets we will find our uploaded dataset among the list. Just drag it to the design surface. Also remember the search box at the top: by typing part of an object name (and a data set is one of the many objects we have in AzureML) we get a filtered list which makes it easier to find an object. Now that we have our data in AzureML we can start interacting with it, such as simply visualizing our data: click in the circle under the data set and select Visualize: This will open up the overview screen, showing basic statistical information on each data field: Reading data Another way to get data in an AzureML experiment is by first uploading your data in a Azure SQL Database, an Hadoop cluster (such as HDInsight) or upload the files with data (same data types as we had in the previous paragraph) into an Azure blob store. In this case you do not need to create a data set, but you can immediately create a new experiment. In this experiment, locate the Reader under Data Input and Output and drag it into the experiment. When we click in the Reader, we get on the right-hand side all the configurable properties of this Reader. The most important property is the data source type. This one determines which other properties are needed. Select over here the location where your data can be found and configure the other properties appropriately When we now run the experiment, we can visualize the data from this reader, just as we could we an uploaded data set. But we have an extra option. By clicking Save as dataset, we can permanently store this data in AzureML. This speeds up the runtime of an experiment, but it increases the storage cost (we store another redundant copy of the data). In a next blog post, I will discuss data preprocessing.

First Steps in Azure Machine Learning

Today Microsoft announces the availability of machine learning (data mining) in Azure. As you can assume, you need an Azure account to get started with this, but there are free trial accounts… you can try before you buy. To get started with the machine learning preview go to and log in with your azure account. In the list of options, close to the bottom, you will find Machine Learning: Click the Create an ML workspace link. Currently there is only a Quick Create option available. Invent a unique workspace name. The Workspace owner must be a valid LiveID account. Location is easy: the machine learning is currently only available in South Central US. I guess I as an European will just need to be a little more patient . If you already have an South Central US storage account you can reuse that, but I put all my storage accounts in Europe, so I now will need to create one on US soil. Those who need to keep there data within Europe for legal reasons will need to wait, because I assume Microsoft will make this service available later on in Europe as well. My final configuration looks like this: Now is the time to start reading the tutorial at while Azure is creating your Machine Learning workspace. Once the workspace is created we can click the right arrow next to it. Then click on the DashBoard link at the top, and next click the Sign-in to ML Studio under quick glance: And now we arrive in the ML Studio: At this point you can get started following the tutorials(, play with the sample data or build experiments from scratch with your own data: Have fun!

ADAL at Techorama

At Techorama I gave a presentation about Active Directory Authentication Libraries (ADAL) and how you can easily use these to add authentication to your mobile apps.The session slides can be found hereADAL.pptx (1.2MB)

Excellent intro to Windows Azure

If you want an excellent introduction into Windows Azure I can recommend “Windows Azure: Step by Step” from Roberto Brunetti. This book will teach you the basic components of Windows Azure and how to build an application with them. It will introduce you to Azure Compute, Azure Storage and Azure AppFabric Servicebus; in clear and easy-to-follow step-by-step instructions you build applications that use the different Azure features. In my opinion this is the best way to learn. Unfortunately this book doesn’t have space to delve deeper into some subjects (or you would need a >1000 pages book), so if you want to learn more about all the other things I recommend our Azure training.

Installing requirements using Azure startup tasks

Windows Azure deploys your azure web or worker role in the cloud, on a machine with Windows Server 2008 and .NET 4 pre-installed. But what if you need an additional requirement? What if you need to install some performance counter, or if you need some other piece of software like the media encoder? Then you can use a startup task to get the job done. In this blog post you will create a simple web role using ASP.NET MVC 3, then add a startup task to ensure MVC 3 is also installed on the Azure instance. For this walkthrough you’ll need Visual Studio 2010 and ASP.NET MVC 3. You’ll also need the standalone MVC 3 installer, which you can find here. Step 1: Create the Azure solution. Start by creating a new Cloud project, call it UsingStartupTasks. Click Ok. Don’t add any role just yet, so click Ok in the next screen. MVC 3 is not available from the “New Windows Azure project” dialog, so we’ll need to use another way to get an ASP.NET project in Azure… Now add a new ASP.NET MVC 3 project, calling it HelloMVC3. Select the Internet Application template, leave the rest to its defaults, then press Ok. Right-click the Roles folder beneath your cloud project and select Add->Web Role Project in Solution… Select the HelloMVC3 project in the next screen and hit Ok. Adding the Startup task Add a new folder StartupTasks to your MVC project and add the MVC installer AspNetMVC3Setup.exe to it. Open notepad.exe (don’t add the following file using Visual Studio because it will add a Byte Order Mark and the Azure runtime doesn’t like that) and create a new batch file called installmvc.cmd in the StartupTasks folder. To add it to the Visual Studio project first click on the Show All Files button in the solution explorer, and then right-click the installmvc.cmd file and select Include In Project. Do the same for the AspNetMVC3Setup.exe installer. We’ll use this batch file to execute the installer as follows: enter following in installmvc.cmd: %~dp0AspNetMVC3Setup.exe /q /log %~dp0mvc3_install.htm exit /b 0   The %~dp0 actually returns the install folder for your azure project, so the first line will run the standalone MVC3 installer, this will write any install problems to a log file called mvc3_install.htm. The %~dp0 is used to get the directory containing the startup tasks (an azure server local copy of the StartupTasks folder). The first statement will do a silent (quiet) install of MVC3, and the next line will return a success error code. Make sure both files have a build action of “none” and Copy to Output Directory set to “Copy Always”. Editing the Service definition file Finally you need to open the ServiceDefinition.csdef file and add the task to it: <?xml version="1.0" encoding="utf-8"?> <ServiceDefinition name="UsingStartupTasks" xmlns=""> <WebRole name="HelloMVC3"> <Sites> <Site name="Web"> <Bindings> <Binding name="Endpoint1" endpointName="Endpoint1" /> </Bindings> </Site> </Sites> <Endpoints> <InputEndpoint name="Endpoint1" protocol="http" port="80" /> </Endpoints> <Imports> … </Imports> <Startup> <Task commandLine="StartupTasks\installmvc.cmd" executionContext="elevated" taskType="simple" /> </Startup> </WebRole> </ServiceDefinition>   In this <Startup> element you can add any number of Task elements. Each represents a commandLine that will be executed on the installing instance prior to installing your azure project. You also get a couple of options, first you can specify the executionContext: this can be elevated or limited. Elevated gives you administrator-like privileges and is ideal for installers (which normally only work with admin privileges). Limited gives you “normal user” privileges. You can also choose the taskType. This is how the task will be executed. You get three options: simple, background or foreground. Simple means that the installer will wait for this task to complete before continuing with the next task (or the actual installation of your azure role). Background and foreground means the same thing like in threading. Background will not block the installer from continuing with the next task, and these will run in parallel. So to get MVC3 installed we need to run elevated, and we don’t want installation to continue before MVC3 has been installed, so we choose simple as the task type… Deploying to Azure Right-click on the Cloud project and select Publish… The Deploy Windows Azure project dialog opens: Select your credentials, environment and storage account. You may need to do some setup for this to work… Optionally you can configure remote desktop connections in case something went wrong, this will make it easier to see if MVC 3 was indeed installed. Click on Ok and wait… Deployment in Visual Studio should start: Wait some more till complete (because the startup tasks are executing this will take a long time): Click on the Url. You should now see the MVC 3 screen! In my next blog post I will show you how to turn this startup task into an azure startup plugin, which will make it easier to re-use this startup task.