Creating connectors for your Logic Apps/Flow (Part 3 – More triggers)

03 May 2017 Michael-Van-Wesemael API Apps, Azure, Flow, Logic Apps
So, let's recap: we had regular connectors, we had poll triggers. So, we're still missing the push triggers. A poll trigger needs to be polled regularly by the logic app to see if it can continue. A push trigger does not have to be polled, it will tell the logic app when it needs to continue, passing the necessary information. At this moment a push trigger needs to be called by a webhook-action. Your trigger needs two function : Subscribe and Unsubscribe. In the Webhook-action these functions will automatically be called when the Logic App starts or stops. The WebHook action needs to pass the callback-url to your Subscribe/Unsubscribe-action. This callback-url is the url your API app will call for starting the Logic App. Your functions should look like this : public static List subscriptions = new List();[HttpPost, Route("api/webhooktrigger/subscribe")]public HttpResponseMessage Subscribe([FromBody] string callbackUrl){ subscriptions.Add(callbackUrl); return Request.CreateResponse();}[HttpPost, Route("api/webhooktrigger/unsubscribe")]public HttpResponseMessage Unsubscribe([FromBody] string callbackUrl){ subscriptions.Remove(callbackUrl); return Request.CreateResponse();} .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } The idea is that you call back the logic apps using the list of subscriptions as this: using (HttpClient client = new HttpClient()){ foreach (string callbackUrl in subscriptions) await client.PostAsync(callbackUrl, @"{""trigger"":""fired""}", new JsonMediaTypeFormatter(), "application/json");} After this your Logic app continues. For using the pushtrigger you need a webhook-connector in your logic app. This needs to call the subscribe and unsubscribe-methods of your API App. For the body you need to pass the callback url.     But still it won't work. When going to the Trigger History you can find that you get a failure because of an error with the content-Type. It is simple to solve: Show the advanced options of your webhook-connector and add the content-type to your subscribe- and unsubscribe-headers as this: Problem solved !

Creating connectors for your Logic Apps/Flow (Part 2 – Triggers)

07 March 2017 Michael-Van-Wesemael API Apps, Azure, Flow, Logic Apps
  In part 1, we saw how to add regular connectors for using in Azure Logic Apps. Remember: the purpose of these blog posts is just for pointing out some gaps in the documentation you'll find on the internet. Next to regular connectors, we can also create connectors to be used as connectors, i.e. connectors that can initiate the logic apps. There are two types of connectors : poll and push connectors. The difference in short is : Poll connectors will be regularly called to see if the logic app should start, using a polling mechanism. Push connectors will call the Logic app to start, using a webhook mechanism.   Poll triggers As mentioned before: poll triggers will be polled on a regular interval to see if something is changed, starting the logic app when this is the case. So how is this polling working ? If you add your API app as first connector in your logic app, he'll show you your triggerfunctions. Just select your trigger, and specify a polling-interval.The triggerstate can be left blank, but will be used for keeping track of the last poll-time. What will happen now: The Logic App will poll your API app, by calling the Get-method. The triggerstate will initially be empty. The Trigger should return HTTP 202 Accepted. The triggerstate should also be returned, containing the time of calling. next polls will contain a value for the triggerstate. Here we can have two options: if we do not need to trigger (i.e. no need to start the Logic App), the App just needs to return HTTP 202 again, with no changes. If the Logic app now has to start, the app needs to return HTTP 200 OK, with a modified triggerstate, and with, of course, the information you would like to pass to the Logic App In my case, I want to trigger the logic app, when the weather changed. The information to pass looks like this : public class WeatherInfo { public int Temperature { get; set; } = 10; public string WeatherCondition { get; set; } = "Stormy"; public int ChanceOfPrecipitation { get; set; } = 80; } .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; } Again : on the get-function, make sure to specify what will be the output. Otherwise your output will not be usable/visible in the logic app. [SwaggerResponse(HttpStatusCode.Accepted)] [SwaggerResponse(HttpStatusCode.OK, "triggered", typeof(WeatherInfo))] [SwaggerOperation("CheckWeatherReport")] For being able to send back the information, ánd the triggerstate, I created a simple helperfunction. private HttpResponseMessage GenerateResponse( HttpStatusCode code, string triggerState, string retryAfter, WeatherInfo data = null) { HttpResponseMessage responseMessage = Request.CreateResponse(code, data); responseMessage.Headers.Add("location", $"{Request.RequestUri.Scheme}://{Request.RequestUri.Host}/api/weather?" + $"triggerState={HttpUtility.UrlEncode(triggerState)}"); responseMessage.Headers.Add("retry-after", retryAfter); return responseMessage; }   After deploying the API app to AZure (don't forget the metadata and the CORS-settings!), I add my API app to my logic app by first selecting "App Services", and then your API App.After that, start using your output, e.g. in a mail. For testing it out I created some view, that allows me to "change" the weather. If that happens, the logic apps starts working (and sends me a mail). For completeness I'll give the full code for my API polltrigger underneath. Next time : Push triggers !! public class WeatherController : ApiController { [SwaggerResponse(HttpStatusCode.Accepted)] [SwaggerResponse(HttpStatusCode.OK, "triggered", typeof(WeatherInfo))] [SwaggerOperation("CheckWeatherReport")] public HttpResponseMessage Get(string triggerState = "") { if (string.IsNullOrEmpty(triggerState)) { triggerState = DateTime.UtcNow.ToString(); return GenerateResponse(HttpStatusCode.Accepted, triggerState, "15"); } else { if (DateTime.Parse(triggerState) < DateTime.UtcNow && WeatherReport.Changed) { WeatherReport.Changed = false; triggerState = DateTime.UtcNow.AddMinutes(2).ToString(); WeatherInfo info = new WeatherInfo() { ChanceOfPrecipitation = 80, Temperature = 8, WeatherCondition = "Storm" }; return GenerateResponse(HttpStatusCode.OK, triggerState, "15", info); } else { return GenerateResponse(HttpStatusCode.Accepted, triggerState, "15"); } } } private HttpResponseMessage GenerateResponse( HttpStatusCode code, string triggerState, string retryAfter, WeatherInfo data = null) { HttpResponseMessage responseMessage = Request.CreateResponse(code, data); responseMessage.Headers.Add("location", $"{Request.RequestUri.Scheme}://{Request.RequestUri.Host}/api/weather?" + $"triggerState={HttpUtility.UrlEncode(triggerState)}"); responseMessage.Headers.Add("retry-after", retryAfter); return responseMessage; } }

Creating connectors for your Logic Apps/Flow (Part 1–regular Connectors)

24 February 2017 Michael-Van-Wesemael Office 365, SharePoint, Azure, Logic Apps, API Apps, Flow
  Azure logic apps have been around for a while now, most of the time as a preview. These apps allow you to create workflow-like apps, by connecting so called Connectors together. These connectors are actually just REST-services, with some extra's, like the exposure of metadata by using Swagger. These Logic apps have also been made available through Office 365, where it is known as "Flow". Office 365 Flow, or Azure Logic apps have access to a library of Connectors, like connectors for Twitter, Facebook, Outlook Mail and Calendar, SharePoint and many more. At the moment I write this there are 84 connectors available. But of course we can add our own connectors. You can find information on how to do this on the internet, however I found myself struggling with Azure because of incomplete information, even in the official Azure documentation. So, I decided to complete the information. From a developer perspective, there are two types of connectors we can create: "regular" connectors and triggers. A regular connector can just be used within your flow, but not at the start. It can take input from connectors preceding it in the flow, and it can return output to the flow.  A trigger can be used in the beginning of a flow and is responsible for starting the flow. Creating a regular Connector Connectors are REST services. In my example I will do this with Web API, but it can be done in any other web-technology. While creating a web application just choose Azure API App. I will create a simple example in which I just reverse an incoming string. The code is pretty simple : [SwaggerResponse(HttpStatusCode.OK,Type =typeof(StringManipulator))] [SwaggerOperation("GetReversedString")] public IHttpActionResult GetReversedString(string id) { var temp = ""; foreach (var item in id) { temp = item + temp; } return Ok(new StringManipulator { Original = id, Reversed = temp }); } .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; } However, the second argument in your SwaggerResponse-attribute, the Type, is super important. Without this argument Logic apps, using your API, will just assume your function returns nothing ! It took me some time to find this out, since it used to work fine without Type in an earlier version of Logic apps, and it's nor really clear from the documentation. So, beware.  After deploying your API app to Azure, there are two more things to do. Locate your API app in the Azure portal, and go to the API settings. The API definition should point to your swagger metadata. Make sure you refer to it using https !!!.Also make sure that CORS is enabled for your logic app, or just use * here. After that you need to restart your app. Otherwise your logic app will not pick it up. There you go. If you want to use it in a Logic app just add the Action App Services, select you API app, and select the action you want to use. And there you go, the following connectors have access to the output of your API app. Pretty straightforward, if you don't forget the Type in your SwaggerResponse-attribute. Using API apps as triggers is somewhat more work. We'll discuss that in part 2.

Loading data in Azure Machine Learning

24 July 2014 Nico-Jacobs Azure, Business Intelligence, Data Mining, AzureML
In July 2014 Microsoft made their cloud-based data mining environment (known as Azure Machine Learning, or AzureML) available to the public. With this platform users can analyze large amounts of data without the need to install and configure special software: A browser and a credit card is all you need . With the increasing number of people in a number-crunching job (data scientists) it is nice to see Microsoft focusing on this. In a previous blog post (see http://blogs.u2u.be/u2u/post/2014/07/14/First-Steps-in-Azure-Machine-Learning.aspx) I show how to get started with setting up an AzureML environment. In this blog post we take a look at loading data in AzureML. Supported data formats Currently AzureML is focusing on the most common formats used in the world of machine learning: Text files containing comma separated values (CSV), tab-separated values (TSV), the Attribute-Relation File Format (ARFF) which was introduced by the open-source Weka machine learning framework, RData files or the SVMLight format. Database tables: Hive tables (Hadoop), Azure Tables and SQL Databases in Azure Since AzureML runs in the Azure cloud, all your data must be in the cloud as well. Either you already uploaded your data to Azure (e.g. your data is stored in an Azure SQL Database) or you will upload it explicitly for this project. In both cases be careful to store your data in the same region as where you’re running your AzureML, since in the preview period, AzureML only runs from the South Central US data center. If you store your data in another data center it will be slower and more expensive to run your experiments. Let’s first consider the scenario where you upload your data from a local file directly into AzureML (Uploading a DataSet), then we cover the scenario where your data is already somewhere in Azure (Reading data). Uploading a DataSet A lot of sample machine learning data sets are already available out-of-the-box in Azure ML. But after some experimenting with public data, you probably want to play with your own data. If you didn’t have your data anywhere on Azure yet, you can upload it as a new dataset in AzureML directly. But before we start adding data sets, first a warning: In the current preview we cannot delete uploaded datatsets. We can override an existing data set with new data, but if you create 1001 data sets, they will be in the list forever (that is: until Microsoft fixes this limitation). Because of this, if your dataset is not yet fixed, consider uploading the data file(s) into a custom Azure blob store and then load them with the reader from within your experiment. To add a new dataset, click the +New button at the bottom left of the ML Studio screen, and select DataSet –> From local file. In the next dialog box, we can pick the file to upload, provide a name (choose well, it cannot be altered later on), select the type of data in the file and provide an optional description: If you select the checkbox you select an existing dataset, who’s content will be overwritten by the file you select. It is impossible to delete or rename a datset, but you can always upload an empty file ‘as a new version’ of a large data set to truncate it. If we now want to use this data, we create a new experiment by clicking the +New button. In this new experiment under the Saved Datasets we will find our uploaded dataset among the list. Just drag it to the design surface. Also remember the search box at the top: by typing part of an object name (and a data set is one of the many objects we have in AzureML) we get a filtered list which makes it easier to find an object. Now that we have our data in AzureML we can start interacting with it, such as simply visualizing our data: click in the circle under the data set and select Visualize: This will open up the overview screen, showing basic statistical information on each data field: Reading data Another way to get data in an AzureML experiment is by first uploading your data in a Azure SQL Database, an Hadoop cluster (such as HDInsight) or upload the files with data (same data types as we had in the previous paragraph) into an Azure blob store. In this case you do not need to create a data set, but you can immediately create a new experiment. In this experiment, locate the Reader under Data Input and Output and drag it into the experiment. When we click in the Reader, we get on the right-hand side all the configurable properties of this Reader. The most important property is the data source type. This one determines which other properties are needed. Select over here the location where your data can be found and configure the other properties appropriately When we now run the experiment, we can visualize the data from this reader, just as we could we an uploaded data set. But we have an extra option. By clicking Save as dataset, we can permanently store this data in AzureML. This speeds up the runtime of an experiment, but it increases the storage cost (we store another redundant copy of the data). In a next blog post, I will discuss data preprocessing.

First Steps in Azure Machine Learning

15 July 2014 Nico-Jacobs Azure, Data Mining, SQL Server BI
Today Microsoft announces the availability of machine learning (data mining) in Azure. As you can assume, you need an Azure account to get started with this, but there are free trial accounts… you can try before you buy. To get started with the machine learning preview go to http://manage.windowsazure.com and log in with your azure account. In the list of options, close to the bottom, you will find Machine Learning: Click the Create an ML workspace link. Currently there is only a Quick Create option available. Invent a unique workspace name. The Workspace owner must be a valid LiveID account. Location is easy: the machine learning is currently only available in South Central US. I guess I as an European will just need to be a little more patient . If you already have an South Central US storage account you can reuse that, but I put all my storage accounts in Europe, so I now will need to create one on US soil. Those who need to keep there data within Europe for legal reasons will need to wait, because I assume Microsoft will make this service available later on in Europe as well. My final configuration looks like this: Now is the time to start reading the tutorial at http://azure.microsoft.com/en-us/documentation/articles/machine-learning-create-experiment/ while Azure is creating your Machine Learning workspace. Once the workspace is created we can click the right arrow next to it. Then click on the DashBoard link at the top, and next click the Sign-in to ML Studio under quick glance: And now we arrive in the ML Studio: At this point you can get started following the tutorials(http://azure.microsoft.com/en-us/documentation/articles/machine-learning-create-experiment/), play with the sample data or build experiments from scratch with your own data: Have fun!

ADAL at Techorama

10 June 2014 Peter-Himschoot .NET Development, Azure
At Techorama I gave a presentation about Active Directory Authentication Libraries (ADAL) and how you can easily use these to add authentication to your mobile apps.The session slides can be found hereADAL.pptx (1.2MB)