Using Conditional XAML in Universal Windows Apps

28 January 2015 Diederik-Krols Universal Windows Apps
This article covers two solutions for working with conditional XAML in a Universal Windows app. I created a sample app that illustrates the capabilities. It’s a standard Hub based universal app with a shared project, a Windows app project, and a Windows Phone project. Each platform has its own title, background image, and logo on the first HubSection, while the ToggleSwitch in the second HubSection modifies the visual XAML tree. However, the platform specific projects do not contain any XAML and the solution code does not contain any C# value converters. All is done through conditional XAML in the shared project. Here’s how the app looks like: Conditional XAML Preprocessing The XAML Conditional Compilation (XCC) project by Koen Zwikstra defines a set of custom XAML namespaces. These namespaces allow you to mimic the behavior of the standard symbols for the C# preprocessor #if directive, such as DEBUG, RELEASE, WINDOWS_APP, and WINDOWS_PHONE_APP. The project comes with an extra MSBuild task that does the heavy lifting - you find more details on the concept here. XCC is extremely useful when you want to share XAML files between the Windows and the Phone projects in a universal app solution. You can drop the XAML in the shared project of a universal app solution, and decorate the platform specific elements with the win81 or wp81 namespaces. Just add the XCC Nuget package to the solution, and you’re ready to go: You should add these declaration at the top of the page (mind: the order matters):<Page x:Class="XamlBrewer.Universal.ConditionalXaml.MainPage" ... xmlns:win81="condition:WINDOWS_APP" xmlns:wp81="condition:WINDOWS_PHONE_APP" mc:Ignorable="d win81 wp81" ... mc:ProcessContent="win81:* wp81:*"> .csharpcode { font-size: small; font-family: consolas, "Courier New", courier, monospace; color: black; background-color: #ffffff } .csharpcode pre { font-size: small; font-family: consolas, "Courier New", courier, monospace; color: black; background-color: #ffffff } .csharpcode pre { margin: 0em } .csharpcode .rem { color: #008000 } .csharpcode .kwrd { color: #0000ff } .csharpcode .str { color: #006080 } .csharpcode .op { color: #0000c0 } .csharpcode .preproc { color: #cc6633 } .csharpcode .asp { background-color: #ffff00 } .csharpcode .html { color: #800000 } .csharpcode .attr { color: #ff0000 } .csharpcode .alt { width: 100%; margin: 0em; background-color: #f4f4f4 } .csharpcode .lnum { color: #606060 } Here’s a first example of XCC. Depending on the platform, we go for a different background picture (a portrait or a landscape one). While the assets themselves are in the platform specific project, the XAML is in the shared project:<!-- Win Background --> <win81:Image Source="Assets/tintin_horizontal.jpg" Grid.RowSpan="2" Stretch="UniformToFill" VerticalAlignment="Bottom" HorizontalAlignment="Right" /> <!-- Phone Background --> <wp81:Image Source="Assets/tintin_vertical.jpg" Grid.RowSpan="2" Stretch="UniformToFill" VerticalAlignment="Bottom" HorizontalAlignment="Right" /> The namespace prefixes can not only be applied to main UI elements, but also to child elements and attributes. In the following example we adapt the looks of a TextBlock to the platform. It’s a part of the XAML Brewer logo, which –as you see in the screenshots- looks different on the tablet and the phone. However both looks come from the same XAML: .csharpcode { font-size: small; font-family: consolas, "Courier New", courier, monospace; color: black; background-color: #ffffff } .csharpcode pre { font-size: small; font-family: consolas, "Courier New", courier, monospace; color: black; background-color: #ffffff } .csharpcode pre { margin: 0em } .csharpcode .rem { color: #008000 } .csharpcode .kwrd { color: #0000ff } .csharpcode .str { color: #006080 } .csharpcode .op { color: #0000c0 } .csharpcode .preproc { color: #cc6633 } .csharpcode .asp { background-color: #ffff00 } .csharpcode .html { color: #800000 } .csharpcode .attr { color: #ff0000 } .csharpcode .alt { width: 100%; margin: 0em; background-color: #f4f4f4 } .csharpcode .lnum { color: #606060 } <Run Text="XAML" /> <wp81:LineBreak /> <Run Text="Brewer" Foreground="Silver" win81:Foreground="Black" wp81:Foreground="DarkOrange" FontWeight="Light" /> .csharpcode { font-size: small; font-family: consolas, "Courier New", courier, monospace; color: black; background-color: #ffffff } .csharpcode pre { font-size: small; font-family: consolas, "Courier New", courier, monospace; color: black; background-color: #ffffff } .csharpcode pre { margin: 0em } .csharpcode .rem { color: #008000 } .csharpcode .kwrd { color: #0000ff } .csharpcode .str { color: #006080 } .csharpcode .op { color: #0000c0 } .csharpcode .preproc { color: #cc6633 } .csharpcode .asp { background-color: #ffff00 } .csharpcode .html { color: #800000 } .csharpcode .attr { color: #ff0000 } .csharpcode .alt { width: 100%; margin: 0em; background-color: #f4f4f4 } .csharpcode .lnum { color: #606060 } Visual Studio’s designer ignores the wp81 and win81 prefixes. But you can still use non-prefixed properties. They will be overridden on compilation. Here's a screenshot of the designer in action: Here’s the full XAML for that Hub header. It contains a text for the Visual Studio designer, a text for the Windows app, and a complete data template for Windows Phone:<Hub Foreground="Yellow" Header="Header" win81:Header="Universal App Conditional XAML"> <wp81:Hub.HeaderTemplate> <wp81:DataTemplate> <wp81:TextBlock Style="{StaticResource SubheaderTextBlockStyle}" Text="Universal Conditional XAML" Margin="0 20 0 0" /> </wp81:DataTemplate> </wp81:Hub.HeaderTemplate> ... </Hub> .csharpcode { font-size: small; font-family: consolas, "Courier New", courier, monospace; color: black; background-color: #ffffff } .csharpcode pre { font-size: small; font-family: consolas, "Courier New", courier, monospace; color: black; background-color: #ffffff } .csharpcode pre { margin: 0em } .csharpcode .rem { color: #008000 } .csharpcode .kwrd { color: #0000ff } .csharpcode .str { color: #006080 } .csharpcode .op { color: #0000c0 } .csharpcode .preproc { color: #cc6633 } .csharpcode .asp { background-color: #ffff00 } .csharpcode .html { color: #800000 } .csharpcode .attr { color: #ff0000 } .csharpcode .alt { width: 100%; margin: 0em; background-color: #f4f4f4 } .csharpcode .lnum { color: #606060 } Pretty powerful stuff, right? If–Then–Else as XAML syntax The second project is Conditional XAML by Samuel Blanchard. It adds if-then-else syntax to your XAML. This one is also available as a Nuget package: Again, start with declaring the namespace on top of the page or control:<Page x:Class="XamlBrewer.Universal.ConditionalXaml.MainPage" ... xmlns:cond="using:SamuelBlanchard.Controls.Statements" ...> .csharpcode { font-size: small; font-family: consolas, "Courier New", courier, monospace; color: black; background-color: #ffffff } .csharpcode pre { font-size: small; font-family: consolas, "Courier New", courier, monospace; color: black; background-color: #ffffff } .csharpcode pre { margin: 0em } .csharpcode .rem { color: #008000 } .csharpcode .kwrd { color: #0000ff } .csharpcode .str { color: #006080 } .csharpcode .op { color: #0000c0 } .csharpcode .preproc { color: #cc6633 } .csharpcode .asp { background-color: #ffff00 } .csharpcode .html { color: #800000 } .csharpcode .attr { color: #ff0000 } .csharpcode .alt { width: 100%; margin: 0em; background-color: #f4f4f4 } .csharpcode .lnum { color: #606060 } Now you’re able to use if-then-else in XAML. Here’s how to show a specific control based on the value of a ToggleSwitch. That ToggleSwitch is called SettingsSwitch and it is two-way bound to Preferences.IsMetric in the viewmodel. The data binding is only there because I wanted to test this project in a more or less representative environment (i.e. using MVVM). Both the BooleanConditions in the next code snippet will do the trick:<cond:Statement> <cond:If> <!-- Bind to UI Element --> <!--<cond:BooleanCondition Value1="{Binding ElementName=SettingsSwitch, Path=IsOn}" Operator="Equal" Value2="True" />--> <!-- Bind to viewmodel --> <cond:BooleanCondition Value1="{Binding Source={StaticResource Preferences}, Path=IsMetric}" Operator="Equal" Value2="True" /> <cond:If.Then> <toolkit:Gauge Unit="Km/s" Value="{Binding Speed}" Maximum="15000" wp81:Height="200" /> </cond:If.Then> <cond:If.Else> <toolkit:Gauge Unit="Miles/s" Value="{Binding SpeedInMiles}" Maximum="10000" wp81:Height="200" /> </cond:If.Else> </cond:If> </cond:Statement> .csharpcode { font-size: small; font-family: consolas, "Courier New", courier, monospace; color: black; background-color: #ffffff } .csharpcode pre { font-size: small; font-family: consolas, "Courier New", courier, monospace; color: black; background-color: #ffffff } .csharpcode pre { margin: 0em } .csharpcode .rem { color: #008000 } .csharpcode .kwrd { color: #0000ff } .csharpcode .str { color: #006080 } .csharpcode .op { color: #0000c0 } .csharpcode .preproc { color: #cc6633 } .csharpcode .asp { background-color: #ffff00 } .csharpcode .html { color: #800000 } .csharpcode .attr { color: #ff0000 } .csharpcode .alt { width: 100%; margin: 0em; background-color: #f4f4f4 } .csharpcode .lnum { color: #606060 } Here’s how the app looks like when ‘Metric’ is selected (the above code snippet refers to the top Gauge): And here’s the non-metric option. Please mind that these are actually two different gauges: The package contains more conditions than just BooleanCondition. Here’s an overview: BooleanCondition DoubleCondition StringCondition DateTimeCondition DateTimeOffsetCondition TimeSpanCondition ScreenOrientationCondition IsDesignTimeCondition If you add multiple conditions inside an If statement, then all of them should evaluate to true before the Then takes place. So the And is implemented. On top of that, there is explicit syntax support for the Or, Xor and Not Boolean operators. You find a lot more samples here. I guess these could replace a lot of value converters and visual state managers. Unfortunately the Statement element can not be used inside other controls, so you can not use apply it to attributes. I tried to simulate this, however. You can give a Statement a name, and refer to the result in an element binding. In the next code snippet I defined three reusable conditions (cool!), and use these to slightly change the look and feel of a single control:<!-- Conditional Properties --> <cond:Statement x:Name="HeightUnit" Visibility="Collapsed"> <cond:If Then="Meter" Else="Feet"> <cond:BooleanCondition Value1="{Binding Source={StaticResource Preferences}, Path=IsMetric}" Operator="Equal" Value2="True" /> <!--<cond:BooleanCondition Value1="{Binding ElementName=SettingsSwitch, Path=IsOn}" Operator="Equal" Value2="True" />--> </cond:If> </cond:Statement> <cond:Statement x:Name="MaxHeight" Visibility="Collapsed"> <cond:If Then="150" Else="500"> <cond:BooleanCondition Value1="{Binding Source={StaticResource Preferences}, Path=IsMetric}" Operator="Equal" Value2="True" /> <!--<cond:BooleanCondition Value1="{Binding ElementName=SettingsSwitch, Path=IsOn}" Operator="Equal" Value2="True" />--> </cond:If> </cond:Statement> <cond:Statement x:Name="HeightValue" Visibility="Collapsed"> <cond:If Then="{Binding Height}" Else="{Binding HeightInFeet}"> <cond:BooleanCondition Value1="{Binding Source={StaticResource Preferences}, Path=IsMetric}" Operator="Equal" Value2="True" /> <!--<cond:BooleanCondition Value1="{Binding ElementName=SettingsSwitch, Path=IsOn}" Operator="Equal" Value2="True" />--> </cond:If> </cond:Statement> <toolkit:Gauge Unit="{Binding ElementName=HeightUnit, Path=Content}" Maximum="{Binding ElementName=MaxHeight, Path=Content}" Value="{Binding ElementName=HeightValue, Path=Content}" TickBrush="Transparent" ScaleTickBrush="Black" ValueBrush="White" wp81:Height="200" VerticalAlignment="Center" /> .csharpcode { font-size: small; font-family: consolas, "Courier New", courier, monospace; color: black; background-color: #ffffff } .csharpcode pre { font-size: small; font-family: consolas, "Courier New", courier, monospace; color: black; background-color: #ffffff } .csharpcode pre { margin: 0em } .csharpcode .rem { color: #008000 } .csharpcode .kwrd { color: #0000ff } .csharpcode .str { color: #006080 } .csharpcode .op { color: #0000c0 } .csharpcode .preproc { color: #cc6633 } .csharpcode .asp { background-color: #ffff00 } .csharpcode .html { color: #800000 } .csharpcode .attr { color: #ff0000 } .csharpcode .alt { width: 100%; margin: 0em; background-color: #f4f4f4 } .csharpcode .lnum { color: #606060 } This works ... almost. The bindings in the Gauge control are only triggered when the switch is toggled. So here’s how the app looks like on startup. Observe the lack of initialization in the bottom Gauge: Here are some attempts to solve this issue. I tried refreshing the data context and trigger property change notifications, all in vain. The Gauge is only properly initialized when the main property to which everything is bound (Preferences.IsMetric) is really changed:public MainPage() { this.InitializeComponent(); this.GaugeSection.Loaded += this.GaugeSection_Loaded; } /// <summary> /// Too early. /// </summary> // protected override void OnApplyTemplate() // { // base.OnApplyTemplate(); // this.GaugeSection_Loaded(this, null); // } private void GaugeSection_Loaded(object sender, RoutedEventArgs e) { // Reset data context. Not good enough. // var context = this.GaugeSection.DataContext; // this.GaugeSection.DataContext = null; // this.GaugeSection.DataContext = context; // Too early. // var rocket = this.GaugeSection.DataContext as TintinRocket; // rocket.Height = 130; // Trigger change notification to activate the conditional bindings. var preferences = this.Resources["Preferences"] as Preferences; // This is not enough. // preferences.Refresh(); // It looks like you really have to change bound properties. preferences.IsMetric = !preferences.IsMetric; } .csharpcode { font-size: small; font-family: consolas, "Courier New", courier, monospace; color: black; background-color: #ffffff } .csharpcode pre { font-size: small; font-family: consolas, "Courier New", courier, monospace; color: black; background-color: #ffffff } .csharpcode pre { margin: 0em } .csharpcode .rem { color: #008000 } .csharpcode .kwrd { color: #0000ff } .csharpcode .str { color: #006080 } .csharpcode .op { color: #0000c0 } .csharpcode .preproc { color: #cc6633 } .csharpcode .asp { background-color: #ffff00 } .csharpcode .html { color: #800000 } .csharpcode .attr { color: #ff0000 } .csharpcode .alt { width: 100%; margin: 0em; background-color: #f4f4f4 } .csharpcode .lnum { color: #606060 } I’m not sure if this behavior is a bug, or if I’m doing something wrong. Anyway this conditional XAML project still remains valuable. It’s a ‘young’ Nuget package that is regularly updated, so I suggest to keep an eye on it… More screenshotsAll screenshots in the article came from the Windows app. Here’s how the phone version looks like: Conclusions The conclusions are simple: When you’re building a Universal Windows app for both platforms, then XAML Conditional Compilation is mandatory. When you’re building a Universal Windows app and are drowning in value converters, visual state managers, and custom controls to tweak the visual tree, then you must consider Conditional XAML.There seems to be a way to achieve 'one XAML to rule them all' ... Source Code The source code of the sample app is right here on GitHub. Enjoy! XAML Brewer

A Speech Dialog Box for Universal Windows Phone apps

03 December 2014 Diederik-Krols Universal Windows Apps, Windows Phone
In this article I present a Universal Control to assist in text-to-speech and speech-to-text use cases on Windows Phone 8.1. The Speech Dialog Box is a templatable custom control in a shared universal project. Its default look is inspired by the Cortana UI: a text block with a microphone button attached to it: The Speech Input Dialog is an evolution of the Speech Input Box that I used in the previous two blog posts. This newer version of the control has a more ambitious purpose: it is designed to not only recognize and repeat speech input, but to engage a full conversation. Here’s a class diagram: Style The Speech Dialog Box comes as a localizable and templatable Custom Control. Its default style lives in the generic.xaml file. This style embeds a MediaElement, which is necessary for speaking, and a different grid for each relevant state. The control’s states are Default: the control displays its current content (Question or Text) in a TextBlock, and also a Button to start listening. Listening: the control shows in a TextBlock that he’s listening to you, and also a Button to cancel. Typing: the control accepts typed input through a TextBox. Thinking: the control shows in a TextBlock that he’s trying to recognize the input, and also a Button to cancel. The Speaking state has no particular UI. You don’t need to override this template to make the control blend into your design. The following properties are available to tweak its look: Foreground: the color of the text in default and typing modes, becomes background color in other modes Background: the background color in default and typing modes Highlight: the secondary color, used for the border in typing mode and for the text and icons in listening mode ButtonBackground: the background color for the button in default mode Here’s and overview of the control in the different states, using the default color scheme: Managing a Conversation The Speech Dialog Box comes with the following public members that help you to set up a two-way conversation: VoiceGender: sets the gender to use for the voice (note: the language is taken from the UI culture). Question: sets the question that the control will ask you. Constraints: the list of constraints for speech recognition, e.g. a Semantic Interpretation for Speech Recognition (SISR) – there’s a nice example of this here. StartListening: sets the control to listening mode. Text: the recognized text. TextChanged: this event is raised when the control has finished the text recognition. ResponsePattern: the string format that specifies how the control will reply the recognized text to you, e.g. “I understood {0}”. Speak: lets the control repeat the text. Speak(string text): lets the control speak the specified text in its current voice. SpeakSsml(string ssml): lets the  control speak the specified Speech Synthesis Markup Language (SSML) text in its current voice. An example The sample app contains buttons that demonstrate all of the control’s features. Here’s how the Speech Dialog Box is defined in XAML:<controls:SpeechDialogBox x:Name="SpeechDialogBox" Background="White" Foreground="Black" ButtonBackground="DimGray" Highlight="DarkOrange" /> .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; } Here’s the code behind the ‘conversation’ button:private async void ConversationButton_Click(object sender, RoutedEventArgs e) { // Set the question. this.SpeechDialogBox.Question = "What's your favorite color?"; // Let the control ask the question out loud. await this.SpeechDialogBox.Speak("What is your favorite color?"); // Reset the control when it answered (optional). this.SpeechDialogBox.TextChanged += this.SpeechInputBox_TextChanged; // Teach the control to recognize the colors of the rainbow in a random text. var storageFile = await StorageFile.GetFileFromApplicationUriAsync(new Uri("ms-appx:///Assets//ColorRecognizer.xml")); var grammarFileConstraint = new SpeechRecognitionGrammarFileConstraint(storageFile, "colors"); this.SpeechDialogBox.Constraints.Clear(); this.SpeechDialogBox.Constraints.Add(grammarFileConstraint); // Format the spoken response. this.SpeechDialogBox.ResponsePattern = "What a coincidence. {0} is my favorite color too."; // Start listening this.SpeechDialogBox.StartListening(); } private async void SpeechInputBox_TextChanged(object sender, EventArgs e) { this.SpeechDialogBox.TextChanged -= this.SpeechInputBox_TextChanged; await this.SpeechDialogBox.Reset(); } .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; } Here’s how the conversation may look like: Phone: “What is your favorite color?” Person: “I think it’s blue today.” Phone: “What a coincidence: blue is my favorite color too.” At this moment (at least before the Reset) the Text property of the control has the value “blue”, so you can continue the conversation with it. Cool, isn’t it? Source Code After many years, I decided to stop attaching ZIP files to my blog posts. All the newer stuff will be shared –and updated- through GitHub. This solution was created with Visual Studio 2013 Update 4. Enjoy! Xaml Brewer

Integrating Cortana in your Universal Windows Phone app

21 October 2014 Diederik-Krols Universal Windows Apps, Windows Phone
This article describes how to register and use voice commands to start your Universal Windows Phone app, and how to continue the conversation within your app. The attached sample app comes with the following features: It registers a set of voice commands with Cortana, it recognizes a simple ‘open the door’ command, it discovers whether it was started by voice or by text, it recognizes a natural ‘take over‘ command, whit lots of optional terms, it recognizes a complex ‘close a colored something’, where ‘something’ and ‘color’ come from a predefined list, it modifies one of these lists programmatically, it requests the missing color when you ask it to ‘close something’, and it comes with an improved version of the Cortana-like SpeechInputBox control. Here are some screenshots of this app. It introduces my new personal assistant, named Kwebbel (or in English ‘Kwebble’): Kwebble is written as a Universal app, I dropped the Windows app project because Cortana is not yet available on that platform. I you prefer to stick to Silverlight, check the MSDN Voice Search sample. Registering the Voice Command Definitions Your app can get activated by voice only after it registered its call sign and its list of commands with Cortana. This is done through an XML file with Voice Command Definitions (VCD). It contains different command sets – one for each language that you want to support. This is how such a command set starts: with the command prefix (“Kwebble”) and the sample text that will appear in the Cortana overview:<!-- Be sure to use the v1.1 namespace to utilize the PhraseTopic feature --> <VoiceCommands xmlns="http://schemas.microsoft.com/voicecommands/1.1"> <!-- The CommandSet Name is used to programmatically access the CommandSet --> <CommandSet xml:lang="en-us" Name="englishCommands"> <!-- The CommandPrefix provides an alternative to your full app name for invocation --> <CommandPrefix> Kwebble </CommandPrefix> <!-- The CommandSet Example appears in the global help alongside your app name --> <Example> Close the door. </Example> .csharpcode { font-size: small; font-family: consolas, "Courier New", courier, monospace; color: black; background-color: #ffffff } .csharpcode pre { font-size: small; font-family: consolas, "Courier New", courier, monospace; color: black; background-color: #ffffff } .csharpcode pre { margin: 0em } .csharpcode .rem { color: #008000 } .csharpcode .kwrd { color: #0000ff } .csharpcode .str { color: #006080 } .csharpcode .op { color: #0000c0 } .csharpcode .preproc { color: #cc6633 } .csharpcode .asp { background-color: #ffff00 } .csharpcode .html { color: #800000 } .csharpcode .attr { color: #ff0000 } .csharpcode .alt { width: 100%; margin: 0em; background-color: #f4f4f4 } .csharpcode .lnum { color: #606060 } Later in this article, we cover the rest of the elements. You don’t have to write this from scratch, there’s a Visual Studio menu to add a VCD file to your project: Here’s how to register the file through the VoiceCommandManager. I factored out all Cortana-related code in a static SpeechActivation class:/// <summary> /// Register the VCD with Cortana. /// </summary> public static async void RegisterCommands() { var storageFile = await StorageFile.GetFileFromApplicationUriAsync(new Uri("ms-appx:///Assets//VoiceCommandDefinition.xml")); await VoiceCommandManager.InstallCommandSetsFromStorageFileAsync(storageFile); } .csharpcode { font-size: small; font-family: consolas, "Courier New", courier, monospace; color: black; background-color: #ffffff } .csharpcode pre { font-size: small; font-family: consolas, "Courier New", courier, monospace; color: black; background-color: #ffffff } .csharpcode pre { margin: 0em } .csharpcode .rem { color: #008000 } .csharpcode .kwrd { color: #0000ff } .csharpcode .str { color: #006080 } .csharpcode .op { color: #0000c0 } .csharpcode .preproc { color: #cc6633 } .csharpcode .asp { background-color: #ffff00 } .csharpcode .html { color: #800000 } .csharpcode .attr { color: #ff0000 } .csharpcode .alt { width: 100%; margin: 0em; background-color: #f4f4f4 } .csharpcode .lnum { color: #606060 } If your commands are registered and you ask Cortana “What can I say”, then the app is listed and the sample command is shown: Universal apps cannot –yet- define the icon to be displayed in that list. A simple command Let’s take a look at the individual commands. Each command comes with Name: a technical name that you can use in your code, Example: an example that is displayed in the Cortana UI, One or more ListenFor elements: the text to listen for, the feedback from Cortana when she recognized the command, and Navigate: the page to navigate to when the app is activated through the command. The  Navigate element is required by the XSD, but it is used by Silverlight only: it is ignored by Universal apps. Here’s an example of a very basic ‘open the door’ command, it’s just an exhaustive enumeration of the alternatives.<Command Name="DoorOpen"> <!-- The Command example appears in the drill-down help page for your app --> <Example> Door 'Open' </Example> <!-- ListenFor elements provide ways to say the command. --> <ListenFor> Door open </ListenFor> <ListenFor> Open door </ListenFor> <ListenFor> Open the door </ListenFor> <!--Feedback provides the displayed and spoken text when your command is triggered --> <Feedback> Opening the door ... </Feedback> <!-- Navigate specifies the desired page or invocation destination for the Command--> <!-- Silverlight only, WinRT and Universal apps deal with this themselves. --> <!-- But it's mandatory according to the XSD. --> <Navigate Target="OtherPage.xaml" /> </Command> .csharpcode { font-size: small; font-family: consolas, "Courier New", courier, monospace; color: black; background-color: #ffffff } .csharpcode pre { font-size: small; font-family: consolas, "Courier New", courier, monospace; color: black; background-color: #ffffff } .csharpcode pre { margin: 0em } .csharpcode .rem { color: #008000 } .csharpcode .kwrd { color: #0000ff } .csharpcode .str { color: #006080 } .csharpcode .op { color: #0000c0 } .csharpcode .preproc { color: #cc6633 } .csharpcode .asp { background-color: #ffff00 } .csharpcode .html { color: #800000 } .csharpcode .attr { color: #ff0000 } .csharpcode .alt { width: 100%; margin: 0em; background-color: #f4f4f4 } .csharpcode .lnum { color: #606060 } Here you see the list of commands in the Cortana UI, and the feedback:   Since ‘Kwebble’ is not really an English word, Cortana has a problem recognizing it. I’ve seen the term being resolved into ‘Grebel’ (as in the screenshot), ‘Pueblo’, ‘Devil’ and very often ‘Google’. But anyway, ‘something that sounds like ‘Kwebble’ followed by ‘open the door’ starts the app appropriately. Strangely enough that’s not what happens with text input. If I type ‘Kwebbel close the door’ –another valid command- the app’s identifier is not recognized and I’m redirected to Bing:   How the app reacts A Universal app can determine if it is activated by Cortana in the OnActivated event of its root App class. If the provided event argument is of the type VoiceCommandActivatedEventArgs, then Cortana is responsible for the launch:protected override void OnActivated(IActivatedEventArgs args) { var rootFrame = EnsureRootFrame(); // ... base.OnActivated(args); #if WINDOWS_PHONE_APP Services.SpeechActivation.HandleCommands(args, rootFrame); #endif // Ensure the current window is active Window.Current.Activate(); } .csharpcode { font-size: small; font-family: consolas, "Courier New", courier, monospace; color: black; background-color: #ffffff } .csharpcode pre { font-size: small; font-family: consolas, "Courier New", courier, monospace; color: black; background-color: #ffffff } .csharpcode pre { margin: 0em } .csharpcode .rem { color: #008000 } .csharpcode .kwrd { color: #0000ff } .csharpcode .str { color: #006080 } .csharpcode .op { color: #0000c0 } .csharpcode .preproc { color: #cc6633 } .csharpcode .asp { background-color: #ffff00 } .csharpcode .html { color: #800000 } .csharpcode .attr { color: #ff0000 } .csharpcode .alt { width: 100%; margin: 0em; background-color: #f4f4f4 } .csharpcode .lnum { color: #606060 } /// <summary> /// Verify whether the app was activated by voice command, and deal with it. /// </summary> public static void HandleCommands(IActivatedEventArgs args, Frame rootFrame) { if (args.Kind == ActivationKind.VoiceCommand) { VoiceCommandActivatedEventArgs voiceArgs = (VoiceCommandActivatedEventArgs)args; // ... } } .csharpcode { font-size: small; font-family: consolas, "Courier New", courier, monospace; color: black; background-color: #ffffff } .csharpcode pre { font-size: small; font-family: consolas, "Courier New", courier, monospace; color: black; background-color: #ffffff } .csharpcode pre { margin: 0em } .csharpcode .rem { color: #008000 } .csharpcode .kwrd { color: #0000ff } .csharpcode .str { color: #006080 } .csharpcode .op { color: #0000c0 } .csharpcode .preproc { color: #cc6633 } .csharpcode .asp { background-color: #ffff00 } .csharpcode .html { color: #800000 } .csharpcode .attr { color: #ff0000 } .csharpcode .alt { width: 100%; margin: 0em; background-color: #f4f4f4 } .csharpcode .lnum { color: #606060 } The Result property of this VoiceActivatedEventArgs is a SpeechRecognitionResult instance that contains detailed information about the command. In Silverlight there’s a RuleName property that references the command. In Universal apps this is not there. I was first tempted to parse the full Text to figure out what command was spoken or typed, but that would become rather complex for the more natural commands. It’s easier and safer to walk through the RulePath elements – the list of rule identifiers that triggered the command. Here’s another code snippet from the sample app, the ‘Take over’ command guides us to the main page, the other commands bring us to the OtherPage. We conveniently pass the event arguments to the page we’re navigating to, so it can also access the voice command:// First attempt: // if (voiceArgs.Result.Text.Contains("take over")) // Better: if (voiceArgs.Result.RulePath.ToList().Contains("TakeOver")) { rootFrame.Navigate(typeof(MainPage), voiceArgs); } else { rootFrame.Navigate(typeof(OtherPage), voiceArgs); } .csharpcode { font-size: small; font-family: consolas, "Courier New", courier, monospace; color: black; background-color: #ffffff } .csharpcode pre { font-size: small; font-family: consolas, "Courier New", courier, monospace; color: black; background-color: #ffffff } .csharpcode pre { margin: 0em } .csharpcode .rem { color: #008000 } .csharpcode .kwrd { color: #0000ff } .csharpcode .str { color: #006080 } .csharpcode .op { color: #0000c0 } .csharpcode .preproc { color: #cc6633 } .csharpcode .asp { background-color: #ffff00 } .csharpcode .html { color: #800000 } .csharpcode .attr { color: #ff0000 } .csharpcode .alt { width: 100%; margin: 0em; background-color: #f4f4f4 } .csharpcode .lnum { color: #606060 } Optional words in command phrases The ListenFor elements in the command phrases may contain optional words. These are wrapped in square brackets. Here’s the TakeOver command from the sample app. It recognizes different natural forms of the ‘take over’ command, like ‘would you please take over’ and ‘take over my session please’:<Command Name="TakeOver"> <!-- The Command example appears in the drill-down help page for your app --> <Example> Take over </Example> <!-- ListenFor elements provide ways to say the command, including [optional] words --> <ListenFor> [would] [you] [please] take over [the] [my] [session] [please]</ListenFor> <!--Feedback provides the displayed and spoken text when your command is triggered --> <Feedback> Thanks, taking over the session ... </Feedback> <!-- Navigate specifies the desired page or invocation destination for the Command--> <!-- Silverlight only, WinRT and Universal apps deal with this themselves. --> <!-- But it's mandatory according to the XSD. --> <Navigate Target="MainPage.xaml" /> </Command> .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; } When the command is fired, Kwebble literally takes over and starts talking. Since the VoiceCommandActivated event argument was passed from the app to the page, the page can further analyze it to adapt its behavior:/// <summary> /// Invoked when this page is about to be displayed in a Frame. /// </summary> protected async override void OnNavigatedTo(NavigationEventArgs e) { if (e.Parameter is VoiceCommandActivatedEventArgs) { var args = e.Parameter as VoiceCommandActivatedEventArgs; var speechRecognitionResult = args.Result; // Get the whole command phrase. this.InfoText.Text = "'" + speechRecognitionResult.Text + "'"; // Find the command. foreach (var item in speechRecognitionResult.RulePath) { this.InfoText.Text += ("\n\nRule: " + item); } // ... if (speechRecognitionResult.RulePath.ToList().Contains("TakeOver")) { await this.Dispatcher.RunAsync( Windows.UI.Core.CoreDispatcherPriority.Normal, () => Session_Taken_Over(speechRecognitionResult.CommandMode())); } } } .csharpcode { font-size: small; font-family: consolas, "Courier New", courier, monospace; color: black; background-color: #ffffff } .csharpcode pre { font-size: small; font-family: consolas, "Courier New", courier, monospace; color: black; background-color: #ffffff } .csharpcode pre { margin: 0em } .csharpcode .rem { color: #008000 } .csharpcode .kwrd { color: #0000ff } .csharpcode .str { color: #006080 } .csharpcode .op { color: #0000c0 } .csharpcode .preproc { color: #cc6633 } .csharpcode .asp { background-color: #ffff00 } .csharpcode .html { color: #800000 } .csharpcode .attr { color: #ff0000 } .csharpcode .alt { width: 100%; margin: 0em; background-color: #f4f4f4 } .csharpcode .lnum { color: #606060 } Detecting the command mode If the app was started through a spoken Cortana command, it may start talking. If the command was provided in quiet mode -typed in the input box through the keyboard- then the app should also react quietly. You can figure out the command mode –Voice or Text- by looking it up in the Properties of the SemanticInterpretation of the speech recognition result. Here’s a static method that returns the command mode for a speech recognition result:/// <summary> /// Returns how the app was voice activated: Voice or Text. /// </summary> public static CommandModes CommandMode(this SpeechRecognitionResult speechRecognitionResult) { var semanticProperties = speechRecognitionResult.SemanticInterpretation.Properties; if (semanticProperties.ContainsKey("commandMode")) { return (semanticProperties["commandMode"][0] == "voice" ? CommandModes.Voice : CommandModes.Text); } return CommandModes.Text; } /// <summary> /// Voice Command Activation Modes: speech or text input. /// </summary> public enum CommandModes { Voice, Text } .csharpcode { font-size: small; font-family: consolas, "Courier New", courier, monospace; color: black; background-color: #ffffff } .csharpcode pre { font-size: small; font-family: consolas, "Courier New", courier, monospace; color: black; background-color: #ffffff } .csharpcode pre { margin: 0em } .csharpcode .rem { color: #008000 } .csharpcode .kwrd { color: #0000ff } .csharpcode .str { color: #006080 } .csharpcode .op { color: #0000c0 } .csharpcode .preproc { color: #cc6633 } .csharpcode .asp { background-color: #ffff00 } .csharpcode .html { color: #800000 } .csharpcode .attr { color: #ff0000 } .csharpcode .alt { width: 100%; margin: 0em; background-color: #f4f4f4 } .csharpcode .lnum { color: #606060 } Here’s how the sample app reacts to the ‘take over’ command. In Voice mode it loads an SSML document and starts talking, in Text mode it just updates the screen:if (mode == CommandModes.Voice) { // Get the prepared text. var folder = Windows.ApplicationModel.Package.Current.InstalledLocation; folder = await folder.GetFolderAsync("Assets"); var file = await folder.GetFileAsync("SSML_Session.xml"); var ssml = await Windows.Storage.FileIO.ReadTextAsync(file); // Say it. var voice = new Voice(this.MediaElement); voice.SaySSML(ssml); } else { // Only update the UI. this.InfoText.Text += "\n\nBla bla bla ...."; } .csharpcode { font-size: small; font-family: consolas, "Courier New", courier, monospace; color: black; background-color: #ffffff } .csharpcode pre { font-size: small; font-family: consolas, "Courier New", courier, monospace; color: black; background-color: #ffffff } .csharpcode pre { margin: 0em } .csharpcode .rem { color: #008000 } .csharpcode .kwrd { color: #0000ff } .csharpcode .str { color: #006080 } .csharpcode .op { color: #0000c0 } .csharpcode .preproc { color: #cc6633 } .csharpcode .asp { background-color: #ffff00 } .csharpcode .html { color: #800000 } .csharpcode .attr { color: #ff0000 } .csharpcode .alt { width: 100%; margin: 0em; background-color: #f4f4f4 } .csharpcode .lnum { color: #606060 } Here’s a screenshot of the app in both modes (just images, no sound effects):   Natural command phrases Do not assume that the user will try to launch your app by just saying the call sign and a command name: “computer, start simulation” is so eighties. Modern speech recognition API’s can deal very well with natural language and Cortana is no exception. The Voice Definition Command file can have more than just fixed and optional words (square brackets), it can also deal with so-called phrase lists and phrase topics. These are surrounded with curly brackets. The Kwebble app uses a couple of phrase lists. For an example of a phrase topic, check the already mentioned MSDN Voice Commands Quick start. The following command recognizes the ‘close’ action, followed by a color from a list, followed by the thing to be closed, also from a list. This ‘close’ command will be triggered by phrases like ‘close the door’, ‘close the red door’, and ‘close the yellow window’.<Command Name="Close"> <!-- The Command example appears in the drill-down help page for your app --> <Example> Close door </Example> <!-- ListenFor elements provide ways to say the command, including references to {PhraseLists} and {PhraseTopics} as well as [optional] words --> <ListenFor> Close {colors} {closables} </ListenFor> <ListenFor> Close the {colors} {closables} </ListenFor> <ListenFor> Close your {colors} {closables} </ListenFor> <!--Feedback provides the displayed and spoken text when your command is triggered --> <Feedback> Closing {closables} </Feedback> <!-- Navigate specifies the desired page or invocation destination for the Command--> <!-- Silverlight only, WinRT and Universal apps deal with this themselves. --> <!-- But it's mandatory according to the XSD. --> <Navigate Target="MainPage.xaml" /> </Command> <PhraseList Label="colors"> <Item> yellow </Item> <Item> green </Item> <Item> red </Item> <!-- Fake item to make the color optional --> <Item> a </Item> </PhraseList> <PhraseList Label="closables"> <Item> door </Item> <Item> window </Item> <Item> mouth </Item> </PhraseList> .csharpcode { font-size: small; font-family: consolas, "Courier New", courier, monospace; color: black; background-color: #ffffff } .csharpcode pre { font-size: small; font-family: consolas, "Courier New", courier, monospace; color: black; background-color: #ffffff } .csharpcode pre { margin: 0em } .csharpcode .rem { color: #008000 } .csharpcode .kwrd { color: #0000ff } .csharpcode .str { color: #006080 } .csharpcode .op { color: #0000c0 } .csharpcode .preproc { color: #cc6633 } .csharpcode .asp { background-color: #ffff00 } .csharpcode .html { color: #800000 } .csharpcode .attr { color: #ff0000 } .csharpcode .alt { width: 100%; margin: 0em; background-color: #f4f4f4 } .csharpcode .lnum { color: #606060 } You can add optional terms to the ListenFor elements so that sentences like ‘Would you be so kind to close your mouth, please?’ would also trigger the close command. What you can not do, is define a phrase list as optional. Square brackets and curly brackets cannot surround the same term. As a workaround I added a dummy color called ‘a’. The fuzzy recognition logic will map ‘close door’ to ‘close a door’ and put ‘a’ and ‘door’ in the semantic properties of the speech recognition result. Here’s how the sample app evaluates these properties to figure out how to proceed:var semanticProperties = speechRecognitionResult.SemanticInterpretation.Properties; // Figure out the color. if (semanticProperties.ContainsKey("colors")) { this.InfoText.Text += (string.Format("\nColor: {0}", semanticProperties["colors"][0])); } // Figure out the closable. if (semanticProperties.ContainsKey("closables")) { this.InfoText.Text += (string.Format("\nClosable: {0}", semanticProperties["closables"][0])); } .csharpcode { font-size: small; font-family: consolas, "Courier New", courier, monospace; color: black; background-color: #ffffff } .csharpcode pre { font-size: small; font-family: consolas, "Courier New", courier, monospace; color: black; background-color: #ffffff } .csharpcode pre { margin: 0em } .csharpcode .rem { color: #008000 } .csharpcode .kwrd { color: #0000ff } .csharpcode .str { color: #006080 } .csharpcode .op { color: #0000c0 } .csharpcode .preproc { color: #cc6633 } .csharpcode .asp { background-color: #ffff00 } .csharpcode .html { color: #800000 } .csharpcode .attr { color: #ff0000 } .csharpcode .alt { width: 100%; margin: 0em; background-color: #f4f4f4 } .csharpcode .lnum { color: #606060 } .csharpcode { font-size: small; font-family: consolas, "Courier New", courier, monospace; color: black; background-color: #ffffff } .csharpcode pre { font-size: small; font-family: consolas, "Courier New", courier, monospace; color: black; background-color: #ffffff } .csharpcode pre { margin: 0em } .csharpcode .rem { color: #008000 } .csharpcode .kwrd { color: #0000ff } .csharpcode .str { color: #006080 } .csharpcode .op { color: #0000c0 } .csharpcode .preproc { color: #cc6633 } .csharpcode .asp { background-color: #ffff00 } .csharpcode .html { color: #800000 } .csharpcode .attr { color: #ff0000 } .csharpcode .alt { width: 100%; margin: 0em; background-color: #f4f4f4 } .csharpcode .lnum { color: #606060 } Continuing the conversation inside the app Cortana’s responsibilities stop when it started up your app via a spoken or typed command. If you want to continue the conversation (e.g. for asking more details) then you have to do this inside your app. When the Kwebble sample app is started with a ‘close the door’ command without a color, then she will request for the missing color and evaluate your answer. Here’s how she detects the command with the missing color (remember: ‘a’ is the missing color):// Ask for the missing color, when closing the door. if (speechRecognitionResult.RulePath[0] == "Close" && semanticProperties["closables"][0] == "door" && semanticProperties["colors"][0] == "a") { // ... if (speechRecognitionResult.CommandMode() == CommandModes.Voice) { var voice = new Voice(this.MediaElement); voice.Say("Which door do you want me to close?"); voice.Speaking_Completed += Voice_Speaking_Completed; } } .csharpcode { font-size: small; font-family: consolas, "Courier New", courier, monospace; color: black; background-color: #ffffff } .csharpcode pre { font-size: small; font-family: consolas, "Courier New", courier, monospace; color: black; background-color: #ffffff } .csharpcode pre { margin: 0em } .csharpcode .rem { color: #008000 } .csharpcode .kwrd { color: #0000ff } .csharpcode .str { color: #006080 } .csharpcode .op { color: #0000c0 } .csharpcode .preproc { color: #cc6633 } .csharpcode .asp { background-color: #ffff00 } .csharpcode .html { color: #800000 } .csharpcode .attr { color: #ff0000 } .csharpcode .alt { width: 100%; margin: 0em; background-color: #f4f4f4 } .csharpcode .lnum { color: #606060 } The Kwebble app comes with a Speech Input Box control, an improved version from the one I introduced in my previous blog post. I exposes its Text and its Constraints collection so you can change these. You can now fluently continue the conversation by triggering the ‘listening’ mode programmatically (skipping an obsolete Tap). And there’s more: I added the Cortana sound effect when the control starts listening. Here’s what happens just before Kwebble ask you which door to close. The speech input box is prepared to recognize a specific answer to the question:var storageFile = await StorageFile.GetFileFromApplicationUriAsync(new Uri("ms-appx:///Assets//SpeechRecognitionGrammar.xml")); var grammarFileConstraint = new Windows.Media.SpeechRecognition.SpeechRecognitionGrammarFileConstraint(storageFile, "colors"); this.SpeechInputBox.Constraints.Clear(); this.SpeechInputBox.Constraints.Add(grammarFileConstraint); this.SpeechInputBox.Question = "Which door?"; .csharpcode { font-size: small; font-family: consolas, "Courier New", courier, monospace; color: black; background-color: #ffffff } .csharpcode pre { font-size: small; font-family: consolas, "Courier New", courier, monospace; color: black; background-color: #ffffff } .csharpcode pre { margin: 0em } .csharpcode .rem { color: #008000 } .csharpcode .kwrd { color: #0000ff } .csharpcode .str { color: #006080 } .csharpcode .op { color: #0000c0 } .csharpcode .preproc { color: #cc6633 } .csharpcode .asp { background-color: #ffff00 } .csharpcode .html { color: #800000 } .csharpcode .attr { color: #ff0000 } .csharpcode .alt { width: 100%; margin: 0em; background-color: #f4f4f4 } .csharpcode .lnum { color: #606060 } She only starts listening after she finished asking the question. Otherwise she starts listening to herself. Seriously, I am not kidding!private void Voice_Speaking_Completed(object sender, EventArgs e) { this.SpeechInputBox.StartListening(); } .csharpcode { font-size: small; font-family: consolas, "Courier New", courier, monospace; color: black; background-color: #ffffff } .csharpcode pre { font-size: small; font-family: consolas, "Courier New", courier, monospace; color: black; background-color: #ffffff } .csharpcode pre { margin: 0em } .csharpcode .rem { color: #008000 } .csharpcode .kwrd { color: #0000ff } .csharpcode .str { color: #006080 } .csharpcode .op { color: #0000c0 } .csharpcode .preproc { color: #cc6633 } .csharpcode .asp { background-color: #ffff00 } .csharpcode .html { color: #800000 } .csharpcode .attr { color: #ff0000 } .csharpcode .alt { width: 100%; margin: 0em; background-color: #f4f4f4 } .csharpcode .lnum { color: #606060 } Natural phrases in the conversation I already mentioned that the Voice Command Definitions for Cortana activation are quite capable of dealing with natural language, When you take over the conversation in your app, it gets even better. Using a SpeechRecognitionFileConstraint you can explain the SpeechInputBox (and the embedded SpeechRecognizer) the specific pattern to listen for, in Speech Recognition Grammar Specification (SRGS) XML format. When Kwebble asks you which door to close, she’s only interested in phrases that contain one of the door colors (red, green, yellow). Here’s the SRGS grammar that recognizes these, it just listens for color names, and ignores all the rest:<grammar xml:lang="en-US" root="colorChooser" tag-format="semantics/1.0" version="1.0" xmlns="http://www.w3.org/2001/06/grammar"> <!-- The following rule recognizes any phrase with a color. --> <!-- It's defined as the root rule of the grammar. --> <rule id="colorChooser"> <ruleref special="GARBAGE"/> <ruleref uri="#color"/> <ruleref special="GARBAGE"/> </rule> <!-- The list of colors that are recognized. --> <rule id="color"> <one-of> <item> red <tag> out="red"; </tag> </item> <item> green <tag> out="green"; </tag> </item> <item> yellow <tag> out="yellow"; </tag> </item> </one-of> </rule> </grammar> .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; } Here are some screenshots from the conversation. Kwebbel recognizes the missing color, then asks for it and start listening for the answer; then she recognizes the color:   That’s all folks! Here’s the full code of the sample app: the XAML, the C#, and last but not least the different powerful XML files. The solution was created with Visual Studio 2013 Update 3: U2UC.WinUni.Cortana.zip (164.6KB) Enjoy! XAML Brewer

Speech Recognition and Speech Synthesis in Universal Windows Apps

21 September 2014 Diederik-Krols Universal Windows Apps, Windows Phone, WinRT
This article introduces you to speech recognition (speech-to-text, STT) and speech synthesis (text-to-speech, TTS) in a Universal Windows XAML app. It comes with a sample app – code attached at the end- that has buttons for opening the standard UI for speech recognition, opening a custom UI for speech recognition, speaking a text in a system-provided voice of your choice, and speaking a text from a SSML document. On top of that, the Windows Phone version of the sample app comes with a custom control for speech and keyboard input, based on the Cortana look-and-feel. Here are screenshots from the Windows and Phone versions of the sample app: Speech Recognition For speech recognition there’s a huge difference between a Universal Windows app and a Universal Windows Phone app: the latter has everything built in, the former requires some extra downloading, installation, and registration. But after that, the experience is more or less the same. Windows Phone For speech-to-text, Windows Phone comes with the SpeechRecognizer class, just create an instance of it: it will listen to you, think for a while, and then come up with a text string. You can make the recognition easier by giving the control some context in the form of Constraints, like a predefined or custom Grammar – words and phrases that the control understands, or a topic, or just a list of words. Contradictory to the documentation, you *have* to provide constraints, and a call to CompileConstraintsAsync is mandatory. You can trigger the default UI experience with RecognizeWithUIAsync. This opens the UI and starts waiting for you to speak. Eventually it returns the result as a SpeechRecognitionResult that holds the recognized text, together with extra information such as the confidence of the answer (as a level in an enumeration, and as a percentage). Here’s the full use case:/// <summary> /// Opens the speech recognizer UI. /// </summary> private async void OpenUI_Click(object sender, RoutedEventArgs e) { SpeechRecognizer recognizer = new SpeechRecognizer(); SpeechRecognitionTopicConstraint topicConstraint = new SpeechRecognitionTopicConstraint(SpeechRecognitionScenario.Dictation, "Development"); recognizer.Constraints.Add(topicConstraint); await recognizer.CompileConstraintsAsync(); // Required // Open the UI. var results = await recognizer.RecognizeWithUIAsync(); if (results.Confidence != SpeechRecognitionConfidence.Rejected) { this.Result.Text = results.Text; // No need to call 'Voice.Say'. The control speaks itself. } else { this.Result.Text = "Sorry, I did not get that."; } } .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; } .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; } This is how the default UI looks like. It takes the top half of the screen:   If you don’t like this UI, then you can start a speech recognition session using RecognizeAsync. The recognized text is revealed in the Completed event of the resulting IAsyncOperation. Here’s the whole UI-less story:/// <summary> /// Starts a speech recognition session. /// </summary> private async void Listen_Click(object sender, RoutedEventArgs e) { this.Result.Text = "Listening..."; SpeechRecognizer recognizer = new SpeechRecognizer(); SpeechRecognitionTopicConstraint topicConstraint = new SpeechRecognitionTopicConstraint(SpeechRecognitionScenario.Dictation, "Development"); recognizer.Constraints.Add(topicConstraint); await recognizer.CompileConstraintsAsync(); // Required var recognition = recognizer.RecognizeAsync(); recognition.Completed += this.Recognition_Completed; } /// <summary> /// Speech recognition completed. /// </summary> private async void Recognition_Completed(IAsyncOperation<SpeechRecognitionResult> asyncInfo, AsyncStatus asyncStatus) { var results = asyncInfo.GetResults(); if (results.Confidence != SpeechRecognitionConfidence.Rejected) { await Dispatcher.RunAsync(Windows.UI.Core.CoreDispatcherPriority.Normal, new DispatchedHandler( () => { this.Result.Text = results.Text; ; })); } else { this.Result.Text = "Sorry, I did not get that."; } } .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; } This was just an introduction, there’s a lot more in the Windows.Media.SpeechRecognition namespace. Windows App The SpeechRecognizer class is only native to Windows Phone apps. But don’t worry: if you download the Bing Speech Recognition Control for Windows 8.1, you get roughly the same experience. Before you can actually use the control, you need to register an application in the Azure Market Place to get the necessary credentials. Under the hood the control delegates the processing to a web service that relies on OAUTH. Here’s a screenshot of the application registration pages: If you’re happy with the standard UI then you just drop the control on a page, like this:<sp:SpeechRecognizerUx x:Name="SpeechControl" /> Before starting a speech recognition session, you have to provide your credentials:var credentials = new SpeechAuthorizationParameters(); credentials.ClientId = "YourClientIdHere"; credentials.ClientSecret = "YourClientSecretHere"; this.SpeechControl.SpeechRecognizer = new SpeechRecognizer("en-US", credentials); .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; } The rest of the story is similar to the Windows Phone code. Here’s how to open the standard UI:/// <summary> /// Activates the speech control. /// </summary> private async void OpenUI_Click(object sender, RoutedEventArgs e) { // Always call RecognizeSpeechToTextAsync from inside // a try block because it calls a web service. try { var result = await this.SpeechControl.SpeechRecognizer.RecognizeSpeechToTextAsync(); if (result.TextConfidence != SpeechRecognitionConfidence.Rejected) { ResultText.Text = result.Text; var voice = new Voice(); voice.Say(result.Text); } else { ResultText.Text = "Sorry, I did not get that."; } } catch (Exception ex) { // Put error handling here. } } .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; } This is how the UI looks like. I put a a red box around it: If you want to skip (or replace) the UI part, then you don’t need the control in your XAML (but mind that you still have to download and install it). Just prepare a SpeechRecognizer instance:// The custom speech recognizer UI. SR = new SpeechRecognizer("en-US", credentials); SR.AudioCaptureStateChanged += SR_AudioCaptureStateChanged; SR.RecognizerResultReceived += SR_RecognizerResultReceived; .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; } And call RecognizeSpeechToTextAsync when you’re ready:/// <summary> /// Starts a speech recognition session through the custom UI. /// </summary> private async void ListenButton_Click(object sender, RoutedEventArgs e) { // Always call RecognizeSpeechToTextAsync from inside // a try block because it calls a web service. try { // Start speech recognition. var result = await SR.RecognizeSpeechToTextAsync(); // Write the result to the TextBlock. if (result.TextConfidence != SpeechRecognitionConfidence.Rejected) { ResultText.Text = result.Text; } else { ResultText.Text = "Sorry, I did not get that."; } } catch (Exception ex) { // If there's an exception, show the Type and Message. ResultText.Text = string.Format("{0}: {1}", ex.GetType().ToString(), ex.Message); } } /// <summary> /// Cancels the current speech recognition session. /// </summary> private void CancelButton_Click(object sender, RoutedEventArgs e) { SR.RequestCancelOperation(); } /// <summary> /// Stop listening and start thinking. /// </summary> private void StopButton_Click(object sender, RoutedEventArgs e) { SR.StopListeningAndProcessAudio(); } /// <summary> /// Update the speech recognition audio capture state. /// </summary> private void SR_AudioCaptureStateChanged(SpeechRecognizer sender, SpeechRecognitionAudioCaptureStateChangedEventArgs args) { this.Status.Text = args.State.ToString(); } /// <summary> /// A result was received. Whether or not it is intermediary depends on the capture state. /// </summary> private void SR_RecognizerResultReceived(SpeechRecognizer sender, SpeechRecognitionResultReceivedEventArgs args) { if (args.Text != null) { this.ResultText.Text = args.Text; } } .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; } For Windows as well as Phone projects that require speech-to-text, don’t forget to enable the ‘Microphone’ capability. That also means that the user will have to give consent (only once): Speech Synthesis You now know how to let your device listen to you, it’s time to give it a voice. For speech synthesis (text-to-speech) both Phone and Windows apps have access to a Universal SpeechSynthesizer class. The SynthesizeTextToStreamAsync method transforms text into an audio stream (like a *.wav file). You can optionally provide the voice to be used; by selecting from the list of voices (SpeechSynthesizer.AllVoices) that are installed on the device. Each of these voices has it own gender and language. The voices depend on the device and your cultural settings: my laptop only speaks US and UK English, but my phone seems to be fluent in French and German too. When the speech synthesizer's audio stream is complete, you can play it via a MediaElement on the page. Note that Silverlight 8.1 apps do not need a MediaElement, they can call the SpeakTextAsync method, which is not available for Universal apps. Here’s the full flow. I wrapped it in a Voice class in the shared project (in an MVVM app this would be a 'Service'):/// <summary> /// Creates a text stream from a string. /// </summary> public void Say(string text, int voice = 0) { var synthesizer = new SpeechSynthesizer(); var voices = SpeechSynthesizer.AllVoices; synthesizer.Voice = voices[voice]; var spokenStream = synthesizer.SynthesizeTextToStreamAsync(text); spokenStream.Completed += this.SpokenStreamCompleted; } /// <summary> /// The spoken stream is ready. /// </summary> private async void SpokenStreamCompleted(IAsyncOperation<SpeechSynthesisStream> asyncInfo, AsyncStatus asyncStatus) { var mediaElement = this.MediaElement; // Make sure to be on the UI Thread. var results = asyncInfo.GetResults(); await mediaElement.Dispatcher.RunAsync(Windows.UI.Core.CoreDispatcherPriority.Normal, new DispatchedHandler( () => { mediaElement.SetSource(results, results.ContentType); })); } .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; } In a multi-page app, you must make sure that there is a MediaElement on each page. In the sample app, I reused a technique from a previous blog post. I created a custom template for the root Frame:<Application.Resources> <!-- Injecting a media player on each page --> <Style x:Key="RootFrameStyle" TargetType="Frame"> <Setter Property="Template"> <Setter.Value> <ControlTemplate TargetType="Frame"> <Grid> <!-- Voice --> <MediaElement IsLooping="False" /> <ContentPresenter /> </Grid> </ControlTemplate> </Setter.Value> </Setter> </Style> </Application.Resources> .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; } And applied it in app.xaml.cs:// Create a Frame to act as the navigation context and navigate to the first page rootFrame = new Frame(); // Injecting media player on each page. rootFrame.Style = this.Resources["RootFrameStyle"] as Style; // Place the frame in the current Window Window.Current.Content = rootFrame; .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; } The voice class can now easily access the MediaElement from the current page:public Voice() { DependencyObject rootGrid = VisualTreeHelper.GetChild(Window.Current.Content, 0); this.mediaElement = (MediaElement)VisualTreeHelper.GetChild(rootGrid, 0) as MediaElement; } /// <summary> /// Gets the MediaElement that was injected into the page. /// </summary> private MediaElement MediaElement { get { return this.mediaElement; } } .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; } The speech synthesizer can also generate an audio stream from Speech Synthesis Markup Language (SSML). That’s an XML format in which you can not only write the text to be spoken, but also the pauses, the changes in pitch, language, or gender, and how to deal with dates, times, and numbers, and even detailed pronunciation through phonemes. Here’s the document from the sample app:<?xml version="1.0" encoding="utf-8" ?> <speak version='1.0' xmlns='http://www.w3.org/2001/10/synthesis' xml:lang='en-US'> Your reservation for <say-as interpret-as="cardinal"> 2 </say-as> rooms on the <say-as interpret-as="ordinal"> 4th </say-as> floor of the hotel on <say-as interpret-as="date" format="mdy"> 3/21/2012 </say-as>, with early arrival at <say-as interpret-as="time" format="hms12"> 12:35pm </say-as> has been confirmed. Please call <say-as interpret-as="telephone" format="1"> (888) 555-1212 </say-as> with any questions. <voice gender='male'> <prosody pitch='x-high'> This is extra high pitch. </prosody > <prosody rate='slow'> This is the slow speaking rate. </prosody> </voice> <voice gender='female'> <s>Today we preview the latest romantic music from Blablabla.</s> </voice> This is an example of how to speak the word <phoneme alphabet='x-microsoft-ups' ph='S1 W AA T . CH AX . M AX . S2 K AA L . IH T'> whatchamacallit </phoneme>. <voice gender='male'> For English, press 1. </voice> <!-- Language switch: does not work if you do not have a french voice. --> <voice gender='female' xml:lang='fr-FR'> Pour le français, appuyez sur 2 </voice> </speak> .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; } Here’s how to generate the audio stream from it:/// <summary> /// Creates a text stream from an SSML string. /// </summary> public void SaySSML(string text, int voice = 0) { var synthesizer = new SpeechSynthesizer(); var voices = SpeechSynthesizer.AllVoices; synthesizer.Voice = voices[voice]; var spokenStream = synthesizer.SynthesizeSsmlToStreamAsync(text); spokenStream.Completed += this.SpokenStreamCompleted; } .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; } Speech Recognition User Control The Windows Phone project in the sample app contains a SpeechInputBox. It’s a user control for speech input through voice or keyboard that is inspired by the Cortana look-and-feel. It comes with dependency properties for the text and a highlight color, and it raises an event when the input is processed: The control behaves like the Cortana one: it starts listening when you tap on the microphone icon, it opens the onscreen keyboard if you tap on the text box, it notifies you when it listens to voice input, the arrow button puts the control in thinking mode, and the control says the result out loud when the input came from voice (not from typed input). The implementation is very straightforward: depending on the state the control shows some UI elements, and calls the API’s that were already discussed in this article. There’s definitely room for improvement: you could add styling, animations, sound effects, and localization. Here’s how to use the control in a XAML page. I did not use data binding in the sample, the main page hooked an event handler to TextChanged:<Controls:SpeechInputBox x:Name="SpeechInputBox" Highlight="Cyan" /> .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; } Here are some screenshot of the SpeechInputBox in action, next to its Cortana inspiration: If you intend to build a Silverlight 8.1 version of this control, you might want to start from the code in the MSDN Voice Search for Windows Phone 8.1 sample (there are quite some differences with Universal apps: in the XAML as well as in the API calls). If you want to roll your own UI, make sure you follow the Speech Design Guidelines. Code Here’s the code, it was written in Visual Studio 2013 Update 3. Remember that you need to register an app to get the necessary credentials to run the Bing Speech Recognition control: U2UC.WinUni.SpeechSample.zip (661.3KB). Enjoy! XAML Brewer

Universal Windows Apps: a Tale of Two Calendars.

31 July 2014 Diederik-Krols Universal Windows Apps
It was the best of times, it was the worst of times, it was the age of wisdom, it was the age of foolishness, it was the server side calendar on the tablet, it was the client side calendar on the smart phone. This article explains two ways of managing appointments in your calendar through a Universal Windows app. I created a small sample MVVM app that hosts a Calendar service with the following functionality against the user’s default calendar: Open the calendar at a specific date and time, create a new appointment, open the calendar at the newly created appointment, delete the newly created appointment, display the number of appointments that were created by the sample app, display the date and time of the appointments that were created by the sample app, and delete all appointments that were created by the sample app. Here’s a screenshot of it – the orange icons are the buttons that fire the commands: The Windows Phone part was built upon the Windows 8.1 Appointments API that is elaborated in this article on the Windows App Builder blog: Build apps that connect with People and Calendar, Part 2: Appointments. That API hosts the static AppointmentManager class, which interacts with the user’s Appointments provider app -by default the Calendar app- at the UI level. Unfortunately, most of the AppointmentManager’s methods are only applicable to the phone and cannot be called from a Store app on a tablet or PC. Here are the methods that are called in the phone project of the sample app. The one to show the details of an appointment cannot be used in a Store app, the others are really universal: ShowTimeFrameAsync: displays a time frame from an appointments calendar through the Appointments provider app's primary UI. ShowAddAppointmentAsync: opens the Appointments provider Add Appointment UI, to enable the user to add an appointment. ShowAppointmentDetailsAsync: opens the Appointments provider Appointment Details UI on the specified appointment. ShowRemoveAppointmentAsync: opens the Appointments provider Remove Appointment UI, to enable the user to remove an appointment. Don’t forget to declare the Appointments capability in the Phone app, or it won’t work. Strangely enough that capability does not exist for Store apps, there you can always use [the supported parts of] this API. The appointments are not created or removed by the calling app itself, that task is delegated to the Appointments provider. The Appointments provider acts on the local calendar on the device. The local appointment identifiers are passed back to the app, and the appointments themselves are later synced up to the user’s default calendar –most probably Outlook- where they get their ‘real’ identifier (and yes, that’s a different ID). The API is nice and easy. Here’s the call to open the user’s calendar app at a specific date and time. I embedded it in the Calendar service in the sample app: public async static Task Open(DateTimeOffset dto, TimeSpan ts) { await AppointmentManager.ShowTimeFrameAsync(dto, ts); } .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; } Here’s how the ViewModel opens the calendar:private async void Open_Executed() { await Calendar.Open(DateTimeOffset.Now.AddDays(-7), TimeSpan.FromHours(1)); } .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; } And this is how it looks like on the lightest phone emulator: To add or replace an appointment using the Windows 8.1 Appointments API, you need to provide an instance of the Appointment class. Here’s how the ViewModel of the sample app creates one of these:var appt = new Appointment(); appt.Subject = "Exterminate Enemy"; appt.Details = "Destroy an extraterrestrial race of human-sized pepper shakers, each equipped with a single mechanical eyestalk mounted on a rotating dome, a gun mount containing an energy weapon and a telescopic manipulator arm which is usually tipped by an appendage resembling a sink plunger."; appt.Location = "That planet on which silence will fall when the oldest question in the universe is asked."; appt.Invitees.Add(new AppointmentInvitee() { DisplayName = "That impossible girl.", Address = "All around." }); appt.Duration = TimeSpan.FromMinutes(50); appt.StartTime = DateTimeOffset.Now.AddDays(7); .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; } These are the methods of the sample Calendar service that allow you to add a new appointment. Depending on the device, the Appointments provider will take the whole screen, or appear in a Popup – that’s why you need to provide the Rectangle parameter:public async static Task<string> Add(Appointment appt) { var selection = new Rect(new Point(Window.Current.Bounds.Width / 2, Window.Current.Bounds.Height / 2), new Size()); return await Add(appt, selection); } public async static Task<string> Add(Appointment appt, Rect selection) { var id = await AppointmentManager.ShowAddAppointmentAsync(appt, selection, Placement.Default); AddAppointmentId(id); if (String.IsNullOrEmpty(id)) { Toast.ShowInfo("The appointment was not added."); } return id; } These calls return the local identifier of the appointment, which is only valid on the current device. This might be the first show stopper against using this API in your app. Furthermore, the API has no means for querying the calendar – like ‘give me all appointment id’s for appointments that have Time Travel as subject’. So if you require that in your app, please look for another API – there are plenty of them. I decided to keep the list of identifiers in a semicolon-separated string in the app’s Roaming Settings:public static void AddAppointmentId(string appointmentId) { if (String.IsNullOrEmpty(appointmentId)) { return; } string ids = ApplicationData.Current.RoamingSettings.Values["AppointmentIds"] as string; if (String.IsNullOrEmpty(ids)) { ids = appointmentId; } else { ids += ";" + appointmentId; } ApplicationData.Current.RoamingSettings.Values["AppointmentIds"] = ids; }public static List<string> AppointmentIds { get { string ids = ApplicationData.Current.RoamingSettings.Values["AppointmentIds"] as string; if (String.IsNullOrEmpty(ids)) { return new List<string>(); } return new List<string>(ids.Split(';')); } } .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; } To view or delete an appointment, you need to provide the local identifier. Here’s the code from the Calendar service that opens the Appointment provider on a to-be-deleted appointment. The second parameter enables an On Error Resume Next scenario – useful when you want to delete all appointments that were created:public async static Task Delete(string appointmentId, bool ignoreExceptions = false) { var selection = new Rect(new Point(Window.Current.Bounds.Width / 2, Window.Current.Bounds.Height / 2), new Size()); try { var success = await AppointmentManager.ShowRemoveAppointmentAsync(appointmentId, selection); } catch (Exception) { if (!ignoreExceptions) { throw; } } } .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; } Here are some screen shots of AppointmentManager calls on the phone emulator – respectively to add, show the details of, and remove an appointment: Unfortunately the ShowAppointmentDetailsAsync and the ShowEditNewAppointmentAsync calls apply only the phone. You can’t use them in Windows Store apps – although they’re in a Universal API. In some use cases, that’s good enough. If you built a Universal Store app that comes with an accompanying phone app that adds appointment management, then the Windows 8.1 Appointments API is what you need. If you want the exact same functionality on all device types, then you have to look for another API. The only AppointmentManager call that is really useful in a Windows Store app, is ShowTimeFrameAsync - the call to open the calendar to a specific date and time. This is how the result looks like in the sample app: This is how a call to ShowAddAppointmentAsync looks like. Some information of the to-be-inserted appointment appears in a Popup that carries the color scheme of the Calendar app. The only thing you can control is the position of the Popup: So some parts of the Windows 8.1 Appointments API don’t work on a tablet or PC, while other parts simply look ugly on these platforms. Now, there are a lot of alternative appointment API’s available: you can use the Universal AppointmentStore and AppointmentCalendar classes, or the non-universal citizens of the Microsoft.Phone.UserData namespace – including a very promising Appointments class that comes with query capabilities. Unfortunately these API’s have one thing in common: they only apply to the phone. For the Windows Store app part of the sample app, I decided to directly appeal to the root of the user’s calendar –Outlook.com- and use the Live Connect API for this. This starts with installing the Live SDK, and associating the app with the Store. Just read the first paragraphs of my article on OneDrive integration for more details on this procedure. Since you’re directly talking to the calendar(s) on the server(s), you need to be logged in:private static async Task LogIn() { if (NetworkInformation.GetInternetConnectionProfile().GetNetworkConnectivityLevel() >= NetworkConnectivityLevel.InternetAccess) { LiveAuthClient auth = new LiveAuthClient(); var loginResult = await auth.LoginAsync(new string[] { "wl.calendars", "wl.calendars_update", "wl.events_create" }); if (loginResult.Session != null) { client = new LiveConnectClient(loginResult.Session); } isLoggedIn = (loginResult.Status == LiveConnectSessionStatus.Connected); } else { isLoggedIn = false; } } .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; } You also have to know the identifier of the user’s default calendar. Here’s how to find it:private async static Task<string> FetchCalendarId() { if (!isLoggedIn) { await LogIn(); } if (client == null || !isLoggedIn) { return string.Empty; } try { LiveOperationResult lor = await client.GetAsync("me/calendars"); dynamic result = lor.Result; foreach (dynamic calendar in result.data) { // We assume that the first one is the default. string id = calendar.id; return id; } } catch (Exception ex) { Debugger.Break(); return string.Empty; } return string.Empty; } .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; } The HTTP POST call to add an appointment takes a dictionary as parameter. So here’s an extension method to format an Appointment instance:public static Dictionary<string, object> AsDictionary(this Appointment appt) { var calendarEvent = new Dictionary<string, object>(); calendarEvent.Add("name", appt.Subject); calendarEvent.Add("description", appt.Details); calendarEvent.Add("start_time", appt.StartTime.ToString("u")); calendarEvent.Add("end_time", appt.StartTime.Add(appt.Duration).ToString("u")); calendarEvent.Add("location", appt.Location); calendarEvent.Add("is_all_day_event", appt.AllDay); return calendarEvent; } .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; } Here’s the full call to add an appointment to the user’s default calendar. It’s not exactly a one-liner: we fetch the calendar id, ensure we’re logged on, add the appointment, store the appointment id in the Roaming Settings, open the local calendar app at the appropriate day, and finally remind the user that he might need to sync it - since we added the appointment on the server-side:public async static Task<string> Add(Appointment appt) { var calendarId = await FetchCalendarId(); if (!isLoggedIn) { await LogIn(); } if (client == null || !isLoggedIn) { Toast.ShowInfo("Sorry, I could not open your calendar."); return string.Empty; } try { string parm = string.Format("{0}/events", calendarId); LiveOperationResult lor = await client.PostAsync(parm, appt.AsDictionary()); dynamic result = lor.Result; string id = result.id; AddAppointmentId(id); await Open(appt.StartTime.Date, TimeSpan.FromDays(1)); Toast.ShowInfo("You may need to sync the calendar."); return id; } catch (Exception ex) { Debugger.Break(); return string.Empty; } } .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; } This is how it looks like in the sample app: When the user taps on the appointment, its details are displayed, with the possibility to edit and delete: The Live Connect API comes with an HTTP GET to fetch the appointment details if you know its identifier. Here’s how the sample app fetches the date and time of an appointment:public async static Task<DateTime> FetchAppointmentDate(string eventId) { if (!isLoggedIn) { await LogIn(); } if (client == null || !isLoggedIn) { return DateTime.MinValue; } try { LiveOperationResult lor = await client.GetAsync(eventId); dynamic result = lor.Result; string dt = result.start_time; return DateTime.Parse(dt); } catch (Exception ex) { // Debugger.Break(); return DateTime.MinValue; } } .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; } For showing the details of an appointment, I’m only interested in getting its date and time. I use this to open the calendar at the appropriate date:public async static Task Display(string appointmentId) { var date = await FetchAppointmentDate(appointmentId); if (date != DateTime.MinValue) { await Open(date.Date, TimeSpan.FromDays(1)); } else { // Not Found. Toast.ShowError("Sorry, I could not find the appointment."); } } Here’s the result in the sample app, this UI should not come as a surprise: For deleting an appointment, you have two options: open the calendar to the appointment’s date and let the user delete it, or delete it directly with a HTTP DELETE call. Here’s the code for the latter:public async static Task Delete(string appointmentId, bool ignoreFailure = false) { try { LiveOperationResult lor = await client.DeleteAsync(appointmentId); RemoveAppointmentId(appointmentId); } catch (Exception ex) { if (ignoreFailure) { // On Error Resume Next ... RemoveAppointmentId(appointmentId); } else { Toast.ShowError("Sorry, something went wrong."); Debugger.Break(); } } } .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; } For the sake of completeness, here’s the shared code that generates the welcome message for Store and Phone apps in the ViewModel. We cannot query for the list of appointments with the used API’s, but we can count the number of identifiers in the Roaming Settings: public async Task UpdateMessage() { message = string.Format("You have {0} appointment(s).", Calendar.AppointmentIds.Count); #if WINDOWS_PHONE_APP this.OnPropertyChanged("Message"); return; #else foreach (var apptId in Calendar.AppointmentIds) { var dt = await Calendar.FetchAppointmentDate(apptId); if (dt == DateTime.MinValue) { message += "\n - (not found)"; } else { message += "\n -" + dt.ToString(); } } this.OnPropertyChanged("Message"); #endif } .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; } Here’s the solution’s structure. The Calendar service is spread over all projects as a partial class (with a large portion in the Shared part), while the XAML of the phone box UI is shared as a user control: Depending on your use case, you may choose one or the other API for managing appointments in your universal app. But don’t mix them as I did in the sample app. The Windows 8.1 Appointments API works with device-specific identifiers against a local calendar, while the Live Connect API works with global identifiers. The appointments end up in the same place, but there seems to be no way to match the identities. Here’s the full code of the sample app, it was written with Visual Studio 2013 Update 2. Remember to associate the Windows app with the Store to test the Live Connect API: U2UC.WinUni.Appointments.zip (139.4KB) Enjoy! XAML Brewer

Playing sounds in a Universal Windows MVVM app

10 July 2014 Diederik-Krols Universal Windows Apps
This article describes how to play foreground and background sounds in a XAML-based Universal Windows app. I know that this sounds easy (pun intended). But I will do this while sticking to the MVVM pattern, which makes it a bit more challenging. I’m not targeting a specific framework here. The described architecture can be applied in MVVM Light, Universal Prism, Caliburn.Micro as well as in your home brewed MVVM library. Here’s how the attached sample app looks like. It comes with two buttons that trigger a different sound effect, a switch to mute the background sound, and a button to navigate to a new page (very uncommon in a single-page app ): What's the challenge for an MVVM solution to play sounds? Well, a universal XAML app can only make noise through a MediaElement, and that MediaElement should be part of the runtime structure of XAML objects known as the visual tree. As a result of that, only a small number of app components -the currently active Views- have access to it. In most cases however it’s the business logic in the Model or the ViewModel that knows which specific sound effect should be played at which particular moment. So the ‘Sound’ functionality should not be restricted to a some Views, but it should be generally available to all application components and component types. In most MVVM ecosystems, this type of global functionality ends up in a so-called Service (other examples include logging, authorization, and toast notification). And so does the SoundPlayer: it’s a global service that comes with a Play method. That method has two parameters: a reference to the sound effect (from a developer-friendly enumeration), and a boolean indicating whether the sound should be played in the foreground (once) or in the background (in a loop). Since a MediaElement can only play one sound at a time, the SoundPlayer is connected to two of them – one for the foreground and one for background. The background player can be disabled (muted) by the app. Here’s an class diagram of the SoundPlayer, together with the Visual Studio solution structure. The platform specific projects for Windows 8.1 and Windows Phone 8.1 are collapsed, since they’re empty – except for the tile images. All the code is in the Shared project: This is how everything was set up. The first challenge is to make sure that every Page of the app is decorated with two MediaElement instances. An easy way to do this, is to put these elements in the app’s root frame. This can be done through a custom style, e.g. in the App.xaml file:<Application.Resources> <!-- Injecting media players on each page --> <Style x:Key="RootFrameStyle" TargetType="Frame"> <Setter Property="Template"> <Setter.Value> <ControlTemplate TargetType="Frame"> <Grid> <!-- Foreground Player --> <MediaElement IsLooping="False" /> <!-- Background Player --> <MediaElement IsLooping="True" /> <ContentPresenter /> </Grid> </ControlTemplate> </Setter.Value> </Setter> </Style> </Application.Resources> Make sure that the style is applied when the app is launched, by adding an extra line of code in the standard OnLaunched method in App.xaml.cs:.protected override void OnLaunched(LaunchActivatedEventArgs e) { Frame rootFrame = Window.Current.Content as Frame; if (rootFrame == null) { rootFrame = new Frame(); // Injecting media players on each page. rootFrame.Style = this.Resources["RootFrameStyle"] as Style; // ... } // ... } As long as you navigate within the frame, the media elements remain available. But when you programmatically switch to another root Frame, you have to make sure to apply this style to it. That’s what the navigate button in the sample app does:var rootFrame = new Frame(); rootFrame.Style = App.Current.Resources["RootFrameStyle"] as Style; Window.Current.Content = rootFrame ; rootFrame.Navigate(typeof(MainPage)); .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; } Achievement unlocked: every page will always two MediaElement instances on it. We just have to make them available to the SoundPlayer service. This assignment is done in the Initialize call in the SoundPlayer class itself. Make sure to preserve the state since we might be hosted in a new frame with brand new UI elements at their default -unmuted- state:public void Initialize() { // Register media elements to the Sound Service. try { DependencyObject rootGrid = VisualTreeHelper.GetChild(Window.Current.Content, 0); var foregroundPlayer = (MediaElement)VisualTreeHelper.GetChild(rootGrid, 0) as MediaElement; var backgroundPlayer = (MediaElement)VisualTreeHelper.GetChild(rootGrid, 1) as MediaElement; SoundPlayer.ForegroundPlayer = foregroundPlayer; // Keep the state. var isMuted = this.IsBackgroundMuted; SoundPlayer.BackgroundPlayer = backgroundPlayer; this.IsBackgroundMuted = isMuted; } catch (Exception) { // Most probably you forgot to apply the custom root frame style. SoundPlayer.ForegroundPlayer = null; SoundPlayer.BackgroundPlayer = null; } } .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; } .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; } Every Page should make a call to this Initialize after loading, so I factored out the call into a common base class for all of the app’s Page-type views. public class ViewBase : Page { public ViewBase() { this.Loaded += this.OnLoaded; } protected virtual void OnLoaded(object sender, RoutedEventArgs e) { SoundPlayer.Instance.Initialize(); } } .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; } .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; } public sealed partial class MainPage : ViewBase { // ... } .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; } To make the SoundPlayer service globally accessible, it was implemented as a Singleton. A static class would not work, since property-changed notification requires an instance. In most MVVM frameworks this instance would be served to you by Dependency Injection, or it would be listening to a Messenger of some sort. Here’s the core class definition:internal class SoundPlayer : BindableBase { private static SoundPlayer instance = new SoundPlayer(); private static MediaElement ForegroundPlayer { get; set; } private static MediaElement BackgroundPlayer { get; set; } public static SoundPlayer Instance { get { return instance; } } public bool IsBackgroundMuted { get { if (BackgroundPlayer == null) { return false; } return BackgroundPlayer.IsMuted; } set { if (BackgroundPlayer != null) { BackgroundPlayer.IsMuted = value; this.OnPropertyChanged("IsBackgroundMuted"); } } } public async Task Play(Sounds sound, bool inBackground = false) { var mediaElement = inBackground ? BackgroundPlayer : ForegroundPlayer; if (mediaElement == null) { return; } string source = string.Format("ms-appx:///Assets/{0}.mp3", sound.ToString()); mediaElement.Source = new Uri(source); await mediaElement.Dispatcher.RunAsync(CoreDispatcherPriority.Normal, () => { mediaElement.Stop(); mediaElement.Play(); }); } } .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; } The Play method was made asynchronous to keep the UI responsive. The list of available sound effects is contained in an enumeration that maps the physical mp3 assets that you can see in the previous screen shot. That makes it easy for all components to select the sound effect they want to play:public enum Sounds { Nature, Sweep, Bell } By the way, if you’re looking for royalty-free sound effects, you may want to check SoundGator or SoundBible. The SoundPlayer is now accessible from the View components. So you could decide to start some background music after a page is loaded: protected async override void OnLoaded(object sender, RoutedEventArgs e) { base.OnLoaded(sender, e); await SoundPlayer.Instance.Play(Sounds.Nature, true); } The SoundPlayer is also accessible from the (View)Model components. Here's the code for the Bell command in the MainViewModel:private async void Bell_Executed() { await SoundPlayer.Instance.Play(Sounds.Bell); } .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; } None of these app components are actually aware of the MediaElement instances that are doing the work. That knowledge is encapsulated in the custom Frame style and the SoundPlayer itself. Here’s the full source code, it was written with Visual Studio 2013 Update 2. Feel free to adapt it to your favorite MVVM framework: U2UC.WinUni.Sound.zip (1.3MB) Enjoy! XAML Brewer