Configure Xamarin.Forms iOS to use Caliburn.Micro

Update 2017/10/15: This walk through is now available as a Visual Studio Template! More details here: https://marketplace.visualstudio.com/items?itemName=BryanBCook.XamarinFormsCaliburnMicroStarterKit

We’re continuing with our series of getting started with Xamarin.Forms and Caliburn.Micro, check out these previous posts if you’re interested:

• Getting Started with Xamarin.Forms and Caliburn.Micro
• Configure Xamarin.Forms Droid to use Caliburn.Micro

Today, we’ll set up our Xamarin.Forms iOS project. Unlike the Droid project, we can get our project up and running with very few changes to the default template. So we'll start this post with configuring our Visual Studio instance to talk to our Mac.

And before you ask, unfortunately, although it's C# you absolutely need a Mac to compile and run the solution. On the plus side, you can do most of your development on your PC.

Verify Prerequisite Software on your Mac

You will need the following installed on your Mac:

• XCode 7+
• Xamarin Studio
• Xamarin.iOS SDK

Setup your Mac for Remote Access

Before you can configure Visual Studio to use your Mac as a Build Host, you'll need to configure your Mac to allow users to remotely login to your machine. You can get the full details following this post, but in short we can simply set up our mac using the following:

• Press Command ⌘ + Space to bring up Spotlight Search
• Type Remote Login to open Sharing
• Enable Remote Login and then specify Allow access for: All Users 

Configure your Build Host

Now that your Mac will accept remote logins, we can now configure Visual Studio on the PC to use your Mac as a build host.

• From your PC, open our XF.CaliburnMicro project.
• In the toolbar, click on the Xamarin Mac Agent icon (also Tools –> iOS –> Xamarin Mac Agent):
  
  
  
  
  
• If your mac is running, it should appear in the dialog:
  
  
  
  
  
• Select your Mac, click Connect…
• You’ll be prompted to login, so provide the credentials you use when logging into the Mac
• Once connected, the toolbar changes colour and provides the option to launch the simulator:
  
  
  

Note this assumes that both the Mac and the PC are on the same network and that all required software is installed and configured for use.

Create and configure IoC Container

Now that we have our environment dependencies out of the way, we can turn our attention to modifying the template code to leverage Caliburn.Micro.

Similar to our Droid project, we need to set up our IoC container. For iOS, we do this by using a CaliburnAppDelegate which looks almost identical to the Application class we created for Android.

namespace XF.CaliburnMicro1.iOS
{
    using System;
    using System.Collections.Generic;
    using System.Reflection;
    
    using Caliburn.Micro;
    
    using XF.CaliburnMicro1.ViewModels;

    class CaliburnAppDelegate : CaliburnApplicationDelegate
    {
        private SimpleContainer _container;

        public CaliburnAppDelegate()
        {
            Initialize();
        }

        protected override void Configure()
        {
            _container = new SimpleContainer();
            _container.Instance(_container);

            _container.Singleton<App>();

            // TODO: Register all platform services here
        }

        protected override void BuildUp(object instance)
        {
            _container.BuildUp(instance);
        }

        protected override IEnumerable<object> GetAllInstances(Type service)
        {
            return _container.GetAllInstances(service);
        }

        protected override object GetInstance(Type service, string key)
        {
            return _container.GetInstance(service, key);
        }

        protected override IEnumerable<Assembly> SelectAssemblies()
        {
            return new[]
            {
                GetType().Assembly,
                typeof(MainViewModel).Assembly
            };
        }
    }
}

We then modify the AppDelegate to leverage our CaliburnAppDelegate, and change the start-up routine to use our App singleton.

namespace XF.CaliburnMicro1.iOS
{
    using Foundation;
    using UIKit;
    
    using Caliburn.Micro;
    using Xamarin.Forms.Platform.iOS;

    [Register("AppDelegate")]
    public partial class AppDelegate : FormsApplicationDelegate
    {
        private readonly CaliburnAppDelegate appDelegate = new CaliburnAppDelegate();

        public override bool FinishedLaunching(UIApplication app, NSDictionary options)
        {
            global::Xamarin.Forms.Forms.Init();

            LoadApplication(IoC.Get<App>());

            return base.FinishedLaunching(app, options);
        }
    }
}

Build and Run

To run your application, you’ll need to change the start project, select a target and the Device. Here I’ve selected XF.CaliburnMicro.iOS project, specified the iPhoneSimulator target and the iPhone 6s iOS 10.x simulator.

If everything worked, you should now see our Xamarin.Forms app backed by our Caliburn.Micro ViewModel.

Next Steps

So there you go! We’ve got both Android and iOS projects running. Our next post we’ll continue down this path and our Xamarin.Forms project to leverage the Universal Windows Platform which will target tablets, phones and PCs.

• Configure Xamarin.Forms UWP to use Caliburn.Micro

Configure Xamarin.Forms Droid to use Caliburn.Micro

Update 2017/10/15: This walk through is now available as a Visual Studio Template! More details here: https://marketplace.visualstudio.com/items?itemName=BryanBCook.XamarinFormsCaliburnMicroStarterKit

In my last post, we created a Xamarin.Forms project and added necessary components for taking advantage of my favourite MVVM framework, Caliburn.Micro. However, the solution didn't compile because we needed to make some platform specific changes. Today we'll address this.

Add an Application

The main entry point into our Android application is the Application class as defined in the AndroidManifest.xml. By default, the Xamarin.Forms template does not include an Application class, but we’ll need one in order to hold some global state.

To register our custom application, we subclass Caliburn’s CaliburnApplication class and mark it with an [Application] attribute:

namespace XF.CaliburnMicro1.Droid
{
    using System;
    using Android.App;
    using Android.Runtime;
    using Caliburn.Micro;

    [Application]
    public class Application : CaliburnApplication
    {
        public Application(IntPtr javaReference, JniHandleOwnership transfer)
              : base(javaReference, transfer)
        {
        }

        public override void OnCreate()
        {
            base.OnCreate();

            Initialize();
        }
    }
}

Create and configure IoC

One of the main features of the CaliburnApplication is to create our platform-specific Inversion of Control (IoC) container and register any platform-specific abstractions.

There are four primary methods that we need to implement in our Application (SelectAssemblies, BuildUp, GetAllInstances, GetInstance). These methods are the backing implementation for Caliburn's IoC:

It's important to note that we register the Xamarin.Forms App as a singleton before we activate it. This is largely because we want the App to have the same lifetime as our Android Application, which avoids a few scenarios where the Android application lifecycle causes the App class to be reinitialized more than once.

Our Application class now looks like this:

namespace XF.CaliburnMicro1.Droid
{
    using System;
    using System.Collections.Generic;
    using System.Reflection;
    
    using Android.App;
    using Android.Runtime;
    
    using Caliburn.Micro;
    
    using XF.CaliburnMicro1.ViewModels;

    [Application]
    public class Application : CaliburnApplication
    {
        private SimpleContainer _container;

        public Application(IntPtr javaReference, JniHandleOwnership transfer)
              : base(javaReference, transfer)
        {
        }

        public override void OnCreate()
        {
            base.OnCreate();

            Initialize();
        }

        protected override void Configure()
        {
            _container = new SimpleContainer();

            // make the container available for resolution
            _container.Instance(_container);

            // CRITICAL! make sure our Xamarin.Forms App 
            // can only be initilized once!
            _container.Singleton<App>();

            // TODO: Register any Platform-Specific abstractions here
        }

        protected override IEnumerable<Assembly> SelectAssemblies()
        {
            // Get a list of all assemblies that will be used
            // by the IoC container
            return new[]
            {
                GetType().Assembly,
                typeof (MainViewModel).Assembly
            };
        }

        protected override void BuildUp(object instance)
        {
            _container.BuildUp(instance);
        }

        protected override IEnumerable<object> GetAllInstances(Type service)
        {
            return _container.GetAllInstances(service);
        }

        protected override object GetInstance(Type service, string key)
        {
            return _container.GetInstance(service, key);
        }
    }
}

Modify MainActivity

The last modification we need to make is a slight change to our existing MainActivity. Rather than instantiating our App class, we resolve it from the IoC container.

namespace XF.CaliburnMicro1.Droid
{
    using Android.App;
    using Android.Content.PM;
    using Android.OS;
    
    using Xamarin.Forms.Platform.Android;
    
    using Caliburn.Micro;

    [Activity(
        Label = "XF.CaliburnMicro1", 
        Icon = "@drawable/icon", 
        MainLauncher = true, 
        ConfigurationChanges = ConfigChanges.ScreenSize | ConfigChanges.Orientation)]
    public class MainActivity : FormsApplicationActivity
    {
        protected override void OnCreate(Bundle bundle)
        {
            base.OnCreate(bundle);

            global::Xamarin.Forms.Forms.Init(this, bundle);

            // Resolve the App class to use the singleton we created
            LoadApplication(IoC.Get<App>());
        }
    }
}

Compile and Run

At this point, the android project should be good to go. Set the project as the default target, and run (F5). The project should automatically be configured to deploy the app to the emulator.

Next Steps

So we have our Android application running our Caliburn.Micro flavoured Xamarin.Forms app. The next step is to configure our iOS project with similar customizations.

• Configure Xamarin.Forms iOS to use Caliburn.Micro
• Configure Xamarin.Forms UWP to use Caliburn.Micro 

Getting Started with Xamarin.Forms and Caliburn.Micro

Update 2017/10/15: This walk through is now available as a Visual Studio Template! More details here: https://marketplace.visualstudio.com/items?itemName=BryanBCook.XamarinFormsCaliburnMicroStarterKit

Let me say this: Xamarin.Forms is awesome.

Xamarin.Forms is a great way to build native iOS and Android applications using C# and XAML. But if you’re just starting out, you’ll find that the most of the documentation out there, including Charles Petzold’s fantastic book, are lacking great examples on how to do MVVM, which is XAMLs strongest feature.

Last year when Caliburn.Micro finally released version 3.0 with support with Xamarin.Forms I was disappointed that there wasn’t much documentation to support it as well.

So with that said, lets build a Xamarin.Forms app using Caliburn.Micro.

Create Project

Good news here is that Visual Studio will do most of the work for us. To create a blank Xamarin.Forms project, choose File –> New Project. Then select the template Visual C# –> Cross Platform –> Blank App (Xamarin.Forms Portable).

If you specify the project name XF.CaliburnMicro, Visual Studio will create several projects:

• XF.CaliburnMicro (Portable) – This is the PCL library that will hold our Views and ViewModels and core services and interfaces
• XF.CaliburnMicro.Droid – Our Android application
• XF.CaliburnMicro.iOS – Our iPhone / iPad application
• XF.CaliburnMicro.UWP (Universal Windows) – Our Windows 10 desktop/tablet/mobile application.
• XF.CaliburnMicro.Windows (Windows 8.1) – A Windows 8.1 app, for those of us that didn’t take the free upgrade.
• XF.CaliburnMicro.WinPhone (Windows Phone 8.1) – A Windows Phone 8.1 Silverlight/XAML application.

Note: You’ll likely be prompted to specify a Windows 10 version. For now, don’t worry about this, just click OK.

Also note that I’m only going to focus on Android, iOS and UWP, so feel free to delete XF.CaliburnMicro.Windows and XF.Caliburn.WinPhone.

Compile and Run

Now would be a good time to make sure everything compiles and your machine is good to go. Xamarin.Forms requires a few dependencies to get up and running, so before we start messing things up this would be a good time for a sanity test.

For simplicity sake, let’s focus only on getting our Android emulator running. If you’re able to see the default Forms page, we’re good.

Moving along…

Reference NuGet Packages

It’s time to add Caliburn.Micro to the project. There are three key NuGet packages we’re interested in:

• Caliburn.Micro
• Caliburn.Micro.Core
• Caliburn.Micro.XamarinForms

What seems a bit counter intuitive is that the Caliburn.Micro package is platform specific, so it cannot be referenced by the PCL library.

The best way to add these to your project is to right-click the solution and choose Manage NuGet Packages for Solution. Search for “Caliburn.Micro.XamarinForms” and add it to all projects. It’ll bring in the Caliburn.Micro.Core dependency.

Search for “Caliburn.Micro.Core” and add it to the PCL “XF.CaliburnMicro (Portable)” project.

Setup basic View and ViewModel

Caliburn uses conventions to automatically map Views to ViewModels and vice-versa, so you’ll need to create two folders Views and ViewModels in the XF.XamarinForms (Portable) project.

The ViewModels\MainViewModel is pretty straight forward:

using Caliburn.Micro;
using System.Runtime.CompilerServices;

namespace XF.CaliburnMicro1.ViewModels
{
    public class MainViewModel : Screen
    {
        private string _mainText;

        public MainViewModel()
        {
           MainText = "Hello World!";
        }         

        public string MainText
        {
            get { return _mainText; }
            set
            {
                SetField(ref _mainText, value);
            }
        }

        protected bool SetField<T>(ref T field, T value, [CallerMemberName] string propertyName = null)
        {
            if (!object.Equals(field, value))
            {
                field = value;
                NotifyOfPropertyChange(propertyName);
                return true;
            }
            return false;
        }
    }
}

And the Views\MainView.xaml is an empty screen created using Project –> Add New Item –> Forms Xaml Page:

<?xml version="1.0" encoding="utf-8" ?>
<ContentPage xmlns="http://xamarin.com/schemas/2014/forms"
             xmlns:x="http://schemas.microsoft.com/winfx/2009/xaml"
             x:Class="XF.CaliburnMicro1.Views.MainView">
  <Label Text="{Binding MainText}" VerticalOptions="Center" HorizontalOptions="Center" />
</ContentPage>

Introduce Caliburn.Micro

If you’ve worked with Caliburn.Micro in the past, you would normally configure a Bootstrapper as the primary entry point for the application. In Xamarin.Forms, the closest equivalent is the App class that is created in the PCL project. We’ll use it to setup our Inversion of Control (IoC) container and register ViewModels, services and abstractions.

Caliburn.Micro ships with a simple IoC container, aptly named SimpleContainer, which offers simple functionality to register and resolve dependencies. It is possible to replace this container with a different one if you prefer, but in my experience the SimpleContainer does exactly what we need.

namespace XF.CaliburnMicro1
{
    using Caliburn.Micro;
    using Caliburn.Micro.Xamarin.Forms;
    using Xamarin.Forms;
    using XF.CaliburnMicro1.ViewModels;

    public class App : FormsApplication
    {
        private readonly SimpleContainer container;

        public App(SimpleContainer container)
        {
            this.container = container;

            // TODO: Register additional viewmodels and services
            container
                .PerRequest<MainViewModel>();

            Initialize();

            DisplayRootViewFor<MainViewModel>();
        }

        protected override void PrepareViewFirst(NavigationPage navigationPage)
        {
            container.Instance<INavigationService>(new NavigationPageAdapter(navigationPage));
        }

        protected override void OnStart()
        {
            // Handle when your app starts
        }

        protected override void OnSleep()
        {
            // Handle when your app sleeps
        }

        protected override void OnResume()
        {
            // Handle when your app resumes
        }
    }
}

Next Steps

At this point, the solution doesn’t compile, and that’s because we’ve changed the signature of the App class and we’ll have to adjust our iOS, Android and UWP projects to suit. Admittedly, it’s an awkward place to stop a blog post, but there’s a lot more to cover. Over the next few blog posts, we’ll setup the platform specific projects with their Caliburn parts and we’ll update the links below.

• Configure Xamarin.Forms Droid to use Caliburn.Micro
• Configure Xamarin.Forms iOS to use Caliburn.Micro
• ConfiguConfigure Xamarin.Forms UWP to use Caliburn.Micro

This time on a mac

There’s been less activity than normal around here. Looking back, I missed last year completely. That’s more than a blip, that’s a complete year. Not sure what happened, I think it was a perfect storm...

Those that know me know that I started practicing Karate in 2015 and my facebook feed has been filled with belt exams and karate videos. I’m highly motivated to obtain my first degree in under three years, so three times a week in evening classes and weekends makes it difficult to find a few extra cycles to experiment and blog. On the plus side, I’m probably the most fit I’ve ever been, I've dropped nearly twenty pounds, toned up and dangerous.

The other major shift for me was professionally work wasn’t selling the projects that suited my skills. I kinda had a crisis of faith of sorts, where I felt I had to choose a new religion. I switched into managing more people, bought a mac and learned Swift, and explored leveraging my .net and WPF strengths in Xamarin. Work would eventually pick up later in the year, but in true agency fashion it involved dusting off my Unity3D skills, learning how to rock a HoloLens and learn Python and Ruby while I was at it. There’s loads to blog about it.

I want to touch on the macbook reference above. I feel like I’ve made a career out of bashing apple. (I ended many arguments in the 90s with apple fanboys with my favourite argument killer: “...but it’s a mac…”) Buying a mac for me was huge slice of humble pie. I still use a PC at work, but my mac is now my playground and go to device when I want to do something that isn’t “work”. I’m writing this on a mac, trying out blogo today.

One last thing, to accompany my macbook I switched to an iPhone and parted ways with my Windows Phone. I’m no stranger to iOS as I’ve had an iPad and iPhone in the past, but I was on Windows Phone for years with three different devices. I loved the platform, and still do, but I got tired of waiting for apps that the rest of the planet had. Apply Pay and wallet are awesome. I may still buy a lumina to play with UWP with Xamarin.

So what should you expect this year around here? Well, hopefully I can make good on promises to blog more, but expect to see a combination of development and agile delivery practices. You’ll likely see some Xamarin, some UI automation, maybe a HoloLens. From an agile delivery perspective, expect some tips and tricks to bring to your team as well as a focus on DevOps and Test Automation.

List of C# Compiler Warnings

Generally, you want to fix compiler warnings as they come up, but you may need to suppress some of them for legitimate reasons. A list of warnings and errors can be found here, but there’s no high-level list.
Here is a summarized list of warnings.

 

Warning	Description	More Info
Level 1 Warnings (Severe)
CS0420	‘identifier’: a reference to a volatile field will not be treated as volatile	https://msdn.microsoft.com/en-us/library/4bw5ewxy.aspx
CS0465	Introducing a ‘Finalize’ method can interfere with destructor invocation.	https://msdn.microsoft.com/en-us/library/02wtfwbt.aspx
CS1058	A previous catch clause already catches all exceptions	https://msdn.microsoft.com/en-us/library/ms228623.aspx
CS1060	Use of possibly unassigned field 'name'. Struct instance variables are initially unassigned if struct is unassigned.	https://msdn.microsoft.com/en-us/library/bb384216.aspx
CS1598	XML parser could not be loaded for the following reason: 'reason'. The XML documentation file 'file' will not be generated.	https://msdn.microsoft.com/en-us/library/szfbfb7e.aspx
CS1607	A warning was generated from the assembly-creation phase of the compilation.	https://msdn.microsoft.com/en-us/library/4a0640cd.aspx
CS1616	Option 'option' overrides attribute 'attribute' given in a source file or added module	https://msdn.microsoft.com/en-us/library/0b104xt8.aspx
CS1658	'warning text'. See also error 'error code' The compiler emits this warning when it overrides an error with a warning.	https://msdn.microsoft.com/en-US/library/ft0k10fs.aspx
CS1683	Reference to type 'Type Name' claims it is defined in this assembly, but it is not defined in source or any added modules	https://msdn.microsoft.com/en-US/library/6b351z64.aspx
CS1685	The predefined type 'System.type name' is defined in multiple assemblies in the global alias; using definition from 'File Name'	https://msdn.microsoft.com/en-us/library/8xys0hxk.aspx
CS1690	Accessing a member on 'member' may cause a runtime exception because it is a field of a marshal-by-reference class	https://msdn.microsoft.com/en-us/library/x524dkh4.aspx
CS1691	'number' is not a valid warning number	https://msdn.microsoft.com/en-us/library/7z6tx3wa.aspx
CS1699	Use command line option "compiler_option" or appropriate project settings instead of "attribute_name"	https://msdn.microsoft.com/en-us/library/xh3fc3x0.aspx
CS1762	A reference was created to embedded interop assembly '<assembly1>' because of an indirect reference to that assembly from assembly '<assembly2>'. Consider changing the 'Embed Interop Types' property on either assembly.	https://msdn.microsoft.com/en-us/library/ff183282.aspx
CS1956	Member 'name' implements interface member 'name' in type 'type'. There are multiple matches for the interface member at run-time. It is implementation dependent which method will be called.	https://msdn.microsoft.com/en-us/library/bb882526.aspx
CS3003	Type of 'variable' is not CLS-compliant	https://msdn.microsoft.com/en-us/library/ax5w23a6.aspx
CS3007	Overloaded method 'method' differing only by unnamed array types is not CLS-compliant	https://msdn.microsoft.com/en-us/library/3w1d36e0.aspx
CS3009	'type': base type 'type' is not CLS-compliant	https://msdn.microsoft.com/en-us/library/tdd79w48.aspx
CS4014	Because this call is not awaited, execution of the current method continues before the call is completed. Consider applying the 'await' operator to the result of the call.	https://msdn.microsoft.com/en-us/library/hh873131.aspx
Level 2 Warnings
CS0108	'member1' hides inherited member 'member2'. Use the new keyword if hiding was intended.	https://msdn.microsoft.com/en-US/library/3s8070fc.aspx
CS0467	Inherited members from different interfaces that have the same signature cause an ambiguity error.	https://msdn.microsoft.com/en-us/library/ms228509.aspx
CS0618	'member' is obsolete: 'text'	https://msdn.microsoft.com/en-us/library/x5ye6x1e.aspx
CS1701	Assuming assembly reference "Assembly Name #1" matches "Assembly Name #2", you may need to supply runtime policy	https://msdn.microsoft.com/en-us/library/2h4x8b08.aspx
Level 3 Warnings (Less Severe)
CS0675	Bitwise-or operator used on a sign-extended operand; consider casting to a smaller unsigned type first	https://msdn.microsoft.com/en-us/library/wdc6717a.aspx
CS1700	Assembly reference Assembly Name is invalid and cannot be resolved	https://msdn.microsoft.com/en-us/library/ac36c25f.aspx
Level 4 Warnings (Informational)
CS0429	Unreachable expression code detected	https://msdn.microsoft.com/en-us/library/2ch1a3w5.aspx
CS1591	Missing XML comment for publicly visible type or member 'Type_or_Member'	https://msdn.microsoft.com/en-us/library/zk18c1w9.aspx
CS1610	Unable to delete temporary file 'file' used for default Win32 resource -- resource	https://msdn.microsoft.com/en-US/library/z4aytccf.aspx

Configure SourceTree to Rebase by default

Here’s a quick tip for Git for Windows users that are using Atlassian’s SourceTree as their visual interface.

If you’re using git flow as a branching strategy, a good habit to form is to use git pull --rebase when you update your local branch. This technique updates your local branch and then “replays” your changes on top, effectively putting your changes at the head of the branch.

The SourceTree Pull dialog supports this feature with a checkbox, but it’s not enabled by default.

To make this checkbox selected by default:

1. Tools –> Options
2. Git
3. Check the “use rebase instead of merge by default for tracked branches” option

Init Methods are Crap

This title of this post pretty much sums up how I feel about Init methods. Let’s take a look at an example why.

I’ve been on a soap-box yelling at anyone who’ll listen that “work in the constructor” is perhaps the worst thing you can do as a developer. You’ve drank the cool-aid and you have some fairly important initialization logic that needs to be called, so you move the logic out of the constructor and into a separate method. Your code now looks like this:

public class MenuProvider : IMenuProvider
{
    private Array<MenuItem> _menuItems;
    
    public void Initialize()
    {
        // do work to populate menu items, serialize from file, etc...
        _menuItems = // some value;
    }

    public IEnumerable<MenuItem> MenuItems
    {
        get
        {
            return _menuItems;
        }
    }

    public IEnumerable<MenuItem> InActiveMenuItems
    {
        get
        {
	    return _menuItems.Where(i => i.Active == false);
        }
    }

}

It's a contrived example, but in the above code MenuItems will be null and InActiveMenuItems throws an error unless you call Initialize() first. Certainly, there must be a better design than this? (If you answered put the work in the constructor, please come see me after class, I.. uh,.. have something for you.)

Putting the onus on the developer to call methods in a certain order is a very backward approach. You, the class designer, should make it difficult for people to do things the wrong way. We can take on some of that onerous effort inside the class, something like this should work nicely:

public class MenuProvider : IMenuProvider
{
   private bool _isInitialized;
   private object _lock = new object();
   private Array<MenuItem> _menuItems;

   public IEnumerable<MenuItem> GetMenuItems()
   {
        EnsureInitialized();
    
        return _menuItems;
   }

   public IEnumerable<MenuItem> GetInactiveMenuItems()
   {
        EnsureInitialized();

        return _menuItems.Where(i => i.Active == false);
   }

   private void EnsureInitialized()
   {
        if (!_initialized)
        {
            Initialize();
        }
   }
   
   private void Initialize()
   {
        lock(_lock)
        {
            if (!_initialized)
            {
                // do initialization work
                _initialized = true;
            }
        }
   }
}

In this design, I've made a few minor adjustments. For stylistic purposes, I converted the properties into methods because properties look awkward on interfaces. Properties also aren't a good choice if there's work involved in calculating them, so methods feel better here.

Outside of the cosmetic changes, one of the most notable changes is that my Initialize method is now private. I've also added some plumbing logic to ensure that the Initialize method only does work the first time it's called, and since this is good defensive programming, this might have already existed and shouldn't be a surprise.

What should stand out to you, is that each method is responsible for ensuring that the initialization logic is called on the user's behalf using the EnsureInitialized method before doing any other work. Some developers may grumble that this seems like a simple thing to forget when adding new methods and thus has potential for silly bugs. That’s fair, but because the API for the class has been simplified the tests to catch this should be much easier to write and find.

Alternatively, we can take this approach further by splitting the concerns into smaller parts. One part could read the file contents and the other part could be responsible for serving the data to consumers. This is effectively just moving the Initialize method to another class. Something like this:

public class MenuProvider : IMenuProvider
{
    private MenuReader _reader;

    public MenuProvider(MenuReader reader)
    {
        _reader = reader;
    }

    public IEnumerable<MenuItem> GetMenuItems()
    {
        return _reader.GetAll();
    }

    public IEnumerable<MenuItem> GetInactiveMenuItems()
    {
        return _reader.GetAll().Where(i => i.Active == false);
    }
}

public abstract class MenuReader
{
    public abstract IEnumerable<MenuItem> GetAll();
}

public class StaticFileMenuReader : MenuReader
{
    private Array<MenuItem> _items;

    public IEnumerable<MenuItem> GetAll()
    {
        if (_items != null)
        {
            // do serialization work
        }
        return _items;
    }
}

This small change greatly simplifies the MenuProvider and the extra step to move the serialization logic into another class may seem familiar to others as a Repository pattern. The repository pattern in this example also affords us additional test flexibility as the file-system can be abstracted away with a test-double.

To sum up, it seems a bit odd to design a class that requires methods to be called in a specific order. Initialization in general is something that the class designer should be well equipped to handle, so don’t push this responsibility to others – a few small changes to your class makes your code more usable, responsive and easier to test.

MEF Hacks: Constructors with parameters

Often, there’s a need to register an object into your dependency container that takes constructor arguments. There are a few ways to accomplish this with MEF.

You’re mileage will vary, but my favorite approach is a bit of a hack that leverages a lesser-known feature of MEF: Property Exports.

public class ComplexObjectExportFactory
{
    [Export]
    public MyComplexObject ComplexObject
    {
        get
        {
            return new MyComplexObject("config.xml");
        }
    }
}

This "simply works" and doesn't require you to do anything intrusive like sub-classing a dependency or registering named parameters on application start-up. The technique is especially useful if you need to provide any kind of conditional logic based on settings and it can even build upon other MEF registered parts, for example:

internal class ComplexObjectExportFactory
{
    private readonly IApplicationSettingsProvider _settings;
    private readonly IEventAggregator _eventAggregator;

    [ImportingConstructor]
    public ComplexObjectExportFactory(
            IApplicationSettingsProvider settings, 
            IEventAggregator eventAggregator)
    {
        _settings = settings;
        _eventAggregatore = eventAggregator;
    }

    [Export]
    public MyComplexObject ComplexObject
    {
        get
        {
            string configFile = _settings.GetSetting("ComplexObjectConfigFile");
            int timeout = _settings.GetSetting<int>("ComplexObjectTimeoutSeconds");
            
            return new MyComplexObject(_eventAggregator, configFile, timeout);
        }
    }
}

The only real down-fall to this overall approach is that the ComplexObjectExportFactory will never have any usages, so tools like Resharper will think this is dead-code, but this can be solved with a few well placed #pragmas and some comments. The other reason this feels like a hack, and this is more of an esthetics or stylistic difference, is that this feels like it should be a method (eg, factory.Create()) but MEF treats exported methods very differently.

Despite the awkwardness of this class, I’d prefer this to sub-classing third-party dependencies or bloating the bootstrapper with additional initialization logic. Maybe you think so too?

Happy coding.

Application Services for WPF Applications

Suppose you’re building a WPF application and have a number of actions that you want to perform on application start-up. Here’s a pattern that I’ve used successfully for the last few years that works with Caliburn.Micro or other MVVM frameworks…

If you're using Caliburn.Micro, the framework provides a Bootstrapper as an entry point to your application and the overridable Configure method is a great place to initialize your application-specific services.

You might be tempted to do something like this:

protected override void Configure()
{
    Container = new CompositionContainer(
       new AggregateCatalog(AssemblySource.Instance.Select(x => new AssemblyCatalog(x)).OfType<ComposablePartCatalog>()));

    var batch = new CompositionBatch();

    // initialize Caliburn.Micro services and register with container
    EventAggregator = new EventAggregator();

    batch.AddExportedValue<IWindowManager>(new WindowManager());
    batch.AddExportedValue<IEventAggregator>(EventAggregator);
    batch.AddExportedValue(Container);
    Container.Compose(batch);

    // initialize and register all my custom components
    var myService = new MyService();
    myService.Configure("config.xml");
    // repeat for all other services...

}

While this strategy works for one or two services, you'll find that as you add additional services your Bootstrapper logic becomes quite bloated and simply owns too many responsibilities.

To simplify things, let's define a simple interface that describes all of our application services:

public interface IApplicationService : IDispose
{
    void Initialize();
}

And now we can generically initialize all services that implement this interface:

protected void Configure()
{
   // snip...
   Container.Compose(batch);

   InitializeApplicationServices();
}

protected void InitializeApplicationServices()
{
   var services = Container.GetExports<IApplicationService>();
   foreach (Lazy<IApplicationService> exportedService in services)
   {
       IApplicationService service = exportedService.Value;
       Log.InfoFormat("Initializing IApplicationService:{0}.", service.GetType().Name);
       service.Initialize();
   }
}

protected override OnExit(object sender, EventArgs e)
{
    Container.Dispose();

    base.OnExit(sender, e);
}

[Export(typeof(IApplicationService))]
public class MyApplicationService : IApplicationService
{
    private MyService _service;        

    public void Initialize()
    {
       _service = new MyService();
       _service.Configure("config.xml"); 
    }

    public void Dispose()
    {
       GC.SuppressFinialize(this);
       Dispose(true);
    }
    protected virtual void Dispose(bool disposing)
    {
       _service.Dispose();       
    }
}

There! That’s a lot cleaner. Now the Bootstrapper doesn’t know anything about our services and we can add new services without having to further extend this class. There are a few additional benefits:

1. If you’re using MEF, all classes that are export IApplicationService are singletons by default.
2. All services have a predefined start (Initialize) and end (Dispose), thereby avoiding work that is often put in the constructor.
3. Perhaps not obvious, but if you’re using MEF, calling Container.Dispose() when the application exits automatically calls Dispose on all our application services, too.

So there you have a basic initialization routine for all your background services. My next few posts will demonstrate how to really take advantage of this pattern.

P.S: Astute readers of my blog might recognize that I'm using an Initialize() method which I've called out as a bad habit on my On Notice Board. While “Init methods” is high on my list, the context here is different. In this context, the only method I know about is Initialize(), whereas the “Init method” smell is a required method that must be called before calling other methods on the same class. I’ll blog about how to remove Init methods later.

Debugging Google Authentication + Azure MobileServices

Let’s say you’re building a mobile application using Azure MobileServices with Google Authentication but you want to debug and run the solution on your local developer environment. This post will walk you through setting up your local environment and provide an example of how to test your local services.

Before we start, the getting started documentation on how to add Google Authentication to your mobile application is pretty good, and if you follow the samples it’s fairly easy to get up and running. For the purposes of this post, I’m not going to repeat those setup instructions so if you’re trying to get it running you should definitely read this article before reading any further. I should also point out that I’m using a .NET Backend for my MobileService, so there are a few minor variances if you’re using Node.

Step 1: add your Google Client ID and Client Secret to your web.config. By default, the configuration for your MobileService is defined through the Azure MobileServices dashboard so local values in your web.config are ignored in the Azure environment. In order to debug Google Authentication locally, simply provide values for MS_GoogleClientID, MS_GoogleClientSecret in your web.config:

Step 2: modify the settings of your Google Project to support your local environment. At this point, attempts to authenticate in your local environment will fail because the Redirect Url for your local environment doesn’t match the settings defined in the Google Developer Console. Fortunately, the good folks at Google had the smarts to allow multiple redirect URLs. Simply provide the URL of your local developer environment. In my case, I’m running this in IIS Express and I’m using the URL http://localhost:61915

1. Navigate to https://console.developers.google.com/project and select your Project
2. In the Credentials section, click the Edit Settings button and add your local URL to the Authorized Redirect URIs.

Testing it out

So we’ve made a few simple changes and now it’s time to test it out.

1. Navigate to http://localhost:61915/login/google in your browser.
2. If all things are properly configured you should be redirected to Google’s oAuth page to provide consent to your application. Provide consent, dummy.
3. Your browser will be redirected to a special URL (http://localhost:61915/done#token=<json-data>).
4. Copy the URL and decode it. I like to use this online tool http://meyerweb.com/eric/tools/dencoder/
5. Once decoded, make note of the authentication-token.

1. Now using your favourite REST client (mine's the Chrome Advanced REST Client), supply the header X-ZUMO-AUTH: + your auth token and call any web-API method that would require authentication.

Cheers!

Mobile Services: Apply Database Migrations on Startup

As per my last post, if you’re using Entity Framework with Azure Mobile Services you’ll want to move away from Automatic Database upgrades and move to Code Based migrations. There are several strategies posted online, here’s the strategy that worked best for our project.

One of the challenges that my team identified early in the process was that our database content changed more frequently than our database schema. The default database initializer strategies only run the database migration and seeder scripts when the schema changes, so we found it more useful to manually apply the database migrations on start-up. This adds a bit of performance overhead to the start-up process, but since our database seeder scripts were relatively small, this seemed like an acceptable trade-off.

First, you’ll want to enable Code Based Migrations if you haven’t already. This involves opening the Package Manager Console window and typing the following:

Enable-Migrations

This power-shell command will add a few additional NuGet packages and make a few changes to your project. Most notably, the script will add a Migrations folder to your project and a Configuration class. We’ll make a few changes to the Configuration class by allowing automatic migrations and allowing data loss. The data loss feature will allow you to drop database columns even if they have data, so pay attention to your migration scripts.

internal sealed class Configuration : DbMigrationsConfiguration<MyServiceContext>
{
  public Configuration()
  {
    ContextKey = "MyProject.Models.MyServiceContext";
   
    AutomaticMigrationsEnabled = true;
    AutomaticMigrationDataLossAllowed = true;
  }

}

Next you’ll want to override the Seed method and populate your database with starter data. I highly recommend moving all your seeding logic to a static method so that you can reuse this logic in your test scripts, etc. Also note that the System.Data.Entity.Migrations namespace adds an AddOrUpdate extension method for DbSet<T> which greatly simplifies your database seed. Following this approach will allow you to run the seed as many times as you want without introducing duplicates.

protected override void Seed(MyServiceContext context)
{
    //  This method will always be called.

    //  You can use the DbSet<T>.AddOrUpdate() helper extension method 
    //  to avoid creating duplicate seed data. E.g.
    //
    //    context.People.AddOrUpdate(
    //      p => p.FullName,
    //      new Person { FullName = "Nigel Tufnel" },
    //      new Person { FullName = "David St. Hubbins" },
    //      new Person { FullName = "Derek Smalls" }
    //    );
    //
    MyDatabaseSeeder.Seed(context);
}

Finally, to apply the database migrations on start-up you’ll need to add the following to your WebApiConfig.Register() method:

public static void Register()
{
   
   Database.SetInitializer<MyServiceContext>(null);

   var migrator = new DbMigrator(new Configuration());
   migrator.Update();

}

With this in place, the DbMigrator will apply any database changes when there are available and reliably call the Seed method with every deployment.

Happy Coding.

Mobile Services: Sharing a database schema between environments

I recently completed a project where we used Azure Mobile Services as the backend system for an Android application. I was pleased at how easy it was to setup and deploy to an environment, but there are a few details for production deployments that make things a bit tricky.

As your development team sprints full steam towards the finish line one major hurdle you’ll have to cross is database versioning. The default entity framework strategy is to use a database initializer that drops your database on start-up or any time the model changes, which obviously is not good and you can’t deploy to production with this enabled. The solution for this is to disable automatic upgrades and use code-first database migrations, which are also a bit tricky (I might blog about later). Here’s an initial primer on database migrations.

Before you run off to enable database migrations and start scripting out your objects into code, there’s something you should know. If you’re like me and you have separate azure environments for testing and production, the scripted objects will contain the schema name of your development environment and you’ll most likely get an error about the “__MigrationHistory” table on start-up.

You can avoid this hassle by making a small change to your DatabaseContext to ensure both environments use the same schema. The default boilerplate code uses the name of the Azure Mobile Service in your application-settings:

// part of MyMobileServiceContext : DbContext
protected override void OnModelCreating(DbModelBuilder modelBuilder)
{
    string schema = ServiceSettingsDictionary.GetSchemaName();
    if (!string.IsNullOrEmpty(schema))
    {
        modelBuilder.HasDefaultSchema(schema);
    }

    modelBuilder.Conventions.Add(
        new AttributeToColumnAnnotationConvention<TableColumnAttribute, string>(
            "ServiceTableColumn", (property, attributes) => attributes.Single().ColumnType.ToString()));
}

This simple change to use the same schema name ensures that your scripted objects and runtime time stay in sync:

protected override void OnModelCreating(DbModelBuilder modelBuilder)
{
    //string schema = ServiceSettingsDictionary.GetSchemaName();
    string schema = "MySchemaName";
    if (!string.IsNullOrEmpty(schema))
    {
        modelBuilder.HasDefaultSchema(schema);
    }

    modelBuilder.Conventions.Add(
        new AttributeToColumnAnnotationConvention<TableColumnAttribute, string>(
            "ServiceTableColumn", (property, attributes) => attributes.Single().ColumnType.ToString()));
}

Happy coding.

Head Tracking with Kinect v2

This is yet another post in my series about the new Kinect using the November 2013 developer preview SDK. Today we’re going to have some fun by combining the color, depth and body data streams (mentioned in my last few posts, here, here and here) and some interesting math to create an image that magically tracks the user’s head.
This is an early preview of the new Kinect for Windows, so the device, software and documentation are all preliminary and subject to change.

If you recall from my last post, I used the CoordinateMapper to translate the coordinates of the user’s joint information on top of the HD color image. The magic ingredient converts the Joint’s Position to a ColorSpacePoint.

Joint headJoint = body.Joints[JointType.Head];

ColorSpacePoint colorSpacePoint = 
    _sensor.CoordinateMapper.MapCameraSpaceToColorPoint(headJoint.Position);

If we take the X & Y coordinates from this ColorSpacePoint and the wonderful extension methods of the WriteableBitmapEx project, we can quickly create a cropped image of that joint.

int x = (int)Math.Floor(colorSpacePoint.X + 0.5);
int y = (int)Math.Floor(colorSpacePoint.Y + 0.5);

int size = 200;

WriteableBitmap faceImage = _bmp.Crop(new Rect(x,y,size,size));

Wow, that was easy! Although this produces an image that accurately tracks my head, the approach is somewhat flawed as it doesn’t scale based on the user’s position from the camera: if you stand too close to the camera you’ll only see a portion of your face; Stand too far and you’ll see my face and torso. We can fix this by calculating the desired size of the image based on the depth of the joint. To do this, we’ll need to obtain a DepthSpacePoint for the Joint and a simple trigonometric formula…
The DepthSpacePoint by itself doesn’t contain the depth data. Instead, it contains the X & Y coordinates from the depth image which we can use to calculate the index in the array of depth data. I’ve outlined this in a previous post, but for convenience sake here’s that formula again:

// get the depth image coordinates for the head
DepthSpacePoint depthPoint =
    _sensor.CoordinateMapper.MapCameraPointToDepthSpace(headJoint.Position);

// use the x & y coordinates to locate the depth data
FrameDescription depthDesc = _sensor.DepthFrameSource.FrameDescription;
int depthX = (int)Math.Floor(depthPoint.X + 0.5);
int depthY = (int)Math.Floor(depthPoint.Y + 0.5);
int depthIndex = (depthY * depthDesc.Width) + depthX;

ushort depth = _depthData[depthIndex];

To calculate the desired size of the image, we need to determine the width of the joint's pixel in millimeters. We do this using a blast from the past, our best friend from high-school trigonometry, Soh-Cah-Toa.

Given that the Kinect’s Horizontal Field of View is 70.6°, we bisect this in half to form a right-angle triangle. We then take the depth value as the length of the adjacent side in millimeters. Our goal is to calculate the opposite side in millimeters, which we can accomplish using the TOA portion of the mnemonic:

tan(0) = opposite / adjacent
opposite = tan(0) * adjacent

Once we have the length of the opposite, we divide it by the number of pixels in the frame which gives us the length in millimeters for each pixel. The algorithm for calculating pixel width is shown here:

private double CalculatePixelWidth(FrameDescription description, ushort depth)
{
    // measure the size of the pixel
    float hFov = description.HorizontalFieldOfView / 2;
    float numPixels = description.Width / 2;

    /* soh-cah-TOA
     * 
     * TOA = tan(0) = O / A
     *   T = tan( (horizontal FOV / 2) in radians )
     *   O = (frame width / 2) in mm
     *   A = depth in mm
     *   
     *   O = A * T
     */
 
    double T = Math.Tan((Math.PI * 180) / hFov);
    double pixelWidth = T * depth;

    return pixelWidth / numPixels;
}

Now that we know the length of each pixel, we can adjust the size of our head-tracking image to be a consistent “length”. The dimensions of the image will change as I move but the amount of space around my head remains consistent. The following calculates a 50 cm (~19”) image around the tracked position of my head:

double imageSize = 500 / CalculatePixelWidth(depthDesc, depth);

int x = (int)(Math.Floor(colorPoint.X + 0.5) - (imageSize / 2));
int y = (int)(Math.Floor(colorPoint.Y + 0.5) - (imageSize / 2));

WriteableBitmap faceImage = _bmp.Crop(new Rect(x,y, imageSize, imageSize));

Happy Coding.

Drawing Kinect V2 Body Joints

So far the posts in my Kinect for Windows v2 series have concentrated on the Depth, Color and BodyIndex data sources. Today I want to highlight how to access the Body data stream.

This is an early preview of the new Kinect for Windows, so the device, software and documentation are all preliminary and subject to change.
The new Kinect sensor and SDK has improved skeleton tracking so significantly that the Microsoft team has changed the name of their Skeleton class to Body. In addition to being able to track six skeletons (instead of 2), the API is shaping up to include features for Activities (facial features such as left eye open), Expressions (happy or neutral), Appearance (wearing glasses), Leaning (left or right) and the ability to track if the user is actively looking at the sensor. I’ll dive into those in up coming posts, but for now I want to focus on Joint information.
Generally speaking, the body tracking capability and joint positions are improved over previous version of the SDK. Specifically, the positions for hips and shoulders are more accurate. Plus version 2 has introduced 5 new joints (Neck, Hand-Tip-Left, Thumb-Left, Hand-Tip-Right, Thumb-Right) bringing the total number of joints to 25.

Getting the Body data from the sensor is slightly different than the other data streams. First you must initialize an array of Body with a specific size and then pass it into the GetAndRefreshBodyData method on the BodyFrame. The SDK uses a memory saving optimization that updates the items in this array rather than creating a new set each time. So if you want to hang onto an instance of Body between frames, you need to copy it to another variable and replace the item in the Array with a null value.
The following shows how setup and populate an array of Body objects per frame:

private void SetupCamera()
{
    _sensor = Sensor.Default;
    _sensor.Open();

    _bodies = new Body[_sensor.BodySouceFrame.BodyCount];

    _reader = _sensor.BodySourceFrame.OpenReader();
    _reader.FrameArrived += FrameArrived;
}

private void FrameArrived(object sender, BodyFrameArrivedEventArgs e)
{
    using (BodyFrame frame = e.Frame.AcquireFrame())
    {
        if (frame == null)
            return;

        frame.GetAndRefreshBodyData(_bodies);
    }
}

Once you have a Body to work with, the joints and other features are provided to us as dictionaries which allows us to access the joints by name. For example:

Joint head = body.Joints[JointType.Head];

Each Body and Joint is also equipped with tracking confidence. Given this information, we can loop through the joints and only display the items that we know are actively being tracked. The following example iterates over the Body and Joint collections, and uses the CoordinateMapper to translate the joint position to dimensions on our color image. For simplicity sake, I’m simply coloring the surrounding pixels to illustrate their position.

private void DrawBodies(BodyFrame bodyFrame, int colorWidth, int colorHeight)

    bodyFrame.GetAndRefreshBodyData(_bodies);

    foreach(Body body in _bodies)
    {
        if (!body.IsTracked)
            continue;

        IReadOnlyDictionary<JointType, Joint> joints = body.Joints;

        var jointPoints = new Dictionary<JointType, Point>();

        foreach(JointType jointType in joints.Keys)
        {
            Joint joint = joints[jointType];
            if (joint.TrackingState == TrackingState.Tracked)
            {
                ColorSpacePoint csp = _coordinateMapper.MapCameraPointToColorSpace(joint.Position);
                jointPoints[jointType] = new Point(csp.X, csp.Y);                                                                        
            }
        }

        foreach(Point point in jointPoints.Values)
        {
            DrawJoint(ref _colorData, point, colorWidth, colorHeight, 10);
        }
    }
}

private void DrawJoint(ref byte[] colorData, Point point, int colorWidth, int colorHeight, int size = 10)
{
    int colorX = (int)Math.Floor(point.X + 0.5);
    int colorY = (int)Math.Floor(point.Y + 0.5);

    if (!IsWithinColorFrame(colorX, colorY, colorWidth, colorHeight))
        return;

    int halfSize = size/2;

    // loop through pixels around the point and make them red
    for (int x = colorX - halfSize; x < colorX + halfSize; x++)
    {
        for(int y = colorY - halfSize; y < colorY + halfSize; y++)
        {
            if (IsWithinColorFrame(x,y, colorWidth, colorHeight))
            {
                int index = ((colorWidth * y) + x) * bytesPerPixel;
                colorData[index + 0] = 0;
                colorData[index + 1] = 0;
                colorData[index + 2] = 255;
            }
        }
    }
}

private bool IsWithinColorFrame(int x, int y, int width, int height)
{
    return (x >=0 && x < width && y >=0 && y < height);
}

Happy Coding.

Mapping between Kinect Color and Depth

Continuing my series of blog posts about the new Kinect v2, I’d like to build upon my last post about the HD color stream with some of the depth frame concepts that I've used in the last few posts. Unlike the previous version of the Kinect, the Kinect v2 depth stream is not the same dimensions as the new color stream. This post will illustrate how the two are related and how you can map depth data to the color stream using the CoordinateMapper.
This is an early preview of the new Kinect for Windows, so the device, software and documentation are all preliminary and subject to change.
While the Kinect’s color and depth streams are represented as arrays of information, you simply cannot compare the x & y coordinates between the sets of data equally: the bytes of the color frame represent color pixels from the color camera; the bits of the depth frame represent Cartesian distance from the depth camera. Fortunately, the Kinect SDK ships with a mapping utility that can convert data between the different “spaces”.
For this post I’m going to use the MultiFrameSourceReader to access both the depth and color frames at the same time and we’ll use the CoordinateMapper.MapDepthFrameToColorSpace method to project the depth data into an array of ColorSpacePoint. Here’s the skeleton for processing frames as they arrive:

private void FrameArrived(object sender, MultiSourceFrameArrivedEventArgs e) 
{ 
    var reference = e.FrameReference; 

    MultiSourceFrame multiSourceFrame = null;
    ColorFrame colorFrame = null; 
    DepthFrame depthFrame = null; 

    try 
    { 
        using (_frameCounter.Increment()) 
        { 
            multiSourceFrame = reference.AcquireFrame(); 
            if (multiSourceFrame == null) 
                return; 

            using (multiSourceFrame) 
            { 
                colorFrame = multiSourceFrame.ColorFrameReference.AcquireFrame();
                depthFrame = multiSourceFrame.DepthFrameReference.AcquireFrame(); 

                if (colorFrame == null | depthFrame == null) 
                    return; 

                // initialize color frame data 
                var colorDesc = colorFrame.FrameDescription; 
                int colorWidth = colorDesc.Width; 
                int colorHeight = colorDesc.Height; 

                if (_colorFrameData == null) 
                { 
                    int size = colorDesc.Width * colorDesc.Height; 
                    _colorFrameData = new byte[size * bytesPerPixel]; 
                } 

                // initialize depth frame data 
                var depthDesc = depthFrame.FrameDescription; 

                if (_depthData == null) 
                { 
                    uint depthSize = depthDesc.LengthInPixels; 
                    _depthData = new ushort[depthSize]; 
                    _colorSpacePoints = new ColorSpacePoint[depthSize]; 
                } 

                // load color frame into byte[] 
                colorFrame.CopyConvertedFrameDataToArray(_colorFrameData, ColorImageFormat.Bgra); 

                // load depth frame into ushort[] 
                depthFrame.CopyFrameDataToArray(_depthData); 

                // map ushort[] to ColorSpacePoint[] 
                _sensor.CoordinateMapper.MapDepthFrameToColorSpace(_depthData, _colorSpacePoints); 

                // TODO: do something interesting with depth frame 

                // render color frame 
                _bmp.WritePixels( 
                    new Int32Rect(0, 0, colorDesc.Width, colorDesc.Height), 
                    _colorFrameData, 
                    colorDesc.Width * bytesPerPixel, 
                    0); 
            } 
        } 
    } 
    catch { } 
    finally 
    { 
        if (colorFrame != null) 
            colorFrame.Dispose(); 

        if (depthFrame != null) 
            depthFrame.Dispose(); 

    }
}

The MapDepthFrameToColorSpace method copies the depth data into an array of ColorSpacePoint where each item in the array corresponds to the items in the depth data. We can use the X & Y coordinates of the ColorSpacePoint to find the color data as demonstrated below. There’s one caveat: not all points in the depth array contain data that can be mapped to color pixels. Some points might be too close or too far, or there’s no depth data because it’s a shadow or reflective material.
The following snippet shows us how to locate the color bytes from the ColorSpacePoint:

// we need a starting point, let's pick 0 for now
int index = 0;

ushort depth = _depthData[index];
ColorSpacePoint point = _colorSpacePoints[index];

// round down to the nearest pixel
int colorX = (int)Math.Floor(point.X + 0.5);
int colorY = (int)Math.Floor(point.Y + 0.5);

// make sure the pixel is part of the image
if ((colorX >= 0 && (colorX < colorWidth) && (colorY >= 0) && (colorY < colorHeight))
{

    int colorImageIndex = ((colorWidth * colorY) + colorX) * bytesPerPixel;

    byte b = _colorFrameData[colorImageIndex];
    byte g = _colorFrameData[colorImageIndex + 1];
    byte r = _colorFrameData[colorImageIndex + 2];
    byte a = _colorFrameData[colorImageIndex + 3];

}

If we loop through the depth data and use the above technique we can draw our depth data on top of our color frame. For this image, I’m drawing the depth data using an intensity technique described in an earlier post. It looks like this:

You may notice that the pixels are far apart and don’t go to the entire edge of the image. This makes sense because our depth data is a smaller resolution (424 x 512 compared to 1080 x 1920) and tighter viewing angle (70.6° compared to 84°). The mapping also isn’t perfect in this release (remember this is a developer preview!)
We can use the same technique to draw each pixel of the depth frame using values from the color frame, like so:

// clear the pixels before we color them
Array.Clear(_pixels, 0, _pixels.Length);

for (int depthIndex = 0; depthIndex < _depthData.Length; ++depthIndex)
{
    ColorSpacePoint point = _colorSpacePoints[depthIndex];

    int colorX = (int)Math.Floor(point.X + 0.5);
    int colorY = (int)Math.Floor(point.Y + 0.5);
    if ((colorX >= 0) && (colorX < colorWidth) && (colorY >= 0) && (colorY < colorHeight))
    {
        int colorImageIndex = ((colorWidth * colorY) + colorX) * bytesPerPixel;
        int depthPixel = depthIndex * bytesPerPixel;

        _pixels[depthPixel] = _colorData[colorImageIndex];
        _pixels[depthPixel + 1] = _colorData[colorImageIndex + 1];
        _pixels[depthPixel + 2] = _colorData[colorImageIndex + 2];
        _pixels[depthPixel + 3] = 255;
    }
}

…which results in the following image:

So as you can see, we can easily map between the two coordinate spaces. I intended to build upon this further in upcoming posts, so if you haven’t already, add me to your favorite RSS reader.
Happy coding.

Accessing Kinect Color Data

One of the new features of the Kinect v2 is the upgraded color camera which now provides images with 1920 x 1080 resolution. This post will illustrate how you can get access to this data stream.

This is an early preview of the new Kinect for Windows, so the device, software and documentation are all preliminary and subject to change.

Much like the other data streams, accessing the color image is done using a reader:

private void SetupCamera();

    var sensor = KinectSensor.Default;
    sensor.Open();
    ColorFrameReader reader = sensor.ColorFrameSource.OpenReader();
    reader.FrameArrived += ColorFrameArrived;
}

In this example, I'm simply copying the frames to a WriteableBitmap which is bound to the view.

private readonly int bytesPerPixel = (PixelFormats.Bgr32.BitsPerPixel + 7) / 8;
private readonly WriteableBitmap _bmp = new WriteableBitmap(1920, 1080, 96, 96, PixelFormats.Bgra32, null);
Byte[] _frameData = null;

private void ColorFrameArrived(object sender, ColorFrameArrivedEventArgs e)
{
    var reference = e.FrameReference;

    try
    {
        var frame = reference.AcquireFrame();
        if (frame == null) return;

        using(frame)
        {
            FrameDescription desc = frame.FrameDescription;
            var size = desc.Width * desc.Height;
            
            if (_frameData == null)
            {
                _bmp = new WriteableBitmap(desc.Width, desc.Height, 96, 96, PixelFormats.Bgr32, null);
                _frameData = new byte[size * bytesPerPixel];
            }

            frame.CopyConvertedFrameDataToArray(_frameData, ColorImageFormat.Bgra);

            _bmp.WritePixels(
                new Int32Rect(0, 0, desc.Width, desc.Height),
                _frameData,
                desc.Width * bytesPerPixel,
                0);
        }
    }
    catch
    {}

}

As you’d expect, this updates the view every 30 frames per second with a HD image. The wide-angle lens of the camera (84°) captures a fair amount of my office.

Happy Coding.

PhotoBooth (Free Code)

I’ve been blogging recently about some of my experiments with the Kinect for Windows v2 developer preview. Most of my experiments have been based on the supplied samples, with a lot of copy and paste action. I’ve pulled some of the repeated code into some re-usable classes.

Here’s some code that I’ve moved out of the samples into a reusable component that takes a screen shot of the Kinect output. As an added bonus, I’ve added a count-down timer. The code provided here is based on the original samples, modified slightly. Free as in beer.

Here’s an example of how to use it.

First, add the visual for the count-down timer into your UI. I’m putting mine below the image that shows my Kinect output.

<ViewBox>
    <Image Source="{Binding ImageSource}" Stretch="UniformToFill" />
    <TextBlock 
        Text="{Binding CountDownTimer}"
        HorizontalAlignment="Center"
        VerticalAlignment="Center"
        FontSize="96"
        Foreground="White" />
</ViewBox>

<Button 
    Style="{StaticResource ScreenshotButton}"
    Content="Screenshot"
    Click="ScreenshotButton_Click" />

Next, initialize the PhotoBooth in your ViewModel. Note that I’m being very lazy and I’m using the code-behind of the XAML as my ViewModel. You know I’m not a big fan of this, I’m just following the lead of the Kinect examples so let’s pretend I didn’t do that. Do proper MVVM, kids.

public class MainWindow : Window, INotifyPropertyChanged
{
    private KinectSensor _sensor;
    private DepthFrameReader _reader;
    private WriteableBitmapImage _bmp;

    // see my last post on how to use the framecounter!
    private FrameCounter _frameCounter;
    private PhotoBooth _photoBooth;

    private string _statusText;
    private string _countDownTimer;


    public MainWindow()
    {
        InitializeComponent();

        _sensor = KinectSensor.Default;
        _sensor.Open();
        _reader = _sensor.DepthFrameSource.OpenReader();
        _reader.FrameArrived += FrameArrived;

        _photoBooth = new PhotoBooth();
        _photoBooth.PropertyChanged += (o,e) => 
            CountDownTimer = _photoBooth.TimeDisplay;       

        _frameCounter = new FrameCounter();
        _frameCounter.PropertyChanged += (o,e) =>
            StatusText = String.Format("{0} FPS", _frameCounter.FramesPerSecond);

        DataContext = this;
    }

    public string StatusText { /* get/set property changed omitted */ }
    public string CountDownTimer { /* get/set property changed omitted */ }
    public BitmapImage ImageSource { /* get/set property changed omitted */ }

    async void ScreenshotButton_Click(object sender, EventArgs e)
    {

        string fileName = await _photoBooth.TakePhoto(_bmp);

        if (!String.IsNullOrEmpty(fileName)
        {
            StatusText = string.Format("Snapshot created: {0}", fileName);
        }
        else
        {
            StatusText = "Couldn't create snapshot :(";
        }

        // allow our message to show for 5 seconds...
        _frameCounter.DeferFrameCount(TimeSpan.FromSeconds(5));
    }

    void FrameArrived(object sender, DepthFrameSourceEventArgs e)
    {
        // create bitmap, write depth data to bitmap
    }
}

By default, the Photobooth will create a new PNG in your Pictures folder with a random guid as the file name. You can change both of these to suit your needs.

// default directory is My Pictures
_photoBooth.BaseDirectory = "C:\Snaps";

// specify the file naming convention. will add .png to the end...
_photoBooth.CreateFileName = () => {
    string time = System.DateTime.Now.ToString("hh'-'mm'-'ss", CultureInfo.CurrentUICulture.DateTimeFormat);
    return String.Format("KinectScreenshot-PhotoBoothSample-{0}", time);    
};

Click to expand to get the full source.

public class PhotoBooth : INotifyPropertyChanged
{
    private int _timeLeft;
    private Func<string> _createFileName;
    private string _baseDirectory;
    private TimeSpan _waitInterval;

    public PhotoBooth()
    {
        BaseDirectory = Environment.GetFolderPath(Environment.SpecialFolder.MyPictures);
        WaitInterval = TimeSpan.FromSeconds(5);
    }

    public string BaseDirectory
    {
        get { return _baseDirectory; }
        set { _baseDirectory = value; }
    }

    public TimeSpan WaitInterval
    {
        get { return _waitInterval; }
        set { _waitInterval = value; }
    }

    public string TimeDisplay
    {
        get
        {
            if (TimeLeft > 0)
            {
                return TimeLeft.ToString();
            }

            return String.Empty;
        }
    }

    public int TimeLeft
    {
        get { return _timeLeft; }
        protected set
        {
            if (_timeLeft != value)
            {
                _timeLeft = value;
                var handler = PropertyChanged;
                if (handler != null)
                {
                    handler(this, new PropertyChangedEventArgs("TimeLeft"));
                    handler(this, new PropertyChangedEventArgs("TimeDisplay"));
                }
            }
        }
    }

    public async Task<string> TakePhoto(WriteableBitmap bitmap, string path = null, int seconds = -1)
    {
        if (string.IsNullOrEmpty(path))
        {
            path = BaseDirectory;
        }

        if (seconds == -1)
        {
            seconds = (int)WaitInterval.TotalSeconds;
        }

        if (seconds > 0)
        {
            TimeLeft = seconds;
            while (TimeLeft > 0)
            {
                await Task.Delay(1000);
                TimeLeft = TimeLeft - 1;
            }
        }

        string fileName = Path.Combine(path, CreateFileName() + ".png");

        bool success = SaveBitmap(fileName, bitmap);

        return success ? fileName : String.Empty;
    }

    public Func<string> CreateFileName
    {
        get
        {
            if (_createFileName == null)
            {
                _createFileName = () => Guid.NewGuid().ToString();
            }
            return _createFileName;
        }
        set
        {
            _createFileName = value;
        }
    }

    public event PropertyChangedEventHandler PropertyChanged;

    private bool SaveBitmap(string path, BitmapSource bitmapSource)
    {
        if (bitmapSource == null)
            return false;

        // create a png bitmap encoder which knows how to save a .png file
        BitmapEncoder encoder = new PngBitmapEncoder();

        // create frame from the writable bitmap and add to encoder
        encoder.Frames.Add(BitmapFrame.Create(bitmapSource));


        // write the new file to disk
        try
        {
            // FileStream is IDisposable
            using (FileStream fs = new FileStream(path, FileMode.Create))
            {
                encoder.Save(fs);
            }

        }
        catch (IOException)
        {
            return false;
        }

        return true;            
    }
}

Happy coding