Abhijit's World of .NET

Home Security System Using Kinect, Azure, Windows Phone and Windows 8

Abhijit Jana — Mon, 16 Apr 2012 13:54:49 +0000

Most the people who were in the TechEd 2012 India would have witnessed the cool demo by me and Jebarson made on the opening Keynotes of the event. And now we would like to share how this solution was developed so that you can also build your own.

Background

Before we could pitch in, let me give a little background on how we decided to build this on our garage. Developer Conference 2011 on Hyderabad was scheduled on October 2011 and myself and Jebarson were looking forward to show something cool on our own interested areas. Jebarson was looking out for something on Windows Azure and Windows Phone while myself was fiddling around Kinect. Ran out of ideas we both met together and thought of integrating everything and that is when we came up with the idea of building our own home security. We started building this application as early as September 2011. And by now, we have already revised the application more than 5 times due to newer SDK release and also due to the addition of newer components.

Overview

So how does the solution work. You should place Kinect devices on the locations you want to secure and keep it connected to a high end PC capable enough to do a video encoding and up streaming. The Kinect devices will be run by a WPF application running on the PC which will look for any human intrusion. While there is no human intrusion, it will remain in “Patrol Mode” where it will upload pictures of its view on regular intervals to Windows Azure. These pictures can be monitored at any point of time through the application running on Windows Phone and Windows 8. Whenever there is an human intrusion detected the Kinect changes itself to “Intrusion Mode”; it then alerts Windows Azure and also starts streaming the video to the Smooth Streaming Server hosted on Windows Azure. The Azure notifies all the devices subscribed (Windows Phone / Windows 8 ) using a toast and will allow those application to view the live streaming. From your Windows Phone / Windows 8 app you can also control the Kinect device.

Windows Azure

Now that we are done with the overview of how the solution works, lets talk about how it is built. Starting with Azure, below is how the application hosted on the cloud.

Windows Azure Management Portal

The Windows Azure Management Portal is created using the default template provided from Windows Azure Toolkit for Windows Phone or Windows Azure Toolkit for Windows Phone. This management portal can be customized later on to support the needed services. To begin with, the management portal by default integrates with all the above mentioned blocks except for the Smooth Streaming server; which means the API which would require for all the communication with these components are already defined. Now lets go with the flow.

Custom WCF Service Methods

In order to support the WPF application running on the PC connected to the Kinect devices, we will expose a custom WCF method called “Upload” allowing the it to upload an object of type “Photo”.

The image is uploaded as a byte array along with the type (Patrol / Alert) and time. As soon as the WCF service gets the message, it serializes it to string and puts it into the blobs and also pushes it to queue with the expiration time set for 1 minute. This will help the user to see the latest picture from his home at any point of time.

Shared Access Signature Service

The portal derived out of the template would require the authentication done through the certificate along with a user name and password for the generation of the token. Internally the portal uses ASP.net membership authentication through which it authenticates the user and provides a mutually agreed token to the client. So doesn’t matter whether the Windows Phone, Windows 8 needs the access to the Azure, it will require this mechanism to get authenticated. The token then will be required to access any methods from the service.

Microsoft Push Notification Service

This service allows the solution to send a notification message to any smart device which is capable of subscribing / receiving push notifications. The application which is written in Windows 8 and Windows Phone subscribes itself for the notification as soon as application is run for the first time. All the device needs is an internet connection for subscribing / receiving the notifications. Once the Azure service receives an alert message from the Kinect, it sends a push notification to all the devices subscribed. The subscription details are stored in the Azure Tables by the management portal.

Smooth Streaming Server

Configuring this server in the Azure would be the toughest part but would be a cake walk if you understand how it is done. Azure Live Streaming on codeplex allows you to do this. The Web role will have a “Startup.cmd” file which will contain the command as below to install the Smooth Streaming Server on the Azure Server.

set msiexec=%systemroot%\system32\msiexec.exe
set appcmd=%systemroot%\system32\inetsrv\appcmd.exe

%msiexec% /i “%~dp0\IISMedia64.msi” /qn ADDLOCAL=ALL /Le startup_media.txt

exit /b 0

The setup file IISMedia64.msi has to be added to the project as a content. The service definition file for the azure will have the below tags

This will allow the batch file to run once the web role starts up.

You will also require to add “.isml” files for configuring the uplink / downlink streaming end point. For more information on how to configure the streaming server in Azure, please follow the documentation. The kinect will up stream the compressed video stream to the uplink endpoint and once the video stream is available on the Azure, it will automatically be available on the downlink to have a live streaming.

Note: You can configure any number of instances for a downlink server but only one instance for uplink server.

AppFabric Service Bus

We are using the appfabric service bus as a remote endpoint for a WCF service hosted in the application which is connected to the device. This will allow anybody to control the Kinect device like tilting just by sending a message to the WCF endpoint. This will eliminate the need of a static IP / DNS and the firewall overheads which may incur.

In our application there is a method exposed called “SendMessage” by the WCF hosted in the WPF application running along the Kinect devices. Once Windows Phone / Windows 8 sends a command to this method namely “up/down”, the device tilts as per the command.

Now that we have talked enough about Azure, lets talk a little about Kinect on how it is going to act as a watch dog for the solution.

Kinect for Windows

Kinect works as Watch Dog at your home which is connected with Windows Azure. The application was built on Kinect for windows SDK ( V 1.0) . We have used the Skeleton detection features to detect the human body motion. As we have already mentioned by default Kinect will be in “Patrol mode” , where it will upload the image with every 10 sec of interval and if there is any skeleton detection by Kinect sensor, it will start the streaming. We have used the expression encoder SDK to do the stream from application. Below are the list of components which we have used in WPF Client application.

Below are the very top level view of code block used for detection of skeleton and uploading it to azure.

Find if Skeleton is tracked and then set a global variable which is being used across the application.

if (this.skeletonData.Where(item => item.TrackingState == SkeletonTrackingState.Tracked).ToList().Count > 0
{
Global.IsSkeletonTracked = true;
}

Create a patrol type image is there is no Skeleton otherwise send the alert image.

if (Global.IsSkeletonTracked)
{
photo  = newPhoto() { CreatedOn = DateTime.Now, Image = filesbyte, Type = PhotoType.Alert };
}
else
{
photo = newPhoto() { CreatedOn = DateTime.Now, Image = filesbyte, Type = PhotoType.Patrol };
}

The type of Photo is defined with our service contract and we are uploading the images as bellow:

KinectReceiverClient client = newKinectReceiverClient("BasicHttpBinding_IKinectReceiver")
ServicePointManager.ServerCertificateValidationCallback = ((obj, certificate, chain, sslPolicyErrors) => true);
client.UploadPhoto(photo);

Windows Phone

As I mentioned earlier, Windows Phone will act as one of the end user device in this solution. To interact with Azure easily and securely, we will be using the Windows Azure Toolkit for Windows Phone client. As mentioned in the Azure, the windows phone app will authenticate itself against the Azure on the start up using a user name and password signed by the certificate. Once the application is authenticated, the app in the first screen will show the latest pictures which is peeked from the queue. Remember that these images are pushed to the cloud by the Kinect in the “Patrol” mode.

When there is an intrusion, the Windows Phone receives a toast notification. There is no need to keep the application running / keep your phone unlocked for you to see the notification as the Windows Phone takes care of all the services needed for listening to the notification. Upon tapping the notification, you will be navigated to the page where you will see the live stream through the Multimedia Player for Windows Phone streaming through the downlink url.

Windows 8

The metro app developed on Windows 8 runs pretty much the same way as Windows Phone except that it uses Windows Azure Toolkit for Windows 8. Since metro apps also supports the notification service, it’s the same functionality you will require to implement.

Application Functioning on Patrol Mode

Application Functioning on Intrusion Mode

Application Functioning on Accessing Kinect Remotely

We haven’t shared the code here while we have sharing everything that you will require to build this application. We would encourage to try building the application with all the knowledge we have shared. If you require any help on building the same, please do drop a note to us and we will be more than happy to help you.

Resources

Showcased

Multiple events inside Microsoft.
Developer Conference Hyderabad (Oct 2011)
TechEd India 2012 (Jan 2012)

Check Out Recorded Session from TechED India 2012

Cheers !

Filed under: .NET 4.0, Azure, Kinect, Windows 8

TechEd India 2012 – Presenting On "Windows Azure With Windows Phone, Windows 8 Metro And Kinect”

Abhijit Jana — Tue, 13 Mar 2012 16:09:39 +0000

I am very happy and excited to announce that this TechEd I will be presenting along with my friend Jebarson. We will be presenting a session which will showcase the integration onWindows Azure, Windows Phone, Windows 8 Metro and Kinect.

For the presentation we have developed a proto solution for Home Security. Well let me explain how we have put all these technologies into action. The Kinect device will serve as a watch dog at the home which is connected to the Azure. The device will be uploading patrol images to the Azure on regular intervals. Once there is a human intrusion detected, the Kinect will send a alert to the Azure and also will start streaming the live video to the Smooth Streaming Server hosted on Azure. Once the Azure, receives the alert, it will send a notification using Microsoft Push Notification Server (MPNS) to the subscribed Windows Phones and Windows 8 Metro apps where you can watch the video streamed live from the Kinect. Well, that’s not the end, we will also demonstrate on controlling Kinect from Windows Phone / Windows 8 Metro app using Azure App Fabric.

We will be discussing all the technologies involved and the architecture of the complete solution extensively along with a live demonstration in the TechEd.

Session Title : Windows Azure meets Kinect, Windows Phone and Windows 8 Metro
Track : Windows Azure
Date : 21 March 2012 1415 Hrs. IST
Venue : LaLiT Ashok’s, Kumara Krupa High Grounds, Bengaluru – 560001

Teaser Video on our session

See you there !

Cheers !
Aj

TechEd India -Presenting On "Windows Azure With Windows Phone, Windows 8 Metro And Kinect" abhijan.me/ywl9IB #techedin @jebarson007—
Abhijit Jana (@AbhijitJana) March 13, 2012

Filed under: .NET Community Events, TechEd

Visual Studio 11 Developer Day– Hyderabad–18th Feb 2012

Abhijit Jana — Mon, 13 Feb 2012 09:16:24 +0000

Microsoft User Group Hyderabad (MUGH ) invites you to Developer Day on 18th Feb 2012 ( 9:30 A.M to 1.30 PM ) at Microsoft Campus , Building 3 MPR Halls , Gachibowli, Hyderabad. Join us to meet Meet Charles Sterling, Senior Program Manager, Visual Studio Team. Charles will deliver 90 min talks on “Application Lifecycle Management with Visual Studio 11”. Additionally, we have two sessions on “Debug production hosted application with Intellitrace and Visual Studio 11” and “SQL Server 2012 Database Unit Testing with Visual Studio 11”.

Check out the Complete Agenda from here : http://mugh.net/vs11devday/agenda.html

See you there !

Cheers !

Abhijit

Filed under: .NET Community Events

Presenting at DevDays – Hyderabad 29th January 2012

Abhijit Jana — Wed, 25 Jan 2012 13:05:43 +0000

Microsoft User Group Hyderabad (MUGH ) invites you to Developer Day on 29th Jan’12 . This is scheduled on this Sunday (29th Jan – from 9:30 A.M to 1.30 PM ) at Microsoft Campus , Building 3 MPR Halls , Gachibowli, Hyderabad.

I am presenting on “Coding for fun on Kinect” . During this1 hr. session, I will be talking about Kinect .NET SDK APIs and will be developing some cool application using Kinect for Windows SDK Beta 2. Yeah, something interesting!

I have few copies of “SQL Server Questions & Answers” book by Pinal Dave and Vinod Kumar to give away during my session.

Here is the complete agenda

It’s a free event, Register here : http://mughdeveloperday01.eventbrite.com/

Looking forward to meet !

Cheers !

Abhijit

Filed under: .NET Community Events

JScript Debugger– “Unable to attach the process. Another debugger might be attached to the process” – while application is running from Visual Studio. [ Solution ]

Abhijit Jana — Fri, 20 Jan 2012 16:57:48 +0000

IE Developer toolbar provide us a fantastic and easiest way to debug JavaScript with in browser. If the web site is hosted on IIS and you are trying debug JavaScript using IE Developer tool bar, it will work fine if you the correct browser setting for debugging. If you are running your web application directly from visual studio, you might came across the problem of “Unable to attach to the process” while click on “Start debugging” button in IE Developer toolbar. In this post I am going to talk about the resolution of the same and will also talk about little bit behind the scene.

Problem:

IE Developer Toolbar – JS Debugger Error “Unable to attach the process. Another debugger might be attached to the process” .

Steps :

1. Run an ASP.NET application from Visual Studio.

2. Once application launched, Press F12 in browser to launch Developer Toolbar.

3. Move to “Script” Tab and Click on “Start Debugging” . Put necessary breakpoints on the script function as required before click.

On click of “Start Debugging”, you will receive below error message. which says, already another debugger attached and you can not attach with current IE Process for debugging.

Solution :

Follow the below steps for resolve this issue.

1. Go back to Visual Studio and Select “Detach All” from “Debug” menu.

2. Now click on “Start Debugging” from IE Developer Toolbar. No error , and you can continue with debugging and other work.

Behind the scene :

Visual Studio automatically instantiate an Internet explorer process to launch the web sites. You can check the process level using “Process Explorer”.

Also, it attached the process “iexplore.exe” automatically during launch of site that helps us debug JavaScript from visual studio itself. You can check it from Solution explorer itself

Which mean, while your site is running in browser, “iexplore.exe” ( Process for Internet Explorer) is already attached . One of the most simplest way to check this out is, navigate to Debug > Windows> Processes . This will list out all attached process with in visual studio . For any typical ASP.NET web application which is running from visual studio will have two process and iexplore is one of them.

That’s why when you are trying to attach the same process from IE Developer tool bar, it is throwing error “Unable to attach the process. Another debugger might be attached to the process” .

So now we know what was the causing the issue and let’s have a look what we did to resolve this.

As a resolution, we used “Detach All” from debug menu, which actually detach all the process which is attached with visual studio except local ASP.NET Development server process ( WebDev.WebServer40.EXE ).

Note : “Detach All” will detach all the attached process apart from ASP.NET Development server. Because detaching the development server is nothing but stopping your application. If you want to stop, either stop debugging or “Terminate” the local development sever process.

“Detach All” will remove iexplore.exe from the list of attached process. So, you can go ahead and start the debugging in IE Developer toolbar.

Detaching a Particular Process Instead “Detach All”

If there are multiple processes attached and if you want to detach only “IE Process”, you can simply do it from process attach window, by selecting “Detach Process” from the content menu of the “iexplore.exe”. This will help you to retain on other process in attached state.

To know more about Processes Window, check out this post How to get list of attached processes in Visual Studio while debugging ?

Hope this helps !

Cheers !

Abhijit

Filed under: .NET 4.0, ASP.NET 4.0, ASP.NET 4.5, Tips and Tricks, Visual Studio, Visual Studio 2010, Visual Studio 2011

I wish you a Happy New Year 2012

Abhijit Jana — Sat, 31 Dec 2011 19:33:22 +0000

Wish you and your family a very happy and prosperous new year 2012. May this new year bring many opportunities your way to explore every joy of life. I Hope this new year fills your life with happiness and perfect health always, much more success and happiness.

As we wind up with year 2011, I would like to express my sincere thanks to all the visitors, readers, followers of my blog for their encouragement, feedbacks and support. Please keep visiting and send me your feedback.

Keep visiting here and follow me @AbhijitJana for quick update.

Again, Thanks you and a very very happy new year 2012 !

Thanks !

Filed under: Announcement

Daily .NET Tips – Completed 1 Year Today

Abhijit Jana — Tue, 27 Dec 2011 07:29:35 +0000

I am very happy to inform that, Daily .NET Tips marks the first anniversary today. 27th Dec 2010, I announced the official launch of this web site ( .NET Tip a Day, Keeps painkillers away! – Live at http://dailydotnettips.com – 27th Dec 2010 ). A year back I started with this site with intention of sharing most useful and handy tip and tricks on .NET that can improve productivity of work while coding and can give solutions of much required problems. This site is completely design for sharing Tips and Tricks, New Features, useful Code Snippet which you can use in daily development work and targeted anything related with .NET.

With the 1st anniversary, I thought to share some interesting facts regarding this web site:

Total numbers of tips shared : 250
Total number of Facebook Fan : 500 + ( Like http://www.facebook.com/DailyDotNetTips )
Total Number of Twitter Follower : 350 + ( Follow @dailydotnettips )
Approximate Page View : 6,86,000 +
Numbers of post has been showcased as “Spotlight” at http://www.asp.net/
Tweet from Scott Gu
This site covered numbers of Tips and Tricks on recent release of ASP.NET 4.5 and Visual Studio 2011

This has been a very great journey, lots of fun and learning over the year and this is just the beginning. I would like to thanks each individual of you who supported to encouraged for this web site. My personal thanks to “Abhishek Sur”, who is the main contributor and administrator of Daily .NET tips. Abhishek shared numbers of tips over the year and taken care of many backend stuff of this web site. Thanks to our valuable contributor Jebarson Jebamony, Atul Verma, Pranay ana, Shravan Kumar and Brij Bhushan Mishra for their contribution with various tips.

And of course, many thanks to my lovely wife Ananya for her priceless support. Today is her birthday as well. Don’t forget to wish here !

Well, this is just the beginning and hopefully will continue with same passion over coming year. Will appreciate your suggestion and feedback to improve.

Thank and keep visiting.

Cheers,

Abhijit

Daily .NET Tips | Founder

Filed under: Announcement, Daily.NETTips, General

Wishing you all Merry Christmas

Abhijit Jana — Sat, 24 Dec 2011 18:38:33 +0000

Here’s wishing you all the joys of the season. Wish you and your family a Merry Christmas and a prosperous New Year in year in advance. May this good times become the golden memories of tomorrow for all of you. Wish you lots of love, joy and happiness.

Best Regards,

Abhijit

Filed under: Announcement, General

Download : “Betty Wallpapers” – Windows Phone App for Bing image search

Abhijit Jana — Wed, 21 Dec 2011 14:46:21 +0000

If you ever wanted an application in windows phone which can search images and wallpapers from internet and download it into your phone, then your wait is over.

My friend Jebarson has developed a very simple app named "Betty Wallpapers" which uses Bing for searching the images / wallpapers from the internet and shows up on the screen. You can download them directly into your library.

The application also exposes search options which allows you to filter your images by size, aspect, color, style,etc.

This is a totally free application except that you will not be able to turn off adult filter in the trial mode considering the fact that minors will not hold a credit card.

Feel free to download it from Market Place . Do not forget to submit your review for the same. If you have additional feedback, directly reach out to Jebarson ( http://jebarson.info / @Jebarson007)

Filed under: Windows Phone

My Upcoming Events : Virtual Tech Days and Technology Conclave Delhi 2011

Abhijit Jana — Fri, 09 Dec 2011 19:00:53 +0000

The 13th edition of Microsoft Virtual Tech Days – being held on 14th to 16th Dec 2011. I’ll be present ing a session on Web Developer Track and will talk about “Inside IIS Request Processing and ASP.NET Page Life Cycle” on 15th Dec 2011 from 3:00pm – 4:00pm (IST) . Understanding the ASP.NET Page Life Cycle is an essential knowledge for developing ASP.NET Web Application, and before Page Life Cycle, IIS does a numbers of internal process with the request coming from client browser. During this session I will discuss how IIS process ASP.NET Requests, what happens when some client request comes to IIS, how IIS decided what kind of request it is and later in internals of ASP.NET Page lifecycle.

This is an live meeting event. You can attend it from any where. To register for this event click here . Find all the details agenda from here .

Tune in for a demo filled session.

Technology Conclave Delhi 2011

Technology Conclave is the premier technical event for technology professionals in Delhi NCR interested in learning, connecting and exploring the latest Microsoft technologies. TechCon 2011 is a 2 day event (Dec 17 & 18, Venue : Auditorium III Siri Fort Auditorium August Kranti Marg New Delhi ) with Day 1 focused on Developer technologies and Day 2 focused on IT management. All the session in Technology Conclave going to be very interactive and have your questions answered by the very people who work on these technologies day-in and day-out, and who can help you plan the features and architecture in support of your business goals. TechCon Delhi came up with a numbers of latest technology talk such as HTML5, ASP.NET 4.5, Windows Azure as an agenda. Check out complete agenda here http://abhijan.me/tor7qy

I will be presenting on “Visual Studio 2011 Developer Preview and ASP.NET 4.5” . This will be a complete demo oriented session where I will be talking on few new enhancement of VS 2011 Developer preview, useful changes for web developer in Visual Studio 2011 and enhancement of ASP.NET 4.5 core runtime enhancement, data binding, new request validation along with new bundling and minification. I will also showcase how new ASP.NET 4.5 supports Async HTTPModule and HTTPHandler.

What else, if you are in Delhi don’t miss this event. Register yourself Today itself. Here is the complete details of Technology Conclave Delhi http://abhijan.me/tor7qy

See You There !!!

That’s all. This is going to be my last event for the year 2011. You can find out all of my past event details here. Will be back again with new year 2012. Wish you and your family a Merry Christmas and a prosperous New Year in year in well advance.

Cheers !

Abhijit

Filed under: .NET Community Events