I am very happy and pleased to announce the availability of my book “Kinect for Windows SDK Programming Guide”. The purpose of this book is to explain how to develop applications using the Kinect for Windows SDK. If you are a beginner and looking to start developing applications using the Kinect for Windows SDK, and if you want to build motion-sensing , speech-recognizing applications with Kinect, this book is for you.  This book uses C# and WPF in the examples, so you need to know the basics of C# and WPF. If you are a student with basic C# programming knowledge and having basic understanding of WPF and interested to learn about Kinect SDK, you can also read this book.

Quick Overview:

• Building application using Kinect for Windows SDK.
• Covers the Kinect for Windows SDK v1.6
• A practical step-by-step tutorial to make learning easy for a beginner.
• A detailed discussion of all the APIs involved and the explanations of their usage in detail
• Procedures for developing motion-sensing applications and also methods used to enable speech recognition

What you will learn from this book

• Understand the Kinect sensor in terms of a device, different components, and how they work.
• Step by step installation guide of SDK, troubleshooting tips and development environment setup.
• Reading the Kinect device information, monitoring and notifying the sensor status change.
• Automatically adjust the sensor elevation angle
• Understand different approaches of capturing data stream from Kinect sensor.
• Learn the uses of color, depth and IR data and how to leverage them in your applications.
• Use skeleton-tracking for interactive applications and leveraging the applications to gesture-enabled application
• Understand usages of Microphone array and build speech-enabled applications with Speech API
• Develop applications using Multiple Kinects
• Understand how to design cross device application with Kinect
  

  What This Book Covers

  • Understanding the Kinect Device   [ Download the PDF version of this chapter ]
    

    Introduces Kinect as a hardware device. You will get an insight into the different components that make up Kinect and the technology behind this device, which makes the components work together. This chapter will also give an overview of the difference between Kinect for Xbox and Kinect for Windows sensor. You will also become familiar with different possibilities of domain specific applications that can be developed using the Kinect sensor.

  • Getting Started
    

    Introduces the Kinect for Windows SDK, its features, and how to start working with the Kinect sensor. In this chapter, you will get to know about the requirements for preparing your development environment. This will also walk you through a step-by-step guide for downloading and installing the SDK. You will delve into the installed components to verify that everything is set up properly. This chapter will also provide you with a quick lap around the different features of the Kinect for Windows SDK as well as introduce the Kinect for Windows Developer Toolkit.

  • Starting to Build Kinect Applications
    

    Explains the step-by-step process of building your first Kinect-based application. You will understand how applications interact with the Kinect sensor using the SDK libraries. This chapter will give you an in-depth guide on how to start building Kinect applications using the Kinect for Windows SDK and Visual Studio. You will also learn how to deal with applications when there is any change in the device status.

  • Getting the Most out of Kinect Camera
    

    Getting the Most Out of Kinect Camera, covers the in-depth discussion of the Kinect color camera and how to use it. In this chapter, you will learn about the different types of image streams and different approaches to retrieve them from the Kinect sensor. You will get an understanding of Color camera stream pipeline and its events. You will also explore the different features of the Kinect for Windows SDK that control the color camera and process the color data. This chapter will give you an understanding of processing color images and applying different effects to the captured images and how to save the image frames. You will also learn how you can use the Kinect camera to capture images in low light.

  • The Depth Data – Making Things Happen
    

    Explores the fundamentals of the Kinect depth sensors and how they produce depth information. This chapter describes how to work with object distances and player indices from the captured depth data. You  will also learn about the capturing of data using the near mode and also get a quick view of generating 3D depth data.

  • Human Skeleton Tracking

  Describes how a Kinect sensor tracks the human skeleton and how you can leverage the features of the Kinect for Windows SDK to play around with tracked skeletons and joints. You will also learn how to change the sensor elevation angle based on the player position. This chapter also explores how skeletons can be tracked in a seated mode. You also learn about details of the skeleton joints and bone hierarchy. The sample application in this chapter will help you to understand the APIs for skeleton tracking in better ways such as using Kinect as an intrusion detector. At the end of this chapter, you will be familiar with a few debugging tips and tricks to boost your development speed.

  • Using Kinect’s Microphone Array
    

    Introduces the microphone array that captures and processes the audio signal. You will learn why Kinect uses an array of microphones rather than a single microphone. In this chapter you will get an insight into the Kinect audio processing pipeline that helps Kinect to capture good-quality audio signals and makes Kinect a highly directional audio device. This chapter provides you with information on how to capture and record audio signals using the Kinect microphone array and process the audio data for better quality. You will also learn about different concepts such as Noise Suppression, Automatic Gain Control, Echo Cancellation, and Beam forming.

  • Speech Recognition

  Introduces the building of speech-enabled applications using Kinect. You will explore how speech recognition works and how Kinect’s microphone array helps Kinect to recognize human speech. This chapter also shows how you can use Kinect as the default speech recognition device for your PC. You will also learn about the Microsoft Speech API and how it is integrated with Kinect for Windows SDK, which helps us to build speech-enabled applications.

  • Building Gesture-controlled Applications

  Building Gesture-controlled Applications, describes how to build applications that can be controlled by human gestures. You will learn different approaches for recognizing gestures and how to apply these approaches in the form of programs to build motion-sensing applications using the Kinect sensor. This chapter will also help you understand how to build some gestured-enabled controls.

  • Developing Applications Using Multiple Kinects

  Explains how multiple Kinect sensors can be placed together and used to build applications. This chapter describes how to set up environments for developing applications using multiple Kinects and walks you through building applications by reading data from multiple devices. You also learn how multiple Kinects work together and different scenarios where multiple Kinects can be used, along with the challenges while developing applications using multiple devices.

  • Putting Things Together

  Introduces us to more advanced developments using Kinect by integrating it with other devices such as Windows Phone, microcontrollers, and so on. This chapter addresses how we can take things up from Kinect to Windows Azure and control the Kinect sensor using Windows Phone via Windows Azure. You will also learn how Kinect can be integrated with the Netduino microcontroller and how you can use a Kinect device for face tracking.

I would like to thanks all of you for your fantastic response on my Kinect Tutorial, Kinect seminars and my other Kinect works, which actually encouraged me to put something more exciting and together in a form of book. Thanks to the Packt publishing for offering me the opportunity to write for them. I couldn’t have written this book without technical help of  the many people and the community. Over the past one year, I was fortunate enough to present many seminars and conference on Kinect. This helped me to interact with many developers, students who are really passionate about programming with Kinect. I also met with many professors who have been researching on computer vision and Kinect technology. I had a great time to talk to each an individual; I have learned a lot. Sincere thanks to the complete editing team, and the reviewers (Atul Gupta, Anoop Madhusdanan and Atul Verma) who helped me to identify and fill the gaps and improve the overall content of this book. My friend, Atul Verma, not only did the technical review of this book, he worked with me on creating demos, designing samples and reviewing each of the things that I did for this book. My friend Jebarson helped me a lot on various things while writing. Many thanks to Arka Bhattacharya for doing offline review of the book. Arka, not only did the review, he helped me to editing contents, given me any feedback.

I would like to thank many bloggers on the various Kinect topics and also the researchers who have been working and experimenting day in and day out with Kinect. On many occasions I have been reading theirs posts, research papers and referring to them while writing the book.

Finally, I have written this book based on my personal experiences, experiment, study, learning from others, peer coding and working on different assignments on Kinect. The information given in this book is my personal and does not reflect the opinions of my employers. If you find any mistakes or have some constructive feedback please drop me mails at abhijitjana@outlook.com. I will try my best to address then as earliest.

Being a community lover and an active blogger, I have been writing blogs, participating in different forums and seminars for the last couple of years; but this is the first I am putting something in the form of book. The credit goes to each one of you have been connected with me and have been my blog reader and supporter.

I would also like to take this opportunity to wish you and your family a very happy and prosperous new year 2013.

Best Regards,
Abhijit