Choosing an Idea#
I really struggled to come up with an idea for a project. Most of the ideas the rest of my class came up with relied on machine learning instead of focusing on the image processing side of things.
One idea from my classmates that caught my attention was detecting driver awareness. At first this seems like a classical image classification problem but I thought that there would be a lot of inventive ways image processing could be used to solve a rudimentary version of this problem.
Near Infrared as a Solution#
One of the ways I thought of solving this problem was by using Backlit Near Infrared photography in conjunction with normal images.
It seems like an odd thing to come up with but I have always had an interest in the applications of Near Infrared and happened to have a device the Microsoft Surface Pro that had this capability built in.
IR is used throughout photography and image processing. Examples ranging from tracking your movements in VR like with the Oculus Quest 1 to face verification using Windows Hello2.
The reason a Near Infrared Camera suites this problem is that when backlit, photos of people contain a very distinct reflection in their eyes an effect very similar to cats eyes.
This type of camera also gives us the benefit of better low light performance.
The photos below were taken of me at the same time in a pitch black room showing the true value of this technology.
Getting Both IR and Color Images#
So now to the painful stuff.
Creating an application to capture these images took an absolute age.
There wasn't a single line of documentation for how to get raw access the integrate NIR camera in the Surface Pro seen as it was never intended to be used outside of Windows Hello.
There was also no easy way to stream MultipleMediaCaptureFrames not to mention the backwards way the API handles asynchronous calls
but I digress...
Eventually I got the UWP Application to work. This allowed me to capture images from both the webcam and the backlit NIR camera simultaneously and to never have to use C Sharp again.
So the next step is to see how me and the team can combine these high resolution color and NIR images accounting for the differences in resolution, aspect ratio and field of view.
Footnotes#
- Official Facebook blog post about the oculus insight technology
- Microsoft's Documentation for Windows Hello Face Authentication
- Cats Eyes Image:
- This file is licensed under Creative Commons Attribution-Share Alike 3.0 Unported and is by Biswapathik