Quick look at stereo USB webcam and stereo vision

Share

Stereo vision isn't something new. It can be used to get 3D data out of imaged scene which can have various usage cases in robotics, science and alike. Basic stereo vision kit would consist of two webcams mounted in one line with some distance from each other. Rarely you could find pre-built webcams of such type.

In this review I'll look at a stereo-webcam KYT-U200-SNF01ND that integrates two sensors into one output image that functions as one camera.

Dual sensor stereo webcams

I got the KYT-U200-SNF01ND webcam from Aliexpress for around $75. Few models is available including bit more expensive USB3 ones. Those webcams consist of two sensors and one USB connector - image from both sensors is combined and returned as one data stream.

The mounting rails allow us to change the distance between sensors as well as mount one of them on the opposite side of the board for a ~360 field of view (non-stereo in such case). For non stereo usages some variants can have a mono and a color sensor, one suited for daylight reinsurance and the other for night time low light condition video surveillance (there is more shops with those dual sensor cameras).

KYT-U200-SNF01ND

This particular model uses two 4MP OmniVision OV4689 sensors that can offer up to 2688 x 1520 at 90 fps or 1920 x 1080 at 120 fps. With USB2 it will be much less. USB3 variants allow for faster framerate.

The webcam is detected and handled like a regular webcam. No drivers or special software needed. Frames are placed side by side in a vertical orientation. Exposure settings change independently and after few seconds usually end up identical or very close.

For more detailed imaging I used SharpCap which offers manual control of all detected settings options. The camera can output the image in various resolutions with much slower max framerate on largest resolutions. The output can be in MJPEG or for smaller resolution also YUY2 (at a much lower framerate).

On larger resolutions compression artifacts are clearly visible so either the camera increases the compression factor or just upscales source image.

Disparity maps

One of usage cases for such images is to get disparity maps - getting apparent pixel difference or motion between a pair of stereo images - how close things are.

I've tried to used few OpenCV solutions to generate them in Python. A lot depends on getting the correct settings. You can get my source files from Github repository. Below are two examples I got:

The most basic OpenCV script would look like this:

import cv2
from matplotlib import pyplot as plt


class DisparityMap:
    def draw(self, left_image, right_image, map_path):
        left = cv2.imread(left_image)
        left = cv2.cvtColor(left, cv2.COLOR_BGR2GRAY)
        right = cv2.imread(right_image)
        right = cv2.cvtColor(right, cv2.COLOR_BGR2GRAY)

        stereo = cv2.StereoBM_create(
            numDisparities=64, blockSize=5
        )
        disparity = stereo.compute(left, right)
        plt.imshow(disparity, 'gray')
        plt.savefig(map_path, dpi=199)
        plt.close()

This however can be extended upon with additional processing or different generation algorithm.

Distance measurement

Stereo webcams can also be used to measure distance from objects. This does require some preparation and calibration as show by the LearnTechWithUs/Stereo-Vision or erget/StereoVision projects. Due to current situation I can't make a chessboard for calibration that easily.

Also do note that most of such code will assume two webcams, not one. The code would have to be modified to generate two data streams from one.

There are other similar projects on GitHub like aliyasineser/stereoDepth or disparity map generation tutorial on albertarmea.com or making a stereo camera from two separate webcams on rdmilligan.wordpress.com.

Software and usage cases

I'll be trying more options at disparity map generation as well as try getting distance measurements. If you know any software or library that could be used or have ideas for usage cases feel free to comment.

Comment article