How to measure the latency of a WebCam with OpenCV

Posted by: Chengwei 6 years ago

(Comments)

In this quick tutorial, you will learn how to measure the latency of your webcam to make the assessment whether it is capable to handle image capturing task with a higher real-time demand compared to doing a still image capturing.

Picture a beverage producing line filling bottles rapidly along a conveyor. A camera is deployed to validate the quality of each bottle like the volume of liquid and shape, every defected product must be sorted out. The real-time requirement for the image capturing is critical since any elongated latency could result in a different bottle being diverted in the wrong path.

Where the latency comes from?

It might seem simple to read a frame from your webcam by calling camera.read() in Python cv2 library, however, lots of stuff happens behind the scene which could introduce the unpredictable latency not desirable the in real-time application.

As seen from the above diagram, the captured image traveled a "long" way to reach your computer then it can be decoded and displayed on the screen.

Measure the latency

To measure the latency between when the actual image is captured and displayed on your screen, we are going to whip up a simple Python script to read the image from the camera as fast as we can and flip the display color as soon as the previous color change is detected at the end of the pipeline.

The previous sequence demonstrates that the latency might be longer than several frames.

Here is the logic diagram for this experiment.

The complete script shown below.

"""
Run this script then
point the camera to look at the window,
watch the color flips between black and white.
Slightly increase "THRESHOLD" value if it doesn't flip.
"""

import cv2
import numpy as np

# Initialize USB webcam feed
CAM_INDEX = 0
# Adjust this value if it doesn't flip. 0~255
THRESHOLD = 50
# Set up camera constants
IM_WIDTH = 1280
IM_HEIGHT = 720
# IM_WIDTH = 640
# IM_HEIGHT = 480

### USB webcam ###
camera = cv2.VideoCapture(CAM_INDEX)
if ((camera == None) or (not camera.isOpened())):
    print('\n\n')
    print('Error - could not open video device.')
    print('\n\n')
    exit(0)
camera.set(cv2.CAP_PROP_FRAME_WIDTH, IM_WIDTH)
camera.set(cv2.CAP_PROP_FRAME_HEIGHT, IM_HEIGHT)
# save the actual dimensions
actual_video_width = camera.get(cv2.CAP_PROP_FRAME_WIDTH)
actual_video_height = camera.get(cv2.CAP_PROP_FRAME_HEIGHT)
print('actual video resolution:{:.0f}x{:.0f}'.format(actual_video_width, actual_video_height))

prev_tick = cv2.getTickCount()
frame_number, prev_change_frame = 0, 0
is_dark = True


while True:
    frame_number += 1

    _, frame = camera.read()
    img = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)

    is_now_dark = np.average(img) < THRESHOLD

    if is_dark != is_now_dark:
        is_dark = is_now_dark
        new = cv2.getTickCount()

        print("{:.3f} sec, {:.3f} frames".format(
            (new - prev_tick) / cv2.getTickFrequency(),
            frame_number - prev_change_frame
        ))
        prev_tick = new

        prev_change_frame = frame_number

        fill_color = 255 if is_dark else 0
        show = np.full(img.shape, fill_color, dtype=img.dtype)

        cv2.imshow('frame', show)

    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

camera.release()
cv2.destroyAllWindows()

By pointing your camera at the popup window and watch the color flips between black and white several times per second which reflects the latency.

You can point your camera lens as close to the screen as possible, it doesn't need to have a clear focus, only the average intensity is necessary to make the color flip.

I have tested it on my Logitech C930e USB webcam and here is the result.

Two image resolutions are tested, and you can see the lower resolution produces slightly lower latency.

The value is measured in milliseconds, with the average around 200ms. Given a conveyor traveling at a constant speed of 1m/s, the bottle has already moved 20cm away.

What is the solution?

If your application demands more predictive and lower latency image capturing, one simple and effective solution is to upgrade your camera to an industrial camera with external trigger signal where a photo eye sensor or digital controlled IO can be hooked up.

In this case, an image can be captured at a more predictive timestamp either triggered by something blocking the photo eye sensor or a real-time signal from the computer.

• Tags:
• tutorial

• ← How to perform Keras hyperparameter optimization x3 faster on TPU for free
• Multi-class classification with focal loss for imbalanced datasets →