DIY Object Detection Doodle camera with Raspberry Pi (part 2)

Posted by: Chengwei 5 years, 8 months ago

(Comments)

The previous post walked through the first several parts of this project, namely the voice activation and object detection. This time, we will wrap up the remaining software and hardware components to build a complete object detection doodling camera.

• Doodle the detected objects
• Prints the drawing with a mini thermal receipt printer
• Add a shutter push button, and an indicator LED to your Pi

Drawing a detected object

After the previous object detection step, the bounding box, class, and score for each detection will be available. The size and center location of an object can be further obtained from the bounding box values. The trick of creating the almost non-repetitive and high-resolution drawings is to apply the Google quick draw datasets where each drawing is recorded as stroke coordinates on a canvas. Take a look at 103,031 cat drawings made by real people on the internet here.

The following code snippet shows you how you can turn the first "cat" drawing strokes into an image.

import gizeh as gz
from matplotlib import pyplot as plt
from PIL import Image
%matplotlib inline

from drawing_dataset import DrawingDataset
dataset = DrawingDataset('./data/quick_draw_pickles/', './data/label_mapping.jsonl')
dataset.setup()

strokes = dataset.get_drawing('cat', 0)
lines_list = []
for stroke in strokes:
    x, y = stroke
    points = list(zip(x, y))
    line = gz.polyline(points=points, stroke=[0, 0, 0], stroke_width=6)
    lines_list.append(line)
lines  =gz.Group(lines_list)
surface = gz.Surface(width=300, height=300, bg_color=(1, 1, 1))
lines.draw(surface)
plt.imshow(surface.get_npimage())

The dataset.get_drawing() method located in ./drawing_dataset/drawingdataset.py file gets a drawing by name and index. In this case, we get the name "cat" with index 0 corresponding to the first cat drawing strokes. Inside each stroke, there are multiple points, they are connected together and drawn on a canvas as a polyline by the Gizeh Python library.

For example, the second stroke for this cat has 3 points drawn from right to left.

Since the SSD lite MobileNet V2 object detection model can only detect limited categories of objects while there are 50 million drawings across 345 categories on quick draw dataset, I decided to pre-process the original dataset and keep the first 100 drawings for each recognizable class. Each class' drawings are saved as a much easier to read Python 3 pickle format to path ./data/quick_draw_pickles/.

Prints with a mini thermal receipt printer

Thermal printers are also known as receipt printers. They're what you get when you go to the ATM or grocery store. It works by heating up dots on thermal paper which is coated with material formulated to change color when exposed to heat, so there is no need for ink.  This printer is ideal for small embedded systems such as Raspberry Pi and even microcontrollers since it communicates with the application processor by TTL serial port and powered by 5-9 VDC @ 1.5Amp during print. As long as your system have an extra serial port, there is no additional printer driver necessary, which means you can use the Python printer library across OS, whether it is Ubuntu PC, Windows PC or Raspberry Pi.

My Python printer library has several improvements compared to the original Adafruit library for both the speed and stability when printing large bitmap images.

An image will first be converted to a binary bitmap then printed row by row. The "denser" a row is, the more heating time it is required by the printer otherwise blank lines could appear. Dynamically calculating the printing time for each line achieves optimal speed and stability.

Take this 8 x 8 binary bitmap as an example, row 1 has 4 dots to print and row 5 has only 2 dots, it requires a little bit longer to print row 1 than row 5. Since the maximum width can be printed on the thermal paper is 384 pixels, we will first flip the drawing image by 90 degree to print it vertically line by line.

It is also necessary to adjust its baud rate to 115200 from its default 19200 or 9600 to achieve faster bitmap printing on a thermal printer since sending a complete 384 by 512 bitmap through a serial port takes a lot of bandwidth. I have attached a tool to complete this on my GitHub.

Add shutter button and indicator LED

Voice activation without any response or feedback can ruin the user experience, that is why we add a single LED to blink in different patterns to show whether a voice command is detected. An additional push button also provides an extra option to trigger the camera capture, object detection, drawing, and printing workflow.

The gpiozero Python library came with the Raspberry Pi system provide a quick solution to interface with its IO pins. It allows you to pulse or blink LEDs in a variety of fashions and handle the timing using a background thread without further interaction. A push button can trigger a function call in simply two lines. 

button = Button(button_pin)
button.when_pressed = callback_function

All the GPIO usages are defined in ./raspberry_io/raspberryio.py file in case you want to change how the LED blinks and pluses.

Going cordless, power by battery

While it is possible to obtain a powerful 5V power bank supply current at 3Amp, it might be too chunky to fit inside our camera box. Using a compact two cells 7.4 v lipo battery to power up Raspberry Pi and the thermal printer at the same time seems like a variable way to go. You can power the thermal printer directly by the 7.4 V battery while Raspberry Pi take 5V power supply, that is where the DC-DC buck module comes to play. It is a switching power supply steps down voltage from its input to its output. I am using an LM2596 DC to DC Buck Converter as you can see on Amazon. 

There is a little potentiometer on the module to adjust the output voltage. With the battery connected to its input power and ground pins, turn the little nob on the potentiometer while monitoring the output voltage with a multimeter until it reaches 5V.

Conclusion and further reading

Here is the final complete diagram shows you how parts are connected.

Feel free to leave a comment if you have a question about building your object detection and doodling camera.

• ← DIY Object Detection Doodle camera with Raspberry Pi (part 1)
• "If I were a girl" - Magic Mirror by StarGAN →