ESP32 CAM Object Detection & Identification with OpenCV

Overview: ESP32 CAM Based Object Detection & Identification
This tutorial introduces the topic of
ESP32 CAM Based Object Detection & Identification
with
OpenCV
. OpenCV is an open-sourced image processing library that is very widely used not just in industry but also in the field of research and development.
Here for object detection, we have used the
cvlib Library
. The library uses a pre-trained AI model on the COCO dataset to detect objects. The name of the pre-trained model is YOLOv3.
In this tutorial, we will go through its features, pins description and the method to program ESP32 Camera Module using FTDI Module. We will also set up the Arduino IDE for the ESP32 Camera Module. We will also upload the
firmware
and then work on the
object detection & identification
part. The script for object detection is written in the python programming language, thus we will also have to
install Python
and its required
Libraries
In an earlier ESP32 CAM Based project we learned about
Face Detection System
& also
Color Detection System
using Python & OpenCV. This project also requires the use of
OpenCV
for Object Detection & Identification.
Bill of Materials
The following is the list of Bill of Materials for building an
ESP32 CAM Based Object Detection & Identification System
. TheESP32 CAM when combined with other
hardware
&
firmware
track and identify the object.You can purchase all these components from Amazon.
S.N.
Components
Quantity
Purchase Links

ESP32-CAM Board AI-Thinker

Amazon
AliExpress

FTDI Module

Amazon
AliExpress

Micro-USB Cable

Amazon
AliExpress

Jumper Wires

Amazon
AliExpress
ESP32 CAM Module
The ESP32 Based Camera Module developed by
AI-Thinker
. The controller is based on a
32-bit CPU
& has a combined
Wi-Fi
+
Bluetooth/BLE
Chip. It has a built-in
520 KB SRAM
with an external
4M PSRAM
. Its GPIO Pins have support like
UART, SPI, I2C, PWM, ADC,
and
DAC
The module combines with the
OV2640 Camera Module
which has the highest Camera Resolution up to
1600 × 1200
. The camera connects to the ESP32 CAM Board using a 24 pins gold plated connector. The board supports an
SD Card
of up to
4GB
. The SD Card stores capture images.
To learn in detail about the ESP32 Camera Module you can refer to our previous
Getting Started Tutorial
ESP32-CAM FTDI Connection
The board doesn’t have a programmer chip. So In order to program this board, you can use any type of
USB-to-TTL Module
. There are so many
FTDI Module
available based on
CP2102
or
CP2104
Chip or any other chip.
Make a following
connection between FTDI Module and ESP32 CAM
module.
ESP32-CAM
FTDI Programmer
GND
GND
5V
VCC
U0R
TX
U0T
RX
GPIO0
GND
Connect the
5V
&
GND
Pin of ESP32 to 5V & GND of FTDI Module. Similarly, connect the
Rx
to
UOT
and
Tx
to
UOR
Pin. And the most important thing, you need to short the
IO0
and
GND
Pin together. This is to put the device in
programming mode
. Once programming is done you can remove it.
Project PCB Gerber File & PCB Ordering Online
If you don’t want to assemble the circuit on a breadboard and you want PCB for the project, then here is the PCB for you. The PCB Board for ESP32 CAM Board is designed using
EasyEDA
online Circuit Schematics & PCB designing tool. The PCB looks something like below.
The Gerber File for the PCB is given below. You can simply download the Gerber File and order the PCB from
ALLPCB
at
1$
only.
Download Gerber File:
ESP32-CAM Multipurpose PCB
You can use this Gerber file to order high quality PCB for this project. To do that visit the
ALLPCB
official website by clicking here:
https://www.allpcb.com/
You can now upload the Gerber File by choosing the Quote Now option. From these options, you can choose the Material Type, Dimensions, Quantity, Thickness, Solder Mask Color and other required parameters.
After filling all details, select your country and shipping method. Finally you can place the order.
You can assemble the components on the PCB Board.
Installing ESP32CAM Library
Here we will not use the general
ESP webserver example
rather another streaming process. Therefore we need to add another ESPCAM library. The esp32cam library provides an object oriented API to use
OV2640 camera
on
ESP32 microcontroller
. It is a wrapper of
esp32-camera library
Go to the following
and download the zip library as in the image
Once downloaded add this zip library to Arduino Libray Folder. To do so follow the following steps:
Open
Arduino
->
Sketch
->
Include Library
->
Add .ZIP Library
… -> Navigate to downloaded zip file ->
add
Source Code/Program for ESP32 CAM Module
Here is a
for Object Detection & Identification with ESP32 Camera & OpenCV. Copy the code and paste it in the Arduino IDE.

#include <WebServer.h>
#include <WiFi.h>
#include <esp32cam.h>
const
char
*
WIFI_SSID
=
"ssid"
;
const
char
*
WIFI_PASS
=
"password"
;
WebServer
server
(

)
;
static
auto
loRes
=
esp32cam
::
Resolution
::
find
(

)
;
static
auto
midRes
=
esp32cam
::
Resolution
::
find
(

)
;
static
auto
hiRes
=
esp32cam
::
Resolution
::
find
(

)
;
void
serveJpg
(
)
{
auto
frame
=
esp32cam
::
capture
(
)
;
if
(
frame
==
nullptr
)
{
Serial
println
(
"CAPTURE FAIL"
)
;
server
send
(

""
""
)
;
return
;
}
Serial
printf
(
"CAPTURE OK %dx%d %db\n"
frame
->
getWidth
(
)
frame
->
getHeight
(
)
static_cast
<
int
>
(
frame
->
size
(
)
)
)
;
server
setContentLength
(
frame
->
size
(
)
)
;
server
send
(

"image/jpeg"
)
;
WiFiClient
client
=
server
client
(
)
;
frame
->
writeTo
(
client
)
;
}
void
handleJpgLo
(
)
{
if
(
!
esp32cam
::
Camera
changeResolution
(
loRes
)
)
{
Serial
println
(
"SET-LO-RES FAIL"
)
;
}
serveJpg
(
)
;
}
void
handleJpgHi
(
)
{
if
(
!
esp32cam
::
Camera
changeResolution
(
hiRes
)
)
{
Serial
println
(
"SET-HI-RES FAIL"
)
;
}
serveJpg
(
)
;
}
void
handleJpgMid
(
)
{
if
(
!
esp32cam
::
Camera
changeResolution
(
midRes
)
)
{
Serial
println
(
"SET-MID-RES FAIL"
)
;
}
serveJpg
(
)
;
}
void
setup
(
)
{
Serial
begin
(
115200
)
;
Serial
println
(
)
;
{
using
namespace
esp32cam
;
Config
cfg
;
cfg
setPins
(
pins
::
AiThinker
)
;
cfg
setResolution
(
hiRes
)
;
cfg
setBufferCount
(

)
;
cfg
setJpeg
(

)
;
bool
ok
=
Camera
begin
(
cfg
)
;
Serial
println
(
ok
?
"CAMERA OK"
:
"CAMERA FAIL"
)
;
}
WiFi
persistent
(
false
)
;
WiFi
mode
(
WIFI_STA
)
;
WiFi
begin
(
WIFI_SSID
WIFI_PASS
)
;
while
(
WiFi
status
(
)
!=
WL_CONNECTED
)
{
delay
(

)
;
}
Serial
(
"http://"
)
;
Serial
println
(
WiFi
localIP
(
)
)
;
Serial
println
(
" /cam-lo.jpg"
)
;
Serial
println
(
" /cam-hi.jpg"
)
;
Serial
println
(
" /cam-mid.jpg"
)
;
server
on
(
"/cam-lo.jpg"
handleJpgLo
)
;
server
on
(
"/cam-hi.jpg"
handleJpgHi
)
;
server
on
(
"/cam-mid.jpg"
handleJpgMid
)
;
server
begin
(
)
;
}
void
loop
(
)
{
server
handleClient
(
)
;
}
Before Uploading the code you have to make a small change to the code. Change the
SSID
and
password
variable and in accordance with your WiFi network.
Now
compile
and
upload
it to the ESP32 CAM Board. But during uploading, you have to follow few steps every time.
Make sure the
IO0 pin
is shorted with the ground when you have pressed the upload button.
If you see the dots and dashes while uploading press the
reset button
immediately
Once the code is uploaded, remove the I01 pin shorting with Ground and press the reset button once again.
If the output is the
Serial monitor
is still not there then press the reset button again.
Now you can see a similar output as in the image below.
Here, copy the IP address visible, we will be using it to edit the URL in python code
Python Libra