ESP32 CAM Based Face & Eyes Recognition System

Overview: ESP32 CAM Face Recognition System
In this project, we will build an
ESP32 CAM Based Face & Eyes Recognition System
. This tutorial introduces everyone to an efficient
video streaming
method wirelessly. Here we have used the
ESP32-CAM module
, which is a small camera module with the
ESP32-S chip
. Besides the OV2640 camera and several GPIOs to connect peripherals, it also features a microSD card slot that can be useful to store images taken with the camera.
We will go through ESP32 CAM features, pins description, and the method to program this device using FTDI Module. Then we will install the
ESP32 CAM Webserver Library
& upload the example Code using Arduino IDE. Apart from this, we will also install the required
Python Libraries
. Later we will go through the
python code
for
Face & Eyes Recognition
This is an essential tutorial as you will be able to use any sort of
Image processing
or
Machine Learning
on the live video without having to write it on Arduino IDE.
Bill of Materials
The following is the list of Bill of Materials for building an
ESP32 CAM Based Face & Eyes Recognition System
. You can purchase all these components from Amazon.
S.N.
Components
Quantity
Purchase Links

ESP32-CAM Board AI-Thinker

Amazon
AliExpress

FTDI Module

Amazon
AliExpress

Micro-USB Cable

Amazon
AliExpress

Jumper Wires

Amazon
AliExpress
ESP32 CAM Module
The ESP32 Based Camera Module developed by
AI-Thinker
. The controller is based on a
32-bit CPU
& has a combined
Wi-Fi
+
Bluetooth/BLE
Chip. It has a built-in
520 KB SRAM
with an external
4M PSRAM
. Its GPIO Pins have support like
UART, SPI, I2C, PWM, ADC,
and
DAC
The module combines with the
OV2640 Camera Module
which has the highest Camera Resolution up to
1600 × 1200
. The camera connects to the ESP32 CAM Board using a 24 pins gold plated connector. The board supports an
SD Card
of up to
4GB
. The SD Card stores capture images.
To learn in detail about the ESP32 Camera Module you can refer to our previous
Getting Started Tutorial
ESP32-CAM FTDI Connection
The board doesn’t have a programmer chip. So In order to program this board, you can use any type of
USB-to-TTL Module
. There are so many
FTDI Module
available based on
CP2102
or
CP2104
Chip or any other chip.
Make a following
connection between FTDI Module and ESP32 CAM
module.
ESP32-CAM
FTDI Programmer
GND
GND
5V
VCC (5V)
U0R
TX
U0T
RX
GPIO 0
GND
Connect the
5V
&
GND
Pin of ESP32 to 5V & GND of FTDI Module. Similarly, connect the
Rx
to
UOT
and
Tx
to
UOR
Pin. And the most important thing, you need to short the
IO0
and
GND
Pin together. This is to put the device in
programming mode
. Once programming is done you can remove it.
Project PCB Gerber File & PCB Ordering Online
If you don’t want to assemble the circuit on a breadboard and you want PCB for the project, then here is the PCB for you. The PCB Board for ESP32 CAM Board is designed using
EasyEDA
online Circuit Schematics & PCB designing tool. The PCB looks something like below.
The Gerber File for the PCB is given below. You can simply download the Gerber File and order the PCB from
ALLPCB
at
1$
only.
Download Gerber File:
ESP32-CAM Multipurpose PCB
You can use this Gerber file to order high quality PCB for this project. To do that visit the
ALLPCB
official website by clicking here:
https://www.allpcb.com/
You can now upload the Gerber File by choosing the Quote Now option. From these options, you can choose the Material Type, Dimensions, Quantity, Thickness, Solder Mask Color and other required parameters.
After filling all details, select your country and shipping method. Finally you can place the order.
You can assemble the components on the PCB Board.
Installing ESP32CAM Library
Here we will not use the general
ESP webserver example
rather another streaming process. Therefore we need to add another ESPCAM library. The esp32cam library provides an object oriented API to use
OV2640 camera
on
ESP32 microcontroller
. It is a wrapper of
esp32-camera library
Go to the following
and download the zip library as in the image
Once downloaded add this zip library to Arduino Libray Folder. To do so follow the following steps:
Open
Arduino
->
Sketch
->
Include Library
->
Add .ZIP Library
… -> Navigate to downloaded zip file ->
add
Arduino Source Code/program
The source code/program
ESP32 CAM Face Recognition
can be found in Library Example. So go to
Files -> Examples -> esp32cam -> WifiCam
You need to change your
WiFi SSID
and
Password

const
char
*
ssid
=
"*********"
;
const
char
*
password
=
"*********"
;
Now go to
tools
. Then select the
ESP32
Board. And from the list select
ESP32Wroover Module
Then connect the FTDI Module to your Computer and select the
COM Port
Now you can upload the code. But while you upload the code the IO0 pin should be shorted with the ground. Once the code is uploaded, remove the shorting Jumper and press the RESET pin.
In the Serial Monitor similar output must be shown.
Copy the IP address visible, we will be using it to edit the URL in python code.
Python Installation & Source Code
For the live stream of video to be visible on our computer we need to write a Python script that will enable us to retrieve the frames of the video. The first step is to install Python. Go to
python.org
and download Python.
Install Python once downloading is completed.
Then Go to the command prompt and install
NumPy
and
OpenCV
libraries.
type:
pip install numpy
and press enter. After the installation is done.
type:
pip install opencv-python
and press enter, close the command prompt.
Now open
Idle code editor
or any
python code editor
and paste the following code.

import
cv2
import
urllib
request
import
numpy
as
np
f_cas
=
cv2
CascadeClassifier
(
cv2
data
haarcascades
+
'haarcascade_frontalface_default.xml'
)
eye_cascade
=
cv2
CascadeClassifier
(
cv2
data
haarcascades
+
'haarcascade_eye.xml'
)
url
=
'http://192.168.1.58/cam-lo.jpg'
##'''cam.bmp / cam-lo.jpg /cam-hi.jpg / cam.mjpeg '''
cv2
namedWindow
(
"Live Transmission"
cv2
WINDOW_AUTOSIZE
)
while
True
:
img_resp
=
urllib
request
urlopen
(
url
)
imgnp
=
np
array
(
bytearray
(
img_resp
read
(
)
)
dtype
=
np
uint8
)
img
=
cv2
imdecode
(
imgnp

)
gray
=
cv2
cvtColor
(
img
cv2
COLOR_BGR2GRAY
)
face
=
f_cas
detectMultiScale
(
gray
scaleFactor
=
1.1
minNeighbors
=

)
for
y
w
h
in
face
:
cv2
rectangle
(
img
(
y
)
(
+
w
y
+
h
)
(

)

)
roi_gray
=
gray
[
y
:
y
+
h
:
+
w
]
roi_color
=
img
[
y
:
y
+
h
:
+
w
]
eyes
=
eye_cascade
detectMultiScale
(
roi_gray
)
for
(
ex
ey
ew
eh
)
in
eyes
:
cv2
rectangle
(
roi_color
(
ex
ey
)
(
ex
+
ew
ey
+
eh
)
(

)

)
cv2
imshow
(
"live transmission"
img
)
key
=
cv2
waitKey
(

)
if
key
==
ord
(
'q'
)
:
break
cv2
destroyAllWindows
(
)
Change the
IP address
copied from Arduino serial monitor and update it in the URL variable in the above code. Then save it and RUN.
ESP32 CAM Face Recognition System
After you run the code, a pop-up window with the name “
Live Transmission
” appears, showing the live video.
red-colored rectangular box covering the whole face and within that if it recognizes eyes. Then
green colored boxes
appear around the eyes.
For the
high resolution
of live
video transmission
, change the URL variable in python code by replacing it with url=’
http://192.168.1.58/cam-hi.jpg
‘.
Save and run the code again.
Benifits of Face Recognition System
The major benefit of this method is, most of our algorithms that involve any processing on images are very heavy to implement on microcontrollers and require more processing, thus making the whole system slow.
Instead, we can implement the difficult and high processing tasks on our computer such as Image processing or Artific