Motion Detection Squid Game using ESP32 CAM & OpenCV

Overview
In this article, we will develop a
Motion Detection
project based on Squid Game using
ESP32 CAM
&
OpenCV
. With the help of the
Python program
and ESP32 Camera Module, we will develop a
Red Light – Green Light Game
. This game is inspired from a famous Netflix TV Series “
Squid Game
”. Here, we will capture the frames of the person moving using ESP32-Cam. If there is any motion detected in the live video stream when the
red light
is turned on then the person is
dead
or game over else
green light
is shown in which the person needs to
move
Red Light = No motion
&
Green Light = Motion
To get started one must have sound knowledge of
Python
, Image processing, Embedded Systems, and IoT. In this project, we will understand how to detect the motion of a person, and what all requirements are needed to run the python program. First, we will test the whole python script with a
webcam
or
internal camera
of a laptop. Later
motion detection program
is implemented with the
ESP32 CAM
. So let’s see how we can build a Motion Detection Project like a Squid Game Red Light Green Light.
You can go through the earlier
projects
where we did
Gesture Recognition
and also the
Face Recognition
project using ESP32 CAM & OpenCV.
Bill of Materials
The following is the list of Bill of Materials for making a
ESP32 CAM Motion Detection Project
. You can purchase all these components from Amazon.
S.N.
Components
Quantity
Purchase Links

ESP32-CAM Board AI-Thinker

Amazon
AliExpress

FTDI Module

Amazon
AliExpress

Micro-USB Cable

Amazon
AliExpress

Jumper Wires

Amazon
AliExpress
Motion Detection Squid Game using PC Camera
Before jumping to the project part, lets build a
Motion Detection based Squid Game
using PC
image recognition technology
Installing Python & Required Libraries
For the live stream of video to be visible on our computer we need to write a
Python script
that will enable us to retrieve the frames of the video. The first step is to
install Python
. Go to python.org and download
Python latest version
Once downloaded and installed open to the command prompt. Now we have to install a few
libraries
. For that run the following commands below one after another until all the libraries are installed.

pip
install
numpy
pip
install
opencv
python
pip
install
mediapipe
pip
install
playsound
==
1.2.2
Once you have entered these commands in the command prompt then these libraries will be installed. Now create a new folder.
Test Code/Program
Inside that folder create a new python file and paste the following code below.

import
mediapipe
as
mp
import
cv2
import
numpy
as
np
import
time
from
playsound
import
playsound
cap
=
cv2
VideoCapture
(

)
cPos
=

startT
=

endT
=

userSum
=

dur
=

isAlive
=

isInit
=
False
cStart
cEnd
=

isCinit
=
False
tempSum
=

winner
=

inFrame
=

inFramecheck
=
False
thresh
=

def
calc_sum
(
landmarkList
)
:
tsum
=

for
i
in
range
(

)
:
tsum
+=
(
landmarkList
[
i
]
x *

)
return
tsum
def
calc_dist
(
landmarkList
)
:
return
(
landmarkList
[

]
y*

landmarkList
[

]
y*

)
def
isVisible
(
landmarkList
)
:
if
(
landmarkList
[

]
visibility
>
0.7
)
and
(
landmarkList
[

]
visibility
>
0.7
)
:
return
True
return
False
mp_pose
=
mp
solutions
pose
pose
=
mp_pose
Pose
(
)
drawing
=
mp
solutions
drawing_utils
im1
=
cv2
imread
(
'im1.jpg'
)
im2
=
cv2
imread
(
'im2.jpg'
)
currWindow
=
im1
while
True
:
_
frm
=
cap
read
(
)
rgb
=
cv2
cvtColor
(
frm
cv2
COLOR_BGR2RGB
)
res
=
pose
process
(
rgb
)
frm
=
cv2
blur
(
frm
(

)
)
drawing
draw_landmarks
(
frm
res
pose_landmarks
mp_pose
POSE_CONNECTIONS
)
if
not
(
inFramecheck
)
:
try
:
if
isVisible
(
res
pose_landmarks
landmark
)
:
inFrame
=

inFramecheck
=
True
else
:
inFrame
=

except
:
(
"You are not visible at all"
)
if
inFrame
==

:
if
not
(
isInit
)
:
playsound
(
'greenLight.mp3'
)
currWindow
=
im1
startT
=
time
time
(
)
endT
=
startT
dur
=
np
random
randint
(

)
isInit
=
True
if
(
endT
startT
)
<=
dur
:
try
:
m
=
calc_dist
(
res
pose_landmarks
landmark
)
if
m
<
thresh
:
cPos
+=

(
"current progress is : "
cPos
)
except
:
(
"Not visible"
)
endT
=
time
time
(
)
else
:
if
cPos
>=

:
(
"WINNER"
)
winner
=

else
:
if
not
(
isCinit
)
:
isCinit
=
True
cStart
=
time
time
(
)
cEnd
=
cStart
currWindow
=
im2
playsound
(
'redLight.mp3'
)
userSum
=
calc_sum
(
res
pose_landmarks
landmark
)
if
(
cEnd
cStart
)
<=

:
tempSum
=
calc_sum
(
res
pose_landmarks
landmark
)
cEnd
=
time
time
(
)
if
abs
(
tempSum
userSum
)
>

:
(
"DEAD "
abs
(
tempSum
userSum
)
)
isAlive
=

else
:
isInit
=
False
isCinit
=
False
cv2
circle
(
currWindow
(
(

+

*
cPos
)

)

(

)

)
mainWin
=
np
concatenate
(
(
cv2
resize
(
frm
(

)
)
currWindow
)
axis
=

)
cv2
imshow
(
"Main Window"
mainWin
)
#cv2.imshow("window", frm)
#cv2.imshow("light", currWindow)
else
:
cv2
putText
(
frm
"Please Make sure you are fully in frame"
(

)
cv2
FONT_HERSHEY_SIMPLEX
0.8
(

)

)
cv2
imshow
(
"window"
frm
)
if
cv2
waitKey
(

)
==

or
isAlive
==

or
winner
==

:
cv2
destroyAllWindows
(
)
cap
release
(
)
break
frm
=
cv2
blur
(
frm
(

)
)
if
isAlive
==

:
cv2
putText
(
frm
"You are Dead"
(

)
cv2
FONT_HERSHEY_SIMPLEX

(

)

)
cv2
imshow
(
"Main Window"
frm
)
if
winner
==

:
cv2
putText
(
frm
"You are Winner"
(

)
cv2
FONT_HERSHEY_SIMPLEX

(

)

)
cv2
imshow
(
"Main Window"
frm
)
cv2
waitKey
(

)
Adding Images & Audio Files
Copy the following images below in the same folder and name them
im1.jpg
and
im2.jpg
respectively.
Also, download the following audio files in the same folder and rename them as
greenlight.mp3
and
redLight.mp3
accordingly.
1. Audio Red Light:
Download
2. Audio Green Light:
Download
Motion Detection Algorithm & Testing
Now to detect motion we are going to use mediapipe library using this we are able to find landmarks of a person standing in front of the camera as in the image below.
Here to detect motion we are going to use landmark no’s 24,23,28, and 27. Because if the person moves then the distance between his hip and alternate foot increases or decreases. For example, if the person moves then the distance between 23 and 28 or 24 and 27 surely changes. Thus, the motion is detected.
Now after arranging all files in a folder run the python code and the game would work for you.
Now that our Python code is running, then we move towards our firmware.
ESP32 CAM Module
The ESP32 Based Camera Module developed by
AI-Thinker
. The controller is based on a
32-bit CPU
& has a combined
Wi-Fi
+
Bluetooth/BLE
Chip. It has a built-in
520 KB SRAM
with an external
4M PSRAM
. Its GPIO Pins have support like
UART, SPI, I2C, PWM, ADC,
and
DAC
The module combines with the
OV2640 Camera Module
which has the highest Camera Resolution up to
1600 × 1200
. The camera connects to the ESP32 CAM Board using a 24 pins gold plated connector. The board supports an
SD Card
of up to
4GB
. The SD Card stores capture images.
To learn in detail about the ESP32 Camera Module you can refer to our previous
Getting Started Tutorial
ESP32-CAM FTDI Connection
The board doesn’t have a programmer chip. So In order to program this board, you can use any type of
USB-t