Opencv image processing has always been an efficient and interesting tool that was used in malicious projects, but it has always been one of these advantages to record information and make changes in the operating system, but if we can, if processing changes in the external environment From the implemented system, it will certainly be very efficient, the purpose of this project is to control the engine server if it detects a human face.
OpenCV is an open-source computer vision and machine learning software library. OpenCV is built to create a common infrastructure for computer vision applications and accelerate the use of machine perception in commercial products. OpenCV, as a BSD-licensed product, makes it easy for businesses to use and modify code. OpenCV or Open Computer Vision Library is a collection of image processing and machine learning programming libraries. This set focuses on real-time image processing. Originally developed and supported by Intel, it is now supported by Willow Garage and Itseez. Supported by Windows, Linux, Mac OS, iOS, and Android operating systems. It also has a programming interface in c / c ++, Python, Java, and MATLAB languages.
Machine Learning Pre-trained Models
In machine learning, there is a topic called model, which is actually the objects and things that we want to recognize, for example, human face recognition, so for this we have to teach our machine that the appearance of a human, female or male, Small or large, old or young. For this, we have to introduce different images to our car to increase the ability to detect our car. This result of this process is called modeling, which is the hallmark of our machine, and then all of these models must be labeled so that the result is visually understandable to the user.
Arduino Board
The Arduino board series is one of the most popular development boards among embedded engineers, which can be provided in various models such as Micro, pro mini, Nano, Uno, as well as Mega, the core of these popular boards is the AtMega328 series. Arduino is an open-source hardware and software platform. As mentioned earlier, the Arduino platform includes an open-source single-control microcontroller that forms part of the Arduino hardware. In addition, the Arduino platform includes Arduino IDE software designed for programming Arduino boards and a software bootloader that loads on the microcontroller.
How this Project Works
In the Servo Motor project with OpenCV image processing, we use two different codes, one code for Arduino which is responsible for commanding the sg90 servo and the second code is a Python code that runs in your system and changes the servo variables. Depending on the position of your face in the image, as well as with the help of Arduino pyserial and Python code, they communicate with each other.
Items Needed
- Arduino board
- Servo motor sg90
Required Libraries
To run the project, we will first install the Servo reference library in the Arduino IDE software. Follow the steps below.
- Follow this path Sketch > Include Library > Manage Libraries
- Search for the word Servo.
- Install the library.
- This library is usually installed by default.
In Python code you need to install multiple libraries in your editor environment, I use pycharm editor. Search and install the packages marked in the image. And keep in mind that for image processing we need Python version 3.7 to be used simultaneously with pyserial.
Project Schematic
In the OpenCV project with Arduino, we use two servo motors for the length and width of the image center with the detected face, one of which is connected to pin 9 and the other to pin 10 in Arduino so that it moves horizontally and vertically. Align your face in the center of the image and follow it.
Project Codes
-
Arduino Code
- In this part of the code, using the serial protocol, the coordinates recognized by the face in the image or x and y are read in Arduino.
|
1 2 3 4 5 6 7 8 9 |
if (Serial.available() > 0) { int x_mid, y_mid; if (Serial.read() == 'X') { x_mid = Serial.parseInt(); if (Serial.read() == 'Y') y_mid = Serial.parseInt(); } |
- This part of the code has the task of setting the servo in the square area if the coordinates are outside the range of its values.
|
1 2 3 4 5 6 7 8 |
if (x_mid > width / 2 + 30) xpos += angle; if (x_mid < width / 2 - 30) xpos -= angle; if (y_mid < height / 2 + 30) ypos -= angle; if (y_mid > height / 2 - 30) ypos += angle; |
- This part of the code, like the above code, is responsible for setting the servo with the square in the image, with the difference that if the degree of the servo is outside the range of values.
|
1 2 3 4 5 6 7 8 |
if (xpos >= 180) xpos = 180; else if (xpos <= 0) xpos = 0; if (ypos >= 180) ypos = 180; else if (ypos <= 0) ypos = 0; |
- Ready to upload code on Arduino.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 |
#include<Servo.h> Servo x, y; int width = 640, height = 480; int xpos = 180, ypos = 180; void setup() { Serial.begin(9600); x.attach(9); y.attach(10); x.write(xpos); y.write(ypos); } const int angle = 2; void loop() { if (Serial.available() > 0) { int x_mid, y_mid; if (Serial.read() == 'X') { x_mid = Serial.parseInt(); if (Serial.read() == 'Y') y_mid = Serial.parseInt(); } if (x_mid > width / 2 + 30) xpos += angle; if (x_mid < width / 2 - 30) xpos -= angle; if (y_mid < height / 2 + 30) ypos -= angle; if (y_mid > height / 2 - 30) ypos += angle; if (xpos >= 180) xpos = 180; else if (xpos <= 0) xpos = 0; if (ypos >= 180) ypos = 180; else if (ypos <= 0) ypos = 0; x.write(xpos); y.write(ypos); } } |
-
Python Code
- In the first part of the Python code, we specify the camera used, if there is no image, change the existing number.
|
1 |
cap=cv2.VideoCapture(2) |
- In this section, we specify the Arduino port we are using, by changing the value in the following code.
|
1 |
ArduinoSerial=serial.Serial('com6',9600,timeout=0.1) |
- This line of code is responsible for sending the x and y values obtained from the image to the Arduino in order to move the servo.
|
1 |
string='X{0:d}Y{1:d}'.format((x+w//2),(y+h//2)) |
- Python code is ready to run in the intended editor.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 |
import cv2 import serial,time face_cascade= cv2.CascadeClassifier('haarcascade_frontalface_default.xml') cap=cv2.VideoCapture(2) ArduinoSerial=serial.Serial('com6',9600,timeout=0.1) time.sleep(1) while cap.isOpened(): ret, frame= cap.read() frame=cv2.flip(frame,1) gray = cv2.cvtColor(frame,cv2.COLOR_BGR2GRAY) faces= face_cascade.detectMultiScale(gray,1.1,6) for x,y,w,h in faces: string='X{0:d}Y{1:d}'.format((x+w//2),(y+h//2)) print(string) ArduinoSerial.write(string.encode('utf-8')) cv2.circle(frame,(x+w//2,y+h//2),2,(0,255,0),2) cv2.rectangle(frame,(x,y),(x+w,y+h),(0,0,255),3) cv2.rectangle(frame,(640//2-30,480//2-30), (640//2+30,480//2+30), (255,255,255),3) cv2.imshow('img',frame) if cv2.waitKey(10)&0xFF== ord('q'): break cap.release() cv2.destroyAllWindows() |
Note: Note that image processing in this project requires predefined models, and in our project, a model for recognizing the image of a human face, for this purpose, use the following file next to the Python code. frontface model
Final Result
Upon uploading the code to the Arduino and executing the Python code, if all of the above is done correctly, a webcam page editor will open for you that will be able to recognize your face, and in the meantime, the values for the Arduino will be serialized. Will be sent, and according to the existing conditions, Arduino will try to change the position of the servos according to the movement of your face in the image by commanding the servo.
The Last Word with Cifer
If you encounter any problems during the setup and implementation of this project, do not hesitate to ask your question at the end of this post. I will answer your problem as soon as possible. Also, if there is a problem in the code or the execution steps, you can inform through this way.