Building an API with Dlib + Python + Heroku
This article aims to build an API for face recognition using Dlib library for Python. To create the API, we are going to use Flask framework for Python 3. For the purpose of being simple, we are going to deploy this API into Heroku environment, which provides simple steps that we can easily follow, even not being any expert in the area. Docker will help us with our heaviest dependencies. Shall we start?
Dlib
Dlib is a library built in C++ which provides a variety of machine learning algorithms, and some of them are related to image processing. The library provides an API for python. In this article, we are going to use Dlib to recognise the points inside a face. We are also going to use OpenCV, which is another library for face recognition, and Dlib will help gathering the points in the face detected.
Flask
Flask is a microframework to build web applications in Python. It’s very easy to install, to use, and doesn’t require extensive configuration as we have in more complete frameworks.
For the face detection, we need a face_recognition_service.py file. We can use structure similar to this:
import cv2
import dlib
import base64
from imutils import resize, face_utils
from urllib.parse import quotedetector = dlib.get_frontal_face_detector()
predictor = dlib.shape_predictor("shape_predictor_68_face_landmarks.dat")def get_image_with_landmarks(file_path: str):
rects = None
gray = None
clone = None
try:
image = cv2.imread(file_path, 1)
image = resize(image, height=240)
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
clone = image.copy()
rects = detector(gray, 1)
except Exception:
return {'error': 'Error while reading the image'}
any_face_was_found = len(rects) > 0
if any_face_was_found:
for (i, rect) in enumerate(rects):
shape = predictor(gray, rect)
shape = face_utils.shape_to_np(shape)
for point in range(1, 68):
coords = shape.tolist()[point]
cv2.circle(clone, (coords[0], coords[1]), 1, (0, 0, 255), thickness=2)
else:
return {'error': 'No face was detected in the image provided'}
retval, buffer = cv2.imencode('.jpg', clone)
image_as_text = base64.b64encode(buffer)
return {'image_with_landmarks': 'data:image/png;base64,{}'.format(quote(image_as_text))}
In order to use Dlib for face landmarks recognition, we need to provide a trained model named shape_predictor_68_face_landmarks.dat. We can obtain this file from the website, or in the repository.
For the Flask framework, we need to define a controller app.py file, like this:
import os
from flask import Flask, render_template, request
from face_recognition_service import get_image_with_landmarks
app = Flask(__name__)
app.config['UPLOAD_FOLDER'] = './'
@app.route('/', methods=["GET", "POST"])
def index():
if request.method == "GET":
return render_template("index.html", result={})
else:
image = request.files["image"]
path = os.path.join(app.config['UPLOAD_FOLDER'], image.filename)
image.save(path)
result_from_landmarks = get_image_with_landmarks(path)
os.remove(path)
return render_template("index.html", result=result_from_landmarks)
if __name__ == '__main__':
port = int(os.environ.get("PORT", 5000))
app.run(host='0.0.0.0', port=port)For the UI, we can provide a simple HTML. This file should be inside a templates folder, and we used index.html for the filename. For the annotations that we use inside the HTML, we can look for Jinja documentation.
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Testing API with Dlib</title>
</head>
<body>
<h2>Face recognition with Dlib</h2>
<form method="post" enctype="multipart/form-data">
<input type="file" name="image" placeholder="Upload an image">
<button>Send</button>
</form>
<div style="background: #f6f6f6; padding: 20px; margin-top: 20px;">
<strong>Result with landmarks</strong>
{% if result and result.error %}
<p>{{ result.error }}</p>
{% elif result and result.image_with_landmarks %}
<p><img src="{{ result.image_with_landmarks }}" alt="face with landmarks"/></p>
{% else %}
<p>The results will be placed here</p>
{% endif %}
</div>
</body>
</html>Heroku
Heroku is a cloud platform that helps us to deploy, create pipelines for different environments, monitor our apps, among many other things. It’s easy to use and to configure.
For this configuration, we need to provide a requirements.txt file. In this file, we can add any dependency that we have in the project. We can see that Flask is the main dependency in our case.
click==6.7
Flask==1.1.2
numpy==1.20.1
Dlib==19.21.1
opencv-python==4.5.1.48
itsdangerous==0.24
Jinja2>=2.10.1
MarkupSafe==1.0
Werkzeug>=0.15
imutils==0.5.3For Heroku, we usually provide a Procfile in order to define routines to start the application. In our case, Procfile is not needed because we are going to use another configuration file to build with Docker.
The reason why we don’t simply deploy this only as an ordinary git repository on Heroku is that Dlib takes a considerable time to be installed in a cloud environment, due to a lot of sub-dependencies. Dlib uses c++ dependencies such as Cmake, Boost and XQuartz. In fact, when Heroku tries to install Dlib using Procfile, it takes a lot of time trying to install and doesn’t complete at all. It’s a machine learning library, so it’s okay to expect that.
Docker
We use Docker to build independent containers with an operation system and the softwares that we need in our applications. These components are independent, powerful, lightweight, and we can benefit of not needing to manually set virtual machines in the cloud, like we used to do some time ago.
For that, we need to set a Dockerfile in our project, as it follows:
FROM jhonatans01/python-dlib-opencv
COPY . /app
WORKDIR /app
RUN pip3 install -r requirements.txt
CMD ["python3", "app.py"]In this file, we pull an image from the DockerHub named jhonatans01/python-dlib-opencv. This image contains the Dlib + OpenCV dependencies using Python 3.6. Basically, in these 5 lines of code we are:
- Pulling this image
- Copying everything in the project to the app folder
- Setting this app folder as our workspace
- Installing our dependencies (other than Dlib or OpenCV) with pip
- Running our python main file for Flask, which is app.py.
For Heroku configuration, we still need a heroku.yml file, in order to define that this is an app to be deployed with Docker every time you commit and push your changes to the git repository.
build:
docker:
web: DockerfileIt’s very simple, and for this project we only need to set the path for the Dockerfile.
Deploying the API
By this time, it’s assumed that you already have an Heroku account. If you don’t have yet, go ahead and sign up. Furthermore, you need to install Heroku CLI in order to run the following commands.
To build the image that will be deployed, you need to install Docker in your local machine.
With everything installed, you need to create a project. Heroku UI platform can help you with that. Let’s login to heroku in the terminal, build and push the docker container.
$ heroku login
$ heroku container:login
$ heroku container:push web -a name_of_your_app
$ heroku container:release web -a name_of_your_appAfter you finish these steps, you should be able to open the app online and see something similar to this:
If you are using a git repository linked to your Heroku application, any new changes that you commit will automatically trigger the Docker build and will be deployed afterwards. You just worry about the heroku container commands mentioned above the first time you need to deploy this app.
This code is present at my github. In the repository there’s a docker-compose.yml file that will allow you to run this app locally with one command.
docker-compose up --buildHope this helps you somehow :)
