“Zero-base” facial expression recognition

1. Introduction

In recent years, artificial intelligence (AI) technology has been developing continuously. From intelligent news recommendations to song recognition, to the widely used face recognition technology, AI can be seen everywhere. It has to be said that AI has gradually penetrated into people's daily lives and become an indispensable part. So, how to realize an AI function from "zero"? Let UCloud help you.

[[204495]]

UCloud has launched the AI ​​as a Service platform. Based on UCloud's rich computing resources and distributed system practical experience, the platform is committed to providing cheap, highly reliable, highly elastic, and easy-to-use AI online services, freeing customers from the complicated platform system development and operation and maintenance work. Taking online facial expression recognition as an example, it explains how to use UAI-Service and open source algorithms to easily implement online services and get started with AI from "zero foundation". In addition, the performance of AI online services is evaluated and compared with GPU performance, so that users can more intuitively understand the performance advantages of AI online services.

2. Implementation steps

The entire implementation process mainly consists of two parts. First, use tensorflow1.1.0 to train the required model files, and then deploy the online service according to the instructions for using UAI-SERVICE. The specific steps are as follows:

1. Model training

(1) Install TensorFlow 1.1.0

The installation environment is ubuntu14.04.5, the python version is 2.7.6, and tensorflow is installed directly with pip. The instructions are as follows:

 bash
 pip install tensorflow = 1.1.0

(2) Choose a suitable database

UCloud chose to use the largest public database for facial expression recognition, fer2013, which contains 35,887 face images, including 28,709 training sets, 3,589 validation sets, and 3,589 test sets. The samples in the database have relatively large differences in age, facial orientation, etc., which has certain practical significance and makes expression recognition more challenging.

At the same time, all the images in the database are grayscale images with a size of 48*48 pixels. The samples are divided into seven categories: angry, disgusted, fearful, happy, neutral, sad, and surprised. The distribution of various types is basically even. (The database is actually data provided by a Kaggle competition project. The official file format is csv, which was manually converted into image format.)

(3) Data preprocessing

The TF-Slim experimental library provided by tensorflow is actually used. For details, please refer to the official documentation of TF_Slim.

TF-Slim is a new lightweight API for tensorflow to define, train, and evaluate complex models. It provides a collection of widely used convolutional neural network image classification model codes and pre-trained models, as well as running scripts. With it, you can quickly get started, either training models from scratch or fine-tuning already trained network weights.

TF-Slim provides code for converting datasets into tfrecord format. After adjusting the code, the fer2013 dataset used can be converted into tfrecord format.

The data format is as follows, where labels.txt contains the category mapping:

(4) Training

This training uses a larger model, inception_v3, to fine-tune the official pre-trained model. Since the purpose of training the model is only to try out online services, the training process does not involve much parameter adjustment.

The following are examples of calling instructions:

 bash
 TRAIN_DIR = ./train_log
 DATASET_DIR = ./fer2013
 PRETRAINED_CHECKPOINT_DIR =./pretrain_model 
 
 python train_image_classifier.py \
 --train_dir =${TRAIN_DIR} \
 --dataset_name = fer2013 \
 --dataset_split_name = train \
 --dataset_dir =${DATASET_DIR} \
 --model_name = inception_v3 \
 --checkpoint_path =${PRETRAINED_CHECKPOINT_DIR}/inception_v3.ckpt \
 --checkpoint_exclude_scopes = InceptionV3 /Logits,InceptionV3/AuxLogits \
 --trainable_scopes = InceptionV3 /Logits,InceptionV3/AuxLogits \
 --max_number_of_steps = 1000 \
 --batch_size = 32 \
 --learning_rate = 0.01 \
 --learning_rate_decay_type = fixed \
 --save_interval_secs = 60 \
 --save_summaries_secs = 60 \
 --log_every_n_steps = 100 \
 --optimizer = rmsprop \
 --weight_decay = 0.00004​

When fine-tuning the model, the system automatically retains the last five generated models. If an interruption occurs, fine-tuning will continue based on the last model.

The training model files are as follows:

2. Deploy online services

Once the model is trained, it’s ready to be deployed!

According to the official document (AI Online Service UAI-Service), the main steps of online deployment are as follows (due to personal preference, I chose to use command line deployment this time. The official also provides instructions for using Console deployment).

(1) Install UCloud UFile SDK and UAI SDK

• The former is the UCloud object storage SDK (http://sdk.ufile.ucloud.com.cn/python_sdk.tar.gz), which is mainly used to upload the model and code files required for the deployment service to the UCloud object storage space. This involves first creating a private space of your own. The specific steps are not repeated here.
• The latter is the SDK of UAI online service (https://gitlab.ucloudadmin.com/uai-service/uai-sdk/tree/master), which provides command line tools for deploying online services.

(2) Write inference code based on the code framework in the SDK toolkit

The code is as follows:

 Python
 #fer_inference.py
 import numpy as np
 import tensorflow as tf 
 
 from PIL import Image
 from inception_v3 import *
 from uai.arch.tf_model import TFAiUCloudModel 
 
 class FerModel(TFAiUCloudModel):
 def __init__(self, conf):
      super(FerModel, self).__init__(conf) 
 
 def load_model(self):
 sess = tf .Session()
 input_tensor = tf .placeholder(tf.float32, [None, 299, 299, 3])
 arg_scope = inception_v3_arg_scope ()
      with slim.arg_scope(arg_scope):
          logits, end_points = inception_v3 (input_tensor,
 is_training = False ,
 num_classes = 7 )
 saver = tf.train.Saver ()
 params_file = tf .train.latest_checkpoint(self.model_dir) 
 
      saver.restore(sess, params_file)
      self.output['sess'] = sess
      self.output['input_tensor'] = input_tensor
      self.output['logits'] = logits
      self.output['end_points'] = end_points 
 
  def execute(self, data, batch_size):
 sess = self .output['sess']
 input_tensor = self .output['input_tensor']
 logits = self .output['logits']
 end_points = self .output['end_points']
 ims = []
      for i in range(batch_size):
 im = Image .open(data[i]).resize((299, 299))
 im = np.array (im) / 255.0
 im im = im.reshape(299, 299, 3)
 ims.append(im)
 ims = np.array (ims)
      predict_values, logit_values ​​= sess .run(
          [end_points['Predictions'], logits], feed_dict ={input_tensor: ims})
 ret = []
      for val in predict_values:
 ret_val = np .array_str(np.argmax(val)) + '\n'
 ret.append(ret_val)
 return ret

(3) Package and upload the required models and code files

File directory structure:

• Packaging directory
• Model file directory (under the checkpoint_dir directory)

Package upload files:

 bash
 python tf_deploy.py pack --public_key = MY_PUBLIC_KEY   --private_key = MY_PRIVATE_KEY   --bucket = MY_BUCKET   --pack_file_path =/Users/littleape1022/Desktop/fer_uaiservice --main_file = fer_inference   --main_class = FerModel   --model_dir = checkpoint_dir   --code_files = fer_inference .py,inception_v3.py,inception_utils.py --upload_name = fer_uaiservice .tar --ai_arch_v = tensorflow -1.1.0

(4) Establishment of agency services

Create a service:

 bash
 python tf_deploy.py create --public_key = MY_PUBLIC_KEY   --private_key = MY_PRIVATE_KEY   --service_name = fer_uaiservice   --cpu = 8   --memory = 8  

After creating the service, the following is returned:

(5) Deployment service

Deployment service:

 bash
 python tf_deploy.py deploy --service_id = uaiservice -av4p1c --public_key = MY_PUBLIC_KEY   --private_key = MY_PRIVATE_KEY   --ai_arch_v = tensorflow -1.1.0 --ufile_url = "MY_UFILE_URL"   --pip = pillow  

After successful deployment, the following is returned:

You can see that the URL of the service has been returned, but notice that the status is "ToStart". After starting, you can use the URL to access the service.

(6) Start the service

Start the service:

 bash
 python tf_deploy.py start --public_key = MY_PUBLIC_KEY   --private_key = MY_PRIVATE_KEY   --service_name = fer_uaiservice   --service_version = SERVICE_VERSION   --paas_id = Srv_PAAS_ID  

After successful startup, the following is returned:

3. Testing

After completing the above steps, it indicates that the facial expression recognition online service has been deployed successfully, and online facial expression recognition can be realized!

1. URL Testing

The URL can be accessed through the cloud host, as follows:

The above results show that the URL given after deploying the online service through UAI is valid and can be used to perform sentiment classification on the input images.

The figure divides the input image "happy.jpg" into category "4", which corresponds to the "neutral" class, indicating that the recognition rate of the model needs to be improved.

2. Online service performance testing

The performance of the service was evaluated with the help of ab test, and compared with local test and GPU (K80). (The local test method is introduced on the official website. If you are interested, you can click TensorFlow local code test method)

The test results are as follows, and we can see that:

• When the number of concurrency increases to 8, the performance of AI online services is basically close to that of GPUs, that is, the performance of 8 UAI-Service nodes is equivalent to the performance of a single K80 core.
• Under the premise of concurrency, the performance of AI online services is generally higher than that of 8-core 8G cloud hosts.

Artificial intelligence (AI) will be an important part of UCloud's "CBA" strategy. Using the AI ​​as a Service platform launched by UCloud can help artificial intelligence companies quickly commercialize artificial intelligence algorithms, while also providing all-round guarantees in resource management and resource scheduling.

[This article is an original article by 51CTO columnist "Big U's Technology Classroom". For reprinting, please contact the author through the WeChat public account (ucloud2012)]

Click here to read more articles by this author