CNN 分类器实践-面部情绪识别

本教程使用 TensorFlow 构建一个卷积神经网络（CNN）对 48×48 灰度人脸图片进行 7 种情绪分类，再结合 OpenCV 的 Haar Cascade 从电影海报中检测人脸并识别情绪。

Part 1：情绪分类模型训练
#

1. 数据集下载
#

从 Kaggle 下载面部情绪数据集：

# download dataset
!wget https://www.kaggle.com/api/v1/datasets/download/dilkushsingh/facial-emotion-dataset?datasetVersionNumber=8

--2024-08-14 06:50:35--  https://www.kaggle.com/api/v1/datasets/download/dilkushsingh/facial-emotion-dataset?datasetVersionNumber=8
Resolving www.kaggle.com (www.kaggle.com)... 35.244.233.98
Connectin...(truncated)

解压数据：

# unzip dataset
!unzip 'facial-emotion-dataset?datasetVersionNumber=8'

  inflating: train_dir/sad/img_161024360.jpg  
  inflating: train_dir/sad/img_161024361.jpg  
  inflating: train_dir/sad/img_161024362....(truncated)

2. 数据清洗
#

检查训练集（29,417 张）和测试集（7,340 张）中的损坏图片并移除：

# check and remove bad files
from pathlib import Path
from tensorflow.io import read_file
from tensorflow.image import decode_image

def check(data_dir):
  cnt = 0
  for image in sorted(data_dir.glob('*/*')):
    cnt += 1
    try:
      img = read_file(str(image))
      img = decode_image(img)
      if img.ndim != 3:
        raise Exception('ndim != 3')
    except Exception as e:
      print('bad file: ', str(image))
      image.unlink()
  print('checked {} files'.format(cnt))

check(Path.cwd()/'train_dir')
check(Path.cwd()/'test_dir')

checked 29417 files
checked 7340 files

训练集 29,417 张，测试集 7,340 张，均无损坏文件。

3. 数据加载与预处理
#

使用 image_dataset_from_directory 加载 48×48 灰度图，转换为 float32 并添加 cache/prefetch 优化：

# read dataset
import tensorflow as tf
from tensorflow.keras.preprocessing import image_dataset_from_directory

ds_train_ = image_dataset_from_directory(
    '/content/train_dir',
    labels = 'inferred',
    label_mode = 'categorical',
    color_mode = 'grayscale',
    image_size = [48, 48],
    batch_size = 32,
    shuffle = True,
)

ds_valid_ = image_dataset_from_directory(
    '/content/train_dir',
    labels = 'inferred',
    label_mode = 'categorical',
    color_mode = 'grayscale',
    image_size = [48, 48],
    batch_size = 32,
    shuffle = True,
)

def convert_to_float(image, label):
  image = tf.image.convert_image_dtype(image, dtype=tf.float32)
  return image, label

AUTOTUNE = tf.data.experimental.AUTOTUNE

ds_train = (
    ds_train_.
    map(convert_to_float)
    .cache().
    prefetch(buffer_size = AUTOTUNE)
)
ds_valid = (
    ds_valid_
    .map(convert_to_float)
    .cache()
    .prefetch(buffer_size = AUTOTUNE)
)

Found 29417 files belonging to 7 classes.
Found 29417 files belonging to 7 classes.

情绪标签
#

7 种情绪类别：

emotions = ['angry', 'disgust', 'fear', 'happy', 'neutral', 'sad', 'surprise']

4. 数据预览
#

查看部分训练样本及其对应标签：

# preview dataset
import matplotlib.pyplot as plt
import numpy as np

ds = ds_train.take(1)
for item in ds:
  images, labels = item
  plt.figure(figsize = (12, 8))
  for i in range(6):
    plt.subplot(2, 3, i+1)
    plt.axis('off')
    plt.imshow(images[i], cmap='gray')
    plt.title(emotions[np.argmax(labels[i].numpy())])

5. 构建 CNN 模型
#

模型结构：多层 Conv2D + Dropout 防止过拟合，最后通过 Dense + softmax 输出 7 类概率。

# define model

from tensorflow import keras
from tensorflow.keras import layers
from tensorflow.keras.callbacks import EarlyStopping

model = keras.Sequential([
    layers.Input([48, 48, 1]),

    layers.Conv2D(filters=32, kernel_size=3, activation='relu', padding='same'),
    layers.MaxPool2D(),

    layers.Conv2D(filters=64, kernel_size=3, activation='relu', padding='same'),
    layers.Dropout(0.2),
    layers.Conv2D(filters=64, kernel_size=3, activation='relu', padding='same'),
    layers.Dropout(0.2),

    layers.Conv2D(filters=64, kernel_size=3, activation='relu', padding='same'),
    layers.Dropout(0.2),
    layers.Conv2D(filters=64, kernel_size=3, activation='relu', padding='same'),
    layers.Dropout(0.2),

    layers.Conv2D(filters=128, kernel_size=3, activation='relu', padding='same'),
    layers.Conv2D(filters=128, kernel_size=3, activation='relu', padding='same'),
    layers.Dropout(0.2),

    layers.Conv2D(filters=128, kernel_size=3, activation='relu', padding='same'),
    layers.Conv2D(filters=128, kernel_size=3, activation='relu', padding='same'),
    layers.Dropout(0.2),

    layers.Conv2D(filters=256, kernel_size=3, activation='relu', padding='same'),
    layers.Dropout(0.2),
    layers.Conv2D(filters=256, kernel_size=3, activation='relu', padding='same'),
    layers.Dropout(0.2),

    layers.Flatten(),
    layers.Dense(64, input_dim=8, activation='relu'),
    layers.Dropout(0.2),
    layers.Dense(7, activation='softmax'),
])

model.compile(
    optimizer = tf.keras.optimizers.Adam(epsilon=0.01),
    loss = 'categorical_crossentropy',
    metrics = ['accuracy']
)

early_stopping = EarlyStopping(
    min_delta = 0.001,
    patience = 20,
    restore_best_weights = True,
)

history = model.fit(
    ds_train,
    validation_data = ds_valid,
    callbacks = [early_stopping],
    epochs = 100
)

Epoch 1/100
920/920 ━━━━━━━━━━━━━━━━━━━━ 66s 55ms/step - accuracy: 0.2055 - loss: 1.9932 - val_accuracy: 0.2174 - val_loss: 1.9872
Epoch 2/100
920/920 ━━━━━━━━━━━━━...(truncated)

使用 EarlyStopping（patience=20）防止过拟合，最多训练 100 个 epoch。

6. 模型预测测试
#

在验证集上测试模型的预测效果：

import numpy as np

ds = ds_valid.take(1)
for item in ds:
  prediction = model.predict(item)
  print(prediction[3])
  print(np.argmax(prediction[3]))

  images, labels = item
  plt.figure(figsize = (10, 20))
  for i in range(images.shape[0]):
    plt.subplot(8, 4, i+1)
    plt.axis('off')
    plt.imshow(images[i], cmap='gray')
    plt.title(emotions[np.argmax(labels[i])])

1/1 ━━━━━━━━━━━━━━━━━━━━ 1s 551ms/step
[2.8921450e-02 1.4205185e-04 1.8577275e-05 9.5940810e-01 1.7444472e-04
 1.1266178e-02 6.9209687e-05]
3

预测结果中最高概率对应的索引为 3，即 neutral（中性）。

7. 模型保存
#

from datetime import datetime
now = datetime.now().strftime("%Y-%m-%d-%H-%M-%S")
model.save('face-emotion-{}.keras'.format(now))

Part 2：人脸检测与情绪识别
#

8. 下载电影海报数据集
#

从 Kaggle 下载电影海报数据集，用于人脸检测测试：

# download movie poster dataset
!wget https://www.kaggle.com/api/v1/datasets/download/raman77768/movie-classifier?datasetVersionNumber=1 -O posters.zip

--2024-08-14 06:09:10--  https://www.kaggle.com/api/v1/datasets/download/raman77768/movie-classifier?datasetVersionNumber=1
Resolving www.kaggle.com (www.kaggle.com)... 35.244.233.98
Connecting to www...(truncated)

解压海报数据：

# unzip dataset
!unzip posters.zip

整理目录结构：

!mv Multi_Label_dataset/Images/ posters && rm -rf Multi_Label_dataset/

9. 下载 Haar Cascade 人脸检测模型
#

OpenCV 预训练的 Haar Cascade 级联分类器，用于检测图片中的人脸：

# download haarcascade_frontalface_default.xml
!wget --no-check-certificate \
    https://raw.githubusercontent.com/computationalcore/introduction-to-opencv/master/assets/haarcascade_frontalface_default.xml \
    -O haarcascade_frontalface_default.xml

--2024-08-14 06:30:10--  https://raw.githubusercontent.com/computationalcore/introduction-to-opencv/master/assets/haarcascade_frontalface_default.xml
Resolving raw.githubusercontent.com (raw.githubuse...(truncated)

10. 批量人脸检测与裁剪
#

遍历所有电影海报，用 Haar Cascade 检测人脸并裁剪保存：

import os
import cv2
import matplotlib.pyplot as plt
from pathlib import Path

face_cascade = cv2.CascadeClassifier('haarcascade_frontalface_default.xml')
for fl in sorted((Path.cwd()/'posters').glob('*.jpg')):
  img = cv2.imread(fl)
  img = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
  detects = face_cascade.detectMultiScale(img, 1.3, 5)
  faces = [img[y:y+h, x:x+w] for (x,y,w,h) in detects]
  for i in range(len(faces)):
    cv2.imwrite('faces/{}_{}.jpg'.format(fl.name[:-4], i+1), faces[i])

10.1 人脸检测效果预览
#

查看单张海报的人脸检测结果：

import matplotlib.pyplot as plt
import cv2

img = cv2.imread('/content/posters/tt0086593.jpg')
plt.figure()
face_cascade = cv2.CascadeClassifier('haarcascade_frontalface_default.xml')
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)

detects = face_cascade.detectMultiScale(gray, 1.3, 5)
faces = [gray[y:y+h, x:x+w] for (x,y,w,h) in detects]
for face in faces:
  plt.figure()
  plt.axis('off')
  plt.imshow(face, cmap='gray')

11. 对检测到的人脸进行情绪预测
#

加载裁剪后的人脸图片，用训练好的情绪分类模型进行预测：

ds_test_ = image_dataset_from_directory(
    '/content/faces',
    labels = None,
    color_mode = 'grayscale',
    image_size = [48, 48],
    batch_size = 32,
    shuffle = True,
)

def convert_to_float(image):
  image = tf.image.convert_image_dtype(image, dtype=tf.float32)
  return image

ds_test = (
    ds_test_
    .map(convert_to_float)
    .cache()
    .prefetch(buffer_size = AUTOTUNE)
)

Found 4691 files.

ds = ds_test.take(1)
for item in ds:
  prediction = model.predict(item)
  print(prediction[0])
  print(np.argmax(prediction[0]))

  images = item
  plt.figure(figsize = (10, 20))
  for i in range(images.shape[0]):
    plt.subplot(8, 4, i+1)
    plt.axis('off')
    plt.imshow(images[i], cmap='gray')
    plt.title(emotions[np.argmax(prediction[i])])

1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 20ms/step
[9.5885843e-02 1.2981243e-06 2.5813368e-05 4.4420755e-01 4.5889294e-01
 3.1211486e-05 9.5540209e-04]
4

12. 总结
#

项目	说明
任务类型	图像分类（情绪识别）+ 目标检测（人脸检测）
数据集	Kaggle 面部情绪数据集 — 29,417 张训练 + 7,340 张测试
图片尺寸	48×48 灰度图
情绪类别	7 类：angry, disgust, fear, happy, neutral, sad, surprise
CNN 模型	多层 Conv2D + Dropout + Dense，最终 softmax 输出
优化器	Adam (epsilon=0.01)
损失函数	categorical_crossentropy
早停机制	EarlyStopping(patience=20)
人脸检测	Haar Cascade 级联分类器
检测数据	电影海报 → 裁剪出 4,691 张人脸

本教程完整实现了从训练情绪分类模型，到用 OpenCV 从电影海报中检测人脸并进行实时情绪预测的完整流程。

Part 1：情绪分类模型训练 #

1. 数据集下载 #

2. 数据清洗 #

3. 数据加载与预处理 #

情绪标签 #

4. 数据预览 #

5. 构建 CNN 模型 #

6. 模型预测测试 #

7. 模型保存 #

Part 2：人脸检测与情绪识别 #

8. 下载电影海报数据集 #

9. 下载 Haar Cascade 人脸检测模型 #

10. 批量人脸检测与裁剪 #

10.1 人脸检测效果预览 #

11. 对检测到的人脸进行情绪预测 #

12. 总结 #