Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat/batch predict age and gender #1396

Open
wants to merge 64 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 18 commits
Commits
Show all changes
64 commits
Select commit Hold shift + click to select a range
c42df04
[update] add batch predicting for Age model
NatLee Dec 5, 2024
a4b1b5d
[update] add batch predicting for Gender model
NatLee Dec 5, 2024
b55cb31
[fix] name of model attributes `inputs`
NatLee Dec 6, 2024
29c818d
[fix] line too long
NatLee Dec 6, 2024
27e8fc9
[update] enhance predict methods to support single and batch inputs f…
NatLee Dec 17, 2024
38c0652
[update] enhance predict methods in Emotion and Race models to suppor…
NatLee Dec 17, 2024
b9418eb
[fix] `input` to `inputs`
NatLee Dec 17, 2024
d992428
[update] embed into deepface module
NatLee Dec 17, 2024
e96ede3
[update] add multiple faces testing
NatLee Dec 17, 2024
e1417a0
Update master branch. Merge branch 'master' into feat/batch-predict-a…
h-alice Dec 31, 2024
f8be282
Update master branch. Merge branch 'master' into feat/merge-predicts-…
h-alice Dec 31, 2024
9f3875e
Update master branch. Merge branch 'master' into feat/make-Race-and-E…
h-alice Dec 31, 2024
9c079e9
Merge branch 'feat/batch-predict-age-and-gender' into feat/merge-pred…
h-alice Dec 31, 2024
bba4322
Merge branch 'feat/merge-predicts-functions' into feat/make-Race-and-…
h-alice Dec 31, 2024
4cf43be
Merge pull request #4 from NatLee/feat/add-multi-face-test
NatLee Dec 31, 2024
05dbc80
Merge pull request #2 from NatLee/feat/make-Race-and-Emotion-batch
NatLee Dec 31, 2024
c312684
Merge pull request #1 from NatLee/feat/merge-predicts-functions
NatLee Dec 31, 2024
ffbba7f
Change base class's predict signature.
h-alice Dec 31, 2024
edcef02
[update] remove dummy functions
NatLee Dec 31, 2024
472f146
Avoid recreating `resp_objects`.
h-alice Jan 3, 2025
b69dcfc
Engineering stuff, remove redundant code.
h-alice Jan 3, 2025
0f65a87
Add assertion to verify length of analyzed objects.
h-alice Jan 3, 2025
bb820a7
[update] one-line checking
NatLee Jan 3, 2025
e182285
Fix: Image batch dimension not expanded.
h-alice Jan 3, 2025
69267cd
Merge pull request #5 from NatLee/patch/test-250103
h-alice Jan 3, 2025
5747d96
Predictor.
h-alice Jan 6, 2025
5a18814
Add comment.
h-alice Jan 6, 2025
72b94d1
Add new predictor.
h-alice Jan 6, 2025
c647488
Merge remote-tracking branch 'origin/feat/batch-predict-age-and-gende…
NatLee Jan 6, 2025
e668cd4
Merge remote-tracking branch 'origin/master' into patch/adjustment-01…
NatLee Jan 6, 2025
85e2d8d
[update] modify comment for multi models
NatLee Jan 6, 2025
ba0d0c5
[update] make process to one-line
NatLee Jan 6, 2025
29141b3
[update] add hint for the shape of input img
NatLee Jan 6, 2025
36fb512
[fix] handle between grayscale and RGB image for models
NatLee Jan 6, 2025
431544a
[update] add process for single and multiple image
NatLee Jan 6, 2025
0417732
[fix] model input size -> (n, w, h, c)
NatLee Jan 6, 2025
c44af00
[fix] check for input number of faces
NatLee Jan 6, 2025
ad577b4
[update] refactor response object creation in analyze function
NatLee Jan 6, 2025
52a38ba
[fix] use prediction shape to avoid confuse situation of predictions
NatLee Jan 6, 2025
ba8c651
[fix] 1 img input for the `Emotion` model
NatLee Jan 6, 2025
4284252
Remove obsolete comment.
h-alice Jan 7, 2025
eb7b841
Documentation
h-alice Jan 13, 2025
688fbe6
[fix] lint
NatLee Jan 13, 2025
a442f7a
Merge remote-tracking branch 'origin/master' into patch/adjustment-01…
NatLee Jan 13, 2025
8883b21
Merge pull request #6 from NatLee/patch/adjustment-0103-1
h-alice Jan 13, 2025
fa4044a
patch: Greyscale image prediction condition.
h-alice Jan 13, 2025
910d6e1
patch: fix dimension.
h-alice Jan 13, 2025
72b6db1
patch: fix dimension
h-alice Jan 13, 2025
a23893a
patch: emotion dimension.
h-alice Jan 13, 2025
da4a0c5
patch: Lint
h-alice Jan 14, 2025
c72b474
[update] lint
NatLee Jan 14, 2025
7e719df
Patch: Make Age model capable to handle single or batched input.
h-alice Jan 16, 2025
6a7bbdb
REVERT demography.py
h-alice Jan 16, 2025
6a8d1d9
patch: Lint
h-alice Jan 16, 2025
0d7e151
[update] rm `print`
NatLee Jan 16, 2025
0971fcd
Merge commit '0d7e15147f527edc0ef09dadbd73f38d87972d1f' into feat/bat…
NatLee Jan 16, 2025
db4b749
[update] add emotions batch test
NatLee Jan 20, 2025
95bb92c
Remove redundant squeeze.
h-alice Jan 21, 2025
61b6931
[update] modify test of `emotion` and add client of `age`, `gender` a…
NatLee Jan 21, 2025
6df7b7d
Add support for batched input.
h-alice Jan 22, 2025
b584d29
Refine some tests.
h-alice Jan 22, 2025
ee3093d
Merge branch 'feat/batch-predict-age-and-gender' of https://github.co…
h-alice Jan 22, 2025
4d77931
Merge remote-tracking branch 'origin/master' into feat/batch-predict-…
NatLee Jan 25, 2025
0ab3ac2
[fix] avoid problem of precision in float
NatLee Jan 25, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions deepface/models/Demography.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
from typing import Union
from typing import Union, List
from abc import ABC, abstractmethod
import numpy as np
from deepface.commons import package_utils
Expand All @@ -18,5 +18,5 @@ class Demography(ABC):
model_name: str

@abstractmethod
def predict(self, img: np.ndarray) -> Union[np.ndarray, np.float64]:
def predict(self, img: Union[np.ndarray, List[np.ndarray]]) -> Union[np.ndarray, np.float64]:
pass
69 changes: 63 additions & 6 deletions deepface/models/demography/Age.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,7 @@
# stdlib dependencies

from typing import List, Union

# 3rd party dependencies
import numpy as np

Expand Down Expand Up @@ -37,11 +41,64 @@ def __init__(self):
self.model = load_model()
self.model_name = "Age"

def predict(self, img: np.ndarray) -> np.float64:
# model.predict causes memory issue when it is called in a for loop
# age_predictions = self.model.predict(img, verbose=0)[0, :]
age_predictions = self.model(img, training=False).numpy()[0, :]
return find_apparent_age(age_predictions)
def predict(self, img: Union[np.ndarray, List[np.ndarray]]) -> np.ndarray:
"""
Predict apparent age(s) for single or multiple faces
Args:
img: Single image as np.ndarray (224, 224, 3) or
List of images as List[np.ndarray] or
Batch of images as np.ndarray (n, 224, 224, 3)
Returns:
np.ndarray (n,)
"""
# Convert to numpy array if input is list
if isinstance(img, list):
imgs = np.array(img)
NatLee marked this conversation as resolved.
Show resolved Hide resolved
else:
imgs = img

# Remove batch dimension if exists
imgs = imgs.squeeze()
NatLee marked this conversation as resolved.
Show resolved Hide resolved

# Check input dimension
if len(imgs.shape) == 3:
serengil marked this conversation as resolved.
Show resolved Hide resolved
# Single image - add batch dimension
imgs = np.expand_dims(imgs, axis=0)

# Batch prediction
age_predictions = self.model.predict_on_batch(imgs)
Copy link
Owner

@serengil serengil Dec 31, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

model.predict causes memory issue when it is called in a for loop, that is why we call it as self.model(img, training=False).numpy()[0, :]

in your design, if this is called in a for loop, still it will cause memory problem.

IMO, if it is single image, we should call self.model(img, training=False).numpy()[0, :], it is many faces then call self.model.predict_on_batch

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for sharing your perspective on this matter.

We found the issue you mentioned is also mentioned in this page: tensorflow/tensorflow#44711. We believe it’s being resolved.

Furthermore, if we can utilize the batch prediction method provided in this PR, we may be able to avoid repeatedly calling the predict function within a loop of unrolled batch images, which is the root cause of the memory issue you described.

We recommend that you consider retaining our batch prediction method.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hey, even though this is sorted in newer tf versions, many users using old tf versions raise tickets about this problem. so, we should consider the people using older tf version. that is why, i recommend to use self.model(img, training=False).numpy()[0, :] for single images, and self.model.predict_on_batch for batches.

Copy link

@h-alice h-alice Jan 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi! 👋
Please take a look at our prediction function, which uses the legacy single prediction method you suggested, and also provides batch prediction if a batch of images is provided.

Please let us know if there’s anything else we can improve. Any advice you have is greatly appreciated.

def _predict_internal(self, img_batch: np.ndarray) -> np.ndarray:


# Calculate apparent ages
apparent_ages = np.array(
[find_apparent_age(age_prediction) for age_prediction in age_predictions]
)

return apparent_ages


def predicts(self, imgs: List[np.ndarray]) -> np.ndarray:
NatLee marked this conversation as resolved.
Show resolved Hide resolved
"""
Predict apparent ages of multiple faces
Args:
imgs (List[np.ndarray]): (n, 224, 224, 3)
Returns:
apparent_ages (np.ndarray): (n,)
"""
# Convert list to numpy array
imgs_:np.ndarray = np.array(imgs)
# Remove batch dimension if exists
imgs_ = imgs_.squeeze()
# Check if the input is a single image
if len(imgs_.shape) == 3:
# Add batch dimension if not exists
imgs_ = np.expand_dims(imgs_, axis=0)
# Batch prediction
age_predictions = self.model.predict_on_batch(imgs_)
apparent_ages = np.array(
[find_apparent_age(age_prediction) for age_prediction in age_predictions]
)
return apparent_ages



def load_model(
Expand All @@ -65,7 +122,7 @@ def load_model(

# --------------------------

age_model = Model(inputs=model.input, outputs=base_model_output)
age_model = Model(inputs=model.inputs, outputs=base_model_output)

# --------------------------

Expand Down
58 changes: 49 additions & 9 deletions deepface/models/demography/Emotion.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,6 @@
# stdlib dependencies
from typing import List, Union

# 3rd party dependencies
import numpy as np
import cv2
Expand Down Expand Up @@ -43,16 +46,53 @@ def __init__(self):
self.model = load_model()
self.model_name = "Emotion"

def predict(self, img: np.ndarray) -> np.ndarray:
img_gray = cv2.cvtColor(img[0], cv2.COLOR_BGR2GRAY)
def _preprocess_image(self, img: np.ndarray) -> np.ndarray:
"""
Preprocess single image for emotion detection
Args:
img: Input image (224, 224, 3)
Returns:
Preprocessed grayscale image (48, 48)
"""
img_gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
img_gray = cv2.resize(img_gray, (48, 48))
img_gray = np.expand_dims(img_gray, axis=0)

# model.predict causes memory issue when it is called in a for loop
# emotion_predictions = self.model.predict(img_gray, verbose=0)[0, :]
emotion_predictions = self.model(img_gray, training=False).numpy()[0, :]

return emotion_predictions
return img_gray

def predict(self, img: Union[np.ndarray, List[np.ndarray]]) -> np.ndarray:
"""
Predict emotion probabilities for single or multiple faces
Args:
img: Single image as np.ndarray (224, 224, 3) or
List of images as List[np.ndarray] or
Batch of images as np.ndarray (n, 224, 224, 3)
Returns:
np.ndarray (n, n_emotions)
where n_emotions is the number of emotion categories
"""
# Convert to numpy array if input is list
if isinstance(img, list):
imgs = np.array(img)
else:
imgs = img

# Remove batch dimension if exists
imgs = imgs.squeeze()

# Check input dimension
if len(imgs.shape) == 3:
# Single image - add batch dimension
imgs = np.expand_dims(imgs, axis=0)

# Preprocess each image
processed_imgs = np.array([self._preprocess_image(img) for img in imgs])

# Add channel dimension for grayscale images
processed_imgs = np.expand_dims(processed_imgs, axis=-1)

# Batch prediction
predictions = self.model.predict_on_batch(processed_imgs)

return predictions


def load_model(
Expand Down
58 changes: 53 additions & 5 deletions deepface/models/demography/Gender.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,7 @@
# stdlib dependencies

from typing import List, Union

# 3rd party dependencies
import numpy as np

Expand Down Expand Up @@ -37,10 +41,54 @@ def __init__(self):
self.model = load_model()
self.model_name = "Gender"

def predict(self, img: np.ndarray) -> np.ndarray:
# model.predict causes memory issue when it is called in a for loop
# return self.model.predict(img, verbose=0)[0, :]
return self.model(img, training=False).numpy()[0, :]
def predict(self, img: Union[np.ndarray, List[np.ndarray]]) -> np.ndarray:
"""
Predict gender probabilities for single or multiple faces
Args:
img: Single image as np.ndarray (224, 224, 3) or
List of images as List[np.ndarray] or
Batch of images as np.ndarray (n, 224, 224, 3)
Returns:
np.ndarray (n, 2)
"""
# Convert to numpy array if input is list
if isinstance(img, list):
imgs = np.array(img)
else:
imgs = img

# Remove batch dimension if exists
imgs = imgs.squeeze()

# Check input dimension
if len(imgs.shape) == 3:
# Single image - add batch dimension
imgs = np.expand_dims(imgs, axis=0)

# Batch prediction
predictions = self.model.predict_on_batch(imgs)

return predictions


def predicts(self, imgs: List[np.ndarray]) -> np.ndarray:
NatLee marked this conversation as resolved.
Show resolved Hide resolved
"""
Predict apparent ages of multiple faces
Args:
imgs (List[np.ndarray]): (n, 224, 224, 3)
Returns:
apparent_ages (np.ndarray): (n,)
"""
# Convert list to numpy array
imgs_:np.ndarray = np.array(imgs)
# Remove redundant dimensions
imgs_ = imgs_.squeeze()
# Check if the input is a single image
if len(imgs_.shape) == 3:
# Add batch dimension
imgs_ = np.expand_dims(imgs_, axis=0)
return self.model.predict_on_batch(imgs_)



def load_model(
Expand All @@ -64,7 +112,7 @@ def load_model(

# --------------------------

gender_model = Model(inputs=model.input, outputs=base_model_output)
gender_model = Model(inputs=model.inputs, outputs=base_model_output)

# --------------------------

Expand Down
38 changes: 33 additions & 5 deletions deepface/models/demography/Race.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,6 @@
# stdlib dependencies
from typing import List, Union

# 3rd party dependencies
import numpy as np

Expand Down Expand Up @@ -37,10 +40,35 @@ def __init__(self):
self.model = load_model()
self.model_name = "Race"

def predict(self, img: np.ndarray) -> np.ndarray:
# model.predict causes memory issue when it is called in a for loop
# return self.model.predict(img, verbose=0)[0, :]
return self.model(img, training=False).numpy()[0, :]
def predict(self, img: Union[np.ndarray, List[np.ndarray]]) -> np.ndarray:
"""
Predict race probabilities for single or multiple faces
Args:
img: Single image as np.ndarray (224, 224, 3) or
List of images as List[np.ndarray] or
Batch of images as np.ndarray (n, 224, 224, 3)
Returns:
np.ndarray (n, n_races)
where n_races is the number of race categories
"""
# Convert to numpy array if input is list
if isinstance(img, list):
imgs = np.array(img)
else:
imgs = img

# Remove batch dimension if exists
imgs = imgs.squeeze()

# Check input dimension
if len(imgs.shape) == 3:
# Single image - add batch dimension
imgs = np.expand_dims(imgs, axis=0)

# Batch prediction
predictions = self.model.predict_on_batch(imgs)

return predictions


def load_model(
Expand All @@ -62,7 +90,7 @@ def load_model(

# --------------------------

race_model = Model(inputs=model.input, outputs=base_model_output)
race_model = Model(inputs=model.inputs, outputs=base_model_output)

# --------------------------

Expand Down
Loading
Loading