Keras implementation of the Yahoo Open-NSFW model
MIT License
Detecting Not-Suitable-For-Work (NSFW) content is a high demand task in computer vision. While there are many types of NSFW content, here we focus on the pornographic images and videos.
The Yahoo Open-NSFW model originally developed with the Caffe framework has been a favourite choice, but the work is now discontinued and Caffe is also becoming less popular. Please see the description on the Yahoo project page for the context, definitions, and model training details.
This Open-NSFW 2 project provides a Keras implementation of the Yahoo model, with references to its previous third-party TensorFlow 1 implementation. Note that Keras 3 is compatible with TensorFlow, JAX, and PyTorch. However, currently this model is only guaranteed to work with TensorFlow and JAX.
A simple API is provided for making predictions on images and videos.
Tested with TensorFlow and JAX, for Python 3.9 to 3.12.
A note on PyTorch:
The OpenNSFW 2 model can in fact be run on PyTorch, but the biggest issue is
that the inference output on PyTorch is quite different from
that on TensorFlow and JAX. The reason is still unknown. In addition,
inference is much slower on PyTorch probably because of the issues
discussed here,
i.e., PyTorch uses channels_first
for its image data format, but this model
uses channels_last
(as in TensorFLow and JAX), hence Keras has to
convert the channel order back and forth at each layer.
Therefore, at the moment it is not recommended to use PyTorch for this model.
The best way to install Open-NSFW 2 with its dependencies is from PyPI:
python3 -m pip install --upgrade opennsfw2
Alternatively, to obtain the latest version from this repository:
git clone [email protected]:bhky/opennsfw2.git
cd opennsfw2
python3 -m pip install .
Quick examples for getting started are given below. For more details, please refer to the API section.
import opennsfw2 as n2
# To get the NSFW probability of a single image, provide your image file path,
# or a `PIL.Image.Image` object.
image_handle = "path/to/your/image.jpg"
nsfw_probability = n2.predict_image(image_handle)
# To get the NSFW probabilities of a list of images, provide a list of file paths,
# or a list of `PIL.Image.Image` objects.
# Using this function is better than looping with `predict_image` as the model
# will only be instantiated once and batching is done during inference.
image_handles = [
"path/to/your/image1.jpg",
"path/to/your/image2.jpg",
# ...
]
nsfw_probabilities = n2.predict_images(image_handles)
import opennsfw2 as n2
# The video can be in any format supported by OpenCV.
video_path = "path/to/your/video.mp4"
# Return two lists giving the elapsed time in seconds and the NSFW probability of each frame.
elapsed_seconds, nsfw_probabilities = n2.predict_video_frames(video_path)
import numpy as np
import opennsfw2 as n2
from PIL import Image
# Load and preprocess image.
image_path = "path/to/your/image.jpg"
pil_image = Image.open(image_path)
image = n2.preprocess_image(pil_image, n2.Preprocessing.YAHOO)
# The preprocessed image is a NumPy array of shape (224, 224, 3).
# Create the model.
# By default, this call will search for the pre-trained weights file from path:
# $HOME/.opennsfw2/weights/open_nsfw_weights.h5
# If not exists, the file will be downloaded from this repository.
# The model is a `keras_core.Model` object.
model = n2.make_open_nsfw_model()
# Make predictions.
inputs = np.expand_dims(image, axis=0) # Add batch axis (for single image).
predictions = model.predict(inputs)
# The shape of predictions is (num_images, 2).
# Each row gives [sfw_probability, nsfw_probability] of an input image, e.g.:
sfw_probability, nsfw_probability = predictions[0]
preprocess_image
Apply necessary preprocessing to the input image.
pil_image
(PIL.Image.Image
): Input as a Pillow image.preprocessing
(Preprocessing
enum, default Preprocessing.YAHOO
):(224, 224, 3)
.Preprocessing
Enum class for preprocessing options.
Preprocessing.YAHOO
Preprocessing.SIMPLE
make_open_nsfw_model
Create an instance of the NSFW model, optionally with pre-trained weights from Yahoo.
input_shape
(Tuple[int, int, int]
, default (224, 224, 3)
):weights_path
(Optional[str]
, default $HOME/.opennsfw/weights/open_nsfw_weights.h5
):None
, no weights will be downloaded nor loaded to the model.OPENNSFW2_HOME
can also be used to indicate.opennsfw2/
directory should be located.name
(str
, default opennsfw2
): Model name to be used for the Keras model object.tf.keras.Model
object.predict_image
End-to-end pipeline function from the input image to the predicted NSFW probability.
image_handle
(Union[str, PIL.Image.Image]
):PIL.Image.Image
object.preprocessing
: Same as that in preprocess_image
.weights_path
: Same as that in make_open_nsfw_model
.grad_cam_path
(Optional[str]
, default None
): If not None
, e.g., cam.jpg
,alpha
(float
, default 0.8
): Opacity of the Grad-CAM layer of the plot,grad_cam_path
is not None
.nsfw_probability
(float
): The predicted NSFW probability of the image.predict_images
End-to-end pipeline function from the input images to the predicted NSFW probabilities.
image_handles
(Union[Sequence[str], Sequence[PIL.Image.Image]]
):PIL.Image.Image
objects.batch_size
(int
, default 8
): Batch size to be used for model inference.preprocessing
: Same as that in preprocess_image
.weights_path
: Same as that in make_open_nsfw_model
.grad_cam_paths
(Optional[Sequence[str]]
, default None
): If not None
,predict_image
.alpha
: Same as that in predict_image
.nsfw_probabilities
(List[float]
): Predicted NSFW probabilities of the images.Aggregation
Enum class for aggregation options in video frames prediction.
Aggregation.MEAN
Aggregation.MEDIAN
Aggregation.MAX
Aggregation.MIN
predict_video_frames
End-to-end pipeline function from the input video to predictions.
video_path
(str
): Path to the input video source.frame_interval
(int
, default 8
): Prediction will be done on every thisaggregation_size
(int
, default 8
):frame_interval
),aggregation_size
framesaggregation
(Aggregation
enum, default Aggregation.MEAN
):batch_size
(int
, default 8
, upper-bounded by aggregation_size
):output_video_path
(Optional[str]
, default None
):None
, e.g., out.mp4
,preprocessing
: Same as that in preprocess_image
.weights_path
: Same as that in make_open_nsfw_model
.progress_bar
(bool
, default True
): Whether to show the progress bar.List[float]
, each with length equals to the number of video frames.
elapsed_seconds
: Video elapsed time in seconds at each frame.nsfw_probabilities
: NSFW probability of each frame.frame_interval > 1
, all frames without a predictionThis implementation provides the following preprocessing options.
YAHOO
: The default option which was used in the original(256, 256)
.(224, 224)
.[104, 117, 123]
.SIMPLE
: A simpler and probably more intuitive preprocessing option is also provided,(224, 224)
.[104, 117, 123]
.