camtrapml

{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# CamTrapML\n", "\n", "> CamTrapML is a Python library for Detecting, Classifying, and Analysing Wildlife Camera Trap Imagery.\n", "\n", "## Installation\n", "\n", " $ pip install camtrapml" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Features" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Loading Data\n", "\n", "Search for images in a directory, load an image and create a thumbnail." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "execution": { "iopub.execute_input": "2022-05-04T14:30:58.752773Z", "iopub.status.busy": "2022-05-04T14:30:58.752322Z", "iopub.status.idle": "2022-05-04T14:31:00.031193Z", "shell.execute_reply": "2022-05-04T14:31:00.030927Z" } }, "outputs": [], "source": [ "%load_ext autoreload\n", "%autoreload\n", "\n", "from camtrapml.dataset import ImageDataset\n", "from camtrapml.image.utils import load_image, thumbnail\n", "\n", "imageset = ImageDataset(\n", " name="Test Images",\n", " path="test/fixtures/images",\n", ")\n", "\n", "image_paths = list(imageset.enumerate_images())\n", "\n", "thumbnail(load_image(image_paths[0]))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### EXIF Extraction\n", "\n", "EXIF extraction is a common task in gathering the metadata such as each image's timestamp, camera model, focal length, etc. Some researchers write labelling into the EXIF data. CamTrapML doesn't assist with writing to EXIF. However, there is functionality for extracting it for analysis and building datasets for training new models from previously labelled images.\n", "\n", "ExifTool is required for this package to work. Installation instructions can be found here.\n", "\n", "Three methods are available for extracting EXIF data from images. Each with different performance characteristics." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Method 1: Individual Images" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "execution": { "iopub.execute_input": "2022-05-04T14:31:00.043804Z", "iopub.status.busy": "2022-05-04T14:31:00.043710Z", "iopub.status.idle": "2022-05-04T14:31:00.126414Z", "shell.execute_reply": "2022-05-04T14:31:00.126098Z" } }, "outputs": [], "source": [ "from camtrapml.image.exif import extract_exif\n", "\n", "exif = extract_exif(image_paths[0])\n", "exif" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Method 2: Multiple Images\n", "\n", "extract_multiple_exif passes a list of image paths to ExifTool and returns a list of dictionaries containing the EXIF data. This is faster than extract_exif when multiple images are being processed as it only passes the list of image paths to ExifTool once, rather than spawning a new process for each image." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "execution": { "iopub.execute_input": "2022-05-04T14:31:00.128363Z", "iopub.status.busy": "2022-05-04T14:31:00.128254Z", "iopub.status.idle": "2022-05-04T14:31:00.205496Z", "shell.execute_reply": "2022-05-04T14:31:00.205202Z" } }, "outputs": [], "source": [ "from camtrapml.image.exif import extract_multiple_exif\n", "\n", "exif = extract_multiple_exif(image_paths)\n", "exif[0]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Method 3: Multiple Images, Multiple Processes\n", "\n", "When processing large datasets, it's apparent that the bottleneck in extracting the EXIF information tends to be the CPU. This method spawns multiple versions of ExifTool in parallel, each with a batch of image paths. This is faster than extract_multiple_exif when processing large datasets as it allows for multiple processes to be spawned and the data extracted in parallel." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "execution": { "iopub.execute_input": "2022-05-04T14:31:00.207105Z", "iopub.status.busy": "2022-05-04T14:31:00.207006Z", "iopub.status.idle": "2022-05-04T14:31:00.343766Z", "shell.execute_reply": "2022-05-04T14:31:00.343506Z" } }, "outputs": [], "source": [ "from camtrapml.image.exif import extract_multiple_exif_fast\n", "\n", "exif = extract_multiple_exif_fast(image_paths)\n", "exif[0]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Detection\n", "\n", "Various Detection models are available in the camtrapml.detection subpackage. These currently include MegaDetector (v4.1, v3 and v2) and support for loading in custom Tensorflow v1.x Object Detection Frozen models." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Detection with MegaDetector v4.1" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "execution": { "iopub.execute_input": "2022-05-04T14:31:00.345296Z", "iopub.status.busy": "2022-05-04T14:31:00.345214Z", "iopub.status.idle": "2022-05-04T14:31:00.500790Z", "shell.execute_reply": "2022-05-04T14:31:00.500074Z" } }, "outputs": [], "source": [ "from camtrapml.detection.models.megadetector import MegaDetectorV4_1\n", "from camtrapml.detection.utils import render_detections\n", "\n", "with MegaDetectorV4_1() as detector:\n", " detections = detector.detect(image_paths[0])\n", "\n", "thumbnail(\n", " render_detections(image_paths[0], detections, class_map=detector.class_map)\n", ")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Detection with a custom Tensorflow v1.x Object Detection Frozen model" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "execution": { "iopub.execute_input": "2022-05-04T14:31:00.505336Z", "iopub.status.busy": "2022-05-04T14:31:00.505107Z", "iopub.status.idle": "2022-05-04T14:31:00.800178Z", "shell.execute_reply": "2022-05-04T14:31:00.799735Z" } }, "outputs": [], "source": [ "!cp ~/.camtrapml/models/megadetector/v4.1.0/md_v4.1.0.pb example-custom-model.pb\n", "\n", "from camtrapml.detection.models.tensorflow import TF1ODAPIFrozenModel\n", "from camtrapml.detection.utils import render_detections\n", "from pathlib import Path\n", "\n", "with TF1ODAPIFrozenModel(\n", " model_path=Path("example-custom-model.pb").expanduser(),\n", " class_map={\n", " 1: "animal",\n", " },\n", ") as detector:\n", " detections = detector.detect(image_paths[1])\n", "\n", "thumbnail(\n", " render_detections(image_paths[1], detections, class_map=detector.class_map)\n", ")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Extract Detections" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "execution": { "iopub.execute_input": "2022-05-04T14:31:00.802816Z", "iopub.status.busy": "2022-05-04T14:31:00.802665Z", "iopub.status.idle": "2022-05-04T14:31:00.810082Z", "shell.execute_reply": "2022-05-04T14:31:00.809735Z" } }, "outputs": [], "source": [ "from camtrapml.detection.models.megadetector import MegaDetectorV4_1\n", "from camtrapml.detection.utils import extract_detections_from_image\n", "\n", "with MegaDetectorV4_1() as detector:\n", " detections = detector.detect(image_paths[0])\n", "\n", "list(extract_detections_from_image(load_image(image_paths[0]), detections))[0]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Remove Humans from Images\n", "\n", "In order to reduce the risks of identification of humans in line with GDPR, CamTrapML provides the ability to remove humans from images. This is achieved by using the MegaDetector v3+ models to detect humans in the image, and then replacing all pixels in each human detection." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "execution": { "iopub.execute_input": "2022-05-04T14:31:00.812150Z", "iopub.status.busy": "2022-05-04T14:31:00.812016Z", "iopub.status.idle": "2022-05-04T14:31:00.819177Z", "shell.execute_reply": "2022-05-04T14:31:00.818870Z" } }, "outputs": [], "source": [ "from camtrapml.detection.models.megadetector import MegaDetectorV4_1\n", "from camtrapml.detection.utils import remove_detections_from_image\n", "from camtrapml.image.utils import load_image, thumbnail\n", "from pathlib import Path\n", "\n", "ct_image_with_humans = Path("test/fixtures/human_images/IMG_0254.JPG").expanduser()\n", "\n", "with MegaDetectorV4_1() as detector:\n", " detections = detector.detect(ct_image_with_humans)\n", "\n", "human_class_id = 2\n", "\n", "thumbnail(\n", " remove_detections_from_image(\n", " load_image(ct_image_with_humans),\n", " [\n", " detection\n", " for detection in detections\n", " if detection["category"] == human_class_id and detection["conf"] > 0.5\n", " ],\n", " )\n", ")" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "interpreter": { "hash": "52ee2977380704a66854748a73250e0671a9318bd5b3fd45a3df9f851ae61629" }, "kernelspec": { "display_name": "Python 3.9.0 ('tf')", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.0" } }, "nbformat": 4, "nbformat_minor": 2 }