Fix a deadlock that appears when using a substream

What's Changed

• resolve deadlock when using substream by @roflcoopter in https://github.com/roflcoopter/viseron/pull/302

Full Changelog: https://github.com/roflcoopter/viseron/compare/v1.10.1...v1.10.2

Bugfix release, see autogenerated changelog below

What's Changed

• properly override config for motion detection by @roflcoopter in https://github.com/roflcoopter/viseron/pull/285
• v1.10.1 by @roflcoopter in https://github.com/roflcoopter/viseron/pull/286

Full Changelog: https://github.com/roflcoopter/viseron/compare/v1.10.0...v1.10.1

The work on a UI has started thanks to @danielperna84
As of now you can view your MJPEG streams, watch recordings and also change the config.yaml file from the interface.

There are also new Contribution guidelines which hopefully makes it easier for folks to dive into the project.

Breaking changes

• Decoder h264_nvv4l2dec and hevc_nvv4l2dec is no longer available for the Jetson Nano.
  Use h264_nvmpi or hevc_nvmpi instead.
• Frames that are older than two seconds will now be dropped from the detection queue.
  If you have a slow system you might have to raise this number by using the max_frame_age config option.

New features

• We now have a basic UI!
  Thanks to @danielperna84 the work on a UI has begun.
  Functionality is right now pretty limited, but will be improved upon.
  By releasing this we hope to get some more contributors on board.
• New config option device for edgetpu detector.
  Lets you specify which EdgeTPU device to use if you have multiple.
• New config option mask under a cameras object_detection block.
  Use this to completely ignore objects that are within the mask.
  Closes #199
• There are now multiple kinds of motion detectors. Check out Background subtractor MOG2 which might be more reliable for you.
  • Background subtractor (the default, same motion detection as always)
  • Background subtractor MOG2
• The status sensor for each camera now has a new state, disconnected, when cameras cant be reached.
  Use this information to know if Viseron is connected to all your cameras and possibly send an alarm on disconnects.
• New config option max_frame_age. Frames that are older than this number will be dropped from the detection queue.
  If you have a slow system you might need to raise this number.

Changes

• Jetson Nano image now defaults to h264_nvmpi and hevc_nvmpi decoders.
• Upgrade to CUDA 11.4.1 for amd64-cuda-* images

Fixes

• Fixed a bug that could cause object detection to return duplications across all cameras when running YOLO on CUDA

Docker images will be on Docker Hub shortly

Automatically generated changelog

• Pytest by @roflcoopter in https://github.com/roflcoopter/viseron/pull/222
• list all changed files in ci by @roflcoopter in https://github.com/roflcoopter/viseron/pull/240
• Edgetpu pci permissions by @roflcoopter in https://github.com/roflcoopter/viseron/pull/242
• Object mask by @roflcoopter in https://github.com/roflcoopter/viseron/pull/243
• build nvmpi and pin ffmpeg=4.2 for jetson nano by @roflcoopter in https://github.com/roflcoopter/viseron/pull/244
• pin numpy and install scikit-learn from pip by @roflcoopter in https://github.com/roflcoopter/viseron/pull/245
• Revert "pin numpy and install scikit-learn from pip" by @roflcoopter in https://github.com/roflcoopter/viseron/pull/247
• set max size in detector queue by @roflcoopter in https://github.com/roflcoopter/viseron/pull/248
• Update README.md by @magicmonkey in https://github.com/roflcoopter/viseron/pull/255
• fix libedgetpu1 installation by @roflcoopter in https://github.com/roflcoopter/viseron/pull/265
• Simple UI by @danielperna84 in https://github.com/roflcoopter/viseron/pull/266
• Motion detection revamp by @roflcoopter in https://github.com/roflcoopter/viseron/pull/269
• upgrade to cuda 11.4.1 by @roflcoopter in https://github.com/roflcoopter/viseron/pull/268
• set camera status on disconnect by @roflcoopter in https://github.com/roflcoopter/viseron/pull/273
• Detector duplication fix by @roflcoopter in https://github.com/roflcoopter/viseron/pull/274
• OpenCV 4.5.3 by @roflcoopter in https://github.com/roflcoopter/viseron/pull/275
• implement basic API boilerplate by @roflcoopter in https://github.com/roflcoopter/viseron/pull/272
• fix typo in opencv build command by @roflcoopter in https://github.com/roflcoopter/viseron/pull/276
• try to speed up OpenCV build time by @roflcoopter in https://github.com/roflcoopter/viseron/pull/277
• add dummy setup.py file to make tox happy by @roflcoopter in https://github.com/roflcoopter/viseron/pull/282
• Dev by @roflcoopter in https://github.com/roflcoopter/viseron/pull/283

New Contributors

• @magicmonkey made their first contribution in https://github.com/roflcoopter/viseron/pull/255
• @danielperna84 made their first contribution in https://github.com/roflcoopter/viseron/pull/266

Full Changelog: https://github.com/roflcoopter/viseron/compare/v1.9.0...v1.10.0

This release brings support for DeepStack object detection aswell as face detection.
Another exciting feature is the support for the Jetson Nano!
Other than that I have spent a lot of effort to increase the stability of Viseron.
Viseron now handles missbehaving cameras in a much better way.
If a camera is unavailable during startup, Viseron will continue to try and reconnect. This is to help against temporarily unavailable cameras.
Viseron carefully monitors any subprocesses or threads that are started, and in the case of one crashing, it will be restarted accordingly.

There are a few more goodies in this one, aswell as some breaking changes. Make sure to read the changelog below.

Enjoy!

Breaking changes

• triggers_recording under labels has been deprecated.
  Use trigger_recorder instead. A warning will be printed if you still use triggers_recording, reminding you to change your config.
• The endpoint /<camera>/stream has been renamed to /<camera>/mjpeg-stream. If you still use the old endpoint Viseron will warn you and then redirect to the new URL. In the future this will produce an error.

New features

• DeepStack object detection is now available.
  See the new section in the README
• The post processor class is now more general which means adding new processors is a piece of cake.
• DeepStack face recognition is now available.
  See the new section in the README
• Jetson Nano support.
  An image specifically made for the Jetson Nano is now available: roflcoopter/jetson-nano-viseron.
  Hardware accelerated decoding of cameras is supported, aswell as YOLOv4 with CUDA backend.
• New config option filename_pattern under recorder and thumbnail. Allows you to change the output filename of recordings and thumbnails.
• New config option ffprobe_loglevel
• An alias is created for each ffmpeg process to make it easy to spot which ffmpeg process belongs to which camera.
  If you have a camera called Front Door, your ffmpeg command will start with front_door instead of ffmpeg
• New config option color_logs which controls if logs are colored or not. Enabled by default.
• New config option trigger_recorder under motion_detection which allows motion to start recordings.
• New config option enable under object_detection which allows you to disable object detection if you want to run motion_detection only.

Changes

• All docker images are now based on Ubuntu Focal(20.04) which ships Python 3.8
• OpenCV upgraded to 4.5.2
• FFmpeg upgraded to 4.4
• dlib upgraded to 19.22
• OpenCL upgraded to 21.15.19533
• The detector class is now more general which means adding new detectors is a piece of cake.
• amd64-viseron image now once again defaults to YOLOv3.
  This is done because YOLOv4 has performance issues that are not present when running CUDA.
  YOLOv4 models are still provided so you can pick and choose which one you like if YOLOv3 does not suit you.
• Username and passwords are now redacted from logs in the FFmpeg command
• All long-running theads are now monitored and automatically restarted in case of an exception.
  The FFmpeg process is also monitored. Closes #109. Closes #11. Closes #113
• Permissions are now automatically set when not running as root for EdgeTPU and all hardware acceleration methods. Closes #173. Closes #176
• Viseron now once again runs as root by default. This is to make sure Viseron always works for new users.
  Setting PUID and PGID to something other than root works almost always so if you rather not run as root it should work fine.
• Subprocesses like the one used for segments only is now monitored and restarted accordingly.
• ffmpeg errors will now be logged to the standard python logging instead of being suppressed
• Cameras are now setup completely independent of eachother.
  Previously if one camera was blocking, no other camera would process frames.
• If setup of a camera fails, the setup process will be retried indefinitely.
  This is done to guard against cameras that are temporarily unavailable at startup.

Fixes

• Fix a crash caused by using require_motion: true on a label inside a zone while not tracking it in the entire field of view.

Docker images will be on Docker Hub shortly

1.8.0 - Docker revamp

This release brings a huge revamp to the Docker containers.
This was a very big change and some bugs might still be present.
There is also experimental support for the RPi4 which might not work as expected in this first version.
The support for RPi4 will improve in the coming version, but i need your help with testing and reporting bugs.

Breaking changes

• Docker image names have changed according to this table.
  Note that roflcoopter/viseron is now a multiarch image which means you dont need to pick between the images yourself, unless you want CUDA support.

  Old	New
  roflcoopter/viseron	roflcoopter/amd64-viseron
  roflcoopter/viseron-cuda	roflcoopter/amd64-cuda-viseron
  roflcoopter/viseron-vaapi	Removed, same as roflcoopter/amd64-viseron
  roflcoopter/viseron-rpi3	roflcoopter/rpi3-viseron
  N/A	roflcoopter/aarch64-viseron

• port is now required for substream

Changes and new Features

• New config option require_motion for labels.
  If set to true, the recorder will stop when motion is no longer detected, even if the object still is.
  This is useful to avoid never ending recordings of stationary objects, such as a car on a driveway.

• Complete rewrite of all Dockerfiles(!).
  Multistage builds are now used extensively which dramatically reduces the size of the containers.
  All containers are now built on Azure Pipelines which means i no longer have to build them all locally(!!).
  Cross-building is done using Balenalibs baseimages, which means we now have (experimental) support for the RPi4(!!!).
  This new way of working with containers mean i can easily support different hardware, such as the Jetson Nano in the near future.
  Multiarch images are also in play, which means you dont need to pull different images based on your architecture, unless you want a specific one, like the amd64 CUDA version. Closes #1, closes #66

• Static MJPEG streams can now be configured, which provides better performance due to processing only happening once
  See the new section on Static MJPEG Streams in the README for more information. Closes #23

• An MJPEG stream is now served for each camera.
  A number of query parameters are available to control resolution, what is drawn on the frames etc.
  See the new section in the README for more information. Closes #23

• stream_format and port is now supported for substream. Closes #112

• Viseron no longer runs as root in the containers. You can now set PUID and PGID as environment variables to control the user.
  If you are using docker-compose it might look like this:

  version: "2.4"
  
  services:
    viseron:
      image: roflcoopter/viseron:latest
      container_name: viseron
      volumes:
        - <recordings path>:/recordings
        - <config path>:/config
        - /etc/localtime:/etc/localtime:ro
      environment:
        - PUID=1000
        - PGID=1000
  

• Pin libedgetpu-max to avoid version mismatch

• New config option save_unknown_facesfor dlib face recognition.
  If set, unknown faces will be saved in a folder named unknown next to your regular faces folders.
  These can then be manually moved to the folder of the correct person to improve accuracy. Closes #90, Closes #96

• Audio is now recorded by default if the camera supports it. If you want you can specify the audio codec with audio_codec under recorder. This feature is experimental and i need some testers on it.

• Default to YOLOv4 instead of YOLOv4-tiny. You can go back yo using YOLOv4-tiny by changing model_path.

Fixes

• The labels file was corrupted in the amd64* images, so labels of objects made no sense.
• Better reporting of errors when loading EdgeTPU, fixes #76

Docker images will be on Docker Hub shortly

1.8.0b6

Changes and new Features

• Pin libedgetpu-max to avoid version mismatch

• New config option save_unknown_facesfor dlib face recognition.
  If set, unknown faces will be saved in a folder named unknown next to your regular faces folders.
  These can then be manually moved to the folder of the correct person to improve accuracy. Closes #90, Closes #96

• Audio is now recorded by default if the camera supports it. If you want you can specify the audio codec with audio_codec under recorder. This feature is experimental and i need some testers on it.

• Default to YOLOv4 instead of YOLOv4-tiny

Fixes

• Permissions to access /dev/dri to utilize VAAPI

Previous 1.8.0 betas

Breaking changes

• port is now required for substream

Changes and new Features

• New config option require_motion for labels.
  If set to true, the recorder will stop when motion is no longer detected, even if the object still is.
  This is useful to avoid never ending recordings of stationary objects, such as a car on a driveway.

• Complete rewrite of all Dockerfiles(!).
  Multistage builds are now used extensively which dramatically reduces the size of the containers.
  All containers are now built on Azure Pipelines which means i no longer have to build them all locally(!!).
  Cross-building is done using Balenalibs baseimages, which means we now have (experimental) support for the RPi4(!!!).
  This new way of working with containers mean i can easily support different hardware, such as the Jetson Nano in the near future.
  Multiarch images are also in play, which means you dont need to pull different images based on your architecture, unless you want a specific one, like the amd64 CUDA version. Closes #1, closes #66

• Static MJPEG streams can now be configured, which provides better performance due to processing only happening once
  See the new section on [Static MJPEG Streams(https://github.com/roflcoopter/viseron#static-mjpeg-streams) in the README for more information. Closes #23

• An MJPEG stream is now served for each camera.
  A number of query parameters are available to control resolution, what is drawn on the frames etc.
  See the new section in the README for more information. Closes #23

• stream_format and port is now supported for substream. Closes #112

• Viseron no longer runs as root in the containers. You can now set PUID and PGID as environment variables to control the user.
  If you are using docker-compose it might look like this:

  version: "2.4"
  
  services:
    viseron:
      image: roflcoopter/viseron:latest
      container_name: viseron
      volumes:
        - <recordings path>:/recordings
        - <config path>:/config
        - /etc/localtime:/etc/localtime:ro
      environment:
        - PUID=1000
        - PGID=1000
  

Fixes

• The labels file was corrupted in the amd64* images, so labels of objects made no sense.
• Better reporting of errors when loading EdgeTPU, fixes #76

Docker images will be on Docker Hub shortly

Changes and new Features

• New config option require_motion for labels.
  If set to true, the recorder will stop when motion is no longer detected, even if the object still is.
  This is useful to avoid never ending recordings of stationary objects, such as a car on a driveway.

Fixes

• The labels file was corrupted in the amd64* images, so labels of objects made no sense.

Previous 1.8.0 betas

Breaking changes

• port is now required for substream

Changes and new Features

• Complete rewrite of all Dockerfiles(!).
  Multistage builds are now used extensively which dramatically reduces the size of the containers.
  All containers are now built on Azure Pipelines which means i no longer have to build them all locally(!!).
  Cross-building is done using Balenalibs baseimages, which means we now have (experimental) support for the RPi4(!!!).
  This new way of working with containers mean i can easily support different hardware, such as the Jetson Nano in the near future.
  Multiarch images are also in play, which means you dont need to pull different images based on your architecture, unless you want a specific one, like the amd64 CUDA version. Closes #1, closes #66

• Static MJPEG streams can now be configured, which provides better performance due to processing only happening once
  See the new section on [Static MJPEG Streams(https://github.com/roflcoopter/viseron#static-mjpeg-streams) in the README for more information. Closes #23

• An MJPEG stream is now served for each camera.
  A number of query parameters are available to control resolution, what is drawn on the frames etc.
  See the new section in the README for more information. Closes #23

• stream_format and port is now supported for substream. Closes #112

• Viseron no longer runs as root in the containers. You can now set PUID and PGID as environment variables to control the user.
  I you are using docker-compose it might look like this:

  version: "2.4"
  
  services:
    viseron:
      image: roflcoopter/viseron:latest
      container_name: viseron
      volumes:
        - <recordings path>:/recordings
        - <config path>:/config
        - /etc/localtime:/etc/localtime:ro
      environment:
        - PUID=1000
        - PGID=1000
  

Fixes

• Better reporting of errors when loading EdgeTPU, fixes #76

Docker images will be on Docker Hub shortly

Note: Re-release of 1.8.0b3 due to errors in pipeline.

Breaking changes

• port is now required for substream

Changes and new Features

• Complete rewrite of all Dockerfiles(!).
  Multistage builds are now used extensively which dramatically reduces the size of the containers.
  All containers are now built on Azure Pipelines which means i no longer have to build them all locally(!!).
  Cross-building is done using Balenalibs baseimages, which means we now have (experimental) support for the RPi4(!!!).
  This new way of working with containers mean i can easily support different hardware, such as the Jetson Nano in the near future.
  Multiarch images are also in play, which means you dont need to pull different images based on your architecture, unless you want a specific one, like the amd64 CUDA version. Closes #1, closes #66

• Static MJPEG streams can now be configured, which provides better performance due to processing only happening once
  See the new section on [Static MJPEG Streams(https://github.com/roflcoopter/viseron#static-mjpeg-streams) in the README for more information. Closes #23

• An MJPEG stream is now served for each camera.
  A number of query parameters are available to control resolution, what is drawn on the frames etc.
  See the new section in the README for more information. Closes #23

• stream_format and port is now supported for substream. Closes #112

• Viseron no longer runs as root in the containers. You can now set PUID and PGID as environment variables to control the user.
  I you are using docker-compose it might look like this:

  version: "2.4"
  
  services:
    viseron:
      image: roflcoopter/viseron:latest
      container_name: viseron
      volumes:
        - <recordings path>:/recordings
        - <config path>:/config
        - /etc/localtime:/etc/localtime:ro
      environment:
        - PUID=1000
        - PGID=1000
  

Docker images will be on Docker Hub shortly

Removed, see next release

Fixes

• filter_args now works again @olekenneth (#126 ), fixes #123

Docker images are available on Docker Hub
roflcoopter/viseron:1.7.3
roflcoopter/viseron-cuda:1.7.3
roflcoopter/viseron-vaapi:1.7.3
roflcoopter/viseron-rpi:1.7.3

Changes and new Features

• An MJPEG stream is now served for each camera.
  A number of query parameters are available to control resolution, what is drawn on the frames etc.
  See the new section in the README for more information.

Fixes

• Better reporting of errors when loading EdgeTPU, fixes #76

Docker images are available on Docker Hub
roflcoopter/viseron:1.8.0b1
roflcoopter/viseron-cuda:1.8.0b1
roflcoopter/viseron-vaapi:1.8.0b1
roflcoopter/viseron-rpi:1.8.0b1

Fixes

• Check if MQTT is enabled before trying to publish, fixes #97

Docker images are available on Docker Hub
roflcoopter/viseron:1.7.2
roflcoopter/viseron-cuda:1.7.2
roflcoopter/viseron-vaapi:1.7.2
roflcoopter/viseron-rpi:1.7.2

1.7.1 - Hotfix for saving thumbnail

Fixes

• Correctly save thumbnail to disk, filepath is /recordings/thumbnails//latest_thumbnail.jpg, fixes #91

Docker images are available on Docker Hub
roflcoopter/viseron:1.7.1
roflcoopter/viseron-cuda:1.7.1
roflcoopter/viseron-vaapi:1.7.1
roflcoopter/viseron-rpi:1.7.1

Breaking changes

• Recorder global_args can no longer be specified

Changes and new Features

• FFmpeg segments are now used to record instead of caching frames in memory
• A substream can now be configured which will be used for image processing.
  This can be used to great success to reduce system load.
• Upgrades to CUDA 11 and OpenCV 4.5.0 for CUDA image
• Recordings are encoded to /tmp, then moved to target directory when done. Closes #49
• Installs tzdata package. Closes #55
• A new config option, thumbnail is available under recorder.
  Use this to send a snapshot of the detection that triggered a recording to MQTT, or choose to save to disk
• A new sensor entity is created in Home Assistant which indicates the status of Viseron.
  The state is set to recording, scanning_for_motion or scanning_for_objects
• Cameras are now setup in parallel instead of sequentially. This significantly increases startup speed with many cameras

Fixes

• Fixed a formatting issue with the duplicate log filter
• Returncodes > 0 from MQTT is now logged as errors in text format. Closes #59
• Fixes segments process not starting when using substream

Docker images are available on Docker Hub
roflcoopter/viseron:1.7.0
roflcoopter/viseron-cuda:1.7.0
roflcoopter/viseron-vaapi:1.7.0
roflcoopter/viseron-rpi:1.7.0

1.7.0b3 - Segments and Substream

Changes and new Features

• A new sensor entity is created in Home Assistant which indicates the status of Viseron.
  The state is set to recording, scanning_for_motion or scanning_for_objects
• Cameras are now setup in parallel instead of sequentially. This significantly increases startup speed with many cameras

Docker images are available on Docker Hub
roflcoopter/viseron:1.7.0b3
roflcoopter/viseron-cuda:1.7.0b3
roflcoopter/viseron-vaapi:1.7.0b3
roflcoopter/viseron-rpi:1.7.0b3

1.7.0b2 - Segments and Substream

Changes and new Features

• Installs tzdata package. Closes #55
• A new config option, thumbnail is available under recorder.
  Use this to send a snapshot of the detection that triggered a recording to MQTT, or choose to save to disk

Fixes

• Fixes segments process not starting when using substream

Docker images are available on Docker Hub
roflcoopter/viseron:1.7.0b2
roflcoopter/viseron-cuda:1.7.0b2
roflcoopter/viseron-vaapi:1.7.0b2
roflcoopter/viseron-rpi:1.7.0b2

1.7.0b1 - Segments and Substream

Breaking changes

• Recorder global_args can no longer be specified

Changes and new Features

• FFmpeg segments are now used to record instead of caching frames in memory. Closes #26
• A substream can now be configured which will be used for image processing.
  This can be used to great success to reduce system load. This is huge for smaller devices like an RPi, coupled with the segments above.
• Upgrades to CUDA 11 and OpenCV 4.5.0 for CUDA image
• Recordings are encoded to /tmp, then moved to target directory when done. Closes #49

Fixes

• Fixed a formatting issue with the duplicate log filter
• Returncodes > 0 from MQTT is now logged as errors in text format. Closes #59

Docker images are available on Docker Hub
roflcoopter/viseron:1.7.0b1
roflcoopter/viseron-cuda:1.7.0b1
roflcoopter/viseron-vaapi:1.7.0b1
roflcoopter/viseron-rpi:1.7.0b1

Fixes a crash which was detected during troubleshooting of #62 which occurs when MQTT is not connected.

Note: Due to some unfortunate circumstances the cuda image will not be built with this fix.
Instead the fix will be included in 1.7.0 which should come a long shortly.

Docker images are available on Docker Hub
roflcoopter/viseron:1.6.2
roflcoopter/viseron-vaapi:1.6.2
roflcoopter/viseron-rpi:1.6.2

Fixes #60 through #61

Docker images are available on Docker Hub
roflcoopter/viseron:1.6.1
roflcoopter/viseron-cuda:1.6.1
roflcoopter/viseron-vaapi:1.6.1
roflcoopter/viseron-rpi:1.6.1