The environment

  • windows 10 64bit
  • Anaconda with python 3.7
  • Pyscenedetect 0.5.5

Introduction to the

Those of you who know video clips know the word transition. A transition from one video scene to another, or a transition between scenes, is called a transition. This article introduces a powerful open source tool PySceneDetect, which is a video scene switch detection and analysis tool based on OpencV, project address: github.com/Breakthroug…

The installation

First, we create a completely new virtual environment

Conda create -n pyScenedetect python=3.7 conda activate PyScenedetectCopy the code

Next, install ffMPEG, which you’ll need for video processing. Download the compiled binary file from the official website ffmpeg.org/download.ht… And add its path to the system environment variable

Finally, the actual installation phase, in fact, in one sentence

pip install scenedetect opencv-python
Copy the code

Yes, it’s as simple as that

Command line test

Pyscenedetect provides a command-line tool to view its help information with the -h argument

Usage: scenedetect [OPTIONS] COMMAND1 [ARGS]... [COMMAND2 [ARGS]...] . For example: scenedetect -i video.mp4 -s video.stats.csv detect-content list-scenes Note that the following options represent [OPTIONS] above. To list the optional [ARGS] for a particular COMMAND, type `scenedetect help COMMAND`. You can also combine commands (e.g. scenedetect [...]  detect-content save- images --png split-video). Options: -i, --input VIDEO [Required] Input video file. May be specified multiple times to concatenate several videos together. Also supports image sequences and URLs. -o, --output DIR Output directory for all files (stats file, output videos, images, log files, etc...) . -f, --framerate FPS Force framerate, In frames/ SEC (e.g. -f 29.97). It's a bit uninformative to ensure that all input videos have the same framerates. -d, --downscale N Integer factor to downscale frames by (e.g. 2, 3, 4...) . where the frame is scaled to width/N x height/N (thus -d 1 implies no downscaling). Each increment speeds up processing by a factor of 4 (e.g. -d 2 is 4 times quicker than -d 1). Higher values can be used for high definition content with minimal effect on accuracy. [default: 2 for SD, 4 for 720p, 6 for 1080p, 12 for 4k] -fs, --frame-skip N Skips N frames during processing (-fs 1 skips every other frame, processing 50% of the video, -fs 2 processes 33% of the frames, -fs 3 processes 25%, etc...) . Reduces processing speed at expense of accuracy. [default: 0] -m, --min-scene-len TIMECODE Minimum size/length of any scene. TIMECODE can be specified as exact number of frames, a time in seconds followed by s, or a timecode in the format HH:MM:SS or HH:MM:SS.nnn [default: 0.6s] -- short scenes drop scenes shorter than '--min-scene-len' instead of combining them with neighbors-s, --stats CSV Path to stats file (.csv) for writing frame metrics to. If the file exists, any metrics will be processed, otherwise a new file will be created. Can be used to determine optimal values for various scene detector options, and to cache frame calculations in order to speed up multiple detection runs. -v, --verbosity LEVEL Level of debug/info/error information to show. Setting to none will suppress all output except that generated by actions (e.g. timecode list output). Can be overriden by `-q`/`--quiet`. -l, --logfile LOG Path to log file for writing application logging information, mainly for debugging. Make sure to set `-v debug` as well if you are submitting a bug report. -q, --quiet Suppresses all output of PySceneDetect except for those from the specified commands. Equivalent to setting `--verbosity none`. Overrides the current verbosity level, even if `-v`/`--verbosity` is set. -h, --help Show this message and exit. Commands: about Print license/copyright info. detect-content Perform content detection algorithm on input video(s). detect-threshold Perform threshold detection algorithm on input video(s). export-html Exports scene list to a HTML file.  help Print help for command (help [command]). list-scenes Prints scene list and outputs to a CSV file. save-images Create images for each detected scene. split-video Split input video(s) using ffmpeg or mkvmerge. time Set start/end/duration of input video(s). version Print version of PySceneDetect.Copy the code

Find a video with multiple scenarios to test it and run the command

scenedetect.exe -i lldq.mp4 detect-content split-video
Copy the code

After the script is run, short videos are generated in the current directory

Each video contains only one scene

If you want to start at some point in the video, you can use the time parameter

scenedetect.exe -i lldq.mp4 time -s 5s detect-content split-video
Copy the code

It can also save the scene pictures after detection and generate a statistical file. CSV

scenedetect.exe -i lldq.mp4 -o video_scenes detect-content save-images
Copy the code

Scene cutting algorithm

Pyscenedetect uses two methods of scene cutting, detect-content and detect-threshold, in addition to supporting custom detection algorithms.

  • detect-content

    As the name suggests, this method is based on the content of the images before and after the judgment, and our common sense is the same as the video transition. The algorithm calculates the different area sizes of the two frames of video. If the area is larger than a preset value (30 by default, which can be specified by the –threshold parameter), the scene is considered to have switched.

  • detect-threshold

    This is a more traditional detection method, a bit like the BLACKFrame filter in FFMPEG. It compares the brightness of the data frame with a specific value, and if it is greater than a preset value, the scene is considered switched. In PySceneDetect, this value is calculated from the average RGB of each pixel of the video frame.

  • Custom detection algorithm

    All detection algorithms must inherit from the SceneDetector class

    from scenedetect.scene_detector import SceneDetector

class CustomDetector(SceneDetector):
    """CustomDetector class to implement a scene detection algorithm."""
    def __init__(self):
        pass

    def process_frame(self, frame_num, frame_img, frame_metrics, scene_list):
        """Computes/stores metrics and detects any scene changes.

        Prototype method, no actual detection.
        """
        return

    def post_process(self, scene_list):
        pass
Copy the code

    There are two main methods in the class. Process_frame handles all video frames. Post_process is optional. It is executed after process_frame and is used for post-processing, such as file saving of scenario switching data.

    Let’s focus on the process_frame method, which has several important parameters

    • Frame_num: The number of frames currently being processed
    • Frame_img: The returned frame data in the format ofnumpyAn array of
    • Frame_metrics: Dictionary that holds the results calculated by the detection algorithm
    • Scene_list: indicates the frame list of all scenes in a video

    More implementation details, you can refer to the source directory scenedetect/detectors/content_detector py or scenedetect/detectors/threshold_detector. Py

Use of the Python API

If you need to use PySceneDetect in your own code, pySceneDetect provides a Python-based API in addition to the command-line invocation.

Here is a simple demo. The program reads the video file, uses the Content-Detector algorithm to detect it, and finally prints out the start time, end time and total frame number of all scenes respectively.

from scenedetect.video_manager import VideoManager from scenedetect.scene_manager import SceneManager from scenedetect.stats_manager import StatsManager from scenedetect.detectors.content_detector import ContentDetector def find_scenes(video_path): video_manager = VideoManager([video_path]) stats_manager = StatsManager() scene_manager = SceneManager(stats_manager) # Use contect-detector scene_manager.add_detector(ContentDetector()) try: video_manager.set_downscale_factor() video_manager.start() scene_manager.detect_scenes(frame_source=video_manager) scene_list = scene_manager.get_scene_list() print('List of scenes obtained:') for i, scene in enumerate(scene_list): print( 'Scene %2d: Start %s / Frame %d, End %s / Frame %d' % ( i + 1, scene[0].get_timecode(), scene[0].get_frames(), scene[1].get_timecode(), scene[1].get_frames(),)) finally: video_manager.release() if __name__ == '__main__': find_scenes('lldq.mp4')Copy the code

The result of code execution is this

The resources

  • Github.com/Breakthroug…
  • Pyscenedetect. Readthedocs. IO/projects/Ma…
  • Blog.gdeltproject.org/using-ffmpe…