In SAVANT, we call “frame annotation jobs” any job that requires labelling objects in the frames of a video. If you need to label timestamps in a video, consult our guide on creating video annotation jobs.
Frame batch creation basics
The diagram below shows how SAVANT treats a video to create jobs to annotate individual frames. The process requires the following:
Every video is processed as a collection of frames (still images), which are played at a given frame rate specified in frames per second (FPS).
Users of SAVANT must select a frame rate (FPS) at which they want to annotate the video. This frame rate should be lower or equal to that of the original video. Important: it is not possible for SAVANT to check the original frame rate, if you select a higher frame rate this will result in duplicate frames with no warnings or errors.
SAVANT extracts the required frames from the original video at the FPS requested by the user. This is called “deframing”. In the diagram below, the video is being deframed at half of the original frame rate.
The frames that have been extracted are then grouped into annotation jobs (called “sequences”), again the number of frames to put in each sequences is defined by the user. In the diagram below, the user has selected three frames per sequence. The number of frames in each sequence will define how fast an annotator can finish a sequence and how fast data can be ready for use.
Finally, annotation jobs are grouped into batches for annotation. In the diagram below, the user has selected two sequences per batch. Annotation jobs can be split into batches if, for example, the annotation work is going to be split into separate teams.
To configure this process and create jobs for frame annotations, you can follow the instructions in the next section.
Frame batch creation flow
Where to start the flow
To create annotation jobs for frame sequences you will need to follow the batch creation flow. You can access the flow from three places in the app:
In the video registry, select the videos that you want to annotate, click on the “Actions” button and then “Create Frame Batches”.
In the Files section, go to the folder where you have uploaded your videos, then go to “Registered Videos”, select the videos that you want to annotate and then in the “Actions” button you will see the “Create Frame Batches” options.
Within the Project where you want to create the batches, click on the “Create Batches” button where you will then see the “Frame Batches” option.
The three images below show where to find each of the options.
In the video registry:
In the Files page:
In the project page:
If you start the flow using (1) or (2), you will need to provide a project where the batches will be created, make sure that you have previously created a project. If you select (3) then you will need to have at hand a list of video IDs that you want to annotate.
Configuring the batch creation flow
You will need to specify all the following configurations in order to create frame annotation batches.
Select the batch creation method
The first decision to make is which method you want to use to create batches. We currently offer to options:
Full videos will use the entire duration of the videos in the deframing process (see diagram at the top)
Timestamp annotations will let you select only sections of each video that will be used when deframing. To use this method you should have labelled the sections that you want previously as timestamp annotations. If you haven’t done so, see the guides on how to set up and complete timestamp annotations.
The steps for both flows are the same, except that with (1) you will skip the label selection step.
Project setup
In this page you will need to provide the following fields:
New configuration name. This will help identify all the settings that you are about to provide so you can revisit (or reuse) them later.
Batch description. It’s a short description that will be attached to every batch created from this flow.
Annotation profile. You must specify the profile that will be used in the annotation jobs. This profile must exist in the workspace before-hand.
Video projects source. Specify the name of the project(s) where you have made the annotations that you will use to select the portions of the videos to annotate. Note that this is only available if you are using the timestamp annotations method.
Select labels (only for timestamp annotations method)
For each project that you have selected in the previous step, you must now select the labels that should be used to select the sections of the videos to deframe. The batches of annotation jobs will then be created using only the sections of the videos that are within the labels that you have selected.
Frame configurations
In this section you will configure the deframing, sequence creation and batch creation steps (see the flow at the top of this guide). You will need to specify the following settings:
Sequences per batch. This will define how many sequences of frames to include in each batch. Batches are currently limited to 500 sequences to maintain the performance of the app. In practice, it is rare that you will create this many sequences in a single batch.
Frames per sequence. Defines how many frames will be included in each sequence. Remember that one sequence is equivalent to one annotation job, the more frames per sequence, the longer it will take to complete its annotation. We currently limit sequences to have a maximum of 100 frames in them, this is to ensure the performance of the AI algorithms that we use to assist annotation.
Frame rate. Is used to specify the deframing frame rate. As shown in the diagram at the top this will ultimately define how much data needs to be annotated. We recommend to use frame rates under 10 FPS. If you select a high frame rate, you must ensure that it is lower than the videos' original frame rate, otherwise you will get duplicate frames within the sequences.
Before/After seconds. Allows you to specify whether you want to include frames before the start or after the end of the annotation labels selected. (Only relevant if using Timestamp Annotations method)
Image output configuration
This section is entirely optional, it allows you to specify the desired pixel height of the frames (the width is automatically selected to maintain the original ratio) and whether you want the frames to be saved as JPEG or PNG.
Select video UUIDs
If you have started this flow from the video or files registry, you can skip this step; the list will be pre-populated for you. If you have started the flow from the project page, then you have three options to select the video IDs:
Use all the videos from the video labels projects. With this option you will create batches for annotation from every video in the projects that you have previously selected as source for timestamp annotations.
Use videos from specific batches. With this option you can select only specific batches from the aforementioned projects.
Specify video IDs. If you have a list of video IDs that you would like to annotate, then you can copy and paste them directly into the text box and only those videos will be used.
Once you have selected the video IDs you wish to use, you must click the “Validate” button. You will then see a box appear where you will see which videos are ready for processing (in green) and which videos will not be processed due to some error (in red). Videos shown in red will be left out of the following processing steps and there will be no resulting frames from these in any annotation jobs.
Summary page
Finally, once you have completed all the settings, you will see a summary where you can check the details of your batch creation configuration and how much data is expected to be produced with the settings you have provided. If all the settings are as you expect then you can save the configuration and click on “Process” to start creating the batches of annotation jobs.
Tracking batch creation progress
The process of deframing and creating annotation jobs can take in the order of a few minutes. You can check progress by clicking on the “View processing batches” action button on the project page.
You can then navigate to the “Processing” section to view batches that are still in progress.
When a batch is done processing, it will not immediately appear in your project page. Instead, it will appear in the “Ready for review” tab. This step is intended for you to spot any errors in the batch creation settings before adding all the new batches to your project. For each batch created you can click into it and see all the sequences. If you are happy with the batch, then click on “Approve” and the batch will appear in your project page ready for annotation. If you don’t want to annotate the batch, then click on the trash symbol and it will be deleted.
Managing and re-running batch creation configurations
If you click the “Configurations” button on the project page you will open the batch creation configurations management page. Here you will see a list of the different batch creation configurations that have been created and run in this project. From this page, for each configuration, you can:
Use the configuration as a template. Will launch the batch creation flow with all the fields pre-populated with this configuration. You can then edit any fields you want to create a new configuration.
View runs history. Which will take you to a new page where you can see the settings and results of every run that has used this configuration.
Re-run. Allows you to run the same configuration with new video IDs.
What next
You should now have created the batches of annotation jobs with all the frames you want to annotate. The next step is to use the frame annotation interface to annotate the data.