In this guide, we will walk you through how to use the frame annotation interface to create labels on sequences of frames of a video. Before you start, you should have created at least one batch of jobs for frame annotations and have labels that allow shapes within the profile assigned to those batches. If you need to do those things, consult our guides on how to create an annotation profile and how to create frame annotation batches.
Core functionalities
The annotation interface will change according to what is available to your project and to what you are currently doing in the interface. When you first enter it, it will look like the image below.
The following fields available in the frame annotation interface
Shapes panel is where you will go when you want to annotate a new shape. You can hover over each symbol and click on it to add the shape that you wish. For any shape, if you want to cancel the annotation, press the “esc” key. Depending on what is available in your profile, you will have options to add:
A bounding box is a rectangle shape that is defined by two opposing corners. You annotate this shape by positioning your mouse where you want the first corner to be and clicking, then moving your mouse to where you want the second corner to be and clicking again.
A polygon is a closed shape with an arbitrary number of points. You annotate this shape by clicking to define the outline of the polygon and pressing “enter” to conclude the annotation. The last point you entered will be joined with the first one to form the closed shape.
A line is an open shape with an arbitrary number of points. You annotate this shape in the same way as the polygon, with the difference that the last point will not be joined with the first one.
A keypoint is a single point in the frame that you want to annotate. You annotate it by clicking on where you want the keypoint to be.
If you have enabled polygon annotations in your project you will also see the symbol for AI polygon generation. This uses the model that you have selected in the profile to generate polygons. To generate a polygon, click on the wand and then click on some points within the structure for which you want a polygon to be generated; you can use the left and right arrows to cycle through different options and the slider that will appear in the left toolbar to add or remove detail from the polygon; when you are happy with the polygon press “enter” to keep the shape; alternatively, you can press “esc” to delete the shape without saving it.
Under the shapes panel, you will find some general functionality to interact with labels. Specifically:
Undo the last command.
Show locations where comments have been made.
Show label tags in the frame.
Invert the zoom commands.
Toggle visibility of the timeline.
Timeline showing the places in the video where there are sequences to annotate. Note that this timeline only shows the sequences to annotate in the current batch. If the video has more sequences in a different batch, they will not appear on this timeline.
Information about the current sequence. Note that all the information corresponds to the frame sequence that is currently active. The information shown is:
Button to display sequence information.
Keyboard shortcuts available for annotation.
QA warnings or errors.
Reset zoom.
Indicator of whether the current frame has been seen already by the annotator.
Person assigned to the current workflow step.
The current workflow step.
Navigation controls which enable you to move through the frames in the current sequence or switch between the sequences in a video. You can use the arrows in the UI to move between sequences or you can directly type the number of the frame or sequence that you want to jump to.
Tag and annotation information:
Toggle to select sequence to annotate.
Toggle to add a tag to the current sequence.
Toggle to switch between different versions of annotations for the sequence.
Toggle to add an annotation. Note that once you select the label here, you will then need to select which shape to use for the annotation (see 1)
Toggle to add a skeleton. Skeletons and how to use them will be covered in a different guide.
Annotation management panel. Here you can find functionality to manipulate the annotations that are being made. For each label, you will find buttons to propagate the label (see below), to copy the annotations and to delete them. A single annotation can have multiple shapes in it, to add more shapes you need to select the annotation where you wish to add a polygon and then use the shapes panel to add a shape. If you don’t wish to add another shape to the same label, then you can press “esc” and create a new label.
Image controls. In this section you can change the thickness of the outlines of the shapes and their opacity and you can change the image brightness and contrast.
AI-assisted propagation
This functionality must be enabled in the annotation profile before it shows in the editor. Assuming that you have these enabled, the annotation management panel will display some new functionality:
The left and right arrows will propagate the selected shape or groups of shapes from the current frame until the start and end of the current sequence, respectively, using the AI model that is selected in the profile. A new panel (Propagation) will appear on top of the “Controls” panel where you can refine how you want to propagate: you can select a specific group or subgroup of shapes and which frames from the current sequence to propagate from and to.
Completing an annotation task
Once you have placed all the polygons for all the frames in a sequence, you can click the “finish” button, which will move the sequence to the next annotation step. Note that even though you can see all the sequences in the timeline, the annotation of each sequence must be completed on its own (i.e. click the “Finish” button on every sequence annotated). If your sequence requires you to view all frames, you may not be able to complete the annotation until you have seen every frame at least once.
What next?
Once you have done all the annotations you need, you can move on to exporting sequence data.