Train automated image analysis

Guide to using the "Classify Images" page

An algorithm for automated image analysis runs in the background of ReefCloud to help you extract information from the images, such as benthic cover. This algorithm has been preconfigured using data from AIMS projects to maximise its accuracy. For it to work, you will need to manually analyse (i.e., annotate) several images in your project. This will train the automated algorithm and estimate the labels in the remaining images automatically.

This is an interactive process. The more images you manually analyse the better you train the algorithm. From testing, to adequately training the machine we recommend the user to annotate 15-30% of points. As the algorithm is trained, you can make use of the annotated images to search and review the labelled images using the Classifier 'Filter'.

Please watch the short video below for a step-by-step guide on how to annotate an image.

Note that the location of some buttons and features currently present in ReefCloud may not match the 'How to' video below. We are working on updating these video resources to reflect the updated and improved layout. Thank you for your understanding while this takes place!

Five minute video overview of how to classify images in ReefCloud

ReefCloud Classifier

Images are analysed by annotating benthic categories or labels that correspond to each of the points on a given image in the Classifier. This method is commonly referred to as the Point Count Method, used to estimate the abundance of each identified label in an image (or survey).

Here are a few quick steps to annotate an image in ReefCloud:

1. Click on the “Classify Images” tab from the menu on the left hand side of the screen.

2. Filter the images that you want to annotate using the Filters tool on top left of Classifier screen.

3. Select a point (or multiple points) from the image and select the label to assign from the Label Set panel on the right side of the screen.

Some useful hints...

• Hovering over points will highlight human and machine labels for each point, as well as other information.

• Within the filter options of the Classifier, choose to annotate: all images (default), every 5th image, or every 10th image. This selection allows users to spread their effort more evenly across surveys among images to optimise model training.

• The Classifier Filter provides many options for more targeted and efficient annotations to train the model. Click the Filters button to explore!

The ReefCloud Classifier in Full Image view. See details on buttons and features available below.

Top of Classifier screen

Filters

Filter for images to annotate by Site, Survey, Date, Human v Machine Classified, Group Code, Label, Method, Enabled v Disabled, QC Checked v Not Checked, Country or Local Regions, Reef Names, and Observer (who annotated a given image).

• An option to untick the default 'Only Show Human Annotatable Points' can also be found here, allowing users to see all 50 of the points (including the machine only Transfer Learning Points).

• Image Selection allows users to select which images to annotate in the Classifier from a given survey. This is automatically set to 'All Images' by default, but can be changed to 'Every 5th image' or 'Every 10th image' depending on the total number of images and effort required for optional model training.

  • Watch this space! The ReefCloud team are working on including guidance for users on images to annotate through applying a distance metric which will indicate the most different images in a given survey to better focus annotating effort to train the model.

• To go back to default settings, click .

Disable Image

Disabling an image will keep the image in the project but exclude from analysis.

Enable Image

If an image has been disabled, clicking Enable Image will enable it for inclusion in analysis.

Full Image View

See the whole image with all Human Annotatable Points.

Point View

See all points displayed as patches in a grid format. This also allows users to annotate images in a hybrid format using the f shortcut, and see points alongside the full image.

Image & Point Brightening

Adjust the brightening of images or selected points using the slider. Original brightness can be reset using the button to the right of the slider.

Before and after image brightening in the ReefCloud Classifier.

Image & Point Enhancement

A feature allowing users to enhance image quality while in the Classifier. The feature can be toggled on/off. Note that this feature is at presentation-level in the user-interface and independent of the machine learning model.

Before and after image enhancement in the ReefCloud Classifier.

Annotation Marker

Update or reset your marker settings using this button; including preferences for using the crosshair, secondary colours, and the colour choice, size and opacity for all markers.

Quality Control

Toggle the Quality Control feature on and off using this button. Select a Human Label / Group Code from your label set to start the reviewing the human annotations with your project.

Bottom of Classifier screen (Full Image view)

Show Annotations

Toggle on point annotation information, including:

• Machine: label assigned by the model

• Human: label assigned by a user

• Tag: the tag applied to the point to provide context to an annotation

Show Points

View image points in patch view along the bottom of the page in a hybrid format.

Tag Images

Update image tags using Reserved Tags or Custom Tags. Note images can also be tagged via the Shortcut T .

Label Set panel on right

Search Label Set

Search for the Group Code, Code, Shortcut, or Description of a code to select and use label when annotating points.

Column Display Options

Choose which columns are displayed for a given Label Set, including Group Code, Code, Shortcuts, and Description.

Shortcuts

View information on the keyboard shortcuts available within the Classifier via the Shortcuts button. Use the button if you'd like to have a copy on hand while familiarising yourself or team with the shortcut options.

Tour

Click Tour to quickly explore the features within the Classifier page.

Check out how your training of AI model is going by visiting the Model Report page and dynamic Printable Report which provides guidance what actions you might need to take to improve performance.