Set up your project
ReefCloud Projects define your studies, monitoring programs or research projects. A Project consists of images (photo-quadrats) collected within a set of sites and surveyed once or over repeated periods, and associated annotations and data. Projects can be set up to include Users who can manage the project details (Admin user) or those who can view and analyse the data (General user).
You can use Projects to define data collected for different purposes (e.g., monitoring programs), define specific label-sets and methodology, share data among collaborators, summarise the results, export data in bulk, etc.
Creating a new Project
Once you login into ReefCloud, a list of Projects that either you manage or are a member of is presented in the Projects table. From here you can navigate through your Projects or create a new one.
By clicking on the blue “Create Project” button you will be taken through a wizard that takes you step-by-step through a new project setup, which involves adding project details and choosing a labelset and point sampling method.
Step 1. Project Details.
A Project Name and Project Description are required fields. Try and keep Project Name short and informative, as it may be viewable to other users.
Project Status is used to label your project type. Choose Test Project if you're just trying out ReefCloud—your data might be deleted after a period of inactivity. Choose Active Project for real, ongoing work that needs to be saved long-term.
Project Description is intended to summarise the science behind your project, aims, and any other relevant background information. This will also be viewable by other users, even if your data remain private.
Organisations is an optional field and it is intended to register the different organisations affiliated to this Project. You can select as many organisations as needed from the list. Selecting Organisation can make your project easier for collaborators to find. If you don't see your organisation listed, please email us so we can add you! We're working on an option to request new organisations to be added. If you choose to make your project visible on the ReefCloud Public Dashboard, your organisation will be acknowledged as the contributor.
Global Region captures, at large scale, where the reefs you surveyed are located geographically. We provide a list of Coral Reef Regions defined by the World Resource Institute and you can refer to this link to identify your region or regions of interest. Coral and other benthic species vary regionally, and completing this field helps us to aggregate Projects based on reef biogeography, which can help improve machine learning algorithms for common use. Select all that apply.
Countries is a list to help us identify the domain, in terms of countries, in which your Project operates, and should reflect where your photos were taken. Please select all that apply.
Step 2. Data capture method.
This second section of the Project setup captures metadata on your in-water sampling protocol - your image size, sampling method and the way you sampled across each site.
Sampling Protocol describes the sampling design used for image collection. Select all that apply.
Photo transects represent images positioned along a straight line or following the contour of a natural feature on along the reef, while
Photo quadrats are collections of discrete randomly placed images.
Geo-referenced transects are a special case when every image along a transect line is associated with its own unique set of spatial coordinates (latitude and longitude). Geo-referenced images can be obtained from technology platforms or using a towed GPS unit (see sampling protocols for details).
BRUVS stands for Baited Remote Underwater Video.
Data Collection Method refers to the technique used to collect the images within your project. Images can be collected diving, snorkeling or using remote sensing vehicles (e.g., Autonomous Robots (AUV), Towed Camera, etc). You can select Other and provide details if needed.
Field Of View is a rough estimate of the maximum width of reef area captured within each image. Depending on your camera and survey technique, some scientists take photos very close to the reef, capturing 20-30 cm of reef per image, others might photograph at a greater distance. Providing a value here will help us estimate the footprint of your images to standardise the information we feed to the machine learning algorithm (AI). While in some cases this width is very standard (for example if you use an altimeter or tripod to keep the camera a fixed distance from the substrate) we understand that in other cases this value varies from image to image, and we may not know the exact value. So, we ask for an estimated value as a reference.
Step 3. Setup AI Model
This next section of the Project setup requires you to define how the information will be extracted from the images by creating a label set (your classification scheme) and defining the number and distribution of points.
Choosing a Label Set
Data on the abundance and composition of the benthos are extracted from images by classifying the benthos below a collection of points overlaid on each image. This process is called image annotation and images can be annotated in ReefCloud based on pre-defined classification schemes (your reference taxonomy, referred to here as "Label sets") or by importing you own set of labels into a customised label set.
Defining your label set is an important part of setting up your Project. A user may import their own label set using the Upload Your Own function following the on-screen prompts, or choose to use a default ReefCloud "T1", "T2" and "T3" label sets provided. Editing your label set is possible after project set up.
Large label set risks
There may be a risk for your model by having too many labels (e.g., 1000 labels). Overfitting and underfitting are two common problems across machine learning models. Overfitting occurs when a model learns the training data too well, including noise, and performs poorly on unseen data. Underfitting, on the other hand, happens when a model is too simple and fails to capture the underlying patterns in the data, resulting in poor performance on both training and unseen data. While "too many labels" might seem like it would only lead to overfitting, it can also contribute to underfitting if those labels are not actually informative or representative of the underlying data structure.
Uploading your own label set
New label sets need to be uploaded as a .CSV file and must contain three elements: a Code, a Description and a Functional Group. Download the template to see how it should be set up.
Codes represent a unique short identifier for each of the benthic components you want to include in your analysis.
Description is a longer descriptor of each benthic component, for example a species name, to remind you what your label represents.
Functional Group is a higher-level grouping for labels that will help make larger label sets easier to navigate in when you are working on annotations, and will aid visualisation of your data on the dashboard.
There is also an option to add a "Keyboard Shortcut Code" for each label. This will help you find labels quickly when annotating your photos.
What label set should I use?
Label sets are creatable and customisable for each project and can be as simple or complex as you need. For example, ReefCheck use 10 simple substrate categories to describe for community monitoring, while the Palau International Coral Reef Center employs over 120 labels, including many coral species names.
There is a trade off with the size and complexity of your label set. In general, we recommend users to try to go as in-depth as possible with taxonomy, as information can always be aggregated upwards, but to some extent the machine may learn faster on simpler label sets and this may reduce the number of manual annotations you need to make before the machine is performing well. Bear in mind with inclusions of species that the machine will only be able to identify species if you can confidently identify the species from the photo.
You may already have a label set that your project or organisation uses, for consistency. If you don't, consider following established labels such as the Collaborative and Automated Tools for Analysis of Marine Imagery codes (CATAMI classification), specifically designed to help standardise vocabulary across different studies, commonly used labels such as ReefCheck benthic categories, or the default label sets provided by AIMS.
Drop your .CSV file, and add a Label Set Name and a Label Set Description - a short description of your label set, perhaps including size or intended usage, and then save it to apply to your project.
Once your label set is created it becomes a foundational part of your Project so take your time creating your label set, and check out our FAQ here if you get stuck.
Choosing a default label set
ReefCloud also provides three default label sets, corresponding to different levels of taxonomy with different intended purposes for different monitoring programs, with different levels of intended taxonomy. Once selected, you can modify these labels within your project - adding, removing or editing label descriptions - if you find you need to tailor it towards your specific monitoring objective.
The three default label sets allow us to highlight specific taxonomic units which are important and have these labels defined within the ReefCloud guide for consistency. Please see our Taxonomy Guidelines for more details on the default label sets.
Defining your point sampling approach
Point count methodology, whereby the proportion of randomly overlain points intersecting an organism or substratum is used to estimate its coverage, is a common method for estimating the abundance of benthic organisms on coral reefs.
In ReefCloud, images in your project can be annotated using a different number (e.g., 5 – 50) and distribution of points (e.g., random, equidistant). Use this feature to dictate how many points appear on each photo, and whether they are arranged randomly across photos or in a fixed grid pattern (5-points in an X-shape, or 12 or 20 points in a grid).
Carefully consider the distribution and number of points you select, since this step is part of defining your AI model and can't be changed later on.
How many points per image should I use?
“Human annotatable points” are used to train the model. The optimum number of points is dependent on the number of images, but also on the spread of the images across the habitat you want to sample, and the size, abundance and spatial distribution of the target organisms (see Perkins et al 2016), and also your scientific question and what kind of information you're interested in capturing. Generally, precision is improved with a larger number of images than more points per image.
For broadscale monitoring, 5-10 points are considered adequate for sampling the community, especially where there are plenty of photos within a project. Some research suggests where coral cover is lower a greater number of points are needed (5 points for >60% coral cover, 30 points for <30%). If your goal is to adequately capture all recognisable features within each image >60 points has been recommended (Perkins et al 2016): the machine will give 50 points per image as default. The complexity of your label set will also influence the number of recommended points.
Coming soon, you will be able to define a margin from the edge of the image to exclude points to be considered in this area, and the method used to generate these points.
Step 4. Data Permissions
ReefCloud was developed around the principle of sharing and encourages users to make data available as much as possible. We do recognise that not all data can be made open access because of the nature of data, funding, etc. This section allows users to define how they wish to share data from a project. You can select three categories of data sharing:
Private: Your data will not be visible to other users of ReefCloud. Only the location of sites and contact information will be shared in the public dashboard to allow data to be discoverable and encourage collaboration. Please note that while the data will not be accessible by other users, two exceptions are made:
Your image annotations will be used to improve the state of machine learning that will benefit the science community. This will be only visible to the ReefCloud developers.
The estimation of benthic cover from the images in this project using machine learning may be used to produced aggregated summaries on the state and trend of coral reef communities in your region. These aggregated summaries will be available to the public in a dashboard, but the data on specific locations will not be revealed, only the aggregated summaries (e.g., countrywide, Ocean region, etc).
Visible only: Your data will remain private and no other user, outside of your project users, will be able to access the data. However, the benthic cover estimated from the images in this project will be shown in a public dashboard across surveys and sites. We will also disclose the organisation who contributed in this dashboard in order to encourage collaboration across the science community.
Fully Open access: In this option, you grant ReefCloud a license to openly share the data with other ReefCloud users that request downloading all the data from this project. This data will be protected under a Non-commercial use Creative Commons license that requires users of this data to acknowledge its source.
Important Acknowledgement: Towards the end of this section, you will be asked to agree to an acknowledgement to state how your data will be used within ReefCloud. Please note that you can specify different formats for sharing your data among collaborators, from private to public access. In addition to the data sharing policies that you will manage, the ReefCloud team will be using your data (Images and Annotations) to help the team in the development of ReefCloud and the design of more powerful Machine Learning methods to assist the scientific community. The team is keen to hear your thoughts about this point: [email protected].
Last updated
Was this helpful?