Fall 2021 Updates: Automated Quality Control Tools and More

The Collect Earth Online (CEO) team is excited to announce the launch of several new features that we’ve added to the platform in response to feedback from our community of users. The features include automated quality control tools, improvements to plot navigation, and more. 

✨Automated quality control tools

CEO has implemented automated quality assurance and quality control (QA/QC) tools for projects. 

If you are an institution administrator in CEO, you can now assign members of your institution to plots with the User Assignment feature. You can distribute plots among your members either equally or by percentage. For example, one user can be assigned 50% of plots while two other users can each be assigned 25%. 

Administrators can now assign plots to specific users using the User Assignment feature in the Plot Design tab. Users can have equal assignments or be assigned different percentages of the total number of plots
Administrators can now assign plots to specific users using the User Assignment feature in the Plot Design tab. Users can have equal assignments or be assigned different percentages of the total number of plots.

Institution administrators can also use two new Quality Control modes: Overlap, where a certain percentage of each user’s plots are reviewed by other users, and SME (Subject Matter Expert) Verification, where a percentage of each user’s plots are reviewed by one or more SMEs. The SME might be a project administrator, for example, or someone with deep knowledge of the local landscape. Note that when Quality Control is enabled, the project can no longer support user-drawn samples.

Administrators can choose from two Quality Control modes, Overlap and SME (Subject Matter Expert) Verification.
Administrators can choose from two Quality Control modes, Overlap and SME (Subject Matter Expert) Verification.

🧩 Improvements to plot navigation

Once data for the project has been collected using a quality control approach, administrators will be able to switch to Review Mode in the Data Collection interface. The new QA/QC navigation feature will allow administrators to choose the level of disagreement—specified as a percentage—between the responses provided by different users.  They can then navigate through the plot labels where the disagreement between different interpreters meets or exceeds the specified level.

Administrators can use the new Review Mode to Navigate Through QA/QC plots that have an unacceptable level of disagreement between the answers provided by different users for the same plot.
Administrators can use the new Review Mode to Navigate Through QA/QC plots that have an unacceptable level of disagreement between the answers provided by different users for the same plot.

Administrators can press the View Disagreements button to bring up a new window that compares all users’ answers for the plot’s questions.

This navigation mode is particularly useful for situations where administrators are assessing difficult plots or classes to identify or are holding a group meeting to work through the identification of plots that have a high level of disagreement between interpreters. 

In addition, administrators can use the Navigate Through option to review unanalyzed plots, analyzed plots, flagged plots, low confidence plots, and the plots of a particular user. These new modes give administrators unprecedented flexibility in carrying out project quality control. 

Administrators can now use Review Mode to Navigate Through all unanalyzed plots, analyzed plots, flagged plots, plots with low confidence, and an individual user’s plots.
Administrators can now use Review Mode to Navigate Through all unanalyzed plots, analyzed plots, flagged plots, plots with low confidence, and an individual user’s plots.

Data collection users will also see a more streamlined navigation interface, and will be able to navigate through their own unanalyzed plots, analyzed plots, and flagged plots. The default will be to navigate through unanalyzed plots. Users also will now be warned if they attempt to leave the page without saving their answers.

Options for data collectors to navigate through plots in the CEO Data Collection interface.
Options for data collectors to navigate through plots in the CEO Data Collection interface.

🦺 Improvements to user-drawn samples

CEO now allows the restriction of user-drawn samples to specific geometries. Once the administrator has enabled users to draw their own samples, the administrator can choose which types of sample geometries are allowed. Administrators can choose points, lines, and/or polygons. Note that when the “random,” “gridded,” or “center” sample options are selected, the “points” geometry cannot be disabled. 

In addition to adding more QA/QC options, this approach will help administrators when they want to download data from CEO and then import it into desktop GIS programs such as QGIS or ArcGIS or into web applications like Google Earth Engine. Data from CEO is exported as well-known text (WKT), but these programs allow only one type of geometry for file imports. Restricting the allowed sample geometries will make this process easier for administrators.

Administrators can now restrict the allowed geometries when enabling users to draw their own samples. Geometry types can be restricted to points, lines, and/or polygons.
Administrators can now restrict the allowed geometries when enabling users to draw their own samples. Geometry types can be restricted to points, lines, and/or polygons.

🔔 Other great new features

In addition to our automated quality control features and improvements to plot navigation, CEO has made the following improvements:

  • We now show the version of CEO in the top navigation bar, just under the CEO logo.
  • We now require new users to validate their email upon registration.
  • We have added the plot confidence and flagged reason to CSV output.
  • We have changed the name of “Unpublished” to “Draft Mode” to match previous changes that allow users to edit projects before collecting data.
  • We have made proxying optional for WMS feeds. This should improve performance for users who don’t need to obfuscate a key for their WMS. There is now a Proxy Imagery button that administrators can use if they need to obfuscate their WMS keys.
The WMS Imagery addition interface now has a Proxy Imagery option for users who need to obfuscate their WMS keys.
The WMS Imagery addition interface now has a Proxy Imagery option for users who need to obfuscate their WMS keys.

🛠 Beta testing process

If you’re an experienced CEO user and are interested in participating in our trusted tester program, please email support@collect.earth

 

Our thanks to ongoing funders FAO, NASA SERVIR, and SilvaCarbon. Thanks also to technology partners Norway’s International Climate & Forests Initiative (NICFI) for funding open high-resolution data availability, Planet for providing high-resolution imagery, and the Google Earth Engine team for creating a platform for Earth science data and analysis. 

Collect Earth Online is working constantly to improve the user experience, and your feedback is invaluable. If you have an idea, please let us know by writing to support@collect.earth. Thank you!

 

Search Our Blog

Generic selectors
Exact matches only
Search in title
Search in content