TY - JOUR
T1 - Accuracy and consistency of grass pollen identification by human analysts using electron micrographs of surface ornamentation
AU - Mander, Luke
AU - Baker, Sarah J.
AU - Belcher, Claire M.
AU - Haselhorst, Derek S.
AU - Rodriguez, Jacklyn
AU - Thorn, Jessica L.
AU - Tiwari, Shivangi
AU - Urrego, Dunia H.
AU - Wesseln, Cassandra J.
AU - Punyasena, Surangi W.
N1 - Publisher Copyright:
© 2014 Crawford and Belcher. Published by the Botanical Society of America.
PY - 2014/8
Y1 - 2014/8
N2 - Premise of the study: Humans frequently identify pollen grains at a taxonomic rank above species. Grass pollen is a classic case of this situation, which has led to the development of computational methods for identifying grass pollen species. This paper aims to provide context for these computational methods by quantifying the accuracy and consistency of human identifi cation. Methods: We measured the ability of nine human analysts to identify 12 species of grass pollen using scanning electron microscopy images. These are the same images that were used in computational identifi cations. We have measured the coverage, accuracy, and consistency of each analyst, and investigated their ability to recognize duplicate images. Results: Coverage ranged from 87.5% to 100%. Mean identifi cation accuracy ranged from 46.67% to 87.5%. The identifi cation consistency of each analyst ranged from 32.5% to 87.5%, and each of the nine analysts produced considerably different identifi cation schemes. The proportion of duplicate image pairs that were missed ranged from 6.25% to 58.33%. Discussion: The identifi cation errors made by each analyst, which result in a decline in accuracy and consistency, are likely related to psychological factors such as the limited capacity of human memory, fatigue and boredom, recency effects, and positivity bias. .
AB - Premise of the study: Humans frequently identify pollen grains at a taxonomic rank above species. Grass pollen is a classic case of this situation, which has led to the development of computational methods for identifying grass pollen species. This paper aims to provide context for these computational methods by quantifying the accuracy and consistency of human identifi cation. Methods: We measured the ability of nine human analysts to identify 12 species of grass pollen using scanning electron microscopy images. These are the same images that were used in computational identifi cations. We have measured the coverage, accuracy, and consistency of each analyst, and investigated their ability to recognize duplicate images. Results: Coverage ranged from 87.5% to 100%. Mean identifi cation accuracy ranged from 46.67% to 87.5%. The identifi cation consistency of each analyst ranged from 32.5% to 87.5%, and each of the nine analysts produced considerably different identifi cation schemes. The proportion of duplicate image pairs that were missed ranged from 6.25% to 58.33%. Discussion: The identifi cation errors made by each analyst, which result in a decline in accuracy and consistency, are likely related to psychological factors such as the limited capacity of human memory, fatigue and boredom, recency effects, and positivity bias. .
KW - automation
KW - classifi cation
KW - expert analysis
KW - identifi cation
KW - palynology.
UR - http://www.scopus.com/inward/record.url?scp=84923598731&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=84923598731&partnerID=8YFLogxK
U2 - 10.3732/apps.1400031
DO - 10.3732/apps.1400031
M3 - Article
C2 - 25202649
AN - SCOPUS:84923598731
SN - 2168-0450
VL - 2
JO - Applications in Plant Sciences
JF - Applications in Plant Sciences
IS - 8
M1 - 1400031
ER -