D3436: Using machine learning to study pollination with high temporal and taxonomic resolution

Field based studies of pollinator behavior are frequently limited in scope. The amount of labor required to observe flowers for extended time periods means that many pollinator studies lack the data to accurately estimate pollinator visitation rates and foraging times.
While video based methods for collecting visitation data can avoid some of these issues, considerable storage space and post-processing are required to extract usable data from videos.
To address these problems, I present BugNet, a machine learning algorithm capable of automatically recording and identifying insect pollinators. BugNet uses a You Only Look Once (YOLO) computer vision model to record and crop insects from video data in real time. These cropped images are then identified using a hierarchical classifier capable of determining the probability that the insect is in any given order, family, genus, and species. pollinators are tracked across multiple frames of video to improve classification accuracy, and to provide data on the period of time spent visiting a flower.
Trained on a dataset of 240 species of North American insects consisting of 330,000 images scraped from Inaturalist, BugNet is capable of cropping 98% of insects from video data, and can identify insects to species with 90% precision. Identifications to genus exceed 95% precision. BugNet allows for the collection of pollinator visitation data in the field with high temporal and taxonomic resolution, and can be easily modified studies of insect behavior in the lab.