On most fall days, David Sopjes can be found in the Eel River in Northwestern California, counting fish. As a retired science teacher and citizen scientist, Sopjes has spent the past 10 years monitoring the chinook salmon population in the Eel River, which he says has the state’s third largest watershed. Each fall, Sopjes counts the salmon while waiting for the winter rains to recur.
“They don’t eat anymore. They have one thing in mind, and it’s just sex, ”says Sopjes.
Before acquiring a drone three years ago, Sopjes and his colleagues counted salmon by snorkeling in the river and standing on paddleboards, which greatly disturbed the fish and did not was not very specific.
The drone produced clear photographs of the salmon, but counting the fish in the images using pen and paper was tedious. As he scoured the internet for a better way to count and organize his data, he found a software called DotDotGoose and has been using it ever since.
Designed at the Center for Biodiversity and Conservation at the American Museum of Natural History, DotDotGoose is a free, open-source tool that helps researchers manually count objects in images. Peter Ersts, the centre’s senior software developer, created DotDotGoose in May 2019. He came up with the idea from discussions with colleagues.
[Related: Alexa will tag along on an uncrewed mission to the moon]
Back then, the most popular ways for conservation researchers to identify different categories of animals in photos were very convenient. “A lot of people were still literally projecting images onto a dry-erase board, circling the animals and turning off the projector, then counting them as they wiped. [off the markings]”, says Ersts.” I saw the need for a really simple tool that allows you to quickly and easily put points on an image. ”
Although the tool has only been online for about two and a half years, it is already helping many researchers around the world. Since finding DotDotGoose, Sopjes says he has counted thousands of fish and the accuracy of his data has improved “dramatically”, so much so that the California Department of Fish and Wildlife has gone. interested in the use of its datasets. Accurately recording the total number of fish associated with the drone footage provided Sopjes with a useful way to track each fish.
How it works
DotDotGoose has a very simple interface which allows users to import images they wish to analyze. Then they can divide different objects of the pictures into “classes” or categories. For example, Sopjes defines classes as different stages in the life of a salmon. Each category corresponds to a point color.
To count each class, researchers can click on each object in the image to place the point. DotDotGoose counts the number of points per class as they are placed. Users can add custom notes, latitude and longitude coordinates, or other data points to describe the image.
DotDotGoose was originally intended to count animals for conservation research, but Ersts has seen users reuse it to count inventory in warehouses, components on circuit boards, and even flowers on plant driveways. tomatoes for the Guinness World Record.
Why this is useful
Rochelle Thomas, a graduate student in the Department of Ecology, Evolution, and Environmental Biology at Columbia University, used DotDotGoose with real geese.
From 1995 to 2019, Thomas’ advisor Robert Rockwell had taken aerial photographs of flocks of lesser snow geese in the Hudson Bay region of Canada. In the early years of the project, Thomas said Rockwell would print the photographs to count the geese by hand.
When Thomas joined the project in 2018, she tried counting geese using Photoshop, but it was hard to count. geese simultaneously by species and by age. She was introduced to Ersts while he was building DotDotGoose and became the beta tester for the program. The name of the program is a nod to his work with lesser snow geese.
“I spent many days putting dots on geese, and it kind of came [to me] to call it DotDotGoose, ”says Ersts.
[Related: AI turned a Rembrandt masterpiece into 5.6 terabytes of data]
Compared to similar software, such as Photoshop and ImageJ, Thomas likes that DotDotGoose was designed with a conservation biologist in mind and allowed him to change the image quality or insert information indicating the presence of water in the photo.
“Conservation biologists and environmentalists only sit on tons and tons of photographic data,” says Thomas.
And while the current manual version of the program already makes the data more manageable for analysis, she believes making the counting in DotDotGoose more automated could help more support research projects like hers.
The future of DotDotGoose
Ersts has been planning to semi-automate the process since its inception.
“If you are able to record these coordinates of your locations on an image, then you basically have a training set that you can use.” [to power] a machine learning model to help automate things in the future, ”says Ersts. “[But] It’s a pretty tough task to automate this when you really start to think about all the different types of data that are out there.
Ersts imagines that the researchers could train him with a few similarly oriented photos specific to their project and containing the same types of objects.
But even a bespoke automated DotDotGoose would have its limits. Images with many objects grouped together would be very difficult to analyze. And while an automated version of the program could free up researchers’ time, Ersts says a human should still be part of the process, at least to verify the work of the computer.