A brand new scientific paper applies computer vision to over 125,000 of the Museum’s digitised Butterfly collection to understand how animals may respond to climate change.
Applying computer vision to digitised natural history collections for climate change research: temperature-size responses in British butterflies has been published in Methods in Ecology and Evolution. It uses the world’s oldest and largest butterfly collection to investigate the impact of climate change on the size of British butterfly species.
As one of the first studies to show that computer vision can accurately measure physical characteristics from digital collections and test species’ responses to climate change. This research is accelerating our potential to understand how life on earth reacts to climate change.
Dr Phillip B Fenberg, Lecturer from the University of Southampton and Science Associate of the Natural History Museum, London his recently graduated PhD student, Dr. Rebecca Wilson (first author), and co-authors from the Museum and UC Berkeley specialise in using museum collections to look at ecological responses to climate change. Such responses to climate change can include:
- Change in geographic range, this could involve species moving towards the poles and/or local or global extinctions.
- Change in seasonal events in the life of organisms (phenology), this could include change in mating times, earlier insect emergence, shorter hibernation.
- Change in body size – there is a widespread phenomenon known as the ‘temperature-size rule.’ This states that individuals of the same species reared at warmer temperatures will be smaller as adults. Thus, if the climate is warming, we can expect specimens to become smaller. However, there is growing evidence that the body sizes of animals can also get larger in response to warming temperatures. This can be seen among terrestrial species with complex life cycles, such as butterflies from temperate regions.
Museum collections can provide historical baselines for studying each of the major responses from living organisms to climate change. Butterflies are incredibly valuable to researchers when looking for responses to climate change because there is a long history of collecting butterflies, which means the Museum has specimens over 200 years old. Most specimens have useful collecting data (where, when, by whom): needed for studying range change and phenological change. When we digitise these specimens we include a scale bar. This makes body sizes easy to measure from images.
Proof of concept with the Silver-Spotted Skipper (Hesperia comma)
Hesperia comma from the Museum’s Data Portal
The group started off by selecting one species that other researchers had already verified a change in geographic range in warmer years. The Silver-Spotted Skipper (Hesperia comma) lives on the chalk grasslands of southern England and has a single generation each year. The species are in their adult form around August, pupal state around July and late larval stage in June. The males of the species have distinctive black scent scales. Lawson et. al (2012) used temperature data from the met office combined with location data from butterflies to demonstrate that warm August temperatures are linked to geographic range expansions of H. comma. Fenberg et al (2016) looked at the changes in seasonal behavior by focusing on the pupal stage in July and found that adults emerge earlier in the year when July temperatures are warmer. Finally change in body size was assessed by measuring the forewing lengths on each specimen. A change was noticed in the males of the species but not of the females. The males of Hesperia comma tend to get bigger during years with warm June temperatures, but females do not. But overall, females are larger than the males. Many butterfly species show a large amount of sexual dimorphism, the males and females can look quite different. The females are often larger than the males, which is thought to be due to the number of eggs that insects lay; a larger body size enables a female insect to lay more eggs.
This proof of concept showed that one species responds to each of the ecological responses to climate warming: change in geographic range, phenology, and body size. Importantly, each of these responses are caused by warming temperatures during specific stages in their life cycle. But this was just one species and took over six months to collect all of the data for this work. There had to be a better way to do this across many more species.
Applying Computer Vision technology: Mothra
Computer vision is a rapidly evolving field in which computers are programmed to identify and measure information from digital images or video. In collaboration with the University of Southampton and the Natural History Museum, London, the Berkeley Institute for Data Science developed a computer vision pipeline called ‘Mothra’ which is freely available to use online. With this, researchers were able to analyse over 125,000 photographed British butterfly specimens from the Museum’s Data Portal. Mothra automatically detects the specimen and measures characteristics including wing features (such as length), orientation (how the specimen is pinned) and identifies the sex. This process substantially reduces the time required to analyse individual specimens, which would otherwise require researchers to physically measure and record manually. Results from this study indicated that there was a near perfect relationship between Mothra and manual measurements.
Scientists paired measurements with monthly temperature records experienced by the immature stages of 24 different British Butterfly species and looked for patterns in the relationship between size and temperature. Out of the 24 species analysed, 17 showed significant results, correlating with an increase in adult size with increase in temperature during the late larval stages. It is thought that years with warm temperatures during the late larval stages will cause adults to be bigger because with there will be more food available for caterpillar growth (i.e., more sunny, and warm days will increase larval food plant growth). Thus, caterpillars will be larger as they enter the pupal stage, resulting in larger adults.
These results were in agreement with what was found using manual measurements from a few species, including Hesperia comma and three other species, which were published by the researchers in 2019. However, these previous studies used manual measurements, which took many months of time spent at the computer. The measurements of the whole of the British butterfly collection (>180,000 specimens) took Mothra only one week to complete using the computer facilities at the Museum. But this time could have been reduced to a mere 30 hours if scientists used the full capacity of the computer cluster. If a single person were to manually measure this many specimens (assuming 9-5 working hours and no holiday time or breaks) it would take well over 2 years of time, and probably much longer and with a high likelihood of developing carpal tunnel syndrome!
Advancements in technology are enabling development software which accurately and rapidly analyses digital specimens. This combined with access to open access digitised collections that allows scientists from all over the globe to be able to more easily use collections, can accelerate research in a more collaborative way than ever before. As the Anthropocene progresses, digitised natural history collections will be further used for understanding the biological impacts of global change.
With 80 million specimens in the Natural History Museum collection alone, the sheer size of natural history collections often makes it difficult to extract this information. This study has shown the value of combining digitisation and machine learning to rapidly release evidence from museum collections, which can be used to conserve species in a changing world. Find out more about this work on the Museum website and follow us on Twitter and Instagram to hear the latest digitisation news.