BuzzNet

Using AI to Fight Mosquito Transmitted Disease

Over the course of 10 months, I will use cutting-edge urban science technologies to investigate mosquito control in Singapore. Singapore is an urban, tropical, and humid nation that is a historical port city. This makes Singapore an ideal natural climate for mosquitoes to thrive. While mosquitoes can be beneficial to ecology, mosquitoes can carry dangerous diseases like dengue. In 2019, there have already been 12 thousand reported cases on dengue in Singapore. Mosquito control is a key urban challenge for this decade. I propose data-based ecologically-friendly tools that help to repel mosquitoes as an alternative to chemical fogging. There is still much work to do!

This website serves as an educational guide, documentation, and an interactive tool. The goal is to demonstrate how anyone can easily use machine learning tools for vector control.

Background

The histories and futures of mosquitoes are linked. After all, mosquitoes may be responsible for about half of all human deaths. To understand humane means of reducing the number of mosquito transmitted diseases, I cover relevant background about mosquitoes, Singapore, urban environments, and existing technologies. Because mosquitoes can grow resistant to insecticides, mosquitoes can only breed in stagnant water, and mosquito-transmitted diseases align with environmental patterns, tools like audio analysis, visual analysis, and geospatial analysis can be effective as predictive and preventative tools in tropical urban environments like Singapore.

In short, computational tools can be used to predict and prevent mosquitoes in urban settings.

Mosquito Biology
Selected information from Wikipedia

Mosquitoes may have been around for over 100 million years. During that time, they adopted various characteristics that help them thrive in urban environments. Prominently, they are adaptive, parasitic, and mobile. There are over 3,500 mosquitoes species, but only a handful are dangerous to humans.

There are a few common characteristics in these mosquito species. For many species, mosquitoes lay eggs in standing water. Sometimes these are puddles, and othertimes, a cupped leaf. Some mosquito species can even pause development if water dries out! Often, the eggs are dispersed throughout a waterbody.

The eggs grow into larvae and pupa, which have a siphon to breate air. They are still in water and move when the water surface is disrupted.

The adult mosquito we are well acquainted with. Male and female mosquitoes have different features. Males are smaller, live about 1 week, and eat nectar. Females are larger, live about 2 weeks, and take blood meals. The blood meal provides protein to the eggs. Mosquitoes are attracted to some combination of chemicals and heat. Many mosquitoes are active at dusk and dawn. Some mosquitoes can travel 12km in one day. Mosquitoes are nearly ubiquitous.

Mosquito saliva contains proteins to help the mosquito get a blood meal without alerting the host. Sometimes, this is how dangerous diseases are spread.

Vector Control in South-East Asia

South-East Asia arguably has the greatest risk of mosquito borne diseases. Dengue, Malaria, and Yellow Fever are common despite strong mosquito-prevention programmes. Countries most affected by mosquito-transmitted disease are Indonesia, Malaysia, and Thailand. Additionally, there are concerns of drug-resistant malaria travelling spreading from Cambodia.

In the midst of this is Singapore, with over 15,000 cases of dengue this year despite a refined mosquito control regime form the National Environmental Agency (NEA). Accross Singapore, there is public and private fogging services. The NEA also performs inspections and fines those who have mosquito breeding environments. Recently, the NEA has been looking at drone-based mosquito detection techniques. There is still much work to be done. There has also been research into Wolbachia bacteria in mosquitoes by the Urban Redevelopment Authority (URA).

Urban Mosquitoes
Mosquitoes can thrive in urban areas. This makes mosquitoes an ever-growing global challenge.

Most obviously, urban areas provide plenty of blood meals for mosquitoes. Female mosquitoes find human blood quite appealing, rivaled only by the proteins in some cows and insects. Mosquitoes also have access to detritus to hide in and puddles to breed. Humans also provide places to breed outdoors in potholes, or inside in flower pots.

Another effect of urbanization is urban verticalization, like skyscrapers. In Singapore, it is common to see vertical greenery on skyscrapers. It is a common misconception that living on upper levels will prevent mosquito-transmitted diseases, but there have been anecdotal stories of mosquitoes breeding vertically, with a female going up a floor during each breeding cycle. Further, many Singaporeans leave their windows open for better ventillation and do not have screens. This allows for mosquitoes to breed.

A final consideration is vector control during construction. While there remains many questions, it appears that mosquiteos thrive in contruction sites. During my interviews, one Singaporean described the construction site near his house as a potential source of his Dengue. Some of the danger is attributable to construction practices. But other issues like open water pits, detritus, and heat seem inherent in the construction process.

Particularly in tropical urban areas, mosquitoes have plenty of opportunity to breed. Mosquitoes enjoy weather above 26 degrees C. Further, mosquitoes transmit viruses better in warmer conditions. Of particular interest is Urban Heat islands, although their effect on mosquitoes is not well studied. One can guess as urban climates become warmer from human activity, mosquiteos will gather the benefits of warmer weather.

Current Technologies

Briefly, the most common vector control strategies are pesticides and personal protection. On the urban-scale, insecticides and growth inhabitors are able to prevent mosquito breeding grounds. However, these chemicals also may be dangerous to humans and beneficial insects. In Singapore, residents describe the fog as a cloud that they try to avoid with a pungent smell. Some residents try to avoid the fog by rescheduling their activities.

Other technologies include personal protection, like wearing long sleves, removing sources of indoor stagnant water, and keeping a clean environment. Many marketed mosquito killers have no

Some more innovative approaches include the use of genetics, bacteria, or lasers targeted towards female mosquitoes.

There have been great advancements recently in machine learning and data science. The power of machine learning is harnessed by providing great amounts of clean data. Singapore has collected historic data on mosquito populations. Further, with advances in Internet-of-Things (IoT) technologies there is a considerable amount of sensor and environmental data around Singapore. This makes machine learning techniques more feasible.

Identified Gaps

There remain three major gaps in effective mosquito control today:

  1. Effective detection: We lack ways to effeciently detect and predict mosquito populations due to their adaptive lifecycle.
  2. Lack of urban focus: Humans do not take proper protective measures to keep safe from mosquitoes in urban environments.
  3. Effective detterence: A solution that deters mosquitoes from human-populated areas.

Buzznet is a first attempt at closing these gaps.

Proposed Solutions

BuzzNet is an initial exploration into how different machine learning techniques can be used to address the mosquito control problem in Singapore. Through this exploration, I hope to identify the most promising leads, provide relevant information for future researchers, and learn how technology can help with vector control.

Below, I provide a brief exploration on audio, visual, and geospatial models.

Audio

One of the first ways we detect mosquitoes is by hearing them. The high pitched buzz is acutally quite complex. Male and females have different wingbeats and produce different sounds. But, when mosquitoes mate, their wingbeats come closer in frequency . For this part of the project, I use data I collected in Singapore, the Wingbeats dataset from Kaggle, and sounds of mosquitoes from YouTube. There is previous work in classifying mosquito species by wingbeats.

Audio is a promising tool for mosquito detection and attraction.

Theory

A common use for audio in machine learning is speech processing. The Google AudioSet provides many sample clips. Common audio classification algorithms use a VGG vector encoding. IBM's audio classifier claims a ~60% accuracy at detecting mosquito sounds.

Audio Clips

Here are brief clips that I recorded from Singapore urban environments. Is there a mosquito in these clips?

(The answer is Yes, No, Yes)

In general, it was hard to detect mosquito presence with my iPhone as a recording device. I need to collect more data before continuing this lead.

The next dataset was the Wingbeats dataset. There are nearly 300,000 wingbeats from 6 different species. Some of the clips have one or two mosquitoes. It is a good dataset to perform transfer learning with. Here are a few samples.

The final is mosquito clips from Youtube. I am not sure exactly which clips were used to build the AudioSet from Google, but this should be a representative sample.

Audio Models

First, I used pre-trained classifiers to detect the moquitoes from my data, the Wingbeats data, and the Youtube data. One model said all of the Wingbeats were meows! There is clearly more work to do.

Coming soon: Pretrained IBM model performance on the datasets.

The second model is using the existing data and the VGG encodings to train new audio models for detection. There are two ways the audio model can work- by inference (ex: sounds of birds imply mosquitoes) or by direct detection (as in the high-pitched wingbeats buzz). These are still in progress.

Reflection

It is great to be able to detect mosquito audio, but what could be a good use for it? One option discussed in depth on the Kaggle website would be to use the audio classification to capture only the mosquitoes (vs. other flying insects) in insect traps. I think the audio mating properties can also be used to create a mosquito attractant. This is a part I am still struggling with. I want to better understand the urban soundscape and how mosquitoes may be attracted/repulsed by different frequencies.

Future Work

Beyond BuzzNet, I imagine that this work can be used to identify other bugs in urban environments by inference. With additional data in Singapore, this would be an important component of an IoT device that detects mosquitoes.

Proposed Solutions

Some leads that I stopped were collecting data in forrested parts of Singapore because it was hard to know when there was a mosquito and I was getting bitten too often. This is important data that I hope will be collected in the future.

Visual

Many animals detect mosquitoes by their movement. Sometimes mosquitoes can also be detected by their appearance, or guessed from environmental conditions. This visual section is in three parts. The first is direct mosquito detection by image, the second is mosquito movement, and finally inference.

Computer Vision techniques aid with direct mosquito obersvation and inference.

Computer Vision

There have been significant advances in computer vision in the past decade. Convolutional Neural Networks are promissing for object detection and classification. Video processing has also made advances, and there is much work on human motion detection and prediction. Finally, with advances in object processing, we can determine which environments have breeding grounds, like open pools of water or potholes. We can build on existing work to apply computer vision techniques to mosquito environments.

Sample Photos

Coming Soon!

Vision Models

The first model was built with Nanonets. I took a sample of 50 puddle images and labeled them with bounding boxes. The baseline from the Nanonets vision algorithm was 0.625. The improvements I'm making are using bitmap annotations, a bigger dataset, more labels, and different algorithms. I'm in the process of evaulating them right now. I also can augment the datasets by using Google Images for photoes.

Reflection

This is the most promising lead, but it will not be effective alone. One future direction is to understand mosquito swarm behavior. Additionally, we can make use of heat imaging. These tools could be useful with drones to determine mosquito breeding grounds.

Future Work

In progress!

Proposed Solutions

Analyzing the movement of mosquitoes is on hold until I can gather more video clips.

Statistical

Geostatistics provide essential information about environmental trends in mosquito populations.

Theory

Coming Soon!

Data Exploration

Coming Soon!

Statistical Models

Coming Soon!

Reflection

Coming Soon!

Future Work

Coming Soon!

Proposed Solutions

Coming Soon!

Next Steps

Nov 12: Current steps are to link the demos to the webpage and bolster the Visual section.

Demos

Upcoming

Datasets

I curated some datasets and used other publicly available ones. As the datasets become complete, I will post them here with the meta-data.

Existing Sets of Interest: Environmental Datasets from https://data.gov.sg/

Analysis

Upcoming

Conclusion

Upcoming