Knowing.NET

ResNet-style CNNs To Predict Freshwater Algae Blooms in Satellite Imagery: Mediocre Results

Although I have no domain experience with satellite imagery, I've used convolutional neural nets with aerial photography to recognize marine debris. So when I saw the DataDriven challenge 'Tick Tick Bloom' I took a glance at the dataset and decided to put a few days of effort into it. One thing I liked about the challenge was that the dataset was very straightforward: you can use satellite visual data, date, latitude & longitude, and a specific feed of weather data. The target variable is equally straightforward: a classification, on a scale of 1-5, of cyanobacteria density. Rather than a classifier, though, I predicted a floating point value and rounded to the nearest integer. The challenge comes with a baseline solution based on LightGBM that produces a mean error of about 1.5 (which is pretty poor, considering that "5" is a rare result, so mostly you're in the business of guessing 1-4!). The baseline uses visible spectrum bands, and my first iteration used the same RGB data in a ResNet-style CNN. Depending on the platform and spectral band, the Landsat and Sentinel satellite data come in 5-, 10, and 20- meter resolutions. Intuitively, I grabbed small areas around the sample point, which made the images tiny, batch sizes big, and training very fast. If I recall correctly, I used a straight ResNet backbone and transfer learning on this first iteration, figuring that the textures and shapes known by the early layers were probably helpful. This initial model did a little bit better than the LightGBM model. Preliminary data evaluation showed the non-surprising result that seasonal imagery varied widely, so in my next iteration, I took the day of the year of the imagery, calculated the sin and cos of that as a percentage of the year, expecting that this might create a reasonable "positional" encoding of an annual cycle. I also normalized the latitude and longitude and passed that to the head as well.
So now I had a model with a ResNet backbone, "time of year," and "location" to pass to the head, which was just a few fully-connected layers outputting the single floating-point "severity" prediction. This model did quite well, putting me in 4th place in the early running and right in the middle of the 2nd-tier competitors (there's 1 competitor who's a good 15% better than any of the others. I expect them to win!). It seemed a no-brainer to add additional spectral bands to the model, as well as the available "quality" masks that alert users to missing data or cloud cover (which is a very common issue when looking at a small spatial area). Downloading all that data was itself a chore that consumed two days of my time budget. I expected to be able to use FastAI's CNN support, basically passing in a [batchsize, channel_count,width,height] tensor with a channel_count of around 8. But it seems like FastAI has a hard-wired expectation for 3 channels (or 1 channel) when it comes to CNNs. So I ended up writing a custom PyTorch module to convolve over the bands. This had the downside that I couldn't get a jumpstart by transfer-loading weights (well, I could, but not easily), but the upside that I ended up with weights "dedicated" to each band and resolution. Since the bands are a little different between Landsat and Sentinel platforms, I ended up creating channels for both and added a band_available mask Tensor. With January flying by and other commitments piling up, I trained up the model with a strict admonition to myself that if it didn't train well enough to re-establish myself in the top tier of the competition, I would have to walk away. Well, it didn't. Much to my surprise, the addition of all those additional bands hardly budged the final error levels. With more competitors developing competitive models, I dropped out of the top dozen scores. Meanwhile, the naive way that I had just stored the band data as Numpy arrays in separate files led to my training being bottlenecked by IO. So even though there is some fairly low-hanging fruit in terms of data augmentation, pre-judging image quality, and architecture search, the first thing I'd have to do is a performance iteration. But... well, it's the season for both FIRST Robotics and the annual humpback whale migration, so it's time to move on!

Sentiment Analysis of Mastodon Toots is Very Easy

The Mastodon API is very straightforward, as is the OpenAI API for its NLP models. I wrote a quick proof-of-concept program to do sentiment analysis of "toots.".
read more
Re-Identifying Manta Rays

My current project is re-identifying individual manta rays (Mobula alfredi and Mobula birostris) by their distinct belly patterns … er… ventral markings.

Every night at two spots on the Big Island of Hawai’i where I live, dive boats shine bright lights that attract plankton. Most nights, the plankton in turn …
read more
What We Talk About When We Talk About Attention

The "attention" in ML is "what you should attend to," not "alertness." In the sentence "They crossed the <???> to get to the other bank." you need to "attend to" the <???> word to disambiguate "bank". If <???> is "street" then it's "bank" as in "financial institution" (most likely). If <???> is "river" then …
read more
How to train/test on a subset of your FastAI data
If you have a large FastAI (v2) DataLoaders and you're trying to debug something at epoch-scale (such as a custom metric), an easy way to train on a small subset of your data is:
```
subset_size = 100 # Or whatever
selected_items = np.random.choice(dls.train_ds.items, subset_size …
```
read more
Large Language Models and the Chinese Room

The Chinese Room is a 1980 thought experiment from the philosopher John Searle. The Wikipedia summarizes the setup:

[S]uppose that artificial intelligence research has succeeded in constructing a computer that behaves as if it understands Chinese. It takes Chinese characters as input and, by following the instructions of a …
read more
PyScript page

Short version for index and feeds read more
How I Failed At Kaggle Happywhale

I just deleted my intermediate data and models for Kaggle’s Happywhale competition. I did terrible, never getting much above pure random guessing. Which was frustrating, because it’s a problem in Machine Learning that I’m very interested in (and want to do more work in).

Happywhale, Sad Human …
read more
Biggest mistake on Kaggle
Don’t join a team too quickly. Once you’re on a Kaggle team:
- You cannot choose to leave
- The team leader cannot choose to remove you
Unless you have a very good sense of what exactly your teammates are bringing to the competition, including their:
- Knowledge level
- Time commitment …
read more
Installing Detectron2 on a Mac in CPU mode
At the risk of saying, “Yeah, it’s in the docs,” this is what I did. I think the crucial thing is installed things in the proper order, so I would advise going step-by-step:
1. Have conda installed (brew install conda if not, I suppose)
2. Create a conda environment with conda …
read more

Page 1 / 235 » ⇉

Happywhale, Sad Human …

blogroll

social