ML Architecture and papers

Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet Really important article, seemingly a breakthough in peering inside LLMs. A nice adjunct to this article is the less dense An Intuitive Explanation of Sparse Autoencoders for Mechanistic Interpretability of LLMs. My big worry about this is …

LLMs are bland. Two new papers, Art or Artifice? Large Language Models and the False Promise of Creativity and A Robot Walks into a Bar: Can Language Models Serve as Creativity Support Tools for Comedy? An Evaluation of LLMs’ Humour Alignment with Comedians ] reinforces the findings of a more rigorous …

BirdCLEF is a bird-call recognition contest that runs yearly on Kaggle. (CLEF stands for Cross Language Education and Function.)

Last year, I participated in BirdCLEF because the domain was native Hawaiian birds, so even though I didn't know much of anything about audio ML, I had a leg up in …

Jeffrey Hawkins is the CEO of Numenta, a company that has been pursuing machine intelligence since the early 2000s. Prior to that he was the founder of Palm Pilot, the most successful Personal Digital Assistant, the sale of which presumably has funded Numenta this part quarter-century.

Hawkins’ approach is iconoclastic …

Felt potentially communicative, might delete later.

Focus on data transformation and scenarios, not math, if you want to get into real-world Machine Learning. Yes, you’ll eventually have to learn some math, but the abstraction level within ML engineering has already moved on.

Learning the theory and math at the foundations of Machine Learning is like …

Mantas Deployed, Time for Whales

Since last Fall, I’ve been working on reidentifying reef manta rays with “low k-shot” (few labeled examples). That project is now deployed to the scientists and, until they start giving me feedback (and hopefully, more labeled data) pretty much at the finish line.

My …

Reading Recommendations: Machine Learning for Non-Coders

The best orientation to machine learning (ML) I could find is Machine Learning Explained from MIT Sloan. That gives a good overall orientation, even if it, like all texts, suffers from underestimating the speed at which ML capabilities are evolving. I don’t think …

ResNet-style CNNs To Predict Freshwater Algae Blooms in Satellite Imagery: Mediocre Results

Although I have no domain experience with satellite imagery, I've used convolutional neural nets with aerial photography to recognize marine debris. So when I saw the DataDriven challenge 'Tick Tick Bloom' I took a glance at the dataset …

The Mastodon API is very straightforward, as is the OpenAI API for its NLP models. I wrote a quick proof-of-concept program to do sentiment analysis of "toots.".