You say "ChatGPT says," I say "Sprently"

A while back I wrote about how we needed a word for citing statements that had been sourced from an LLM but which are low-stakes enough not to have been fact-checked.

I like the word "sprently," for this. It's already an English word that seems more or less interchangeable with …

more ...

There needs to be a word

I found myself writing "s/apparently/according to Claude 3.7 Sonnet/".

There needs to be a word.

(That string is programmer code for "whenever I write 'apparently' switch it to 'according to a possibly-hallucinating LLM.')

The essay in which I found myself writing that substitution doesn't use "apparently," except …

more ...

ChatGPT 04-mini-high for coding? Screw that!

OpenAI claims ChatGPT 04-mini-high is its best model for coding. Screw that. The thing lies about its ability to correct its hallucinating behavior after I gave it every chance to clarify its shortcoming. If I were a junior, much less "vibe" coder, I would likely believe its assurances about doing …

more ...


Recent readings

ML Architecture and papers

Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet Really important article, seemingly a breakthough in peering inside LLMs. A nice adjunct to this article is the less dense An Intuitive Explanation of Sparse Autoencoders for Mechanistic Interpretability of LLMs. My big worry about this is …

more ...


Conformer: An interesting ML architecture that I'm abandoning

BirdCLEF is a bird-call recognition contest that runs yearly on Kaggle. (CLEF stands for Cross Language Education and Function.)

Last year, I participated in BirdCLEF because the domain was native Hawaiian birds, so even though I didn't know much of anything about audio ML, I had a leg up in …

more ...

Thoughts on Hawkins’ “A Thousand Brains”

Jeffrey Hawkins is the CEO of Numenta, a company that has been pursuing machine intelligence since the early 2000s. Prior to that he was the founder of Palm Pilot, the most successful Personal Digital Assistant, the sale of which presumably has funded Numenta this part quarter-century.

Hawkins’ approach is iconoclastic …

more ...

Noodling about with a Neural Network Markup Language

Felt potentially communicative, might delete later.

Some sketches of neural net layer annotations

more ...

Interested in ML?: The math can wait

Focus on data transformation and scenarios, not math, if you want to get into real-world Machine Learning. Yes, you’ll eventually have to learn some math, but the abstraction level within ML engineering has already moved on.

Learning the theory and math at the foundations of Machine Learning is like …

more ...