Prefab: Modifying UIs via Extreme Screen Scraping

This video shows "Prefab": a research tool that screen scrapes applications at the pixel level, allows the UI to be enhanced (changing cursor or mouse behavior, for instance), and then routes the input to the enhanced UI back to the original. The performance is impressively snappy. It's not clear how much programming is involved with some of the more advanced UI behaviors ("parameter spectrum visualization" seems to involve quite a bit of logic), but it's well worth viewing as food for thought.