Interesting Reading in October
Blog posts
Transgender Sticker Fallacy (Yassine Meskhout). Labels and definitions in debates are tricky, because they are used to hide premises and bring unearned connotations. For the specific debate about when transgender people should be grouped with women or not, rather than fighting over the “woman” label, examine whether it makes sense or not on the merits.
Some thoughts on the Sutton interview (Dwarkesh Patel). In the interview, they were talking past each other. Here, Dwarkesh is looking back, understanding better and challenging Sutton ideas better. Sutton thinks LLMs don’t have world models and only do imitation learning while humans do have world models and almost never do imitation learning.
Abriendo el algoritmo (David Bonilla). Software commissioned for the public administration should be open source for multiple reasons like transparency, reuse of something we have paid communally, etc. This doesn’t happen in Spain, being IP or national security the usual pretexts. A new ruling from the Supreme Court establishes that “national security” cannot be used without a detailed justification. Congrats everyone!
The Fatima Sun Miracle: Much More Than You Wanted To Know (Scott Alexander). Scott has a new hobby horse: the Fatima Miracle. He does an encyclopedic effort of putting together evidence, arguments and, reserves a section to remind you about healthy Bayesian epistemics. The article is inconclusive but included calls to action and to gather more information. Waiting for the Astral Codex Ten community to do its magic.
Claude Skills are awesome, maybe a bigger deal than MCP (Simon Willison). Skills are markdown documents explaining how to do something or play a role that get used automatically when it makes sense by your agent.
The PACs are closing in on the almonds (Scott Alexander). PACs are collective platforms to support political candidates in the USA. Some time ago, Scoot compared the amount of money spent on almonds with PACs for comedic effect and now the amounts are getting closer to each other. More importantly, it shows how Marc Andreessen anti AI legislation PAC will face pro AI safety opposition and gives a clear call to action to his fellow Americans.
Which side of the AI safety community are you in? (Max Tegmark). The two camps are “race to super-intelligence safely” (but we don’t know how) and “don’t race to super-intelligence” (but the outgroup is going to win the race).
EU explained in 10 minutes (Martin Sustrik). The EU is a Frankenstein in the middle of transitioning from international cooperation to a federation or political union and—to increase the confusion—the institutions have aspirational names, signalling the role they aspire to play but not the one played.
The “length” of “horizons” (Adam Scholl). Some healthy push back on the extrapolations that the METR horizons paper is inspiring people to make (including the AI 2027 one). The METR benchmark has only SW development tasks, not that many, not that diverse, and restricted to easily verifiable tasks. The benchmark will be as useful as this is correlated to actual agency. Adam is warning us that is not clear how much correlation to expect yet.
Vocabulary
shibboleth (n.)
1. a particular way of speech that distinguishes a certain group
2. a common saying or belief with little current meaning or truth
The term “shibboleth” comes from the Book of Judges, where Gileadites used it as a lethal password to identify enemy Ephraimites trying to escape across the Jordan River. The Ephraimites couldn’t pronounce the “sh” sound in “shibboleth” and said “sibboleth” instead, revealing their identity and leading to their execution. This linguistic death-trap gave us the modern meaning: a word or custom that distinguishes insiders from outsiders.
Excellent analysis of the Sutton interview points. The 'world model' debate for LLMs is so central write now. Are we perhaps underestimating the complex patterns they learn beyond pure imitation?