About
    • AMMDI is an open-notebook hypertext writing experiment, authored by Mike Travers aka mtraven. It's a work in progress and some parts are more polished than others. Comments welcome! More.
Search
Page Contents
Full

Antiformalism Apology

01 Nov 2026 - 01 Nov 2026
Open in Logseq
    • I said I was anti-formalism at a recent meeting, and I wanted to apologize for that, or at least explain. I'm not really anti-formalism; let's say I'm formalism-critical, meaning. I think a lot about the limits or misuses or pitfalls of formalization, in a way I hope is useful.
    • My suspicions of formalisms date back to my roots in the world of GOFAI, good old-fashioned AI, symbolic AI, which basically was a total failure. Don't need to rehash that history, but that AI was was very much about formalism.The idea was that thinking was a matter of logic and deduction, and you made formalisms for representing the knowledge in people's heads and formal deduction processes that worked over them. This was thought to be the road to replicating human intelligence.
    • This project seems exquisitely dumb to me now (and maybe to the world at large since we have seen that statistical learning methods far outpace deduction). But a lot of smart people persued it. At least it threw out some interesting technologies and ideas along the way, some of which might be worth revisiting.
    • GOFAI failed for a variety of reasons but one was the overemphasis on formalism and logic, and bad choices regarding how that worked. I was heavily influenced by the work of Phil Agre and David Chapman. Back in the 1980s, they did a very thorough critique of AI's intellectual approach. Very well informed both technically and and intellectually. The nub of their critique was that intelligence was grounded in concrete situated activity, symbolic reasoning was a secondary phenomenon, and the standard model of action involving symbolic world models and planning algorithms was just wrong (as well as being computationally intractable).
    • Agre wrote a paper, Routines, on the importance of routine activity I considered this work to be brilliantly insightful, it had a lot of impact on me, and I think it might be of interest to the protocol community.
    • So that's where my skepticism about protocol formalisms come from. AI was stuck on a bad formalism, Agre & Chapman tried to fix that, mostly failed, and GOFAI went into the dustbin of history. The problem of Protocol formalization are not exactly same thing as those in AI. there are different different issues at play, but the lesson that formalisms can be fatally misleading I think is worth absorbing.
    • There must be other writings on the topic of misleading formalisms Ask Claude
      • This is a rich topic with a genuine intellectual tradition behind it. Here's a survey of the key literature on misleading or failed formalisms, organized by domain, that should be useful for the study group.

      • The Situated Action / GOFAI Critique Cluster

      • Your Agre & Chapman work sits within a broader intellectual ecosystem worth mapping out:
      • *Lucy Suchman, Plans and Situated Actions* (1987)** — The closest peer to Agre's work, and essential reading here.
      • Suchman argues that the planning model of interaction favoured by the majority of AI researchers does not take sufficient account of the situatedness of most human social behaviour.
      • Her key move:
      • all actions are situated and their characteristics are indexical, and plans systematically filter out the detail that characterizes situated actions.
      • Based on ethnomethodological studies at Xerox PARC, Suchman demonstrated that photocopiers' help systems were built for plans, but their users worked through situated actions, resulting in mismatches and rendering the help systems unhelpful.
      • This is a very direct analog to protocol formalism problems.
      • *Hubert Dreyfus, What Computers Can't Do* (1972) / What Computers Still Can't Do (1992)** — The earlier, philosophically-grounded version of the same critique.
      • His critique was based on the insights of modern continental philosophers such as Merleau-Ponty and Heidegger, and was directed at the first wave of AI research which tried to reduce intelligence to high-level formal symbols.
      • Specifically: early AI equated thought with an explicit rule-book. Dreyfus called this the "psychological assumption"—the belief that experts produce their skill by consciously following rules they could, in principle, recite. Dreyfus argued real expertise works the opposite way: perception outruns articulation, and we justify our moves only after making them.
      • He was largely dismissed by the AI community at the time, but Dreyfus's objections have not been disproved so much as side-stepped by the shift to statistical methods.
      • **Agre's own retrospective: "Toward a Critical Technical Practice" (1997)** —
      • Agre identifies two related precepts that underwrite the AI project: that cognition comprises mental operations translatable as computation, and that those translations can be achieved through techniques of formalization. These precepts are sustained, he argues, rhetorically.
      • The essay reflects on what it means to try to reform a field from within, which is directly relevant to your position as formalism-critical rather than anti-formalist.
      • *Brian Cantwell Smith, On the Origin of Objects* (1996)** — A more technical but deep treatment.
      • Based on a sustained critique of the formal tradition that underlies the reigning views, he presents an argument for an embedded, participatory, "irreductionist," metaphysical alternative.
      • Smith coined the term "inscription error" for the mistake of treating formal representations as capturing something they don't — his analysis of formality as overly disconnected ("discreteness run amok") is particularly pointed.

      • The "Map is Not the Territory" Cluster

      • A separate but complementary tradition:
      • **Alfred Korzybski (1931)** — The original statement.
      • In 1931, Korzybski introduced the phrase "the map is not the territory." His argument was that human beings navigate reality through abstractions — language, models, categories, frameworks — and that these abstractions are not reality itself.
      • Goodhart's Law — The organizational/policy instantiation of this problem.
      • Formulated by economist Charles Goodhart, it posits that "when a measure becomes a target, it ceases to be a good measure." The act of gaming a metric erodes its utility for understanding genuine performance.
      • This is highly relevant to protocol formalization: once you formalize a protocol and make it a target for compliance, the formalism stops tracking what it was meant to track.
      • John Kay, "The Map is Not the Territory: An Essay on the State of Economics" —
      • Kay argues that economists have been led astray by excessive reliance on formal models derived from assumptions that bear too little similarity to the world we live in.
      • A domain-specific application of the general critique.

      • The Planning / State Legibility Cluster

      • *James C. Scott, Seeing Like a State* (1998)** — Probably the most powerful single book on how formalization-for-legibility fails at scale.
      • Scott continually contrasts the abstract, simplified, "legible" but ultimately unrealistic plans of high-modernist administrators with the down-to-earth, complex, "illegible" but realistic approaches of small-scale producers.
      • The key concept is metis (practical local knowledge) vs. formal epistemic knowledge:
      • formal schemes cannot substitute for metis.
      • And the failure mode is active, not just passive:
      • to the degree that authorities insist on replacing this ineffably complex web of activity with formal rules and regulations, they are certain to disrupt the web in ways that they cannot possibly foresee.
      • Scott is explicit that his critique is structural, not anti-planning per se — which maps well onto your "formalism-critical" position.

      • The Philosophical/Logical Limits Cluster

      • **Wittgenstein's Rule-Following Paradox (Philosophical Investigations, §201)** —
      • Wittgenstein states: "This was our paradox: no course of action could be determined by a rule, because any course of action can be made out to accord with the rule." The rule-following paradox threatens ordinary beliefs and practices concerning meaning because it implies that there is no such thing as meaning something by an expression or sentence.
      • The upshot is that conventions cannot explicitly describe or prescribe how users use mathematics and language in general, which implies that linguistic conventions are largely a post-hoc expression of how people decide to use language in practice, rather than the converse.
      • This is directly relevant: a formal protocol specification faces exactly this problem — it cannot fully determine its own application.
      • Gödel's Incompleteness Theorems —
      • Gödel's result states that for any formal system of interesting complexity there is a necessary incompleteness. There are always statements expressible within the system and informally provable to be true, yet not provable according to the strict formal rules of the system itself. New statements are constructible with the same property — informal provability in the face of formal intractability.
      • The logical bedrock of the whole critique.

      • Summary for the Study Group

      • The through-line across all these: formalisms work by stripping away context, and the stripped-away context is often exactly what matters for actual practice. The failure modes are:
      • Incompleteness (Gödel, Wittgenstein) — no formal system can fully specify its own application
      • Situational blindness (Agre, Suchman, Dreyfus) — formal models of action can't capture the grounding in concrete activity
      • Legibility displacement (Scott, Goodhart) — once you formalize for administrative or compliance purposes, the formalism starts optimizing itself rather than the underlying practice
      • Inscription error (B.C. Smith) — the formal representation gets treated as if it is the thing it represents
      • None of these are arguments against formalism as a tool; all of them are arguments against mistaking the formalism for the territory it describes.