Research & References

Academic Foundations

Explore the theoretical underpinnings of resumable AI systems, from continuation-passing style to modern memory architectures.

FoundationsRelated WorkOur ContributionsReferences

Academic Foundations

Theoretical concepts that enable resumability

Continuation-Passing Style (CPS)

Originated in Scheme (1975), CPS transforms programs to make control flow explicit. The call/cc (call-with-current-continuation) operator captures the program's execution state at any point, enabling:

Key Properties

  • •First-class continuations as values
  • •Non-local control flow (early exit, exceptions)
  • •Resumable computation from any point
  • •Multiple invocation of same continuation

Relevance to AI

  • →Cognitive checkpoints = captured continuations
  • →Branching = invoking continuation multiple times
  • →Tool calls = continuation + callback
typescript
// Conceptual parallel: Scheme's call/cc ≈ AI checkpoint
(call/cc (lambda (k)          // k = current continuation
  (set! saved-continuation k)  // save for later
  (k "continue")))             // invoke immediately

// In resumable AI:
const checkpoint = captureState(currentContext);  // capture "continuation"
saveCheckpoint(checkpoint);                        // save for later
restoreState(checkpoint);                          // invoke to resume

Process Checkpointing (CRIU, DMTCP)

Operating systems have long supported process migration through checkpoint/restore. CRIU (Checkpoint/Restore In Userspace) and DMTCP (Distributed MultiThreaded Checkpointing) serialize entire process state including:

Memory State

  • • Heap
  • • Stack
  • • BSS/Data segments

Process Context

  • • Registers
  • • Signal handlers
  • • File descriptors

External State

  • • Sockets
  • • Pipes
  • • Shared memory

Key Insight

Unlike OS processes, AI cognitive state is semantic rather than binary. We don't need to serialize memory addresses — we serialize meaning. This makes cognitive checkpoints portable across model versions and architectures.

Transformer KV-Cache & Attention State

Transformers naturally create a form of "working memory" through Key-Value caching. During autoregressive generation, previously computed attention states are cached to avoid recomputation.

// Simplified attention with KV-cache
for each new token:
K_new = W_k @ token_embedding
V_new = W_v @ token_embedding
cache.keys.append(K_new) // Grow cache
cache.values.append(V_new)
Q = W_q @ token_embedding
attention = softmax(Q @ cache.keys.T / sqrt(d))
output = attention @ cache.values

The problem: KV-caches are model-specific and ephemeral. They exist only during a single inference session and can't be serialized across calls.

Our approach: Cognitive checkpoints operate at a higher level of abstraction. Instead of caching attention weights, we cache the semantic content that produces consistent reasoning — facts, beliefs, intentions, and decisions.

Memory-Augmented Neural Networks

Research into Neural Turing Machines (NTM) and Differentiable Neural Computers (DNC) explored explicit memory modules that networks could read from and write to.

Neural Turing Machine (2014)

  • • External memory matrix M ∈ ℝ^(N×M)
  • • Content-based addressing via softmax
  • • Location-based addressing for sequences
  • • Read/write heads with attention

Differentiable Neural Computer (2016)

  • • Temporal link matrix for ordering
  • • Usage vector for memory allocation
  • • Multiple read heads
  • • Free list for memory reuse

Connection to Resumability

Our epistemic state (facts, beliefs, assumptions) is analogous to the external memory in MANNs. The key difference is that our memory is human-readable and semantically structured, enabling inspection, editing, and transfer between systems.

Our Contributions

Novel aspects of this approach

1

Cognitive State Serialization

We formalize a schema for serializing cognitive state — not just conversation history, but the structured reasoning that emerges from it. This includes:

  • Intent Graphs
    Hierarchical goals with dependencies
  • Decision Trees
    Branching choices with rationale
  • Tool Binding State
    Mid-execution tool call context
  • Confidence Intervals
    Uncertainty quantification
2

Epistemic State Formalization

We introduce a structured representation of "what the AI knows" that goes beyond simple key-value memory:

Facts
Verified information with sources
Assumptions
Unverified premises being used
Questions
Open uncertainties blocking progress
Beliefs
Inferences with confidence levels
3

TOON Serialization Format

Token-Oriented Notation: a compact serialization format optimized for LLM token efficiency. Reduces checkpoint size by 20-40% compared to standard JSON while remaining human-readable.

Design principles:
  • • Abbreviated keys (2-3 chars)
  • • Omit null/undefined values
  • • Inline small objects
  • • Reference deduplication
4

Cognitive Branching

"Git for cognition" — fork from any checkpoint to explore alternative reasoning paths, then optionally merge insights back.

Use cases: A/B testing reasoning strategies, exploring hypotheticals, recovering from errors, collaborative review of AI decisions.

References

Key papers and resources

[1]
Graves, A., Wayne, G., & Danihelka, I. (2014). Neural Turing Machines. arXiv preprint arXiv:1410.5401.
arxiv.org/abs/1410.5401...
[2]
Graves, A., et al. (2016). Hybrid computing using a neural network with dynamic external memory. Nature 538, 471-476.
www.nature.com/articles/nature20101...
[3]
Vaswani, A., et al. (2017). Attention Is All You Need. NeurIPS 2017.
arxiv.org/abs/1706.03762...
[4]
Sussman, G. J., & Steele, G. L. (1975). Scheme: An Interpreter for Extended Lambda Calculus. MIT AI Lab Memo 349.
dspace.mit.edu/handle/1721.1/5794...
[5]
Hevery, M. (2022). Qwik: Resumability vs Hydration. Builder.io Blog.
www.builder.io/blog/resumability-vs-hydr...
[6]
Chase, H. (2022). LangChain: Building LLM applications. GitHub/LangChain.
github.com/langchain-ai/langchain...

"The future isn't better prompts. It's persistent, resumable cognition — AI systems that remember not just what was said, but what was thought."

— The core insight of resumable AI systems

Explore Further

View Implementation See Real Examples