Agentic AI: OpenClaw/MoltBot/ClawdBot’s Memory Architecture Explained

OpenClaw implements a radical departure from conventional AI memory systems: plain Markdown files as the source of truth, layered with hybrid semantic search. This open-source AI agent — created by Austrian developer Peter Steinberger, and now boasting 145,000+ GitHub stars — has captured Silicon Valley’s attention with its “persistent memory” that recalls interactions across weeks. Unlike traditional RAG systems that hide data in vector databases, OpenClaw’s approach prioritises transparency, human readability, and version controllability.

What OpenClaw is and why it matters

OpenClaw (formerly Clawdbot, then Moltbot after Anthropic trademark complaints) is a locally-running autonomous AI agent that connects to LLMs like Claude or ChatGPT through a persistent Gateway process. Users interact via familiar messaging platforms — WhatsApp, Telegram, Discord, Slack, iMessage — while the agent manages emails, calendars, browses the web, summarises documents, and executes shell commands on their behalf.

Two-tier memory design separates ephemeral from durable knowledge

OpenClaw’s memory architecture uses a deliberately simple structure within the default workspace (typically ~/clawd/):

Daily logs (memory/YYYY-MM-DD.md) function as append-only ephemeral memory capturing running context, decisions, and activities. The system automatically creates new files each day and loads today's plus yesterday's logs at session start, providing recent temporal context without overwhelming the context window.

Long-term memory (MEMORY.md) stores curated, stable information—preferences, project conventions, critical decisions. A crucial privacy feature: this file only loads in private sessions, never in group contexts, protecting sensitive information from exposure in shared channels.

Session transcripts (sessions/YYYY-MM-DD-<slug>.md) preserve full conversation histories with LLM-generated descriptive filenames. When enabled viaexperimental.sessionMemory: true, they become searchable, allowing recall of conversations from weeks earlier.

The file-first philosophy means the AI literally “remembers” only what gets written to disk. Everything remains human-readable, editable with any text editor, and version-controllable via Git — a stark contrast to opaque vector database blobs.

OpenClaw Memory Flow

Hybrid search combines semantic understanding with exact matching

OpenClaw’s retrieval system employs weighted score fusion, merging two complementary approaches:

Vector search (default 70% weight) uses cosine similarity with embeddings stored in SQLite via the sqlite-vec extension. This handles conceptual matches where wording differs—"gateway host" semantically matches "machine running gateway."

BM25 keyword search (default 30% weight) uses SQLite’s FTS5 full-text search for exact token matching. This excels at error codes, function names, and unique identifiers where semantic similarity would fail.

The critical implementation detail: OpenClaw uses union, not intersection. Results from either search contribute to the final ranking. A chunk that scores high on vectors but zero on keywords is still included, ensuring comprehensive recall.

finalScore = vectorWeight × vectorScore + textWeight × textScore

BM25 ranks convert to scores via1/(1 + rank), making rank 0 equal 1.0 and rank 9 equal 0.1. Configuration allows adjusting weights, with candidateMultiplier: 4 fetching extra candidates before fusion for better coverage.

Why not Reciprocal Rank Fusion?

RRF flattens score magnitude — OpenClaw needs it. RRF converts everything to ranks, so a vector hit with 0.98 cosine similarity and one with 0.71 both just become “rank 1” and “rank 2.” OpenClaw’s vector search returns meaningful cosine similarity scores, and they want that signal preserved. A near-perfect semantic match should dominate the final ranking, not get reduced to an ordinal position.

Asymmetric weighting is the whole point. OpenClaw deliberately weights vector search at 70% and BM25 at 30% by default. RRF treats all rankers as equal contributors (or requires workarounds like duplicating rankers to simulate weighting). OpenClaw’s formula — vectorWeight × vectorScore + textWeight × textScore — makes the asymmetry explicit and trivially configurable per user.

Embedding providers offer a local-first with a fallback chain.

The system auto-selects embedding providers in priority order: Local → OpenAI → Gemini → BM25-only fallback.

Embedding Provider Fallback Chain for OpenClaw

Local embedding uses node-llama-cpp and auto-downloads GGUF models from HuggingFace on first use. OpenAI's Batch API integration reduces costs by 50% for bulk indexing operations. If all embedding providers fail, the system gracefully degrades to keyword-only BM25 search rather than breaking entirely.

Pre-compaction memory flush prevents context loss

One of OpenClaw’s most innovative features addresses a critical problem: when sessions exceed context limits, compaction discards information, potentially losing valuable context.

The solution: when sessions approach the threshold (calculated as contextWindow - reserveTokensFloor - softThresholdTokens), OpenClaw triggers a silent agentic turn, prompting the model to write durable memories before compaction occurs. For a 200K context window with default settings (20K reserve, 4K soft threshold), this triggers at approximately 176K tokens.

The flush typically responds with NO_REPLY If nothing needs saving, keep interactions seamless. Safeguards prevent double-flushing by tracking memoryFlushCompactionCount in the session state. The feature gracefully skips when the workspace is read-only.

SQLite schema enables efficient storage and retrieval

Memory indices live at ~/.openclaw/memory/{agentId}.sqlite with these core tables:

• files: Tracks file paths, source type, content hash, and update timestamps for delta-based indexing
• chunks: Stores text segments with embeddings, line numbers, content hash, and model information
• embedding_cache: Cross-file deduplication using SHA-256 hashes prevents re-embedding identical content
• chunks_fts: FTS5 virtual table enabling fast lexical search
• vec_chunks: Vector acceleration via sqlite-vec extension (falls back to JS cosine similarity if unavailable)

The chunking algorithm uses ~400 tokens per chunk with 80-token overlap, preserving context across chunk boundaries. Line-aware processing maintains precise source attribution for search results.

How OpenClaw compares to traditional RAG approaches

Openclaw Memory Retrieval vs Traditional RAG

Openclaw Memory Retrieval vs Traditional RAG

Compared to Claude Code’s stateless terminal sessions, OpenClaw maintains persistent stateful sessions with long-term memory spanning hours or days. Versus Meta’s acquired Manus agent, OpenClaw’s open-source nature allows full inspection of the memory implementation code

Third-party plugins extend memory capabilities

The community has developed several memory extensions:

• mem0/openclaw: Long-term memory via Mem0’s managed service
• openclaw-graphiti-memory: Temporal knowledge graphs via Graphiti
• nova-memory: PostgreSQL-based structured memory
• openclaw-supermemory: Cloud-based via Supermemory

These address limitations like cross-device memory sync (the default system keeps memory on a single machine) and structured data requirements.

Security implications of persistent memory deserve attention

Security researchers have identified a “lethal trifecta” of risks: access to private data, exposure to untrusted content, and the ability to perform external communications while retaining memory. Cisco’s analysis found 26% of 31,000 agent skills contained at least one vulnerability.

1Password’s security assessment warns: “A single stolen API token is bad… but a hundred stolen tokens and sessions, plus a long-term memory file that describes who you are, what you’re building, and who you work with, is something else entirely.”

Memory and configuration are stored in plain-text files in predictable locations — convenient for users but equally convenient for infostealers. The documentation acknowledges prompt injection remains “an unsolved industry-wide problem.”

Conclusion

OpenClaw’s memory architecture represents a philosophical choice: prioritising transparency, human control, and graceful degradation over the convenience of managed vector databases. The file-first approach means users own their data completely — every memory, preference, and conversation lives as readable text rather than opaque embeddings.

The hybrid BM25 + vector search addresses real retrieval limitations, while the pre-compaction flush mechanism solves the context window cliff that plagues long-running AI sessions. Whether this “files as source of truth” philosophy scales to enterprise use remains uncertain, but for personal AI assistants running on dedicated Mac Minis, it offers an unusually transparent window into what the AI actually “knows” about you.

References

• Official documentation: https://docs.openclaw.ai/concepts/memory
• GitHub repository: https://github.com/openclaw/openclaw
• DigitalOcean deployment guide: https://www.digitalocean.com/resources/articles/what-is-openclaw
• OpenClaw Session Compaction: https://docs.openclaw.ai/reference/session-management-compaction
• OpenClaw Memory Deepdive: https://docs.openclaw.ai/concepts/memory
• Turing College: https://www.turingcollege.com/blog/openclaw

If you like my posts, don’t forget to subscribe to my profile for more tips and productivity hacks on Claude Code.