AI Agent Long-Term Memory in Python (2026): A Framework-Free Guide

Quick Answer (July 2026): Long-term memory lets your AI agent remember facts across separate sessions, not just inside one conversation. You do not need LangChain, FAISS, or a hosted vector database to build it. Store durable facts in SQLite, embed each one once with an embeddings API, and on every new turn retrieve the few most relevant facts by cosine similarity and inject them into the system prompt. This tutorial builds exactly that in about 120 lines of framework-free Python, using the Anthropic SDK for the agent and OpenAI's text-embedding-3-small for retrieval.

Short-term vs long-term memory: the distinction most tutorials blur

Almost every "give your LLM memory" guide reaches for a framework and a vector service on day one. Before you install anything, get the two kinds of memory straight, because they are solved differently.

If short-term is all you need (a running buffer plus rolling summaries), the TypeScript companion to this post covers it end to end: AI agent memory in TypeScript. This post is the long-term half, in Python: how an agent recalls a fact from a conversation that happened days ago, in a process that has since restarted.

The SERP consensus here (LangChain, FAISS, Mem0, LangMem) is heavy infrastructure. You do not need any of it until you are holding tens of thousands of memories per user. Below is the version you fully own and can read top to bottom.

What we are building

A support-style agent that remembers user facts across sessions. In session one the user mentions their plan and their database. In session two, a brand new process with an empty message history, the agent still answers a question about their setup, because the fact was written to long-term memory and retrieved on demand.

Three pieces, nothing else:

1. A SQLite table of memories (text plus a stored embedding).
2. A writer that, at the end of a session, extracts durable facts with a cheap model, embeds them, and stores them (skipping near-duplicates).
3. A reader that, on each new turn, embeds the user message, retrieves the top matching facts by cosine similarity, and injects them into the system prompt.

Prerequisites

pip install anthropic openai numpy
export ANTHROPIC_API_KEY=sk-ant-...
export OPENAI_API_KEY=sk-...

Models used (July 2026 aliases, never pinned dated ids, so you do not ship a model that quietly goes stale):

• claude-sonnet-5 for the agent
• claude-haiku-4-5 for cheap fact extraction
• text-embedding-3-small for embeddings (see OpenAI's embeddings guide for current 2026 pricing; Anthropic recommends Voyage AI embeddings as an alternative in the Messages API docs)

the store

Plain sqlite3 from the standard library. The embedding is stored as JSON text next to the fact.

# memory.py
import sqlite3

def open_store(path="agent_memory.db"):
    conn = sqlite3.connect(path)
    conn.execute("""
        CREATE TABLE IF NOT EXISTS memories (
            id         INTEGER PRIMARY KEY AUTOINCREMENT,
            user_id    TEXT NOT NULL,
            text       TEXT NOT NULL,
            embedding  TEXT NOT NULL,
            created_at TEXT DEFAULT (datetime('now'))
        )
    """)
    conn.commit()
    return conn

embeddings and cosine similarity

One function to turn text into a vector, one to compare two vectors. text-embedding-3-small returns a 1536-dimension vector; the norm is not guaranteed to be 1, so we normalize inside cosine.

import json
import numpy as np
from openai import OpenAI

oai = OpenAI()

def embed(text: str) -> list[float]:
    resp = oai.embeddings.create(model="text-embedding-3-small", input=text)
    return resp.data[0].embedding

def cosine(a, b) -> float:
    a, b = np.asarray(a), np.asarray(b)
    return float(a @ b / (np.linalg.norm(a) * np.linalg.norm(b)))

retrieve the top matching facts

No index, no vector DB. For one user's memories, a full scan in Python is fast and correct up to tens of thousands of rows. Embed the query once, score every stored fact, return the best k.

def recall(conn, user_id: str, query: str, k: int = 4) -> list[str]:
    q = embed(query)
    rows = conn.execute(
        "SELECT text, embedding FROM memories WHERE user_id = ?", (user_id,)
    ).fetchall()
    scored = [(cosine(q, json.loads(emb)), text) for text, emb in rows]
    scored.sort(reverse=True)
    return [text for _, text in scored[:k]]

extract durable facts, not raw turns

This is the step tutorials skip, and it is the one that decides whether retrieval works. Do not embed raw transcripts. If you dump every message into the store, retrieval drowns in small talk and half-finished thoughts. Instead, run the session transcript through a cheap model and keep only atomic, durable facts.

from anthropic import Anthropic

anthropic = Anthropic()

FACT_PROMPT = (
    "Extract durable facts about the user from this conversation. "
    "Return one atomic fact per line, no numbering, no preamble. "
    "Keep only stable facts worth remembering later: preferences, setup, goals. "
    "Skip small talk and anything transient. If there are none, return nothing."
)

def extract_facts(transcript: str) -> list[str]:
    msg = anthropic.messages.create(
        model="claude-haiku-4-5",   # cheap model, alias not a pinned id
        max_tokens=300,
        system=FACT_PROMPT,
        messages=[{"role": "user", "content": transcript}],
    )
    lines = msg.content[0].text.strip().splitlines()
    return [ln.strip("-* ").strip() for ln in lines if ln.strip()]

def remember(conn, user_id: str, facts: list[str], dedup_threshold: float = 0.95):
    for fact in facts:
        vec = embed(fact)
        existing = conn.execute(
            "SELECT embedding FROM memories WHERE user_id = ?", (user_id,)
        ).fetchall()
        if any(cosine(vec, json.loads(e)) > dedup_threshold for (e,) in existing):
            continue  # near-duplicate, do not store it again
        conn.execute(
            "INSERT INTO memories (user_id, text, embedding) VALUES (?, ?, ?)",
            (user_id, fact, json.dumps(vec)),
        )
    conn.commit()

The dedup_threshold is what stops the store from filling with fifty copies of "user is on the Pro plan." Anything above 0.95 cosine to an existing memory is treated as already known.

the agent loop with memory injection

Now wire it together. On each turn, recall the relevant facts and prepend them to the system prompt. The message history holds short-term memory for the current session; long-term memory arrives through the system prompt regardless of how empty that history is.

def chat(conn, user_id: str, user_msg: str, history: list[dict]) -> str:
    memories = recall(conn, user_id, user_msg)
    system = "You are a helpful support agent. Be concise."
    if memories:
        known = "\n".join(f"- {m}" for m in memories)
        system += f"\n\nKnown facts about this user:\n{known}"

    history.append({"role": "user", "content": user_msg})
    msg = anthropic.messages.create(
        model="claude-sonnet-5",     # alias, no dated id to go stale
        max_tokens=500,
        system=system,
        messages=history,
    )
    reply = msg.content[0].text
    history.append({"role": "assistant", "content": reply})
    return reply

Run it end to end

Two sessions, in two separate processes, sharing only the SQLite file on disk.

# --- session 1 ---
conn = open_store()
history = []
chat(conn, "u_123", "Hi, I'm on the Pro plan and my stack is Postgres.", history)

# persist durable facts before the process exits
transcript = "\n".join(f"{m['role']}: {m['content']}" for m in history)
remember(conn, "u_123", extract_facts(transcript))
conn.close()

# --- session 2, a brand new process, empty history ---
conn = open_store()
print(chat(conn, "u_123", "Remind me which database I use?", []))
# -> "You're on Postgres." The agent had no history, but recall() pulled the
#    stored fact and injected it into the system prompt.

The second call starts with history=[]. Short-term memory is empty, the process restarted, and the agent still answers correctly. That is long-term memory: the fact survived outside the context window and came back by relevance.

Gotchas most long-term-memory tutorials skip

1. Embed facts, not transcripts. Retrieval quality collapses when you store raw turns. Step 4 exists for a reason.
2. Deduplicate on write. Without the cosine check in remember, one repeated preference becomes dozens of rows that crowd out everything else at retrieval time.
3. Handle staleness and contradiction. Facts change ("I switched to MySQL"). The simplest robust policy is append plus prefer the most recent; a periodic consolidation pass that asks a model to merge or retire conflicting facts is the next step up.
4. Similarity is the 80 percent version. The Generative Agents paper (Park et al., 2023) scores each memory on relevance, recency, and importance, not similarity alone. Add a recency term to your score when "what did we just discuss" matters as much as "what is most similar."

When you actually need a real vector database

A full scan over one user's memories in Python stays fast to roughly 10,000 to 100,000 rows. Past that, or once you are querying across many users at once, reach for sqlite-vec, pgvector, or a hosted store. Not before. Shipping a vector service on day one to hold two hundred facts is the most common over-engineering in this space, and it is exactly what the framework-first tutorials nudge you toward.

Where to go next

• The short-term half of the pair: AI agent memory in TypeScript, covering message buffers and rolling summaries.
• The base loop this plugs into: build your first AI agent from scratch.