Building Conversational RAG with Long-Term Memory

Introduction

A classic RAG system processes each query independently. But users expect continuous conversations where the assistant remembers previous exchanges. This guide explains how to implement persistent memory.

Types of Memory

1. Session Memory (Short-Term)

• Duration: One conversation
• Content: Message history
• Usage: Maintain immediate context
• Storage: Redis

2. User Memory (Long-Term)

• Duration: Permanent
• Content: Preferences, learned information
• Usage: Personalization
• Storage: PostgreSQL

3. Episodic Memory

• Duration: Permanent
• Content: Notable past conversations
• Usage: Reference previous exchanges
• Storage: Qdrant (vector)

Architecture

The flow includes: User Message > Memory Retrieval Layer (Session Memory Redis + User Profile Postgres + Episodic Memory Qdrant) > Context Builder > RAG Pipeline > Memory Update Layer.

Implementation

1. Session Memory Management

Use Redis with 24h TTL, keep the N most recent messages to avoid explosion.

DEVELOPERpython
class SessionMemory:
    def __init__(self, redis_client, session_id, max_messages=20):
        self.redis = redis_client
        self.session_id = session_id
        self.max_messages = max_messages
        self.ttl = 86400  # 24h

2. User Fact Extraction

Extract durable facts: profession, expertise level, technologies used, preferences.

3. Vector Episodic Memory

Create memorable session summaries, calculate importance, store in Qdrant.

4. Context Building

Combine user profile + relevant memories + session history.

Best Practices

1. Privacy Management

Implement a forget_user mechanism for GDPR: delete profile, episodic memory, and sessions.

2. Size Limitations

• MAX_FACTS = 20
• MAX_EPISODIC_MEMORIES = 100
• MAX_SESSION_MESSAGES = 50

Regularly clean old memories while keeping the most important/recent ones.

Conclusion

Long-term memory transforms a chatbot into a true personal assistant. Users appreciate not having to repeat themselves, and personalization significantly improves response quality.

Key points:

1. Separate memory types by duration and usage
2. Intelligent extraction of persistent facts
3. Semantic search for episodic memories
4. Privacy respect with forgetting mechanisms