LLM - Memory for Agent

1. Buffer 2. Window 3. Token 4. Summary 5. Long Term 6. Semantic Enter fullscreen mode Exit fullscreen mode BUFFER MEMORY For each chat, we are going to store complete conversation, like below. In-memory --> RAM Persistent Storage --> like --> SQLITE3, POSTGRES For each and every call, whole conversation history is going to be sent to LLM.

For new chat, its going to have a DIFFERENT IDENTIFIER.

What problem can be identified in these 3 above points? ஒரு சின்ன பிரச்சனை இருக்கு, அது என்ன?

Point 2 --> We need to send the complete HISTORY to the LLM --> Tokens gets affected, also CONTEXT Limit gets affected. But you will get the output.

In the code, InMemorySaver() --> Will help to store in the RAM Memory.

With the capacity of RAM storage only it can store.

arised ……………………….Ipsum → இவ்ளோ words இந்த paragraph ஆயிரம் டோக்கன் வெச்சுக்குங்க ( 1000 tokens ), but popularised word is broken, பாதியிலிருந்து அந்த word வந்து எடுத்துக்குது, நம்ம இதுல set பண்ணலாம், பாதி word வெச்சிக்கிட்டு நம்ம என்ன பண்ண முடியும். Even though 1000 tokens are used, to get the proper meaning, it will take ( that means we will set ), take from the start of the sentence. “ It was popularised …… Ipsum.” --> This will be handled by the Human conversation message.

trim_messages() --> Function will do the honour in this.

In scenarios --> It will be useful only for recent messages. OLD messages will be deleted as per the token size.

Anything older than the given budget is lost completely. --> this is the DOWNSIDE.

Model needs memor y? --> They remember nothing: every call is fresh. The app must resend earlier messages.

Buffer memory --> Keep every message. Simple, but it grows forever (cost and size limit).

Window Memory --> Keep the first message plus the last few. Forgets the middle.

Token budget --> Keep the newest messages that fit a token limit (trim_messages). Still forgets old things.

Summary memory --> Replace old messages with a short summary so facts survive. Costs an extra AI call.

Long-term memory --> Facts saved in a Store, shared across a user's conversations. The agent decides what to save.

Semantic memory --> Long-term facts searched by meaning with embeddings. Needs an embedding model.

Checkpointer --> Saves the conversation after each turn (InMemorySaver; SqliteSaver or a database for persistence).

thread_id --> The conversation's file number. Same id continues it; a new id starts fresh.

Middleware (@before_model) --> A function that runs right before each AI call and can edit the message list.

Store vs checkpointe r --> Checkpointer = one conversation. Store = lasting facts about the user across conversations.

user_id in memory tools --> Comes from your app's context, not from the AI, so users can't read each other's memories.