Learn how to deploy a Streamlit-based LLM chatbot whose history is stored in Azure Managed Redis. Setup takes just five minutes, with built-in capabilities like per-user memory, TTL, token counts, and custom system instructions.
Multiple users, each with their chat memory and settings
Running a live token count over the stored chat context
Ability to trim context to the last n messages
User-specific TTL for chat history entries
Configurable system instructions
The demo app consists of three main Azure components working together to deliver a multi-user LLM chatbot with persistent memory:
Azure App Service hosts the Streamlit web app (LLMmemory.py). When a user submits a prompt, the app’s managed identity obtains an Azure AD token and forwards the request to Azure OpenAI.
Azure OpenAI Service (GPT-4o) processes each incoming chat request. The Streamlit app sends the most recent context (based on the “Length of chat history” setting) alongside a system prompt to the OpenAI endpoint, which returns the assistant’s response.
Azure Managed Redis stores every message—user prompts, AI replies, and system instructions—as Redis hashes under keys like mysession:<UserName>:<entry_id>. The redisvl library’s StandardSessionManager abstracts reads and writes, enabling features such as per-user chat history, TTL, and token counting.
Azure Subscription
Azure CLI
Azure Developer CLI
Install/update Azure CLI
brew update
brew install azure-cli
Azure login
az login
This should open a new browser window or tab to authenticate into your Azure account.
Install Azure Developer CLI
brew install azure/azure-dev/azd
If you have trouble installing the Azure Developer CLI from above, try grabbing it from here
curl -fsSL https://aka.ms/install-azd.sh | bash
Verify the version
azd version
You should see azd version 1.x
Clone the demo repository and get into the folder
git clone [email protected]:robertopc1/Redis_LLMmemory.git
cd Redis_LLMmemory
Azure Development CLI login
azd auth login
If you’re not an Admin/Owner of the Azure account you’re using, then before you run the demo, make sure your Azure user has the Cognitive Services contributor role. If you still hit errors while running azd up, being assigned the owner of the resource group unblocks most errors. If you need to troubleshoot with various permissions, resource groups, etc., remember to run azd auth logout and then azd auth login to refresh your session in your terminal after making changes in Azure.
Run:
azd up
Most of the time, everything stands up by itself within five minutes. And you’ll get a URL endpoint to the app in the terminal that looks something like this:
Deploying services (azd deploy)
(✓) Done: Deploying service web
- Endpoint: https://web-fostjta2f5eww.azurewebsites.net/
Occasionally, it may time out while creating/updating resources:
1# Define users 2users = ["Luke", "Leia", "Han"] 3 4@st.cache_resource 5def initSessionManager(_redis_client): 6 session_manager = StandardSessionManager( 7 name='mysession', 8 redis_client=_redis_client 9 ) 10 session_manager.clear() # Clear any existing data 11 for user in users: 12 # Preload a system message separately for each user 13 session_manager.add_message( 14 {"role": "system", "content": "You are a helpful assistant."}, 15 session_tag=user 16 ) 17 return session_manager 18 19# Initialize Redis client and SessionManager 20credential_provider = create_from_default_azure_credential(("https://redis.azure.com/.default",)) 21redis_host = os.getenv("REDIS_HOST") 22redis_port = os.getenv("REDIS_PORT") 23redis_client = redis.Redis( 24 host=redis_host, 25 port=redis_port, 26 ssl=True, 27 credential_provider=credential_provider 28) 29session_manager = initSessionManager(redis_client) 30 31# Default Streamlit state for which user is active 32if "userselectbox" not in st.session_state: 33 st.session_state.userselectbox = "Luke" 34if "contextwindow" not in st.session_state: 35 st.session_state.contextwindow = 5 36 37# Sidebar control to switch active user 38user = st.sidebar.selectbox("Select User", users, key="userselectbox")1def calculate_tokens(text): 2 encoding = tiktoken.get_encoding("cl100k_base") 3 tokens_per_message = 3 4 tokens_per_name = 1 5 num_tokens = 0 6 for message in text: 7 num_tokens += tokens_per_message 8 for key, value in message.items(): 9 num_tokens += len(encoding.encode(value)) 10 if key == "name": 11 num_tokens += tokens_per_name 12 num_tokens += 3 # Add tokens for assistant priming 13 return num_tokens 14 15# Fetch the last N messages and show token count in the sidebar 16chathistory = session_manager.get_recent( 17 top_k=st.session_state.contextwindow, 18 session_tag=st.session_state.userselectbox, 19 as_text=False 20) 21tokens = st.sidebar.metric( 22 label="Chat history tokens", 23 value=calculate_tokens(chathistory) 24) 251# Sidebar slider to select how many messages to keep in context 2contextwindow = st.sidebar.slider("Length of chat history", 1, 20, key="contextwindow") 3 4# When sending a new prompt: 5historylength = st.session_state.contextwindow 6# Retrieve only the last ‘historylength’ messages (skip system at index 0) 7messages = session_manager.get_recent( 8 top_k=historylength, 9 session_tag=st.session_state.userselectbox 10)[1:] 11# Pass `messages` into the LLM API calldef add_ttl(ttl_length, contextwindow, user): # Fetch raw entries for the last `contextwindow` messages messages = session_manager.get_recent( top_k=contextwindow, session_tag=user, raw=True ) # Set expiration on each key for m in messages: key_id = m["id"] redis_client.expire(key_id, ttl_length) # Sidebar controls for TTL ttl_length = st.sidebar.slider("TTL time (seconds)", 1, 600, 60) ttl_submit = st.sidebar.button("Set TTL of chat history") if ttl_submit: add_ttl( ttl_length, st.session_state.contextwindow, st.session_state.userselectbox )1systeminstructions = [ 2 "Standard ChatGPT", 3 "Extremely Brief", 4 "Obnoxious American" 5] 6 7def update_system_instructions(user, systeminstruction): 8 # Retrieve the first (system) message for this user 9 messages = session_manager.get_recent( 10 top_k=100, 11 session_tag=user, 12 raw=True 13 ) 14 systemmessage = messages[0] 15 keyname = systemmessage["id"] 16 17 if systeminstruction == "Standard ChatGPT": 18 redis_client.hset( 19 keyname, 20 "content", 21 "You are a helpful assistant." 22 ) 23 elif systeminstruction == "Extremely Brief": 24 redis_client.hset( 25 keyname, 26 "content", 27 "You are a VERY brief assistant. Keep your responses as short as possible." 28 ) 29 elif systeminstruction == "Obnoxious American": 30 redis_client.hset( 31 keyname, 32 "content", 33 "You are a VERY pro-American assistant. " 34 "Make sure to emphasize how great the good ole’ USA is in your responses." 35 ) 36 37# Sidebar dropdown to select the instruction 38st.sidebar.selectbox( 39 "System Instructions", 40 systeminstructions, 41 key="systeminstructions", 42 on_change=lambda: update_system_instructions( 43 st.session_state.userselectbox, 44 st.session_state.systeminstructions 45 ) 46) 47When you’re ready to shut down the app, remember to tear down your resources:
azd down
