Langchain Architecture

1---
2name: langchain-architecture
3description: Design LLM applications using the LangChain framework with agents, memory, and tool integration patterns. Use when building LangChain applications, implementing AI agents, or creating complex LLM workflows.
4---
5 
6# LangChain Architecture
7 
8Master the LangChain framework for building sophisticated LLM applications with agents, chains, memory, and tool integration.
9 
10## Do not use this skill when
11 
12- The task is unrelated to langchain architecture
13- You need a different domain or tool outside this scope
14 
15## Instructions
16 
17- Clarify goals, constraints, and required inputs.
18- Apply relevant best practices and validate outcomes.
19- Provide actionable steps and verification.
20- If detailed examples are required, open `resources/implementation-playbook.md`.
21 
22## Use this skill when
23 
24- Building autonomous AI agents with tool access
25- Implementing complex multi-step LLM workflows
26- Managing conversation memory and state
27- Integrating LLMs with external data sources and APIs
28- Creating modular, reusable LLM application components
29- Implementing document processing pipelines
30- Building production-grade LLM applications
31 
32## Core Concepts
33 
34### 1. Agents
35Autonomous systems that use LLMs to decide which actions to take.
36 
37**Agent Types:**
38- **ReAct**: Reasoning + Acting in interleaved manner
39- **OpenAI Functions**: Leverages function calling API
40- **Structured Chat**: Handles multi-input tools
41- **Conversational**: Optimized for chat interfaces
42- **Self-Ask with Search**: Decomposes complex queries
43 
44### 2. Chains
45Sequences of calls to LLMs or other utilities.
46 
47**Chain Types:**
48- **LLMChain**: Basic prompt + LLM combination
49- **SequentialChain**: Multiple chains in sequence
50- **RouterChain**: Routes inputs to specialized chains
51- **TransformChain**: Data transformations between steps
52- **MapReduceChain**: Parallel processing with aggregation
53 
54### 3. Memory
55Systems for maintaining context across interactions.
56 
57**Memory Types:**
58- **ConversationBufferMemory**: Stores all messages
59- **ConversationSummaryMemory**: Summarizes older messages
60- **ConversationBufferWindowMemory**: Keeps last N messages
61- **EntityMemory**: Tracks information about entities
62- **VectorStoreMemory**: Semantic similarity retrieval
63 
64### 4. Document Processing
65Loading, transforming, and storing documents for retrieval.
66 
67**Components:**
68- **Document Loaders**: Load from various sources
69- **Text Splitters**: Chunk documents intelligently
70- **Vector Stores**: Store and retrieve embeddings
71- **Retrievers**: Fetch relevant documents
72- **Indexes**: Organize documents for efficient access
73 
74### 5. Callbacks
75Hooks for logging, monitoring, and debugging.
76 
77**Use Cases:**
78- Request/response logging
79- Token usage tracking
80- Latency monitoring
81- Error handling
82- Custom metrics collection
83 
84## Quick Start
85 
86```python
87from langchain.agents import AgentType, initialize_agent, load_tools
88from langchain.llms import OpenAI
89from langchain.memory import ConversationBufferMemory
90 
91# Initialize LLM
92llm = OpenAI(temperature=0)
93 
94# Load tools
95tools = load_tools(["serpapi", "llm-math"], llm=llm)
96 
97# Add memory
98memory = ConversationBufferMemory(memory_key="chat_history")
99 
100# Create agent
101agent = initialize_agent(
102    tools,
103    llm,
104    agent=AgentType.CONVERSATIONAL_REACT_DESCRIPTION,
105    memory=memory,
106    verbose=True
107)
108 
109# Run agent
110result = agent.run("What's the weather in SF? Then calculate 25 * 4")
111```
112 
113## Architecture Patterns
114 
115### Pattern 1: RAG with LangChain
116```python
117from langchain.chains import RetrievalQA
118from langchain.document_loaders import TextLoader
119from langchain.text_splitter import CharacterTextSplitter
120from langchain.vectorstores import Chroma
121from langchain.embeddings import OpenAIEmbeddings
122 
123# Load and process documents
124loader = TextLoader('documents.txt')
125documents = loader.load()
126 
127text_splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=200)
128texts = text_splitter.split_documents(documents)
129 
130# Create vector store
131embeddings = OpenAIEmbeddings()
132vectorstore = Chroma.from_documents(texts, embeddings)
133 
134# Create retrieval chain
135qa_chain = RetrievalQA.from_chain_type(
136    llm=llm,
137    chain_type="stuff",
138    retriever=vectorstore.as_retriever(),
139    return_source_documents=True
140)
141 
142# Query
143result = qa_chain({"query": "What is the main topic?"})
144```
145 
146### Pattern 2: Custom Agent with Tools
147```python
148from langchain.agents import Tool, AgentExecutor
149from langchain.agents.react.base import ReActDocstoreAgent
150from langchain.tools import tool
151 
152@tool
153def search_database(query: str) -> str:
154    """Search internal database for information."""
155    # Your database search logic
156    return f"Results for: {query}"
157 
158@tool
159def send_email(recipient: str, content: str) -> str:
160    """Send an email to specified recipient."""
161    # Email sending logic
162    return f"Email sent to {recipient}"
163 
164tools = [search_database, send_email]
165 
166agent = initialize_agent(
167    tools,
168    llm,
169    agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION,
170    verbose=True
171)
172```
173 
174### Pattern 3: Multi-Step Chain
175```python
176from langchain.chains import LLMChain, SequentialChain
177from langchain.prompts import PromptTemplate
178 
179# Step 1: Extract key information
180extract_prompt = PromptTemplate(
181    input_variables=["text"],
182    template="Extract key entities from: {text}\n\nEntities:"
183)
184extract_chain = LLMChain(llm=llm, prompt=extract_prompt, output_key="entities")
185 
186# Step 2: Analyze entities
187analyze_prompt = PromptTemplate(
188    input_variables=["entities"],
189    template="Analyze these entities: {entities}\n\nAnalysis:"
190)
191analyze_chain = LLMChain(llm=llm, prompt=analyze_prompt, output_key="analysis")
192 
193# Step 3: Generate summary
194summary_prompt = PromptTemplate(
195    input_variables=["entities", "analysis"],
196    template="Summarize:\nEntities: {entities}\nAnalysis: {analysis}\n\nSummary:"
197)
198summary_chain = LLMChain(llm=llm, prompt=summary_prompt, output_key="summary")
199 
200# Combine into sequential chain
201overall_chain = SequentialChain(
202    chains=[extract_chain, analyze_chain, summary_chain],
203    input_variables=["text"],
204    output_variables=["entities", "analysis", "summary"],
205    verbose=True
206)
207```
208 
209## Memory Management Best Practices
210 
211### Choosing the Right Memory Type
212```python
213# For short conversations (< 10 messages)
214from langchain.memory import ConversationBufferMemory
215memory = ConversationBufferMemory()
216 
217# For long conversations (summarize old messages)
218from langchain.memory import ConversationSummaryMemory
219memory = ConversationSummaryMemory(llm=llm)
220 
221# For sliding window (last N messages)
222from langchain.memory import ConversationBufferWindowMemory
223memory = ConversationBufferWindowMemory(k=5)
224 
225# For entity tracking
226from langchain.memory import ConversationEntityMemory
227memory = ConversationEntityMemory(llm=llm)
228 
229# For semantic retrieval of relevant history
230from langchain.memory import VectorStoreRetrieverMemory
231memory = VectorStoreRetrieverMemory(retriever=retriever)
232```
233 
234## Callback System
235 
236### Custom Callback Handler
237```python
238from langchain.callbacks.base import BaseCallbackHandler
239 
240class CustomCallbackHandler(BaseCallbackHandler):
241    def on_llm_start(self, serialized, prompts, **kwargs):
242        print(f"LLM started with prompts: {prompts}")
243 
244    def on_llm_end(self, response, **kwargs):
245        print(f"LLM ended with response: {response}")
246 
247    def on_llm_error(self, error, **kwargs):
248        print(f"LLM error: {error}")
249 
250    def on_chain_start(self, serialized, inputs, **kwargs):
251        print(f"Chain started with inputs: {inputs}")
252 
253    def on_agent_action(self, action, **kwargs):
254        print(f"Agent taking action: {action}")
255 
256# Use callback
257agent.run("query", callbacks=[CustomCallbackHandler()])
258```
259 
260## Testing Strategies
261 
262```python
263import pytest
264from unittest.mock import Mock
265 
266def test_agent_tool_selection():
267    # Mock LLM to return specific tool selection
268    mock_llm = Mock()
269    mock_llm.predict.return_value = "Action: search_database\nAction Input: test query"
270 
271    agent = initialize_agent(tools, mock_llm, agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION)
272 
273    result = agent.run("test query")
274 
275    # Verify correct tool was selected
276    assert "search_database" in str(mock_llm.predict.call_args)
277 
278def test_memory_persistence():
279    memory = ConversationBufferMemory()
280 
281    memory.save_context({"input": "Hi"}, {"output": "Hello!"})
282 
283    assert "Hi" in memory.load_memory_variables({})['history']
284    assert "Hello!" in memory.load_memory_variables({})['history']
285```
286 
287## Performance Optimization
288 
289### 1. Caching
290```python
291from langchain.cache import InMemoryCache
292import langchain
293 
294langchain.llm_cache = InMemoryCache()
295```
296 
297### 2. Batch Processing
298```python
299# Process multiple documents in parallel
300from langchain.document_loaders import DirectoryLoader
301from concurrent.futures import ThreadPoolExecutor
302 
303loader = DirectoryLoader('./docs')
304docs = loader.load()
305 
306def process_doc(doc):
307    return text_splitter.split_documents([doc])
308 
309with ThreadPoolExecutor(max_workers=4) as executor:
310    split_docs = list(executor.map(process_doc, docs))
311```
312 
313### 3. Streaming Responses
314```python
315from langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler
316 
317llm = OpenAI(streaming=True, callbacks=[StreamingStdOutCallbackHandler()])
318```
319 
320## Resources
321 
322- **references/agents.md**: Deep dive on agent architectures
323- **references/memory.md**: Memory system patterns
324- **references/chains.md**: Chain composition strategies
325- **references/document-processing.md**: Document loading and indexing
326- **references/callbacks.md**: Monitoring and observability
327- **assets/agent-template.py**: Production-ready agent template
328- **assets/memory-config.yaml**: Memory configuration examples
329- **assets/chain-example.py**: Complex chain examples
330 
331## Common Pitfalls
332 
3331. **Memory Overflow**: Not managing conversation history length
3342. **Tool Selection Errors**: Poor tool descriptions confuse agents
3353. **Context Window Exceeded**: Exceeding LLM token limits
3364. **No Error Handling**: Not catching and handling agent failures
3375. **Inefficient Retrieval**: Not optimizing vector store queries
338 
339## Production Checklist
340 
341- [ ] Implement proper error handling
342- [ ] Add request/response logging
343- [ ] Monitor token usage and costs
344- [ ] Set timeout limits for agent execution
345- [ ] Implement rate limiting
346- [ ] Add input validation
347- [ ] Test with edge cases
348- [ ] Set up observability (callbacks)
349- [ ] Implement fallback strategies
350- [ ] Version control prompts and configurations
351