Convert any source code repository into a searchable knowledge base with automatic chunking, embedding generation, and intelligent search capabilities. Now with MCP (Model Context Protocol) support for Claude Code and Cursor integration! 📦 Now available on npm! Install with: npm install -g @vezlo/src-to-kb After installation, you'll have access to these commands: - src-to-kb - Generate knowledge
Add this skill
npx mdskills install vezlo/src-to-kbComprehensive documentation for multi-language codebase indexing with AI search, MCP integration, and external server support
1# Source Code to Knowledge Base Generator with MCP Server23[](https://www.npmjs.com/package/@vezlo/src-to-kb)4[](https://www.gnu.org/licenses/agpl-3.0)56Convert any source code repository into a searchable knowledge base with automatic chunking, embedding generation, and intelligent search capabilities. Now with MCP (Model Context Protocol) support for Claude Code and Cursor integration!78📦 **Now available on npm!** Install with: `npm install -g @vezlo/src-to-kb`910## Available Commands1112After installation, you'll have access to these commands:1314- **`src-to-kb`** - Generate knowledge base from source code15- **`src-to-kb-search`** - Search the knowledge base16- **`src-to-kb-upload`** - Upload local knowledge base to external server17- **`src-to-kb-api`** - Start REST API server with Swagger docs18- **`src-to-kb-mcp`** - Start MCP server for IDE integration19- **`src-to-kb-mcp-install`** - Auto-configure Claude Code/Cursor2021## Features2223- 📁 **Multi-language Support**: JavaScript, TypeScript, Python, Java, C++, Go, Rust, and more24- 📝 **Notion Integration**: Import pages and databases directly from Notion (NEW!)25- 🎯 **Answer Modes**: Three modes for different users - End User (simple), Developer (technical), Copilot (code-focused)26- 🌐 **REST API**: Full-featured API with Swagger documentation for integration with external services27- 🔍 **Smart Chunking**: Intelligent code splitting with configurable overlap28- 🧹 **Code Cleaning**: Optional comment removal and whitespace normalization29- 🔢 **Embeddings**: Optional OpenAI embeddings for semantic search30- 📊 **Statistics**: Comprehensive analysis of your codebase31- 🚀 **Fast Processing**: Efficient file scanning and processing32- 💾 **Structured Storage**: Organized JSON output for easy integration33- 🤖 **MCP Server**: Direct integration with Claude Code, Cursor, and other MCP-compatible tools34- 💡 **AI-Powered Search**: Uses OpenAI GPT-5 (latest reasoning model) for intelligent query understanding and helpful answers35- 🔐 **API Authentication**: Optional API key authentication for secure access36- 🌐 **External Server Integration**: Send code to external servers for processing and search via REST API3738## Quick Start3940### 🚀 For Existing Projects (Next.js, React, etc.)4142```bash43# Install globally44npm install -g @vezlo/src-to-kb4546# Generate KB from your project47src-to-kb ./my-nextjs-app --output ./my-kb4849# Start API server50src-to-kb-api5152# Search your codebase53src-to-kb-search search "How does routing work?" --mode developer54```5556**That's it!** Your codebase is now searchable with AI assistance.5758## Processing Modes & Flags5960The following table shows how flags work across different sources and modes:6162| Source | Mode | Default (no flags) | `--chunks-only` | `--with-embeddings` |63|--------|------|-------------------|-----------------|---------------------|64| **Codebase** | Local | Creates chunks | Same (redundant) | Creates chunks + embeddings |65| **Codebase** | Server | Sends raw content | Creates chunks → sends | Creates chunks + embeddings → sends |66| **Notion** | Local | Creates chunks | Same (redundant) | Creates chunks + embeddings |67| **Notion** | Server | Sends raw content | Creates chunks → sends | Creates chunks + embeddings → sends |6869**Note**: Server mode is enabled when `EXTERNAL_KB_URL` environment variable is set.7071## External Server Integration 🌐7273### 🚀 Try It Now with Public Demo Server7475Experience external server integration immediately with our production-ready assistant-server:7677```bash78# Default: Send raw content (server creates chunks)79EXTERNAL_KB_URL=https://your-assistant-server.com/api/knowledge/items src-to-kb ./your-repo8081# Create chunks locally, then send to server82EXTERNAL_KB_URL=https://your-assistant-server.com/api/knowledge/items src-to-kb ./your-repo --chunks-only8384# Create chunks + embeddings locally, then send to server85EXTERNAL_KB_URL=https://your-assistant-server.com/api/knowledge/items OPENAI_API_KEY=sk-... src-to-kb ./your-repo --with-embeddings8687# With API key authentication88EXTERNAL_KB_URL=https://your-assistant-server.com/api/knowledge/items EXTERNAL_KB_API_KEY=your-api-key src-to-kb ./your-repo8990# Search using assistant-server91EXTERNAL_KB_URL=https://your-assistant-server.com/api/knowledge/items \92EXTERNAL_KB_SEARCH_URL=https://your-assistant-server.com/api/search \93src-to-kb-search search "how does authentication work?"94```9596**Assistant Server**: [vezlo/assistant-server](https://github.com/vezlo/assistant-server) - Production-ready Node.js/TypeScript API server with vector search and Docker deployment9798### 📤 Upload Local Knowledge Base99100Upload an existing local knowledge base to an external server:101102```bash103# Default: Upload raw content (reconstructs from chunks)104EXTERNAL_KB_URL=https://your-assistant-server.com/api/knowledge/items \105src-to-kb-upload --kb ./knowledge-base106107# Upload chunks only (server generates embeddings)108EXTERNAL_KB_URL=https://your-assistant-server.com/api/knowledge/items \109src-to-kb-upload --kb ./knowledge-base --chunks-only110111# Upload chunks with embeddings112EXTERNAL_KB_URL=https://your-assistant-server.com/api/knowledge/items \113src-to-kb-upload --kb ./knowledge-base --with-embeddings114```115116**Requirements:**117- `EXTERNAL_KB_URL` environment variable must be set118- Knowledge base must have `documents/` and `chunks/` directories119- For `--with-embeddings`: `embeddings/` directory must exist120121### 🏢 Enterprise Setup122123For production deployments or custom servers:124125📖 **Complete Guide**: [External Server Setup Guide](https://github.com/vezlo/src-to-kb/blob/main/docs/EXTERNAL_SERVER_ENV.md)126127## Notion Integration 📝128129**NEW!** Import your Notion pages and databases directly into your knowledge base! Perfect for combining documentation, project plans, and code knowledge in one searchable system.130131### Quick Start with Notion132133```bash134# 1. Get your Notion API key (see full guide below)135export NOTION_API_KEY=secret_xxx136137# 2. Generate KB from a Notion page138src-to-kb --source=notion --notion-url=https://notion.so/Your-Page-abc123139140# 3. Or fetch all pages from a Notion database141src-to-kb --source=notion --notion-url=https://notion.so/Database-xyz789142143# 4. Search your Notion content144src-to-kb-search search "your query" --kb ./knowledge-base/notion145```146147### With External Server148149Send Notion content directly to your assistant-server:150151```bash152# Set external server URL153export EXTERNAL_KB_URL=http://localhost:3002/api/knowledge/items154export EXTERNAL_KB_API_KEY=your-api-key155156# Send raw content from Notion to server (default)157src-to-kb --source=notion --notion-url=https://notion.so/Your-Page-abc123158159# Send chunks from Notion to server160src-to-kb --source=notion --notion-url=https://notion.so/Your-Page-abc123 --chunks-only161162# Send chunks + embeddings from Notion to server163OPENAI_API_KEY=sk-... src-to-kb --source=notion --notion-url=https://notion.so/Your-Page-abc123 --with-embeddings164165# Search via external server166export EXTERNAL_KB_URL=http://localhost:3002/api/knowledge/items167export EXTERNAL_KB_SEARCH_URL=http://localhost:3002/api/search168src-to-kb-search search "your query"169```170171### Features172173- ✅ **Auto-detection**: Automatically detects if URL is a page or database174- ✅ **Single Page**: Fetch individual Notion pages175- ✅ **Database Support**: Fetch all pages from a Notion database176- ✅ **Rich Content**: Preserves formatting, headings, lists, code blocks, and more177- ✅ **Separate KB**: Notion content saved to `./knowledge-base/notion` by default178- ✅ **External Server**: Send directly to assistant-server for production use179180### Examples181182```bash183# Single page (local KB)184src-to-kb --source=notion --notion-url=https://notion.so/Project-Docs-abc123185186# Database with all pages (local KB)187src-to-kb --source=notion --notion-url=https://notion.so/Team-Wiki-xyz789188189# With API key as parameter190src-to-kb --source=notion --notion-key=secret_xxx --notion-url=https://notion.so/Page-abc123191192# Send to external server (raw content by default)193EXTERNAL_KB_URL=http://localhost:3002/api/knowledge/items \194EXTERNAL_KB_API_KEY=your-key \195src-to-kb --source=notion --notion-url=https://notion.so/Page-abc123196197# Search local Notion KB198src-to-kb-search search "project timeline" --kb ./knowledge-base/notion199200# Search via external server201EXTERNAL_KB_URL=http://localhost:3002/api/knowledge/items \202EXTERNAL_KB_SEARCH_URL=http://localhost:3002/api/search \203src-to-kb-search search "project timeline"204```205206📖 **Complete Notion Guide**: [Notion Integration Documentation](https://github.com/vezlo/src-to-kb/blob/main/docs/NOTION_INTEGRATION.md) - Includes setup instructions, API key creation, sharing pages/databases, and troubleshooting207208### 1. Basic Usage209210Process your repository with default settings:211212```bash213# If installed globally via npm214src-to-kb /path/to/your/repo215216# Or using the script directly217node kb-generator.js /path/to/your/repo218```219220### 2. With Custom Output Directory221222```bash223src-to-kb /path/to/your/repo --output ./my-knowledge-base224```225226### 3. With OpenAI Embeddings227228```bash229# Set your OpenAI API key230export OPENAI_API_KEY=your-api-key-here231232# Generate with embeddings233src-to-kb /path/to/your/repo --with-embeddings234```235236### 4. Search with Answer Modes237238Choose the right answer mode for your needs:239240```bash241# First generate a knowledge base242src-to-kb ./your-project --output ./project-kb243244# Search with different modes:245246# End User Mode - Simple, non-technical answers247src-to-kb-search search "how do I reset password?" --kb ./project-kb --mode enduser248249# Developer Mode - Technical details and architecture (default)250src-to-kb-search search "authentication flow" --kb ./project-kb --mode developer251252# Copilot Mode - Code examples and implementation patterns253src-to-kb-search search "implement user login" --kb ./project-kb --mode copilot254255# View available modes256src-to-kb-search modes257```258259## Answer Modes 🎯260261The search tool adapts its responses based on who's asking:262263| Mode | For | Description | Example Use Case |264|------|-----|-------------|------------------|265| **`enduser`** | Non-technical users | Simple explanations without technical jargon, focuses on features and capabilities | Product managers, business stakeholders asking about features |266| **`developer`** | Software developers | Full technical details including architecture, dependencies, and implementation details | Engineers understanding codebase structure and design patterns |267| **`copilot`** | Coding assistance | Code examples, snippets, and implementation patterns ready to use | Developers looking for code to copy/adapt for their implementation |268269### Mode Examples270271```bash272# CEO asks: "What payment methods do we support?"273src-to-kb-search search "payment methods" --mode enduser274# Returns: Simple list of supported payment options275276# Developer asks: "How is payment processing implemented?"277src-to-kb-search search "payment processing" --mode developer278# Returns: Technical details about payment gateway integration, API endpoints, error handling279280# Developer needs: "Show me payment integration code"281src-to-kb-search search "payment integration" --mode copilot282# Returns: Actual code snippets for payment implementation283```284285### How Modes Work286287- **Filtering**: Each mode filters results differently (e.g., end users don't see test files)288- **AI Prompts**: Custom prompts guide AI to give appropriate responses289- **Formatting**: Answers are formatted based on the audience (code blocks for developers, plain text for end users)290- **Context**: Technical depth is adjusted (high for developers, low for end users)291292## Installation293294### Option 1: Install from npm (Recommended) ✅295296```bash297# Install globally from npm registry298npm install -g @vezlo/src-to-kb299300# Now use the commands anywhere on your system301src-to-kb /path/to/repo # Generate knowledge base302src-to-kb-search search "your query" # Search knowledge base303src-to-kb-mcp # Start MCP server for Claude/Cursor304```305306### Option 2: Use with npx (No Installation)307308```bash309# Run directly without installing310npx @vezlo/src-to-kb /path/to/repo311npx @vezlo/src-to-kb-search search "your query"312npx @vezlo/src-to-kb-mcp313```314315### Option 3: Install in a Project316317```bash318# Add as a project dependency319npm install @vezlo/src-to-kb320321# Use with npx in your project322npx src-to-kb /path/to/repo323```324325### Option 4: Clone from GitHub (For Development)326327```bash328# Clone the repository329git clone https://github.com/vezlo/src-to-kb.git330cd src-to-kb331332# Install dependencies333npm install334335# Run directly336node kb-generator.js /path/to/repo337```338339## CLI Options340341```342Usage: node kb-generator.js <repository-path> [options]343 node kb-generator.js --source=notion [notion-options] [options]344345Options:346 --output, -o Output directory (default: ./knowledge-base)347 --chunk-size Chunk size in characters (default: 1000)348 --chunk-overlap Overlap between chunks (default: 200)349 --max-file-size Maximum file size in MB (default: 10)350 --chunks-only Create chunks locally (and send to server if enabled)351 --with-embeddings Create chunks with embeddings (requires OPENAI_API_KEY)352 --no-comments Exclude comments from code353 --exclude Additional paths to exclude (comma-separated)354 --extensions File extensions to include (comma-separated)355356Notion Options (use with --source=notion):357 --source Source type: code (default) or notion358 --notion-key Notion API integration token (or set NOTION_API_KEY env var)359 --notion-url Notion page or database URL (auto-detects type)360```361362## Complete Example Workflow363364```bash365# 1. Generate knowledge base from your frontend code366src-to-kb ./frontend/ --output ./frontend-kb367368# 2. Different users asking different questions:369370# Product Manager asks about features371src-to-kb-search search "password reset feature" --kb ./frontend-kb --mode enduser372373# Developer investigates technical implementation374src-to-kb-search search "authentication flow" --kb ./frontend-kb --mode developer375376# Developer needs code examples377src-to-kb-search search "login component implementation" --kb ./frontend-kb --mode copilot378379# 3. Get statistics about the codebase380src-to-kb-search stats --kb ./frontend-kb381382# 4. List all TypeScript files383src-to-kb-search type TypeScript --kb ./frontend-kb384385# 5. View available answer modes386src-to-kb-search modes387```388389## More Examples390391### Process Any Repository392393```bash394# Using npm package395src-to-kb /path/to/repo --output ./repo-kb --with-embeddings396397# Or with npx398npx @vezlo/src-to-kb /path/to/repo --output ./repo-kb --with-embeddings399```400401### Process Only JavaScript and TypeScript Files402403```bash404src-to-kb /path/to/repo --extensions .js,.ts,.jsx,.tsx405```406407### Exclude Test and Build Directories408409```bash410src-to-kb /path/to/repo --exclude tests,build,dist,coverage411```412413### Large Repositories with Custom Chunking414415```bash416src-to-kb /path/to/large-repo \417 --chunk-size 2000 \418 --chunk-overlap 400 \419 --max-file-size 20420```421422## Testing423424Run the included test suite to verify functionality:425426```bash427# Run comprehensive tests428node test.js429430# This will:431# 1. Create a test repository with sample files432# 2. Process it into a knowledge base433# 3. Verify the output structure434# 4. Test chunking on large files435# 5. Verify language detection436```437438## REST API Server439440The Source-to-KB REST API provides programmatic access to all functionality with comprehensive Swagger documentation.441442### Starting the API Server443444```bash445# Start with defaults (port 3000, no authentication)446src-to-kb-api447448# With custom port and API key449PORT=8080 API_KEY=your-secret-key src-to-kb-api450451# With all options452PORT=8080 API_KEY=secret OPENAI_API_KEY=sk-... src-to-kb-api453```454455### API Documentation456457Once started, visit: `http://localhost:3000/api/v1/docs` for interactive Swagger UI458459### Key Endpoints460461- `POST /api/v1/knowledge-bases` - Create new knowledge base462- `POST /api/v1/search` - Search with mode selection463- `GET /api/v1/modes` - List available answer modes464- `GET /api/v1/statistics/{id}` - Get KB statistics465- `POST /api/v1/process-file` - Process single file466467### Example API Usage468469```javascript470// Create knowledge base471const response = await fetch('http://localhost:3000/api/v1/knowledge-bases', {472 method: 'POST',473 headers: {474 'Content-Type': 'application/json',475 'X-API-Key': 'your-api-key'476 },477 body: JSON.stringify({478 name: 'My Project',479 sourcePath: '/path/to/project',480 options: { chunkSize: 1500 }481 })482});483484// Search with mode485const searchResponse = await fetch('http://localhost:3000/api/v1/search', {486 method: 'POST',487 headers: {488 'Content-Type': 'application/json',489 'X-API-Key': 'your-api-key'490 },491 body: JSON.stringify({492 query: 'authentication',493 knowledgeBaseId: 'abc123',494 mode: 'developer'495 })496});497```498499For complete API documentation, see [API_DOCUMENTATION.md](https://github.com/vezlo/src-to-kb/blob/main/docs/API_DOCUMENTATION.md)500501## MCP Server for Claude Code502503### Quick Setup 🚀504505```bash506# 1. Install the package globally507npm install -g @vezlo/src-to-kb508509# 2. Find your global npm installation path510npm list -g @vezlo/src-to-kb --depth=0511512# 3. Add to Claude Code (replace the path with your npm global path)513# For macOS/Linux with nvm:514claude mcp add src-to-kb -- node ~/.nvm/versions/node/v22.6.0/lib/node_modules/@vezlo/src-to-kb/mcp-server.mjs515516# For macOS/Linux without nvm:517claude mcp add src-to-kb -- node /usr/local/lib/node_modules/@vezlo/src-to-kb/mcp-server.mjs518519# For Windows:520claude mcp add src-to-kb -- node %APPDATA%\npm\node_modules\@vezlo\src-to-kb\mcp-server.mjs521522# With OpenAI API key for embeddings:523claude mcp add src-to-kb --env OPENAI_API_KEY=your-key -- node [your-path]/mcp-server.mjs524```525526#### Alternative: Using npx (if the above doesn't work)527528```bash529# Try with npx (may not work on all systems)530claude mcp add src-to-kb -- npx -y @vezlo/src-to-kb src-to-kb-mcp531```532533### Managing the MCP Server534535```bash536# Check if installed537claude mcp list538539# Remove if needed540claude mcp remove src-to-kb541542# Get server details543claude mcp get src-to-kb544```545546### After Installation5475481. **Restart Claude Code** completely5492. **Test by asking Claude:**550 - "Generate a knowledge base for this project"551 - "Search for authentication implementations"552 - "What languages does this codebase use?"553 - "Find files similar to config.js"554555See [MCP_SETUP.md](https://github.com/vezlo/src-to-kb/blob/main/docs/MCP_SETUP.md) for manual setup and [MCP_TOOLS_GUIDE.md](https://github.com/vezlo/src-to-kb/blob/main/docs/MCP_TOOLS_GUIDE.md) for detailed tool documentation.556557## Searching the Knowledge Base558559### Answer Modes560561The search tool supports three different answer modes to tailor responses based on your needs:562563- **`enduser`**: Simplified answers for non-technical users, focusing on features and capabilities564- **`developer`**: Detailed technical answers including architecture and implementation details (default)565- **`copilot`**: Code-focused answers with examples and patterns for implementation566567```bash568# Examples with different modes569src-to-kb-search search "how to use API?" --mode enduser # Simple explanation570src-to-kb-search search "authentication flow" --mode developer # Technical details571src-to-kb-search search "login implementation" --mode copilot # Code examples572573# List available modes574src-to-kb-search modes575```576577### AI-Powered Search (with OpenAI)578579When `OPENAI_API_KEY` is set, searches use GPT-5 (OpenAI's latest reasoning model) for intelligent answers:580581```bash582# Set your OpenAI API key583export OPENAI_API_KEY=your-api-key-here584585# Get intelligent, context-aware answers with mode selection586src-to-kb-search search "how does authentication work?" --kb ./project-kb --mode developer587src-to-kb-search search "where is password reset?" --kb ./project-kb --mode enduser588```589590### Basic Search (without OpenAI)591592Without an API key, the tool provides basic keyword search:593594```bash595# Basic search with pattern matching596src-to-kb-search search "authentication" --kb ./project-kb597598# Find all JavaScript files599src-to-kb-search type JavaScript --kb ./project-kb600601# Show statistics602src-to-kb-search stats --kb ./project-kb603604# Find similar files605src-to-kb-search similar src/index.js --kb ./project-kb606```607608### Search Options609610```bash611# Specify knowledge base path612src-to-kb-search search "query" --kb ./my-knowledge-base613614# Select answer mode615src-to-kb-search search "query" --mode enduser|developer|copilot616617# Show detailed evidence618src-to-kb-search search "query" --verbose619620# Get raw search results (old format)621src-to-kb-search search "query" --raw622```623624## Output Structure625626The generator creates the following directory structure:627628```629knowledge-base/630├── documents/ # Document metadata (without content)631│ ├── doc_xxx.json632│ └── ...633├── chunks/ # Document chunks for searching634│ ├── doc_xxx.json635│ └── ...636├── embeddings/ # OpenAI embeddings (if enabled)637│ ├── doc_xxx.json638│ └── ...639└── metadata/ # Summary and statistics640 └── summary.json641```642643### Document Format644645Each document contains:646647```json648{649 "id": "doc_1234567890_abc123",650 "path": "/full/path/to/file.js",651 "relativePath": "src/file.js",652 "fileName": "file.js",653 "extension": ".js",654 "size": 2048,655 "checksum": "sha256-hash",656 "metadata": {657 "createdAt": "2024-01-01T00:00:00.000Z",658 "modifiedAt": "2024-01-01T00:00:00.000Z",659 "lines": 100,660 "language": "JavaScript",661 "type": "code"662 }663}664```665666### Chunk Format667668Each chunk contains:669670```json671{672 "id": "doc_xxx_chunk_0",673 "index": 0,674 "content": "chunk content here...",675 "startLine": 1,676 "endLine": 25,677 "size": 1000678}679```680681## 🔧 Integration with Existing Projects682683### Next.js / React Integration684685Transform your frontend codebase into a searchable knowledge base with AI-powered assistance:686687#### Quick Setup688689```bash690# 1. Generate knowledge base from your project691src-to-kb /path/to/nextjs-app --output ./nextjs-kb692693# 2. Start the API server694src-to-kb-api695696# 3. Query your codebase697curl -X POST http://localhost:3000/api/v1/search \698 -H "Content-Type: application/json" \699 -d '{"query": "How is authentication implemented?", "knowledgeBaseId": "your-kb-id", "mode": "developer"}'700```701702#### React Component Example703704```jsx705// components/CodeSearch.jsx706import { useState } from 'react';707708export default function CodeSearch() {709 const [query, setQuery] = useState('');710 const [result, setResult] = useState(null);711712 const search = async () => {713 const response = await fetch('http://localhost:3000/api/v1/search', {714 method: 'POST',715 headers: { 'Content-Type': 'application/json' },716 body: JSON.stringify({717 query,718 knowledgeBaseId: 'your-kb-id',719 mode: 'developer'720 })721 });722 const data = await response.json();723 setResult(data);724 };725726 return (727 <div>728 <input729 value={query}730 onChange={(e) => setQuery(e.target.value)}731 placeholder="Ask about your codebase..."732 />733 <button onClick={search}>Search</button>734 {result && <div>{result.answer}</div>}735 </div>736 );737}738```739740#### Integration Ideas741742- **🎓 Onboarding Assistant**: Help new developers understand your codebase743- **📖 In-App Documentation**: Provide context-aware help within your application744- **🔍 Code Review Helper**: Find similar patterns and best practices745- **🤖 Development Copilot**: Get AI suggestions based on your existing code746- **📊 Code Analytics Dashboard**: Visualize codebase statistics and complexity747748### CI/CD Integration749750```yaml751# GitHub Actions example752name: Update Knowledge Base753on: [push]754jobs:755 update-kb:756 runs-on: ubuntu-latest757 steps:758 - uses: actions/checkout@v2759 - run: npm install -g @vezlo/src-to-kb760 - run: src-to-kb . --output ./kb761 # Upload KB as artifact or deploy to server762```763764### Enterprise Setup765766For production environments:767768```bash769# Start with authentication and custom port770API_KEY=secure-key PORT=8080 src-to-kb-api771772# Use with Docker773docker run -p 3000:3000 -e API_KEY=secret vezlo/src-to-kb-api774```775776## Use Cases7777781. **Code Documentation**: Generate searchable documentation from your codebase7792. **AI Training**: Prepare code for fine-tuning or RAG systems7803. **Code Analysis**: Analyze patterns and structure across large repositories7814. **Knowledge Extraction**: Extract domain knowledge from source code7825. **Code Search**: Build intelligent code search systems7836. **IDE Integration**: Use directly in Claude Code or Cursor for code understanding7847. **Team Knowledge Sharing**: Create searchable knowledge bases for team onboarding785786## Performance787788- Processes ~1000 files/minute on average hardware789- Memory efficient - streams large files790- Parallel chunk processing791- Configurable file size limits792793## Supported Languages794795- JavaScript (.js, .jsx)796- TypeScript (.ts, .tsx)797- Python (.py)798- Java (.java)799- C/C++ (.c, .cpp, .h, .hpp)800- C# (.cs)801- Go (.go)802- Rust (.rs)803- Ruby (.rb)804- PHP (.php)805- Swift (.swift)806- Kotlin (.kt)807- Scala (.scala)808- And many more...809810## Configuration Files811812Also processes:813- JSON (.json)814- YAML (.yaml, .yml)815- XML (.xml)816- Markdown (.md)817- HTML/CSS (.html, .css, .scss)818- SQL (.sql)819820## Tips8218221. **Chunking Strategy**:823 - Use smaller chunks (500-1000) for precise search824 - Use larger chunks (2000-3000) for more context8258262. **Overlap**:827 - 10-20% overlap helps maintain context between chunks828 - Increase overlap for code with many dependencies8298303. **Exclusions**:831 - Always exclude node_modules, vendor, dist directories832 - Consider excluding auto-generated files8338344. **File Size**:835 - Default 10MB limit prevents processing of large binaries836 - Increase for legitimate large source files837838## Programmatic Usage839840```javascript841const { KnowledgeBaseGenerator } = require('./kb-generator');842843async function generateKB() {844 const generator = new KnowledgeBaseGenerator({845 outputPath: './my-kb',846 chunkSize: 1500,847 generateEmbeddings: true,848 openaiApiKey: 'your-api-key'849 });850851 generator.on('fileProcessed', (data) => {852 console.log(`Processed: ${data.file}`);853 });854855 const result = await generator.processRepository('/path/to/repo');856 console.log(`Generated KB with ${result.documents.length} documents`);857}858859generateKB();860```861862## License863864This software is dual-licensed:865866- **Non-Commercial Use**: Free under AGPL-3.0 license867- **Commercial Use**: Requires a commercial license - contact us for details868869See [LICENSE](LICENSE) file for full details.870871## Contributing872873Feel free to submit issues and enhancement requests!
Full transparency — inspect the skill content before installing.