aibox22/readmeX

The project does not clone repositories — it works exclusively on a local filesystem path provided via CLI arg or interactive input (cli.py:30-34, core.py:267-301). File discovery uses find_files() with glob patterns: SCRIPT_PATTERNS = ['*.py', '*.sh', '*.ipynb'] and DOCUMENT_PATTERNS = ['*.md', '*.txt'] (config.py:235-236). Gitignore patterns are loaded from the project’s .gitignore and merged with a built-in ignore list (DEFAULT_IGNORE_PATTERNS in config.py:211-232) that excludes .git, __pycache__, node_modules, *.md (so markdown files themselves are not re-processed), build/, dist/, virtual envs, IDE dirs, and __init__.py files. A much larger ignore-patterns config lives in config/ignore_patterns.json for the LanguageAnalyzer — hundreds of patterns covering Python, JS, Rust, and more. There is no chunking in the traditional RAG sense: for README generation, each matched file is read in full and its content sent in a prompt to the LLM for a description, using ThreadPoolExecutor (configurable max_workers, default 10) via _get_script_descriptions() (core.py:690-832). For the website path, the CodeRAG.extract_code_blocks() method (code_rag.py:188-217) does AST-level parsing of .py files only — it walks each file’s AST with ast.walk() and extracts FunctionDef, ClassDef, methods, and import statements into individual CodeBlock dataclass instances with type, signature, docstring, complexity score, and dependency lists. No large-repo limits are enforced beyond available memory and thread count.

Retrieval is implemented via the CodeRAG class (code_rag.py:92-948). It has two vectorization paths configured by local_embedding (default true in config.py:197). Local path: loads a SentenceTransformer model, defaulting to Kwaipilot/OASIS-code-embedding-1.5B (code_rag.py:115). Web path: uses the OpenAI embeddings API (text-embedding-3-small) with configurable base_url/api_key (code_rag.py:591-637), sending texts in batches of 100. In either case, embedding text is built from Type, Name, Signature, docstring, code preview (first 500 chars), and dependencies (code_rag.py:526-552). The vectors are stored in a FAISS IndexFlatIP (inner-product similarity) index (code_rag.py:639-649), with L2 normalization applied. semantic_search(query, top_k=5, min_score=0.3) (code_rag.py:651-662) dispatches to either _vector_search() (FAISS or numpy dot-product) or _text_search() (term-matching fallback with weighted keyword scoring). The generate_enhanced_prompt() method (code_rag.py:789-828) wraps retrieved CodeBlock results — type, name, file path, signature, docstring, code snippet (truncated to 300 chars), and dependencies — into a === 相关代码上下文 === section appended to the base page-generation prompt. This is consumed by WebsiteGenerator._generate_page_content() (website_core.py:672-687), which calls code_rag.generate_enhanced_prompt() with a page-specific RAG query (e.g. '项目 {name} 架构 设计 模块结构 类关系 组件' for the architecture page). All code blocks, relations, embeddings, and the FAISS index are cached to pickle and .bin files under .rag_cache/ and loaded from there on subsequent runs (code_rag.py:868-933).

The wiki structure is not LLM-driven — it is hardcoded. WebsiteGenerator._create_mkdocs_config() (website_core.py:2174-2251) produces an mkdocs.yml with a fixed navigation array:

nav:
  - {首页: 'index.md'}
  - {安装: 'installation.md'}
  - {使用说明: 'usage.md'}
  - {API文档: 'api/index.md'}
  - {示例: 'examples.md'}
  - {架构: 'architecture.md'}
  - {贡献指南: 'contributing.md'}
  - {更新日志: 'changelog.md'}

This 8-page structure is always the same regardless of the project. The API docs sub-section expands dynamically because _generate_api_documentation() (website_core.py:347-377) generates one .md file per valuable function/class (filtered by APIDocumentationFilter which skips tests, private symbols, trivial helpers, and low-value files), each saved as api/{module}/{name}.md. The input to the generator is the project_analysis dict built by _analyze_project() (website_core.py:286-327), which contains structure (text tree), functions, classes, modules, dependencies, entry_points, and git_info — all extracted via filesystem traversal and AST parsing. The static nature is deliberate: the progress tracker pre-defines self.stages with those exact 10 stages (website_core.py:38-49) and _generate_pages_in_parallel() (website_core.py:208-255) maps each stage to its generator function explicitly.

Each page type has a dedicated prompt-builder method (website_core.py:1620-2170): _create_installation_prompt, _create_usage_prompt, _create_examples_prompt, _create_architecture_prompt, _create_contributing_prompt, _create_changelog_prompt. These methods receive the project_analysis dict (file tree, AST-extracted functions/classes, dependency lists, git info, commit history) and format it into a detailed markdown prompt. For the home page, _generate_readme_as_homepage() (website_core.py:869-952) reuses the full README-generation pipeline from readmex.core, creating a temporary readmex instance in silent mode. The architecture page generates a draw.io XML diagram via _generate_drawio_diagram() (website_core.py:1409-1515) with LLM retry logic (3 attempts, validation of XML completeness). The changelog page is especially rich: it runs git log --stat to get commit history with file-change counts, groups commits by type (feat/fix/docs/etc.), detects breaking changes, lists contributors, and feeds all of that into the prompt (website_core.py:2036-2171). Pages are generated in parallel using ThreadPoolExecutor (_generate_pages_in_parallel(), website_core.py:208-255) — six page tasks plus the API doc pages (which themselves use ThreadPoolExecutor for individual API pages). When RAG is enabled, the base prompt is augmented with code_rag.generate_enhanced_prompt() (website_core.py:672-687, code_rag.py:789-828) which appends the top-8 semantically related code blocks as context. Caching: CodeRAG caches extracted code_blocks.pkl, relations.pkl, embeddings.pkl, and faiss_index.bin under .rag_cache/ (code_rag.py:868-933). There is no per-page cache — the RAG cache is all-or-nothing. In debug mode, all LLM calls are skipped and placeholder content is returned instead (website_core.py:715-838). No Mermaid diagrams are generated (the project uses draw.io XML for architecture and MkDocs's drawio plugin for rendering).

The single ModelClient class (utils/model_client.py:10-455) handles both LLM and text-to-image (T2I) providers. It supports Standard OpenAI and Azure OpenAI, detected by checking if the base URL contains .openai.azure.com (model_client.py:60-73). The LLM and T2I providers can differ independently — each has its own config block (get_llm_config() and get_t2i_config() in config.py:168-196). Configuration comes from three sources merged by load_config() (config.py:21-98): (1) ~/.readmex/config.json — a JSON file with keys like LLM_API_KEY, LLM_BASE_URL, LLM_MODEL_NAME (default gpt-3.5-turbo), T2I_MODEL_NAME (default dall-e-3); (2) environment variables (LLM_API_KEY, LLM_BASE_URL, etc.) which override the config file; and (3) interactive prompt from validate_config() if required keys are missing (config.py:100-157). For Azure, the _extract_azure_info() method (model_client.py:75-110) parses the Azure endpoint URL to extract azure_endpoint, deployment_name, and api_version, then creates an AzureOpenAI client. For standard OpenAI, it creates OpenAI(base_url=..., api_key=...) — which means any OpenAI-compatible endpoint works (local models via Ollama/vLLM, Doubao, etc.) by pointing base_url at that server. Per-stage model selection: the same ModelClient instance is used for everything — README generation, page content, API docs, architecture diagrams — so there is no per-stage model routing. However, get_answer() accepts an optional model override parameter (model_client.py:188), and _generate_drawio_diagram() controls quality only through retry logic, not model choice. Embeddings use a separate config (EMBEDDING_API_KEY, EMBEDDING_BASE_URL, EMBEDDING_MODEL_NAME), additionally supporting a LOCAL_EMBEDDING boolean to switch between sentence-transformers and the web API (config.py:190-198).

This repository does not implement any interactive Q&A, chat, or conversation features. It is a pure batch documentation generator — it reads a project, generates README.md or an MkDocs website, and exits. There is no streaming, no chat loop, no conversation memory, and no MCP exposure. The code_rag.py module's semantic_search() and generate_enhanced_prompt() are used exclusively to augment LLM prompts during page generation, not for end-user querying. The ModelClient.get_answer() method is a simple one-shot LLM call with retry logic and no conversation history. A typical Q&A feature would require a different architecture entirely (interactive loop, session state, web UI), none of which is present.