Skip to content

docs: correct claims the code does not back - #78

Merged
r0h1tb merged 6 commits into
mainfrom
docs/truth-audit
Oct 6, 2026
Merged

r0h1tb merged 6 commits into
mainfrom
docs/truth-audit

Conversation

@r0h1tb

@r0h1tb r0h1tb commented Aug 24, 2026 •

Copy link
Copy Markdown
Collaborator

Description

Audited README, AGENTS.md, CONTRIBUTING.md and docs/ against the current
source. Every change below was checked by running the thing it describes.

Commands and flags that don't exist as documented

  • README's CLI list had 11 of 20 commands. create-database, call-graph,
    symbol-impact, blocks, lambdas, summarize, analyze-stacktrace,
    cache-stats and stats were undocumented.
  • ast-rag update <path> → Missing option '--from-commit'. Both
    --from-commit and --to-commit are required.
  • ast-rag sandbox <lang> <cmd> → FileNotFoundError: workdir does not exist.
    The second positional is workdir; the command goes in --cmd.
  • QUICKSTART used --from/--to and a --current-branch that doesn't exist.
  • query --verbose and sig --kind → No such option.
  • QUICKSTART used callers to "find what it calls"; that's
    call-graph --direction callees.
  • AGENTS.md described workspace as "Update from git diff"; it shows
    uncommitted changes.

Python API examples that raise

  • get_diff(from_commit=, to_commit=, file_path=) — real signature is
    get_diff(repo_path, from_commit, to_commit=None, limit, offset), no
    per-file filter.
  • full_index(driver, path, commit=) — real is
    full_index(driver, nodes, edges, commit_hash=...) -> None.
  • update_from_git(root=, from_commit=, to_commit=) — real kwargs are
    repo_path, old_commit, new_commit.
  • index_directory, ast_rag.graph_schema, ast_rag.embeddings and
    ast_rag.benchmarks.evaluator do not exist.
  • find_references returns a dict of dicts, not objects with .node.
  • ASTNode.node_type is kind.
  • ProjectConfig.model_validate_json("ast_rag_config.json") takes JSON text,
    not a path.

Config

  • Bare NEO4J_URI / QDRANT_URL / EMBEDDING_URL are read nowhere. The real
    set is the six AST_RAG_* vars in cli.py:132-137; there is no
    embedding-URL override.
  • The example set remote_url with no dimension, which raises
    EmbeddingConfig.dimension must be set when using remote_url. Embeddings are
    local by default and docker-compose.yml starts no embedding server.
  • model_name default is BAAI/bge-m3, not bge-m3.
  • "Tune thresholds in embeddings.py" — no such module. The real knobs are
    vector_weight/keyword_weight.
  • Nothing documented the [mcp] extra, without which ast-rag-mcp and
    ast-rag-watch are on PATH but fail to import.

Shell scripts that never existed

./scripts/index-remaining.sh and index-sequential.sh are referenced across
four docs. git log --all -- 'scripts/*.sh' is empty and scripts/ contains
only README.md.

Graph schema

Both ID formulas were wrong — the node ID includes project_id and the edge ID
includes dep_kind:raw_type_string:confidence. Verified by computing both
hashes. Added the missing Block label and TYPES/DEPENDS_ON/
CROSS_FILE_CALL edges, and noted that HAS_PARAMETER, VIRTUAL_CALL and
LAMBDA_CALL are declared and indexed but never emitted.

Roadmap drift

summarize shipped but was listed as planned; Go was still under "More
languages" in README and CONTRIBUTING after landing in #17.

Related Issue

None — a sweep. #66 was already fixed and closed; only its roadmap half remained.

Type of Change

  • Bug fix
  • New feature
  • Breaking change
  • Documentation update

Checklist

  • My code follows the code style of this project
  • I have added tests — documentation only
  • All new and existing tests passed (unchanged: no source touched)
  • I have updated the documentation accordingly
  • I have run ast-rag evaluate --all — needs live services
  • My changes generate no new warnings

Testing

No source changes. Commands and flags checked against ast-rag <cmd> --help,
API signatures and imports against the source, config fields against
dto/config.py and the EmbeddingManager guard, ID formulas by computing the
hashes both ways.

Not re-measured: the Phase-2 numbers themselves need live Neo4j and Qdrant.
git log -S "F1 Score" -- README.md dates them to 1852590, 2026-02-26, so
they are dated and marked as not re-measured rather than restated as current.
Also left alone with reasons: QUICKSTART's indexing-time estimates, the
roadmap rows with no evidence either way, and the SUMMARIZATION model
recommendations.

Three things left for you

  • scripts/README.md documents both nonexistent shell scripts in detail, and
    README links it as "Indexing utilities". Needs deleting or rewriting — out of
    scope here.
  • docs/STACK_TRACE_MAPPING.md is in Russian while AGENTS.md says
    documentation is English. A policy call, not a false claim; only its imports
    were fixed.
  • AGENTS.md line 8 has a paste artifact: .qweAlways confirm documentation language preference.... Left as-is.

r0h1tb added 4 commits August 25, 2026 02:58
Audited README, AGENTS.md and docs/ against the current source.

- The CLI list was missing nine commands and gave `update` and `sandbox`
  signatures that do not parse.
- `docs/python-api.md` documented `get_diff(from_commit=, to_commit=,
  file_path=)`; `repo_path` is required and there is no per-file filter, so
  the documented call raises TypeError.
- The example config used `"bge-m3"` and an unexplained `remote_url`. The
  default is `BAAI/bge-m3` and embeddings are local; `remote_url` additionally
  requires `dimension`, which EmbeddingManager raises about.
- Nothing said the `ast-rag-mcp` and `ast-rag-watch` entry points need the
  `[mcp]` extra.
- The Phase-2 quality table read as current. Dated it, and said what the
  benchmarks measure and how to re-run them.
- `summarize` shipped but was still listed as planned; Go was still listed
  under "More Languages" after landing.
… source

extract_nodes returns the same 31 nodes for ast_rag_api.py with and without
source, so the note was wrong.
… weights

There is no ast_rag/embeddings.py. The knobs are embedding.vector_weight /
keyword_weight, exposed as --vector-weight / --keyword-weight on query.
@lexasub

lexasub commented Aug 25, 2026

Copy link
Copy Markdown
Owner

docs/STACK_TRACE_MAPPING.md - it's not a big deal that it's in English. If it's a nuisance, you can translate it into English. typo in agents.md, it's best to fix it.

@lexasub lexasub self-assigned this Aug 25, 2026
@r0h1tb

r0h1tb commented Oct 5, 2026

Copy link
Copy Markdown
Collaborator Author

Sorry for the slow reply. Fixed the typo in AGENTS.md (the stray .qwe in the intro).

Also translated STACK_TRACE_MAPPING.md to English, since AGENTS.md asks for English docs anyway. It's a separate commit, so it's easy to drop if you'd rather keep the Russian version.

@lexasub

lexasub commented Oct 5, 2026

Copy link
Copy Markdown
Owner

good commits, in current time i lost interest to this project) need try business in real life)

@r0h1tb
r0h1tb merged commit dbf64d4 into main Oct 6, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

2 participants