Documentation

Usage

Everything from a first local build to a fully wired MCP client, plus the indexing lifecycle and the knobs you can tune.

01 · Install

Requirements and build

Node.js 20 or newer. No native toolchain is required — every grammar ships as WASM.

From source

git clone https://github.com/nithin554/code-crawler.git
cd code-crawler
npm install
npm run build

node dist/index.js crawl /absolute/path/to/project
node dist/index.js serve

As a dependency

npm install @nithin554/code-crawler

npx codecrawler crawl /absolute/path/to/project
npx codecrawler status
npx codecrawler serve

The published package exposes the codecrawler binary and only ships dist/.

better-sqlite3 is a native module. Prebuilt binaries cover common platforms; if install fails, install a matching C++ build toolchain or use a Node LTS release.

02 · CLI

Four commands

CommandPurposeOutput
codecrawler crawl <path> Index a project. Safe to re-run; changed files are refreshed and deleted files pruned. Summary line on stderr
codecrawler status Node/edge/file counts, last crawl timestamp, indexed project path. JSON on stdout
codecrawler query <symbol> Fuzzy symbol lookup by simple or partial name. JSON array on stdout
codecrawler serve Start the MCP server over stdio. This is the default when no command is given. MCP protocol on stdout, logs on stderr

Because stdout carries the MCP protocol in serve mode, all diagnostics go to stderr. Never print to stdout inside a tool handler.

03 · Connect

Register the MCP server

Code-Crawler speaks stdio MCP. Use an absolute path to dist/index.js so the client can spawn it from any working directory.

Skipped the local build? Every card below has an npx equivalent — see Registry install.

Claude Desktop

Edit claude_desktop_config.json:

{
  "mcpServers": {
    "code-crawler": {
      "command": "node",
      "args": [
        "/absolute/path/to/code-crawler/dist/index.js"
      ],
      "env": {
        "CODECRAWLER_DB": "/absolute/path/index.db"
      }
    }
  }
}

macOS: ~/Library/Application Support/Claude/ · Windows: %APPDATA%\Claude\ · Linux: ~/.config/Claude/

Claude Code

One command registers the server:

claude mcp add code-crawler -- \
  node /absolute/path/to/code-crawler/dist/index.js

Verify with claude mcp list, then run /mcp inside a session.

Cursor

Project .cursor/mcp.json or global ~/.cursor/mcp.json:

{
  "mcpServers": {
    "code-crawler": {
      "command": "node",
      "args": [
        "/absolute/path/to/code-crawler/dist/index.js"
      ]
    }
  }
}

VS Code / Copilot

Workspace .vscode/mcp.json uses a servers map:

{
  "servers": {
    "code-crawler": {
      "type": "stdio",
      "command": "node",
      "args": [
        "/absolute/path/to/code-crawler/dist/index.js"
      ]
    }
  }
}

One database per project is the safest default. If you index several repos, give each its own CODECRAWLER_DB path — otherwise the second crawl replaces the first project's file set.

04 · Registry

Install from the MCP registry

Code-Crawler is published to the official MCP registry as io.github.nithin554/code-crawler and to npm as @nithin554/code-crawler, so registry-aware clients can add it without cloning the repository or building it locally.

FieldValue
Registry name io.github.nithin554/code-crawler
npm package @nithin554/code-crawler
Launch command npx -y @nithin554/code-crawler@latest
Transport stdio — a local child process; no ports, no auth, no hosted service
Runtime Node.js 20 or newer
Environment All optional — see Configuration

GitHub Copilot app

Open Customize → MCP. If Code-Crawler appears in the catalog, select it and follow the prompts; otherwise add a custom server named code-crawler with this command:

npx -y @nithin554/code-crawler@latest

The app stores the entry in ~/.copilot/mcp-config.json and lists it under Customize → Installed. Sessions already running keep their current tool set — the server is spawned for the next session or after you toggle it off and on.

Copilot CLI

One command writes the user-level configuration:

copilot mcp add code-crawler -- \
  npx -y @nithin554/code-crawler@latest

Verify with copilot mcp list, or /mcp inside a session. Add env vars with --env CODECRAWLER_DB=....

VS Code

If the catalog lists it, install from the Extensions view with @mcp code-crawler. Otherwise add the entry to .vscode/mcp.json (workspace) or your user MCP settings:

{
  "servers": {
    "code-crawler": {
      "type": "stdio",
      "command": "npx",
      "args": ["-y", "@nithin554/code-crawler@latest"]
    }
  }
}

Other stdio clients

Any client that accepts a command and args works — no local build and no absolute path to dist/index.js:

{
  "mcpServers": {
    "code-crawler": {
      "command": "npx",
      "args": ["-y", "@nithin554/code-crawler@latest"],
      "env": {
        "CODECRAWLER_DB": "/absolute/path/index.db"
      }
    }
  }
}

Versions and updates

npx resolves the latest dist-tag every time the client spawns the server, reusing npm's download cache when the version is already there. A new npm release therefore takes effect on the next server start — restart the client, open a new session, or toggle the server off and on. Nothing upgrades mid-session, and no version is pushed to you.

Pin an exact version for reproducible setups: "@nithin554/code-crawler@0.1.49". Catalog installs use the version recorded in server.json at publish time, which can lag the npm latest tag while the registry ingests a release. Check what you are running with npm view @nithin554/code-crawler version.

05 · Lifecycle

What a crawl actually does

Discover

Walks the project root, skipping CODECRAWLER_EXCLUDE_DIRS and hidden VCS directories. Optional language and fileFilter arguments narrow the crawl.

Parse

Each file is parsed with the tree-sitter grammar for its language — Java, Python, TypeScript, or JavaScript — to extract packages/modules, imports, types, members, modifiers, decorators, and annotations.

Index and resolve

A global type index is built for the whole project, then per-file call sites are resolved to symbol ids. Unresolved calls become __unknown__ phantom nodes with resolved: false.

Persist

All node and edge writes happen in one transaction. Each file's previous slice is replaced; files that disappeared are removed.

Rebuild derived edges

DYNAMIC_DISPATCH and ADVISES edges are deleted and recomputed from the fresh graph, so re-crawls never duplicate or strand them.

When the graph goes stale

The index is a snapshot. After pulling, switching branches, or editing code, re-run crawl_project (or codecrawler crawl). clear_index wipes the database entirely and forces a clean rebuild.

06 · Configuration

Environment variables

VariableDefaultPurpose
CODECRAWLER_DB ~/.code-crawler/index.db Path to the SQLite graph database. Created on first use.
CODECRAWLER_EXCLUDE_DIRS .git,node_modules,dist,build,target,.idea,.venv,.gradle,out,.next,coverage Comma-separated directory names skipped during crawl.
CODECRAWLER_DEFAULT_LIMIT 50 Default result limit for list-returning queries.
CODECRAWLER_MAX_DEPTH 10 Traversal depth cap for recursive queries.
CODECRAWLER_HELPFUL_ERRORS true Attach fuzzy symbol suggestions to “not found” errors.

Set these in the env block of your MCP client config so the server always opens the intended database. Paths must be absolute for GUI clients, which do not inherit your shell environment.

07 · Prompting

A workflow that gives good answers

The tools are precise but literal. Two cheap steps — index, then resolve the symbol id — make every later answer reliable.

1. Index explicitly

Start a session by asking the assistant to crawl the project at an absolute path. It returns file, node, edge, and call-resolution counts you can sanity check.

crawl_project({ path: "C:/work/shop-service" })

2. Resolve names first

Tool arguments take symbol ids, not guesses. Search once, then reuse the returned qualifiedName.

search_symbols({ query: "place" })

3. Ask one graph question at a time

Prefer a specific tool over a vague prompt. “Who calls X?” maps to get_callers; “what runs?” maps to trace_execution.

get_callers({
  symbol: "com.acme.OrderService#place",
  transitive: true
})

4. Ask for file and line proof

Results carry file and callSiteLine (1-based in output). Ask the assistant to cite them before editing.

08 · Semantics

What the model can and cannot know

Modelled

  • Static CALLS edges with call-site lines and confidence.
  • Uniform graph across Java, Python, TypeScript, and JavaScript.
  • Package/module, type, member, and import containment.
  • IMPLEMENTS / EXTENDS hierarchy edges.
  • Virtual dispatch to project overrides (DYNAMIC_DISPATCH).
  • AspectJ/Spring AOP advice and pointcut matching (ADVISES).
  • Annotations and decorators on types, methods, constructors, and fields.
  • Reflection and run-time lookups as REFLECTS edges: Class.forName, ServiceLoader, getattr, importlib, require, dynamic import().
  • Source-visible dependency injection as INJECTS edges: @Autowired, NestJS constructor injection, FastAPI Depends.
  • Framework entry points and HTTP routes as ROUTES edges: Spring MVC, FastAPI, Flask, NestJS, Express.
  • Control flow per call edge: conditional flag, control-flow kinds, guard text.
  • External packages as ext: nodes with a dependency summary.

Not modelled

  • Value flow and branch feasibility. Edges record that a call is conditional and the guard text — not which branch runs, nor what values reach a parameter.
  • Fully dynamic reflection targets. getattr(obj, name) with an unknown name and require(userInput) point at a __reflect__: marker instead of a symbol. Inventing a target would be worse than the marker.
  • DI containers configured outside code — XML, YAML bean wiring, NestJS dynamic module factories. Only injection visible in source is reported.
  • Libraries outside the crawled tree. Third-party code is summarised as ext: packages and never crawled, so traces stop at the boundary.
  • Branch-level dead code. find_dead_code is whole-method, not per-branch.
  • Cross-language edges. Python does not call into Java even when they share a process boundary; the languages share a graph, not a call edge.
  • Types changed at runtime — monkey-patching, __getattr__ proxies, Object.defineProperty tricks.

Treat find_dead_code as a candidate list, not a verdict. Methods invoked only by frameworks, serialization, or reflection will show up as unused unless the crawler recognises the mechanism. Start with find_entry_points, find_routes, and find_reflections before deleting anything.

09 · Troubleshooting

Common failure modes

SymptomLikely causeFix
Every tool says “not found in the index” The server is using a different database path than the crawl. Pin CODECRAWLER_DB in the client env block.
Low resolved-call count Source layout confuses receiver inference, or the crawl included generated code. Use fileFilter or narrow path; expect heuristic misses on unusual call shapes.
No DYNAMIC_DISPATCH edges No type implements the interface inside the crawled tree. Confirm overrides live in the project; libraries are out of scope.
No aspects listed Aspect types are missing @Aspect or pointcuts use unsupported syntax. Supported primitives: execution, call, within, @annotation, @within, target, this, args, &&, ||, !.
Annotations missing The source uses a language level the grammar does not expose as modifiers. Annotations and decorators are read from each grammar's own modifier, decorator, or attribute nodes; verify with get_node and get_decorators.
Stale symbols after a refactor The index predates the change. Re-crawl; deleted files are pruned automatically.

Still stuck? get_schema prints the live node kinds, edge kinds, id scheme, and dynamic-model semantics the server is using.

Next: see it on real questions.

Impact analysis, runtime traces, aspect auditing, decorator scans, and dead-code triage.

Worked examples →