Usage
Everything from a first local build to a fully wired MCP client, plus the indexing lifecycle and the knobs you can tune.
Requirements and build
Node.js 20 or newer. No native toolchain is required — every grammar ships as WASM.
From source
git clone https://github.com/nithin554/code-crawler.git
cd code-crawler
npm install
npm run build
node dist/index.js crawl /absolute/path/to/project
node dist/index.js serve
As a dependency
npm install @nithin554/code-crawler
npx codecrawler crawl /absolute/path/to/project
npx codecrawler status
npx codecrawler serve
The published package exposes the codecrawler binary and only
ships dist/.
better-sqlite3 is a native module. Prebuilt binaries cover
common platforms; if install fails, install a matching C++ build toolchain or
use a Node LTS release.
Four commands
| Command | Purpose | Output |
|---|---|---|
codecrawler crawl <path> |
Index a project. Safe to re-run; changed files are refreshed and deleted files pruned. | Summary line on stderr |
codecrawler status |
Node/edge/file counts, last crawl timestamp, indexed project path. | JSON on stdout |
codecrawler query <symbol> |
Fuzzy symbol lookup by simple or partial name. | JSON array on stdout |
codecrawler serve |
Start the MCP server over stdio. This is the default when no command is given. | MCP protocol on stdout, logs on stderr |
Because stdout carries the MCP protocol in serve mode, all
diagnostics go to stderr. Never print to stdout inside a tool handler.
Register the MCP server
Code-Crawler speaks stdio MCP. Use an absolute path to dist/index.js
so the client can spawn it from any working directory.
Skipped the local build? Every card below has an npx equivalent —
see Registry install.
Claude Desktop
Edit claude_desktop_config.json:
{
"mcpServers": {
"code-crawler": {
"command": "node",
"args": [
"/absolute/path/to/code-crawler/dist/index.js"
],
"env": {
"CODECRAWLER_DB": "/absolute/path/index.db"
}
}
}
}
macOS: ~/Library/Application Support/Claude/ ·
Windows: %APPDATA%\Claude\ ·
Linux: ~/.config/Claude/
Claude Code
One command registers the server:
claude mcp add code-crawler -- \
node /absolute/path/to/code-crawler/dist/index.js
Verify with claude mcp list, then run
/mcp inside a session.
Cursor
Project .cursor/mcp.json or global ~/.cursor/mcp.json:
{
"mcpServers": {
"code-crawler": {
"command": "node",
"args": [
"/absolute/path/to/code-crawler/dist/index.js"
]
}
}
}
VS Code / Copilot
Workspace .vscode/mcp.json uses a servers map:
{
"servers": {
"code-crawler": {
"type": "stdio",
"command": "node",
"args": [
"/absolute/path/to/code-crawler/dist/index.js"
]
}
}
}
One database per project is the safest default. If you index several repos,
give each its own CODECRAWLER_DB path — otherwise the second
crawl replaces the first project's file set.
Install from the MCP registry
Code-Crawler is published to the official MCP registry as
io.github.nithin554/code-crawler and to npm as
@nithin554/code-crawler, so registry-aware clients can add it
without cloning the repository or building it locally.
| Field | Value |
|---|---|
| Registry name | io.github.nithin554/code-crawler |
| npm package | @nithin554/code-crawler |
| Launch command | npx -y @nithin554/code-crawler@latest |
| Transport | stdio — a local child process; no ports, no auth, no hosted service |
| Runtime | Node.js 20 or newer |
| Environment | All optional — see Configuration |
GitHub Copilot app
Open Customize → MCP. If Code-Crawler appears in the
catalog, select it and follow the prompts; otherwise add a custom server
named code-crawler with this command:
npx -y @nithin554/code-crawler@latest
The app stores the entry in ~/.copilot/mcp-config.json and lists
it under Customize → Installed. Sessions already running keep
their current tool set — the server is spawned for the next session or after
you toggle it off and on.
Copilot CLI
One command writes the user-level configuration:
copilot mcp add code-crawler -- \
npx -y @nithin554/code-crawler@latest
Verify with copilot mcp list, or /mcp inside a
session. Add env vars with --env CODECRAWLER_DB=....
VS Code
If the catalog lists it, install from the Extensions view with
@mcp code-crawler. Otherwise add the entry to
.vscode/mcp.json (workspace) or your user MCP settings:
{
"servers": {
"code-crawler": {
"type": "stdio",
"command": "npx",
"args": ["-y", "@nithin554/code-crawler@latest"]
}
}
}
Other stdio clients
Any client that accepts a command and args works — no local build and no
absolute path to dist/index.js:
{
"mcpServers": {
"code-crawler": {
"command": "npx",
"args": ["-y", "@nithin554/code-crawler@latest"],
"env": {
"CODECRAWLER_DB": "/absolute/path/index.db"
}
}
}
}
Versions and updates
npx resolves the latest dist-tag every time the client
spawns the server, reusing npm's download cache when the version is already
there. A new npm release therefore takes effect on the next server start —
restart the client, open a new session, or toggle the server off and on. Nothing
upgrades mid-session, and no version is pushed to you.
Pin an exact version for reproducible setups:
"@nithin554/code-crawler@0.1.49". Catalog installs use the version
recorded in server.json at publish time, which can lag the npm
latest tag while the registry ingests a release. Check what you are
running with npm view @nithin554/code-crawler version.
What a crawl actually does
Discover
Walks the project root, skipping CODECRAWLER_EXCLUDE_DIRS and
hidden VCS directories. Optional language and
fileFilter arguments narrow the crawl.
Parse
Each file is parsed with the tree-sitter grammar for its language — Java, Python, TypeScript, or JavaScript — to extract packages/modules, imports, types, members, modifiers, decorators, and annotations.
Index and resolve
A global type index is built for the whole project, then per-file call
sites are resolved to symbol ids. Unresolved calls become
__unknown__ phantom nodes with resolved: false.
Persist
All node and edge writes happen in one transaction. Each file's previous slice is replaced; files that disappeared are removed.
Rebuild derived edges
DYNAMIC_DISPATCH and ADVISES edges are deleted and
recomputed from the fresh graph, so re-crawls never duplicate or strand them.
When the graph goes stale
The index is a snapshot. After pulling, switching branches, or editing code,
re-run crawl_project (or codecrawler crawl).
clear_index wipes the database entirely and forces a clean rebuild.
Environment variables
| Variable | Default | Purpose |
|---|---|---|
CODECRAWLER_DB |
~/.code-crawler/index.db |
Path to the SQLite graph database. Created on first use. |
CODECRAWLER_EXCLUDE_DIRS |
.git,node_modules,dist,build,target,.idea,.venv,.gradle,out,.next,coverage |
Comma-separated directory names skipped during crawl. |
CODECRAWLER_DEFAULT_LIMIT |
50 |
Default result limit for list-returning queries. |
CODECRAWLER_MAX_DEPTH |
10 |
Traversal depth cap for recursive queries. |
CODECRAWLER_HELPFUL_ERRORS |
true |
Attach fuzzy symbol suggestions to “not found” errors. |
Set these in the env block of your MCP client config so the
server always opens the intended database. Paths must be absolute for GUI
clients, which do not inherit your shell environment.
A workflow that gives good answers
The tools are precise but literal. Two cheap steps — index, then resolve the symbol id — make every later answer reliable.
1. Index explicitly
Start a session by asking the assistant to crawl the project at an absolute path. It returns file, node, edge, and call-resolution counts you can sanity check.
crawl_project({ path: "C:/work/shop-service" })
2. Resolve names first
Tool arguments take symbol ids, not guesses. Search once, then reuse the
returned qualifiedName.
search_symbols({ query: "place" })
3. Ask one graph question at a time
Prefer a specific tool over a vague prompt. “Who calls X?” maps to
get_callers; “what runs?” maps to trace_execution.
get_callers({
symbol: "com.acme.OrderService#place",
transitive: true
})
4. Ask for file and line proof
Results carry file and callSiteLine (1-based in
output). Ask the assistant to cite them before editing.
What the model can and cannot know
Modelled
- Static
CALLSedges with call-site lines and confidence. - Uniform graph across Java, Python, TypeScript, and JavaScript.
- Package/module, type, member, and import containment.
IMPLEMENTS/EXTENDShierarchy edges.- Virtual dispatch to project overrides (
DYNAMIC_DISPATCH). - AspectJ/Spring AOP advice and pointcut matching (
ADVISES). - Annotations and decorators on types, methods, constructors, and fields.
- Reflection and run-time lookups as
REFLECTSedges:Class.forName,ServiceLoader,getattr,importlib,require, dynamicimport(). - Source-visible dependency injection as
INJECTSedges:@Autowired, NestJS constructor injection, FastAPIDepends. - Framework entry points and HTTP routes as
ROUTESedges: Spring MVC, FastAPI, Flask, NestJS, Express. - Control flow per call edge: conditional flag, control-flow kinds, guard text.
- External packages as
ext:nodes with a dependency summary.
Not modelled
- Value flow and branch feasibility. Edges record that a call is conditional and the guard text — not which branch runs, nor what values reach a parameter.
- Fully dynamic reflection targets.
getattr(obj, name)with an unknown name andrequire(userInput)point at a__reflect__:marker instead of a symbol. Inventing a target would be worse than the marker. - DI containers configured outside code — XML, YAML bean wiring, NestJS dynamic module factories. Only injection visible in source is reported.
- Libraries outside the crawled tree. Third-party code is
summarised as
ext:packages and never crawled, so traces stop at the boundary. - Branch-level dead code.
find_dead_codeis whole-method, not per-branch. - Cross-language edges. Python does not call into Java even when they share a process boundary; the languages share a graph, not a call edge.
- Types changed at runtime — monkey-patching,
__getattr__proxies,Object.definePropertytricks.
Treat find_dead_code as a candidate list, not a verdict. Methods
invoked only by frameworks, serialization, or reflection will show up as unused
unless the crawler recognises the mechanism. Start with
find_entry_points, find_routes, and
find_reflections before deleting anything.
Common failure modes
| Symptom | Likely cause | Fix |
|---|---|---|
| Every tool says “not found in the index” | The server is using a different database path than the crawl. | Pin CODECRAWLER_DB in the client env block. |
| Low resolved-call count | Source layout confuses receiver inference, or the crawl included generated code. | Use fileFilter or narrow path; expect heuristic misses on unusual call shapes. |
No DYNAMIC_DISPATCH edges |
No type implements the interface inside the crawled tree. | Confirm overrides live in the project; libraries are out of scope. |
| No aspects listed | Aspect types are missing @Aspect or pointcuts use unsupported syntax. |
Supported primitives: execution, call, within, @annotation, @within, target, this, args, &&, ||, !. |
| Annotations missing | The source uses a language level the grammar does not expose as modifiers. | Annotations and decorators are read from each grammar's own modifier, decorator, or attribute nodes; verify with get_node and get_decorators. |
| Stale symbols after a refactor | The index predates the change. | Re-crawl; deleted files are pruned automatically. |
Still stuck? get_schema prints the live node kinds, edge kinds, id
scheme, and dynamic-model semantics the server is using.