I Point a Local LLM at Every Repo Before Opening It in My Editor

I Point a Local LLM at Every Repo Before Opening It in My Editor



Tópico: I Point a Local LLM at Every Repo Before Opening It in My Editor
Categoria: Tutoriais | Programação & Tecnologia
Idioma Principal: Português (Conteúdo de Tecnologia)

Descrição do Conteúdo / Informações:
-------------------------------------------------------------------------
In May a "recruiter" sent me a take-home project for a Web3 role. Nice README, plausible Next.js structure, a real-sounding company. Buried in the build tooling was a postinstall script that decoded a base64 blob and pulled a second stage from a hardcoded IP. If I had done what 99% of candidates do, git clone then npm install then open it in my editor, an infostealer would have been running on my machine before I read a single line of code.

That was not the last one either. These fake-recruiter lures are an industry now, and the payload almost never lives in src/. It lives in the places you skim: lifecycle scripts, config files, a "utils" file with one weird function. So I changed my default. Every unknown repo now goes through a local LLM triage pass before my editor ever touches it. No code execution, no install, just static reading.

Here's the workflow.



Rule zero: never let the repo run anything


The whole point is that the repo stays inert. Two safe ways to get the files:

# Option 1: clone without checkout, inspect the tree first
git clone --no-checkout https://github.com/some-org/take-home-task.git
cd take-home-task
git ls-tree -r HEAD --name-only

# Option 2: download the tarball, no git hooks, no clone at all
curl -L https://github.com/some-org/take-home-task/archive/refs/heads/main.tar.gz \
| tar -xz -C ./quarantine/

I prefer the tarball. It cannot execute anything, and extracting into a quarantine/ directory keeps me honest. Also worth saying explicitly: do not open the folder in an editor with plugins that auto-run tasks. VS Code will happily execute workspace settings, launch configs, and some extensions will run npm install for you as a favor. Read the files with cat, bat, or the LLM pipeline below.



Step 1: the boring checks that catch most payloads


Before any AI is involved, three cheap checks catch the majority of these campaigns:

# Lifecycle scripts are the #1 delivery mechanism
cat package.json | jq '.scripts | with_entries(
select(.key | test("install|prepare|prepublish|postpack")))'

# Dependencies present in package.json but missing from the lockfile
# (a classic evasion: the malicious dep gets resolved fresh at install time)
jq -r '.dependencies, .devDependencies | keys[]?' package.json | sort > deps.txt
jq -r '.packages | keys[]' package-lock.json | sed 's|node_modules/||' | sort > locked.txt
comm -23 deps.txt locked.txt

# Long encoded blobs anywhere in the tree
rg -n --max-columns=200 '[A-Za-z0-9+/]{120,}={0,2}' --glob '!*.lock' --glob '!*.map'

The lockfile mismatch check matters more than people think. Several campaigns I've dissected ship a clean-looking lockfile and a dirty package.json, or reference a typosquatted package only from a script. When I built Argus Lens (lens.noctis.biz), a scanner for exactly this class of repo, deps-missing-from-lockfile turned out to be one of the highest-signal checks in the whole tool.



Step 2: feed the suspicious files to a local model


Regex gets you candidates. Judgment is where a model earns its keep, and this has to be a local model, because I'm sometimes triaging repos under NDA or repos whose mere URL I don't want leaving my machine.

I run Ollama on WSL2 with qwen2.5-coder in two sizes: 1.5b for the fast pass over everything, 7b when the small one flags something. The prompt is a classifier, not a chat:

triage_file() {
local file="$1"
ollama run qwen2.5-coder:7b <<EOF
You are a supply-chain malware analyst. Classify the following file
from an UNTRUSTED repository. Do not summarize what the code claims
to do. Focus on what it actually does.

Answer in exactly this format:
VERDICT: CLEAN | SUSPICIOUS | MALICIOUS
SIGNALS: <comma-separated list, or "none">
EXPLANATION: <max 3 sentences>

Signals to look for:
- decoding of base64/hex strings followed by eval, Function, or child_process
- network calls to raw IPs or unusual domains at import/build time
- reading of environment variables, keychains, browser profile paths,
.ssh, .aws, or wallet files
- code that only runs during install/build, not at runtime
- obfuscation: string array shuffling, charCode arithmetic, packed code

FILE: ${file}
---
$(cat "$file")
EOF
}

Then it's just a loop over the candidates:

rg -l 'child_process|eval\(|Function\(|fromCharCode|atob|Buffer\.from' \
--glob '!node_modules' quarantine/ | while read -r f; do
echo "=== $f"
triage_file "$f"
done

The strict output format is doing real work here. Small models ramble, and "answer with VERDICT on the first line" turns a rambling model into something you can grep and script against.



What signals actually matter


After feeding a few dozen of these repos through this pipeline (and building spectr-ai, my open-source contract auditor, which taught me a lot about prompting small models for security work), the signals that separate real payloads from noise:

Install-time execution. Legitimate projects rarely need postinstall beyond native module builds. A postinstall that touches the network or decodes strings is close to a guaranteed conviction.

Deps in manifest but not in lockfile. Covered above. It means the attacker wants resolution to happen fresh on your machine.

Encoded blobs plus a decoder. A base64 string alone is often fine (inlined images, test fixtures). A base64 string within reach of eval, new Function, or child_process.exec is not.

Env harvesting. Loops over process.env, or path building toward ~/.ssh, browser extension folders, or wallet data directories like Local Storage/leveldb. There is no honest reason for a take-home CRUD app to know where MetaMask keeps its state.

Effort asymmetry. The app code is boilerplate quality, but one config or helper file is dense, minified, or oddly sophisticated. Attackers copy the app and hand-craft the payload, and the seam shows.



Honest limitations


The 1.5b model misses things. It's fine as a fast filter over many files, but I've watched it label a charCode-obfuscated dropper as "string manipulation utilities." The 7b catches most of what I throw at it, but a determined attacker who tests their payload against open models will eventually get past this too. That's fine. I'm not trying to build a perfect oracle, I'm trying to make sure the lazy, mass-produced lures (which is most of them) get caught in under two minutes without me executing anything.

Also: the model reads what you give it. If you only scan .js files, the payload will be in a .node binary or a build config. Cast the net wide first, then classify.

The whole thing costs me maybe three minutes per unknown repo, runs entirely offline, and has already paid for itself twice. Cheap insurance.

Do you actually inspect repos from strangers before installing, or does npm install still happen on autopilot?