From a Pile of Office Files to a Searchable Local Knowledge Base: I Built an Open-Source Skill

Self-hosted knowledge base skill v4.0 is out: https://github.com/furoute/workshop/tree/main/skills/self-hosted-kb
Companion skill: Everything file search: https://github.com/furoute/workshop/tree/main/skills/everything-http-search

The problem

I had tens of GB of Office/PDF files sitting around: proposals, contracts, notices, monthly reports. When I needed something, all I had was filenames and memory. Even a well-named file can't answer "what was the winning bid for that project?" or "how does XX's proposal handle operations?"

Off-the-shelf knowledge base options either live in the cloud (your data leaves home) or require standing up a stack of services. What I wanted was small: files stay on my own disk, I can find the original text and open the original file, and when I ask a question the model answers with sources. That tiny list of requirements meant building the pipeline myself.

So I made a skill: self-hosted-kb, a complete hands-on record, MIT licensed. Not a product — a workbook you can follow step by step and get running.

The pipeline: seven steps + two retrieval paths

One sentence: files stay in place, read-only; the index and vectors live separately; at query time, two paths retrieve and fuse.

Seven steps (read-only first, changes later):

1. Full probe (read-only): inventory + magic-number sniffing, four-way triage (healthy / format-faked / damaged / encrypted-or-locked)
2. SHA-256 dedup (read-only): exact duplicates only count by hash — same name and size doesn't count
3. Stratified sample reads (read-only): sample 400–500 files by directory × year × size; recompute all SHAs after the reads and require zero changes — proof I didn't touch the files
4. Auto-triage into three piles (labels only): useful / junk / undecided; undecided files get grouped into "file families", because nobody can review tens of thousands of rows
5. Batch adjudication (labels only): 28 rules, one final state per family
6. Entity cards: five entity types, fifteen YAML fields + nine body sections; evidence grades never get inflated, money figures use a fixed convention
7. Presentation layer (optional): index + property views + Q&A pages in Obsidian, for humans

Retrieval stands on two legs:

• FTS5 (keywords): model numbers, IDs, amounts, dates — exact literal matches
• Vectors (semantics): fuzzy questions like "how do I handle this"
• RRF rank fusion: what ranks high on both paths ranks highest

Scale from the real run: the largest batch was 37,782 files / 27.7 GB → after inventory and dedup, 2,066 files / 10.7 GB; 400k chunks, a 950 MB index.

Tuning: the data was in the base, the model said it wasn't

The most valuable part. Once, a user asked about something that was definitely in the base, and the model answered "not found" — but a direct FTS5 query returned 10 precise hits. The data wasn't missing; it was buried. Two root causes:

1. FTS5's default trigram tokenizer kills two-character Chinese words. Trigrams index 3-character sliding windows, so two-character Chinese words like "合同" (contract) or "预算" (budget) can never form a complete trigram — in testing, 11 two-character queries returned 0 hits, all 11 of them. The fix: jieba segmentation + unicode61, tokenize before feeding FTS5. Lesson: on Chinese corpora, the tokenization strategy matters an order of magnitude more than the index structure.

2. One giant Excel outweighed 130 PDFs. One spreadsheet's extracted text cut into 50,000 chunks, while the library-wide average was just 365 words per chunk — no matter the question, that spreadsheet's chunks filled the candidate pool and pushed the right documents down. That's "retrieval pollution", not "retrieval inaccuracy": the fix isn't better scoring, it's stopping big documents from hogging the pool. Five fixes: chunk-count downweighting, table-density downweighting, a cap of 20 chunks per document, directory-path evidence, and cutting query terms from 60 to 12.

Before vs after: latency 96–203 ms → 24–96 ms (it got faster, because the candidate pool shrank from thousands to hundreds); three groups of previously unanswerable questions all hit the right documents.

The mindset in one line: don't "score the right answer higher" — "keep the wrong answers out of the candidate pool".

Everything: finding files is a different layer, don't mix it with retrieval

"Does file X exist?" needs no vectorization at all — Everything reads the NTFS master file table and locates files among hundreds of thousands in milliseconds, zero tokens. The order should be:

• "Does X exist?" → Everything (milliseconds, zero cost)
• "What does X say?" → knowledge base retrieval (chunking + vectors)

80% of "find a file" scenarios need no vectorization at all. The companion skill is published separately: it runs over an HTTP API, one curl line to query; set up autostart + silent tray + always-on HTTP once. Note: this is a Windows + NTFS solution; on Linux/macOS, use grep/find.

Cloud vs local models: the bottleneck is retrieval, not the model

Same retrieval chain, two answering models: cloud DeepSeek API vs local Ollama qwen3.5:9b (RTX 4060 Laptop, network cable unplugged throughout).

• Speed: cloud steady at 0.7–1.1 s; local jittery at 0.9–4.1 s
• Quality: understanding, summarizing, reading proposals — roughly tied; but the local small model has two hard flaws: multi-digit multiplication goes wrong, and tables squeezed into one line get their states mixed up; when information is insufficient, the cloud model leans toward "not sure", the local one leans toward confabulating
• Key finding: questions where retrieval handed over the wrong material were unanswerable on both sides — a bigger model can't conjure a document that never entered the candidate pool. Optimize recall quality, not model size
• One hands-on pitfall: qwen3.5 has thinking on by default, and the chain of thought eats the whole num_predict budget, leaving an empty answer — you have to turn it off explicitly

Data boundary (honest version)

The accurate statement: data doesn't leave home in bulk, but the snippets hit at query time — and your question — do go out.

• Stays home: the original corpus, 400k chunks, the 950 MB index, the vectors — all on my own disk
• Goes out: the top-k snippets selected at query time + the user's question, sent to the cloud model for understanding and organization
• Looking up a technical manual: fine. Looking up a client list, a quote, contract terms: those passages do go out — so judge before putting sensitive material in the base
• The only way to keep everything home is a local model (5–10× slower, one tier weaker in Chinese)

How to use it

This isn't SaaS; there's no one-click installer. The right way: hand the skill folder to your agent and let it follow SKILL.md:

https://github.com/furoute/workshop/tree/main/skills/self-hosted-kb

SKILL.md is a workbook written for an agent to follow, not a manual for humans; companion scripts, references and architecture diagrams are in the same directory. The Everything skill is here:

https://github.com/furoute/workshop/tree/main/skills/everything-http-search

Honest boundaries

• One complete hands-on record, not a general product; a different corpus means re-tuning
• Windows + NTFS is the precondition for the Everything layer; legacy Office formats (.doc/.xls/.ppt) aren't parsed by the current scripts — only their filenames are searchable
• Hands-on ability is a precondition: you run the scripts, build the index, and debug it yourself
• Example data is de-identified; replace personal configuration with your own

Closing

The original intent was simple: my files, my retrieval. However convenient a cloud knowledge base is, once data leaves, it's gone; once this local chain works, finding it, opening it, and citing the source is enough.

Welcome to try it — issues go straight to the repo.

Side note: this wasn't a solo effort — it was built by six AI agents across three machines, with task split, review and merge all running over ACP-lite (Agent Connect Personal), an open-source file-mailbox-style interconnection scheme: https://github.com/furoute/acp-lite