Architecture

How OIKB is put together, and what happens step by step in each mode.

System overview

Component	Role
CLI	Entry point for interactive use. Resolves config, picks the connector from the source string, hands off to the sync engine
Daemon	Long-running server: HTTP API, per-source schedules, webhook routes, per-Knowledge-Base locks
Sync engine	One function shared by every trigger (CLI, schedule, webhook, watcher): manifest → filter → diff → cleanup → mkdir → upload
API client	The only component that talks to Hrida.ai: diff, upload, cleanup, folder creation
Connectors	One per source system. Each provides two operations: list files with checksums, and read one file
File watcher	Turns filesystem events into debounced syncs
Webhook handlers	Verify signatures and start background syncs on push events
Config & env	Hrida.ai URL/key from flags → environment → saved config; Docker secrets via _FILE
History · Metrics · Ignore	SQLite sync log, Prometheus counters, .hrida-oikbignore patterns

---

The sync algorithm

Every trigger runs the same pipeline:

1. Build manifest: the connector lists every file with its path and checksum (SHA-256, or ETag for object storage). No content is downloaded yet.
2. Filter: entries failing the include/exclude globs or the size limit are dropped.
3. Diff: only paths and checksums go to Hrida.ai, which classifies each file as added, modified, deleted or unchanged, and lists folders to create or remove.
4. Clean up first: stale files and folders are removed before new content arrives, avoiding name clashes.
5. Create folders so uploads have somewhere to land.
6. Upload added and modified files in parallel. Server errors retry up to 3 times with exponential backoff; a file that still fails is recorded and the sync carries on.
7. Report the counts. The CLI prints them and exits with 1 if anything failed; the daemon writes them to history and metrics.

What Hrida.ai does with a sync

Studio's side of the sync is its knowledge sync API:

• Matching: a source file and a Knowledge Base file are the same file when their folder and file name match, and they differ when their SHA-256 checksums do.
• Only synced files are deleted. Files someone added to the Knowledge Base by hand are left alone. The one exception: a hand-added top-level file with the same name as a source file is replaced by the synced copy.
• Folders are part of each file's path (docs/api); a folder goes away with its last file.
• Uploads land in the Knowledge Base once Studio has processed them. A file Studio can't read is added anyway, marked as failed, so it isn't uploaded again on every run.
• Access: the API key's user needs write access to the Knowledge Base.
• Pace: Studio accepts 20 uploads a minute from one address. Uploads beyond that fail with 429 and go through on the next run, so a first sync of many files takes several runs (about one per 20 files). The daemon and watch mode run again by themselves.

---

One-time sync (hrida-oikb sync)

---

Daemon mode (hrida-oikb daemon)

Each entry runs on its own timer. The per-Knowledge-Base lock means a webhook or manual trigger never overlaps a scheduled sync of the same Knowledge Base, and the sync runs in a thread so the API stays responsive. On shutdown (SIGINT/SIGTERM) the timers stop cleanly.

---

Webhook-triggered sync

The response goes back before the sync starts, so the provider's delivery log shows success even if the sync later fails. Check the daemon logs or /health for the outcome.

---

Watch mode (hrida-oikb watch)

The debounce stops a git checkout or a burst of saves from triggering dozens of syncs.

---

Manual sync via the API

• A real sync is fire-and-forget: the response only confirms it was started. Poll GET /health for the result.
• A dry run waits and returns real counts, without changing anything.
• Every response is 200. Check the body (triggered, skipped), not the status code.