Data directory - ollim-bot

Every file in the ~/.ollim-bot/ directory and what it does. All persistent data lives in ~/.ollim-bot/. The directory is created on first use and automatically managed as a git repository — storage.py auto-commits after most writes.

Set the OLLIM_DATA_DIR shell env var to relocate the directory — for example, OLLIM_DATA_DIR=~/.ollim-bot-2 ollim-bot runs a second instance with a separate data dir. It must be a shell env var (not in .env) because storage.py is imported before load_dotenv(). See Configuration reference.

The root contains agent-facing files (profile files, routines, reminders, webhooks, and the optional tool-policy.yaml) that the agent reads and writes directly. Infrastructure state (sessions, credentials, ping budget, etc.) lives in the state/ subdirectory, keeping the agent’s working directory clean. The state/ directory is write-protected from agent tool access — a PreToolUse hook blocks Write and Edit calls targeting paths inside state/. See Permissions for details.

~/.ollim-bot/
├── IDENTITY.md                         # Bot persona (bootstrapped on first run)
├── USER.md                             # User context (user-created, optional)
├── tool-policy.yaml                 # Optional YAML tool policy overrides
├── downloads/                       # Saved non-image Discord attachments
├── routines/
│   └── *.md                         # Recurring routine specs
├── reminders/
│   └── *.md                         # One-shot reminder specs
├── webhooks/
│   └── *.md                         # Webhook specs
├── reflections/                     # Per-job background fork execution traces
│   └── <item-id>/*.md
├── skills/                          # Reusable instruction sets
│   └── */SKILL.md
├── .claude/                         # SDK layout (created at startup)
│   ├── agents/                      # Bundled subagent specs (SDK discovery)
│   │   ├── ollim-bot-guide.md
│   │   ├── gmail-reader.md
│   │   ├── history-reviewer.md
│   │   ├── responsiveness-reviewer.md
│   │   └── user-proxy.md
│   └── skills/ →                    # Symlink → ../skills/ (SDK skill discovery)
└── state/
    ├── bot.pid                      # PID file for single-instance check
    ├── credentials.json             # Google OAuth credentials (from Cloud Console)
    ├── token.json                   # Google OAuth token (auto-generated)
    ├── sessions.json                # Current session ID (plain string)
    ├── session_history.jsonl        # Append-only session lifecycle events
    ├── pending_updates.json         # Background fork updates queued for main session
    ├── inquiries.json               # Button inquiry prompts (7-day TTL)
    ├── fork_messages.json           # Discord message → fork session mapping (7-day TTL)
    ├── config.json                  # Runtime configuration (persistent, no git commit)
    └── ping_budget.json             # Ping budget state (ephemeral, no git commit)

Profile files

Two markdown files in the data directory root customize the bot’s persona and provide context about you. Both are prepended to the system prompt — see System prompt for how they compose into the final prompt.

File	Source	Description
`IDENTITY.md`	`bootstrap_identity()` in `profile.py` — created on first run, skipped if file exists	Bot persona and communication style
`USER.md`	User-created	Context about the user

See Customize personality and Tell the bot about yourself for editing guidance and examples.

Downloads

Non-image Discord attachments (PDFs, text files, spreadsheets, etc.) are saved to downloads/ so the agent can read them with its Read tool. Filename collisions are handled with numeric suffixes (file-2.txt, file-3.txt). This directory is created on first use. See File attachments for user-facing details.

Session files

All files in this section and below live in the state/ subdirectory. These files track the agent’s conversation state across restarts.

File	Format	Description
`sessions.json`	Plain string	Active Agent SDK session ID. A raw string, not JSON. Read on startup to resume.
`session_history.jsonl`	JSONL	Append-only log of session lifecycle events. One JSON object per line.

Session events

Each line in session_history.jsonl contains:

Field	Type	Description
`session_id`	`str`	The Agent SDK session ID
`event`	`str`	One of: `created`, `compacted`, `swapped`, `cleared`, `interactive_fork`, `bg_fork`, `isolated_bg`, `restarting`
`timestamp`	`str`	ISO 8601 datetime in the configured timezone (`OLLIM_TIMEZONE`)
`parent_session_id`	`str \| null`	Set for `compacted`, `swapped`, and fork events

session_history.jsonl

{"session_id": "sess_abc", "event": "created", "timestamp": "2026-02-24T10:00:00-08:00", "parent_session_id": null}
{"session_id": "sess_def", "event": "bg_fork", "timestamp": "2026-02-24T10:05:00-08:00",
 "parent_session_id": "sess_abc"}

Compaction is detected two ways: the /compact command detects it via the compact_boundary SystemMessage from the SDK, while save_session_id() catches SDK auto-compaction by detecting session ID changes. A _swap_in_progress flag suppresses false compacted events during swap_client().

Fork and inquiry files

These files track ephemeral Discord interaction state. Both use a 7-day TTL — entries older than 7 days are pruned on every read.

File	Format	Description
`fork_messages.json`	JSON array	Maps Discord message IDs to fork session IDs, enabling reply-to-fork.
`inquiries.json`	JSON object	Maps 8-char hex keys to inquiry prompts. Persists button actions across restarts.

fork_messages.json

[
  {
    "message_id": 1234567890123456789,
    "fork_session_id": "sess_fork_abc",
    "parent_session_id": "sess_main",
    "ts": 1740000000.0
  }
]

Field	Type	Description
`message_id`	`int`	Discord message snowflake ID
`fork_session_id`	`str`	Agent SDK session ID for the fork
`parent_session_id`	`str \| null`	Session the fork branched from
`ts`	`float`	Unix timestamp for TTL pruning

inquiries.json

{
  "a1b2c3d4": {"prompt": "Can you summarize my emails?", "ts": 1740000000.0}
}

Field	Type	Description
(key)	`str`	8-character hex UUID
`prompt`	`str`	The full prompt string for the button inquiry
`ts`	`float`	Unix timestamp for TTL pruning

Pending updates

Background forks write summaries to pending_updates.json via the report_updates MCP tool. The main session reads and clears these on its next turn.

pending_updates.json

[
  {
    "ts": "2026-02-24T10:15:00-08:00",
    "message": "Checked your morning emails — 3 need replies, none urgent."
  }
]

Field	Type	Description
`ts`	`str`	ISO 8601 datetime in the configured timezone (`OLLIM_TIMEZONE`)
`message`	`str`	Summary from the background fork

Concurrent background forks use an asyncio.Lock to protect the read-modify-write cycle on this file, preventing lost updates.

Reflections

After every background fork finishes, a lightweight Haiku meta-fork writes a short execution trace to reflections/<item-id>/<timestamp>.md. These traces are for your own debugging and history review — nothing is sent externally.

reflections/
├── morning-checkin/
│   ├── 2026-04-09T14-30-00Z.md
│   └── 2026-04-10T14-30-00Z.md
└── email-triage/
    └── 2026-04-10T08-00-00Z.md

Each file records the job status, whether the fork filed a report, any error info, and a 1-3 sentence trace of what the task was meant to do. Set reflect: false in a job’s YAML frontmatter to skip this per job. See Reflections for the full behavior.

Ping budget

ping_budget.json stores the refill-on-read token bucket state for proactive pings. This file is not git-committed — it is ephemeral state that resets naturally.

ping_budget.json

{
  "capacity": 5,
  "available": 3.7,
  "refill_rate_minutes": 90,
  "last_refill": "2026-02-24T09:30:00-08:00",
  "critical_used": 1,
  "critical_reset_date": "2026-02-24",
  "daily_used": 2,
  "daily_used_reset": "2026-02-24"
}

Field	Type	Default	Description
`capacity`	`int`	`5`	Maximum pings in the bucket
`available`	`float`	—	Current fractional ping count
`refill_rate_minutes`	`int`	`90`	Minutes per 1 ping refill
`last_refill`	`str`	—	ISO 8601 datetime of last refill computation
`critical_used`	`int`	—	Critical pings used today (resets daily)
`critical_reset_date`	`str`	—	ISO date for `critical_used` reset
`daily_used`	`int`	—	Total pings used today
`daily_used_reset`	`str`	—	ISO date for `daily_used` reset

On every load(), time-based refill is computed and daily counters are reset if the date has changed. The updated state is saved back immediately.

Runtime configuration

config.json stores persistent settings you configure via /config. These include per-context model and thinking defaults, timeouts, and the default permission mode. See Configuration reference for the full key listing. This file is not git-committed — it is local configuration that may vary between instances.

Agent-managed directories

Routines, reminders, webhooks, and skills are each stored as files in their own subdirectory. For routines, reminders, and webhooks, filenames are auto-generated slugs from the message content (max 50 characters) and the id field in YAML frontmatter is authoritative. Skills use a different structure: each skill is a directory named after the skill, containing a SKILL.md file.

Directory	Contents	Description
`routines/`	`*.md`	Recurring cron-scheduled routines
`reminders/`	`*.md`	One-shot and chainable reminders
`webhooks/`	`*.md`	Webhook endpoint specs with prompt templates
`skills/`	`*/SKILL.md`	Reusable instruction sets referenced by routines and reminders

Routines, reminders, webhooks, and skills use the same storage format: YAML frontmatter delimited by ---, followed by a markdown body. See File formats for the full field specs.

routines/morning-briefing.md

---
id: abc123
cron: "30 8 * * 1-5"
description: Morning briefing
---
Review my tasks and calendar for today, then give me a summary.

The agent has direct file access to routines/ and reminders/ — it creates and manages these files without going through CLI commands.

SDK layout directory

The .claude/ directory is the Agent SDK’s expected layout for discovering subagents and skills. _ensure_sdk_layout() in main.py creates it automatically at startup.

Path	Type	Description
`.claude/agents/*.md`	Regular files	Bundled subagent specs copied from the source tree with template expansion (`{USER_NAME}`, `{BOT_NAME}`). Existing files are never overwritten — your customizations persist across updates.
`.claude/skills/`	Symlink → `../skills/`	Lets the SDK discover skills without duplicating the `skills/` directory.

Both paths work because the bot sets cwd=DATA_DIR and setting_sources=["project"] — the SDK resolves .claude/ relative to the working directory.

Google credentials

File	Source	Description
`credentials.json`	Manual — Google Cloud Console	OAuth 2.0 client creds
`token.json`	Auto-generated after `/google-auth`	OAuth refresh and access tokens

Neither file is git-committed. token.json is auto-generated when you run /google-auth.

Process lock

bot.pid contains the PID of the running bot process. On startup, _check_already_running() reads this file and uses os.kill(pid, 0) to check if the process is alive. On Linux, it additionally verifies /proc/<pid>/cmdline contains ollim-bot to avoid false positives from recycled PIDs. Corrupted PID files (non-numeric content or read errors) are silently overwritten. The file is deleted on exit via atexit.

Git tracking

The ~/.ollim-bot/ directory is initialized as a git repository. storage.py auto-commits after writes to most files. Files that are not git-committed:

File	Reason
`state/ping_budget.json`	Ephemeral state that refills over time
`state/bot.pid`	Process lock deleted on exit
`state/credentials.json`	Sensitive OAuth client credentials
`state/token.json`	Sensitive OAuth tokens
`state/sessions.json`	Frequent overwrites on every agent turn
`state/fork_messages.json`	Ephemeral mapping with 7-day TTL
`state/pending_updates.json`	Ephemeral cross-fork messaging
`state/inquiries.json`	Ephemeral button state with 7-day TTL
`state/config.json`	Local runtime preferences
`.claude/agents/*.md`	SDK infrastructure — written once at startup, not managed by `storage.py`
`.claude/skills/`	Symlink — SDK infrastructure

All other files — profile files, session history, routines, reminders, webhooks, and skills — are auto-committed on every write.

All writes use atomic temp-file-then-rename (tempfile.mkstemp + os.replace) to prevent data corruption. Do not manually edit files while the bot is running.

Next steps

File formats

YAML frontmatter specs for routines, reminders, webhooks, and skills.

Configuration reference

Environment variables and .env file setup.

Ping budget

How the refill-on-read token bucket controls proactive pings.

Set up Google integration

Connect Google Tasks, Calendar, and Gmail.

​Profile files

​Downloads

​Session files

​Session events

​Fork and inquiry files

​fork_messages.json

​inquiries.json

​Pending updates

​Reflections

​Ping budget

​Runtime configuration

​Agent-managed directories

​SDK layout directory

​Google credentials

​Process lock

​Git tracking

​Next steps

File formats

Configuration reference

Ping budget

Set up Google integration

Profile files

Downloads

Session files

Session events

Fork and inquiry files

fork_messages.json

inquiries.json

Pending updates

Reflections

Ping budget

Runtime configuration

Agent-managed directories

SDK layout directory

Google credentials

Process lock

Git tracking

Next steps