diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 0000000..a979f6b --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,68 @@ +# AMC (Agent Management Console) + +A dashboard and management system for monitoring and controlling Claude Code and Codex agent sessions. + +## Key Documentation + +### Claude JSONL Session Log Reference + +**Location:** `docs/claude-jsonl-reference/` + +Comprehensive documentation for parsing and processing Claude Code JSONL session logs. **Always consult this before implementing JSONL parsing logic.** + +| Document | Purpose | +|----------|---------| +| [01-format-specification](docs/claude-jsonl-reference/01-format-specification.md) | Complete JSONL format spec with all fields | +| [02-message-types](docs/claude-jsonl-reference/02-message-types.md) | Every message type with concrete examples | +| [03-tool-lifecycle](docs/claude-jsonl-reference/03-tool-lifecycle.md) | Tool call flow from invocation to result | +| [04-subagent-teams](docs/claude-jsonl-reference/04-subagent-teams.md) | Subagent and team message formats | +| [05-edge-cases](docs/claude-jsonl-reference/05-edge-cases.md) | Error handling, malformed input, recovery | +| [06-quick-reference](docs/claude-jsonl-reference/06-quick-reference.md) | Cheat sheet for common operations | + +## Architecture + +### Server (Python) + +The server uses a mixin-based architecture in `amc_server/`: + +| Module | Purpose | +|--------|---------| +| `server.py` | Main AMC server class combining all mixins | +| `mixins/parsing.py` | JSONL reading and token extraction | +| `mixins/conversation.py` | Claude/Codex conversation parsing | +| `mixins/state.py` | Session state management | +| `mixins/discovery.py` | Codex session auto-discovery | +| `mixins/spawn.py` | Agent spawning via Zellij | +| `mixins/control.py` | Session control (focus, dismiss) | +| `mixins/skills.py` | Skill enumeration | +| `mixins/http.py` | HTTP routing | + +### Dashboard (React) + +Single-page app in `dashboard/` served via HTTP. + +## File Locations + +| Content | Location | +|---------|----------| +| Claude sessions | `~/.claude/projects//.jsonl` | +| Codex sessions | `~/.codex/sessions/**/.jsonl` | +| AMC session state | `~/.local/share/amc/sessions/.json` | +| AMC event logs | `~/.local/share/amc/events/.jsonl` | +| Pending spawns | `~/.local/share/amc/pending_spawns/.json` | + +## Critical Parsing Notes + +1. **Content type ambiguity** — User message `content` can be string (user input) OR array (tool results) +2. **Missing fields** — Always use `.get()` with defaults for optional fields +3. **Boolean vs int** — Python's `isinstance(True, int)` is True; check bool first +4. **Partial reads** — When seeking to file end, first line may be truncated +5. **Codex differences** — Uses `response_item` type, `function_call` for tools + +## Testing + +```bash +pytest tests/ +``` + +All parsing edge cases are covered in `tests/test_parsing.py` and `tests/test_conversation.py`. diff --git a/docs/claude-jsonl-reference/01-format-specification.md b/docs/claude-jsonl-reference/01-format-specification.md new file mode 100644 index 0000000..747e5c2 --- /dev/null +++ b/docs/claude-jsonl-reference/01-format-specification.md @@ -0,0 +1,214 @@ +# Claude JSONL Format Specification + +## File Format + +- **Format:** Newline-delimited JSON (NDJSON/JSONL) +- **Encoding:** UTF-8 +- **Line terminator:** `\n` (LF) +- **One JSON object per line** — no array wrapper + +## Message Envelope (Common Fields) + +Every line in a Claude JSONL file contains these fields: + +```json +{ + "parentUuid": "uuid-string or null", + "isSidechain": false, + "userType": "external", + "cwd": "/full/path/to/working/directory", + "sessionId": "session-uuid-v4", + "version": "2.1.20", + "gitBranch": "branch-name or empty string", + "type": "user|assistant|progress|system|summary|file-history-snapshot", + "message": { ... }, + "uuid": "unique-message-uuid-v4", + "timestamp": "ISO-8601 timestamp" +} +``` + +### Field Reference + +| Field | Type | Required | Description | +|-------|------|----------|-------------| +| `type` | string | Yes | Message type identifier | +| `uuid` | string (uuid) | Yes* | Unique identifier for this event | +| `parentUuid` | string (uuid) or null | Yes | Links to parent message (null for root) | +| `timestamp` | string (ISO-8601) | Yes* | When event occurred (UTC) | +| `sessionId` | string (uuid) | Yes | Session identifier | +| `version` | string (semver) | Yes | Claude Code version (e.g., "2.1.20") | +| `cwd` | string (path) | Yes | Working directory at event time | +| `gitBranch` | string | No | Git branch name (empty if not in repo) | +| `isSidechain` | boolean | Yes | `true` for subagent sessions | +| `userType` | string | Yes | Always "external" for user sessions | +| `message` | object | Conditional | Message content (user/assistant types) | +| `agentId` | string | Conditional | Agent identifier (subagent sessions only) | + +*May be null in metadata-only entries like `file-history-snapshot` + +## Content Structure + +### User Message Content + +User messages have `message.content` as either: + +**String (direct input):** +```json +{ + "message": { + "role": "user", + "content": "Your question or instruction" + } +} +``` + +**Array (tool results):** +```json +{ + "message": { + "role": "user", + "content": [ + { + "type": "tool_result", + "tool_use_id": "toolu_01XYZ", + "content": "Tool output text" + } + ] + } +} +``` + +### Assistant Message Content + +Assistant messages always have `message.content` as an **array**: + +```json +{ + "message": { + "role": "assistant", + "type": "message", + "model": "claude-opus-4-5-20251101", + "id": "msg_bdrk_01Abc123", + "content": [ + {"type": "thinking", "thinking": "..."}, + {"type": "text", "text": "..."}, + {"type": "tool_use", "id": "toolu_01XYZ", "name": "Read", "input": {...}} + ], + "stop_reason": "end_turn", + "stop_sequence": null, + "usage": {...} + } +} +``` + +## Content Block Types + +### Text Block +```json +{ + "type": "text", + "text": "Response text content" +} +``` + +### Thinking Block +```json +{ + "type": "thinking", + "thinking": "Internal reasoning (extended thinking mode)", + "signature": "base64-signature (optional)" +} +``` + +### Tool Use Block +```json +{ + "type": "tool_use", + "id": "toolu_01Abc123XYZ", + "name": "ToolName", + "input": { + "param1": "value1", + "param2": 123 + } +} +``` + +### Tool Result Block +```json +{ + "type": "tool_result", + "tool_use_id": "toolu_01Abc123XYZ", + "content": "Result text or structured output", + "is_error": false +} +``` + +## Usage Object + +Token consumption reported in assistant messages: + +```json +{ + "usage": { + "input_tokens": 1000, + "output_tokens": 500, + "cache_creation_input_tokens": 200, + "cache_read_input_tokens": 400, + "cache_creation": { + "ephemeral_5m_input_tokens": 200, + "ephemeral_1h_input_tokens": 0 + }, + "service_tier": "standard" + } +} +``` + +| Field | Type | Description | +|-------|------|-------------| +| `input_tokens` | int | Input tokens consumed | +| `output_tokens` | int | Output tokens generated | +| `cache_creation_input_tokens` | int | Tokens used to create cache | +| `cache_read_input_tokens` | int | Tokens read from cache | +| `service_tier` | string | API tier ("standard", etc.) | + +## Model Identifiers + +Common model names in `message.model`: + +| Model | Identifier | +|-------|------------| +| Claude Opus 4.5 | `claude-opus-4-5-20251101` | +| Claude Sonnet 4.5 | `claude-sonnet-4-5-20241022` | +| Claude Haiku 4.5 | `claude-haiku-4-5-20251001` | + +## Version History + +| Version | Changes | +|---------|---------| +| 2.1.20 | Extended thinking, permission modes, todos | +| 2.1.17 | Subagent support with agentId | +| 2.1.x | Progress events, hook metadata | +| 2.0.x | Basic message/tool_use/tool_result | + +## Conversation Graph + +Messages form a DAG (directed acyclic graph) via parent-child relationships: + +``` +Root (parentUuid: null) +├── User message (uuid: A) +│ └── Assistant (uuid: B, parentUuid: A) +│ ├── Progress: Tool (uuid: C, parentUuid: A) +│ └── Progress: Hook (uuid: D, parentUuid: A) +└── User message (uuid: E, parentUuid: B) + └── Assistant (uuid: F, parentUuid: E) +``` + +## Parsing Recommendations + +1. **Line-by-line** — Don't load entire file into memory +2. **Skip invalid lines** — Wrap JSON.parse in try/catch +3. **Handle missing fields** — Check existence before access +4. **Ignore unknown types** — Format evolves with new event types +5. **Check content type** — User content can be string OR array +6. **Sum token variants** — Cache tokens may be in different fields diff --git a/docs/claude-jsonl-reference/02-message-types.md b/docs/claude-jsonl-reference/02-message-types.md new file mode 100644 index 0000000..65eea18 --- /dev/null +++ b/docs/claude-jsonl-reference/02-message-types.md @@ -0,0 +1,346 @@ +# Claude JSONL Message Types + +Complete reference for all message types in Claude Code session logs. + +## Type: `user` + +User input messages (prompts, instructions, tool results). + +### Direct User Input +```json +{ + "parentUuid": null, + "isSidechain": false, + "userType": "external", + "cwd": "/Users/dev/myproject", + "sessionId": "abc123-def456", + "version": "2.1.20", + "gitBranch": "main", + "type": "user", + "message": { + "role": "user", + "content": "Find all TODO comments in the codebase" + }, + "uuid": "msg-001", + "timestamp": "2026-02-27T10:00:00.000Z", + "thinkingMetadata": { + "maxThinkingTokens": 31999 + }, + "todos": [], + "permissionMode": "bypassPermissions" +} +``` + +### Tool Results (Following Tool Calls) +```json +{ + "parentUuid": "msg-002", + "type": "user", + "message": { + "role": "user", + "content": [ + { + "type": "tool_result", + "tool_use_id": "toolu_01ABC", + "content": "src/api.py:45: # TODO: implement caching" + }, + { + "type": "tool_result", + "tool_use_id": "toolu_01DEF", + "content": "src/utils.py:122: # TODO: add validation" + } + ] + }, + "uuid": "msg-003", + "timestamp": "2026-02-27T10:00:05.000Z" +} +``` + +**Parsing Note:** Check `typeof content === 'string'` vs `Array.isArray(content)` to distinguish user input from tool results. + +## Type: `assistant` + +Claude's responses including text, thinking, and tool invocations. + +### Text Response +```json +{ + "parentUuid": "msg-001", + "type": "assistant", + "message": { + "role": "assistant", + "type": "message", + "model": "claude-opus-4-5-20251101", + "id": "msg_bdrk_01Abc123", + "content": [ + { + "type": "text", + "text": "I found 2 TODO comments in your codebase..." + } + ], + "stop_reason": "end_turn", + "stop_sequence": null, + "usage": { + "input_tokens": 1500, + "output_tokens": 200, + "cache_read_input_tokens": 800 + } + }, + "uuid": "msg-002", + "timestamp": "2026-02-27T10:00:02.000Z" +} +``` + +### With Thinking (Extended Thinking Mode) +```json +{ + "type": "assistant", + "message": { + "role": "assistant", + "content": [ + { + "type": "thinking", + "thinking": "The user wants to find TODOs. I should use Grep to search for TODO patterns across all file types.", + "signature": "eyJhbGciOiJSUzI1NiJ9..." + }, + { + "type": "text", + "text": "I'll search for TODO comments in your codebase." + } + ] + } +} +``` + +### With Tool Calls +```json +{ + "type": "assistant", + "message": { + "role": "assistant", + "content": [ + { + "type": "tool_use", + "id": "toolu_01Grep123", + "name": "Grep", + "input": { + "pattern": "TODO", + "output_mode": "content" + } + } + ], + "stop_reason": null + } +} +``` + +### Multiple Tool Calls (Parallel) +```json +{ + "type": "assistant", + "message": { + "content": [ + { + "type": "text", + "text": "I'll search for both TODOs and FIXMEs." + }, + { + "type": "tool_use", + "id": "toolu_01A", + "name": "Grep", + "input": {"pattern": "TODO"} + }, + { + "type": "tool_use", + "id": "toolu_01B", + "name": "Grep", + "input": {"pattern": "FIXME"} + } + ] + } +} +``` + +## Type: `progress` + +Progress events for hooks, tools, and async operations. + +### Hook Progress +```json +{ + "parentUuid": "msg-002", + "isSidechain": false, + "type": "progress", + "data": { + "type": "hook_progress", + "hookEvent": "PostToolUse", + "hookName": "PostToolUse:Grep", + "command": "node scripts/log-tool-use.js" + }, + "parentToolUseID": "toolu_01Grep123", + "toolUseID": "toolu_01Grep123", + "timestamp": "2026-02-27T10:00:03.000Z", + "uuid": "prog-001" +} +``` + +### Bash Progress +```json +{ + "type": "progress", + "data": { + "type": "bash_progress", + "status": "running", + "toolName": "Bash", + "command": "npm test" + } +} +``` + +### MCP Progress +```json +{ + "type": "progress", + "data": { + "type": "mcp_progress", + "server": "playwright", + "tool": "browser_navigate", + "status": "complete" + } +} +``` + +## Type: `system` + +System messages and metadata entries. + +### Local Command +```json +{ + "parentUuid": "msg-001", + "type": "system", + "subtype": "local_command", + "content": "/usage\n", + "level": "info", + "timestamp": "2026-02-27T10:00:00.500Z", + "uuid": "sys-001", + "isMeta": false +} +``` + +### Turn Duration +```json +{ + "type": "system", + "subtype": "turn_duration", + "slug": "project-session", + "durationMs": 65432, + "uuid": "sys-002", + "timestamp": "2026-02-27T10:01:05.000Z" +} +``` + +## Type: `summary` + +End-of-session or context compression summaries. + +```json +{ + "type": "summary", + "summary": "Searched codebase for TODO comments, found 15 instances across 8 files. Prioritized by module.", + "leafUuid": "msg-010" +} +``` + +**Note:** `leafUuid` points to the last message included in this summary. + +## Type: `file-history-snapshot` + +File state tracking for undo/restore operations. + +```json +{ + "type": "file-history-snapshot", + "messageId": "snap-001", + "snapshot": { + "messageId": "snap-001", + "trackedFileBackups": { + "/src/api.ts": { + "path": "/src/api.ts", + "originalContent": "...", + "backupPath": "~/.claude/backups/..." + } + }, + "timestamp": "2026-02-27T10:00:00.000Z" + }, + "isSnapshotUpdate": false +} +``` + +## Codex Format (Alternative Agent) + +Codex uses a different JSONL structure. + +### Session Metadata (First Line) +```json +{ + "type": "session_meta", + "timestamp": "2026-02-27T10:00:00.000Z", + "payload": { + "cwd": "/Users/dev/myproject", + "timestamp": "2026-02-27T10:00:00.000Z" + } +} +``` + +### Response Item (Messages) +```json +{ + "type": "response_item", + "timestamp": "2026-02-27T10:00:05.000Z", + "payload": { + "type": "message", + "role": "assistant", + "content": [ + {"text": "I found the issue..."} + ] + } +} +``` + +### Function Call (Tool Use) +```json +{ + "type": "response_item", + "payload": { + "type": "function_call", + "call_id": "call_abc123", + "name": "Grep", + "arguments": "{\"pattern\": \"TODO\"}" + } +} +``` + +### Reasoning (Thinking) +```json +{ + "type": "response_item", + "payload": { + "type": "reasoning", + "summary": [ + {"type": "summary_text", "text": "Analyzing the error..."} + ] + } +} +``` + +## Message Type Summary + +| Type | Frequency | Content | +|------|-----------|---------| +| `user` | Per prompt | User input or tool results | +| `assistant` | Per response | Text, thinking, tool calls | +| `progress` | Per hook/tool | Execution status | +| `system` | Occasional | Commands, metadata | +| `summary` | Session end | Conversation summary | +| `file-history-snapshot` | Start/end | File state tracking | diff --git a/docs/claude-jsonl-reference/03-tool-lifecycle.md b/docs/claude-jsonl-reference/03-tool-lifecycle.md new file mode 100644 index 0000000..6cc83d9 --- /dev/null +++ b/docs/claude-jsonl-reference/03-tool-lifecycle.md @@ -0,0 +1,341 @@ +# Tool Call Lifecycle + +Complete documentation of how tool invocations flow through Claude JSONL logs. + +## Lifecycle Overview + +``` +1. Assistant message with tool_use block + ↓ +2. PreToolUse hook fires (optional) + ↓ +3. Tool executes + ↓ +4. PostToolUse hook fires (optional) + ↓ +5. User message with tool_result block + ↓ +6. Assistant processes result +``` + +## Phase 1: Tool Invocation + +Claude requests a tool via `tool_use` content block: + +```json +{ + "type": "assistant", + "message": { + "role": "assistant", + "content": [ + { + "type": "text", + "text": "I'll read that file for you." + }, + { + "type": "tool_use", + "id": "toolu_01ReadFile123", + "name": "Read", + "input": { + "file_path": "/src/auth/login.ts", + "limit": 200 + } + } + ], + "stop_reason": null + }, + "uuid": "msg-001" +} +``` + +### Tool Use Block Structure + +| Field | Type | Required | Description | +|-------|------|----------|-------------| +| `type` | `"tool_use"` | Yes | Block type identifier | +| `id` | string | Yes | Unique tool call ID (format: `toolu_*`) | +| `name` | string | Yes | Tool name | +| `input` | object | Yes | Tool parameters | + +### Common Tool Names + +| Tool | Purpose | Key Input Fields | +|------|---------|------------------| +| `Read` | Read file | `file_path`, `offset`, `limit` | +| `Edit` | Edit file | `file_path`, `old_string`, `new_string` | +| `Write` | Create file | `file_path`, `content` | +| `Bash` | Run command | `command`, `timeout` | +| `Glob` | Find files | `pattern`, `path` | +| `Grep` | Search content | `pattern`, `path`, `type` | +| `WebFetch` | Fetch URL | `url`, `prompt` | +| `WebSearch` | Search web | `query` | +| `Task` | Spawn subagent | `prompt`, `subagent_type` | +| `AskUserQuestion` | Ask user | `questions` | + +## Phase 2: Hook Execution (Optional) + +If hooks are configured, progress events are logged: + +### PreToolUse Hook Input +```json +{ + "session_id": "abc123", + "transcript_path": "/Users/.../.claude/projects/.../session.jsonl", + "cwd": "/Users/dev/myproject", + "permission_mode": "default", + "hook_event_name": "PreToolUse", + "tool_name": "Read", + "tool_input": { + "file_path": "/src/auth/login.ts" + } +} +``` + +### Hook Progress Event +```json +{ + "type": "progress", + "data": { + "type": "hook_progress", + "hookEvent": "PreToolUse", + "hookName": "security_check", + "status": "running" + }, + "parentToolUseID": "toolu_01ReadFile123", + "toolUseID": "toolu_01ReadFile123", + "uuid": "prog-001" +} +``` + +### Hook Output (Decision) +```json +{ + "decision": "allow", + "reason": "File read permitted", + "additionalContext": "Note: This file was recently modified" +} +``` + +## Phase 3: Tool Result + +Tool output is wrapped in a user message: + +```json +{ + "type": "user", + "message": { + "role": "user", + "content": [ + { + "type": "tool_result", + "tool_use_id": "toolu_01ReadFile123", + "content": "1\texport async function login(email: string, password: string) {\n2\t const user = await db.users.findByEmail(email);\n..." + } + ] + }, + "uuid": "msg-002", + "parentUuid": "msg-001" +} +``` + +### Tool Result Block Structure + +| Field | Type | Required | Description | +|-------|------|----------|-------------| +| `type` | `"tool_result"` | Yes | Block type identifier | +| `tool_use_id` | string | Yes | Matches `tool_use.id` | +| `content` | string | Yes | Tool output | +| `is_error` | boolean | No | True if tool failed | + +### Error Results +```json +{ + "type": "tool_result", + "tool_use_id": "toolu_01ReadFile123", + "content": "Error: File not found: /src/auth/login.ts", + "is_error": true +} +``` + +## Phase 4: Result Processing + +Claude processes the result and continues: + +```json +{ + "type": "assistant", + "message": { + "content": [ + { + "type": "thinking", + "thinking": "The login function looks correct. The issue might be in the middleware..." + }, + { + "type": "text", + "text": "I see the login function. Let me check the middleware next." + }, + { + "type": "tool_use", + "id": "toolu_01ReadMiddleware", + "name": "Read", + "input": {"file_path": "/src/auth/middleware.ts"} + } + ] + }, + "uuid": "msg-003", + "parentUuid": "msg-002" +} +``` + +## Parallel Tool Calls + +Multiple tools can be invoked in a single message: + +```json +{ + "type": "assistant", + "message": { + "content": [ + {"type": "tool_use", "id": "toolu_01A", "name": "Grep", "input": {"pattern": "TODO"}}, + {"type": "tool_use", "id": "toolu_01B", "name": "Grep", "input": {"pattern": "FIXME"}}, + {"type": "tool_use", "id": "toolu_01C", "name": "Glob", "input": {"pattern": "**/*.test.ts"}} + ] + } +} +``` + +Results come back in the same user message: + +```json +{ + "type": "user", + "message": { + "content": [ + {"type": "tool_result", "tool_use_id": "toolu_01A", "content": "Found 15 TODOs"}, + {"type": "tool_result", "tool_use_id": "toolu_01B", "content": "Found 3 FIXMEs"}, + {"type": "tool_result", "tool_use_id": "toolu_01C", "content": "tests/auth.test.ts\ntests/api.test.ts"} + ] + } +} +``` + +## Codex Tool Format + +Codex uses a different structure: + +### Function Call +```json +{ + "type": "response_item", + "payload": { + "type": "function_call", + "call_id": "call_abc123", + "name": "Read", + "arguments": "{\"file_path\": \"/src/auth/login.ts\"}" + } +} +``` + +**Note:** `arguments` is a JSON string that needs parsing. + +### Function Result +```json +{ + "type": "response_item", + "payload": { + "type": "function_call_result", + "call_id": "call_abc123", + "result": "File contents..." + } +} +``` + +## Tool Call Pairing + +To reconstruct tool call history: + +1. **Find tool_use blocks** in assistant messages +2. **Match by ID** to tool_result blocks in following user messages +3. **Handle parallel calls** — multiple tool_use can have multiple tool_result + +```python +# Example: Pairing tool calls with results +tool_calls = {} + +for line in jsonl_file: + event = json.loads(line) + + if event['type'] == 'assistant': + for block in event['message']['content']: + if block['type'] == 'tool_use': + tool_calls[block['id']] = { + 'name': block['name'], + 'input': block['input'], + 'timestamp': event['timestamp'] + } + + elif event['type'] == 'user': + content = event['message']['content'] + if isinstance(content, list): + for block in content: + if block['type'] == 'tool_result': + call_id = block['tool_use_id'] + if call_id in tool_calls: + tool_calls[call_id]['result'] = block['content'] + tool_calls[call_id]['is_error'] = block.get('is_error', False) +``` + +## Missing Tool Results + +Edge cases where tool results may be absent: + +1. **Session interrupted** — User closed session mid-tool +2. **Tool timeout** — Long-running tool exceeded limits +3. **Hook blocked** — PreToolUse hook returned `block` +4. **Permission denied** — User denied tool permission + +Handle by checking if tool_use has matching tool_result before assuming completion. + +## Tool-Specific Formats + +### Bash Tool +```json +{ + "type": "tool_use", + "name": "Bash", + "input": { + "command": "npm test -- --coverage", + "timeout": 120000, + "description": "Run tests with coverage" + } +} +``` + +Result includes exit code context: +```json +{ + "type": "tool_result", + "content": "PASS src/auth.test.ts\n...\nCoverage: 85%\n\n[Exit code: 0]" +} +``` + +### Task Tool (Subagent) +```json +{ + "type": "tool_use", + "name": "Task", + "input": { + "description": "Research auth patterns", + "prompt": "Explore authentication implementations...", + "subagent_type": "Explore" + } +} +``` + +Result returns subagent output: +```json +{ + "type": "tool_result", + "content": "## Research Findings\n\n1. JWT patterns...\n\nagentId: agent-abc123" +} +``` diff --git a/docs/claude-jsonl-reference/04-subagent-teams.md b/docs/claude-jsonl-reference/04-subagent-teams.md new file mode 100644 index 0000000..43afb5a --- /dev/null +++ b/docs/claude-jsonl-reference/04-subagent-teams.md @@ -0,0 +1,363 @@ +# Subagent and Team Message Formats + +Documentation for spawned agents, team coordination, and inter-agent messaging. + +## Subagent Overview + +Subagents are spawned via the `Task` tool and run in separate processes with their own transcripts. + +### Spawn Relationship + +``` +Main Session (session-uuid.jsonl) +├── User message +├── Assistant: Task tool_use +├── [Subagent executes in separate process] +├── User message: tool_result with subagent output +└── ... + +Subagent Session (session-uuid/subagents/agent-id.jsonl) +├── Subagent receives prompt +├── Subagent works (tool calls, etc.) +└── Subagent returns result +``` + +## Task Tool Invocation + +Spawning a subagent: + +```json +{ + "type": "assistant", + "message": { + "content": [ + { + "type": "tool_use", + "id": "toolu_01TaskSpawn", + "name": "Task", + "input": { + "description": "Research auth patterns", + "prompt": "Investigate authentication implementations in the codebase. Focus on JWT handling and session management.", + "subagent_type": "Explore", + "run_in_background": false + } + } + ] + } +} +``` + +### Task Tool Input Fields + +| Field | Type | Required | Description | +|-------|------|----------|-------------| +| `description` | string | Yes | Short (3-5 word) description | +| `prompt` | string | Yes | Full task instructions | +| `subagent_type` | string | Yes | Agent type (Explore, Plan, etc.) | +| `run_in_background` | boolean | No | Run async without waiting | +| `model` | string | No | Override model (sonnet, opus, haiku) | +| `isolation` | string | No | "worktree" for isolated git copy | +| `team_name` | string | No | Team to join | +| `name` | string | No | Agent display name | + +### Subagent Types + +| Type | Tools Available | Use Case | +|------|-----------------|----------| +| `Explore` | Read-only tools | Research, search, analyze | +| `Plan` | Read-only tools | Design implementation plans | +| `general-purpose` | All tools | Full implementation | +| `claude-code-guide` | Docs tools | Answer Claude Code questions | +| Custom agents | Defined in `.claude/agents/` | Project-specific | + +## Subagent Transcript Location + +``` +~/.claude/projects///subagents/agent-.jsonl +``` + +## Subagent Message Format + +Subagent transcripts have additional context fields: + +```json +{ + "parentUuid": null, + "isSidechain": true, + "userType": "external", + "cwd": "/Users/dev/myproject", + "sessionId": "subagent-session-uuid", + "version": "2.1.20", + "gitBranch": "main", + "agentId": "a3fecd5", + "type": "user", + "message": { + "role": "user", + "content": "Investigate authentication implementations..." + }, + "uuid": "msg-001", + "timestamp": "2026-02-27T10:00:00.000Z" +} +``` + +### Key Differences from Main Session + +| Field | Main Session | Subagent Session | +|-------|--------------|------------------| +| `isSidechain` | `false` | `true` | +| `agentId` | absent | present | +| `sessionId` | main session UUID | subagent session UUID | + +## Task Result + +When subagent completes, result returns to main session: + +```json +{ + "type": "user", + "message": { + "content": [ + { + "type": "tool_result", + "tool_use_id": "toolu_01TaskSpawn", + "content": "## Authentication Research Findings\n\n### JWT Implementation\n- Located in src/auth/jwt.ts\n- Uses RS256 algorithm\n...\n\nagentId: a3fecd5 (for resuming)" + } + ] + } +} +``` + +**Note:** Result includes `agentId` for potential resumption. + +## Background Tasks + +For `run_in_background: true`: + +```json +{ + "type": "tool_use", + "name": "Task", + "input": { + "prompt": "Run comprehensive test suite", + "subagent_type": "general-purpose", + "run_in_background": true + } +} +``` + +Immediate result with task ID: +```json +{ + "type": "tool_result", + "content": "Background task started.\nTask ID: task-abc123\nUse TaskOutput tool to check status." +} +``` + +## Team Coordination + +Teams enable multiple agents working together. + +### Team Creation (TeamCreate Tool) + +```json +{ + "type": "tool_use", + "name": "TeamCreate", + "input": { + "team_name": "auth-refactor", + "description": "Refactoring authentication system" + } +} +``` + +### Team Config File + +Created at `~/.claude/teams//config.json`: + +```json +{ + "team_name": "auth-refactor", + "description": "Refactoring authentication system", + "created_at": "2026-02-27T10:00:00.000Z", + "members": [ + { + "name": "team-lead", + "agentId": "agent-lead-123", + "agentType": "general-purpose" + }, + { + "name": "researcher", + "agentId": "agent-research-456", + "agentType": "Explore" + } + ] +} +``` + +### Spawning Team Members + +```json +{ + "type": "tool_use", + "name": "Task", + "input": { + "prompt": "Research existing auth implementations", + "subagent_type": "Explore", + "team_name": "auth-refactor", + "name": "researcher" + } +} +``` + +## Inter-Agent Messaging (SendMessage) + +### Direct Message +```json +{ + "type": "tool_use", + "name": "SendMessage", + "input": { + "type": "message", + "recipient": "researcher", + "content": "Please focus on JWT refresh token handling", + "summary": "JWT refresh priority" + } +} +``` + +### Broadcast to Team +```json +{ + "type": "tool_use", + "name": "SendMessage", + "input": { + "type": "broadcast", + "content": "Critical: Found security vulnerability in token validation", + "summary": "Security alert" + } +} +``` + +### Shutdown Request +```json +{ + "type": "tool_use", + "name": "SendMessage", + "input": { + "type": "shutdown_request", + "recipient": "researcher", + "content": "Task complete, please wrap up" + } +} +``` + +### Shutdown Response +```json +{ + "type": "tool_use", + "name": "SendMessage", + "input": { + "type": "shutdown_response", + "request_id": "req-abc123", + "approve": true + } +} +``` + +## Hook Events for Subagents + +### SubagentStart Hook Input +```json +{ + "session_id": "main-session-uuid", + "transcript_path": "/path/to/main/session.jsonl", + "hook_event_name": "SubagentStart", + "agent_id": "a3fecd5", + "agent_type": "Explore" +} +``` + +### SubagentStop Hook Input +```json +{ + "session_id": "main-session-uuid", + "hook_event_name": "SubagentStop", + "agent_id": "a3fecd5", + "agent_type": "Explore", + "agent_transcript_path": "/path/to/subagent/agent-a3fecd5.jsonl", + "last_assistant_message": "Research complete. Found 3 auth patterns..." +} +``` + +## AMC Spawn Tracking + +AMC tracks spawned agents through: + +### Pending Spawn Record +```json +// ~/.local/share/amc/pending_spawns/.json +{ + "spawn_id": "550e8400-e29b-41d4-a716-446655440000", + "project_path": "/Users/dev/myproject", + "agent_type": "claude", + "timestamp": 1708872000.123 +} +``` + +### Session State with Spawn ID +```json +// ~/.local/share/amc/sessions/.json +{ + "session_id": "session-uuid", + "agent": "claude", + "project": "myproject", + "spawn_id": "550e8400-e29b-41d4-a716-446655440000", + "zellij_session": "main", + "zellij_pane": "3" +} +``` + +## Resuming Subagents + +Subagents can be resumed using their agent ID: + +```json +{ + "type": "tool_use", + "name": "Task", + "input": { + "description": "Continue auth research", + "prompt": "Continue where you left off", + "subagent_type": "Explore", + "resume": "a3fecd5" + } +} +``` + +The resumed agent receives full previous context. + +## Worktree Isolation + +For isolated code changes: + +```json +{ + "type": "tool_use", + "name": "Task", + "input": { + "prompt": "Refactor auth module", + "subagent_type": "general-purpose", + "isolation": "worktree" + } +} +``` + +Creates temporary git worktree at `.claude/worktrees//`. + +Result includes worktree info: +```json +{ + "type": "tool_result", + "content": "Refactoring complete.\n\nWorktree: .claude/worktrees/auth-refactor\nBranch: claude/auth-refactor-abc123\n\nChanges made - worktree preserved for review." +} +``` diff --git a/docs/claude-jsonl-reference/05-edge-cases.md b/docs/claude-jsonl-reference/05-edge-cases.md new file mode 100644 index 0000000..b0f5507 --- /dev/null +++ b/docs/claude-jsonl-reference/05-edge-cases.md @@ -0,0 +1,475 @@ +# Edge Cases and Error Handling + +Comprehensive guide to edge cases, malformed input handling, and error recovery in Claude JSONL processing. + +## Parsing Edge Cases + +### 1. Invalid JSON Lines + +**Scenario:** Corrupted or truncated JSON line. + +```python +# BAD: Crashes on invalid JSON +for line in file: + data = json.loads(line) # Raises JSONDecodeError + +# GOOD: Skip invalid lines +for line in file: + if not line.strip(): + continue + try: + data = json.loads(line) + except json.JSONDecodeError: + continue # Skip malformed line +``` + +### 2. Content Type Ambiguity + +**Scenario:** User message content can be string OR array. + +```python +# BAD: Assumes string +user_text = message['content'] + +# GOOD: Check type +content = message['content'] +if isinstance(content, str): + user_text = content +elif isinstance(content, list): + # This is tool results, not user input + user_text = None +``` + +### 3. Missing Optional Fields + +**Scenario:** Fields may be absent in older versions. + +```python +# BAD: Assumes field exists +tokens = message['usage']['cache_read_input_tokens'] + +# GOOD: Safe access +usage = message.get('usage', {}) +tokens = usage.get('cache_read_input_tokens', 0) +``` + +### 4. Partial File Reads + +**Scenario:** Reading last N bytes may cut first line. + +```python +# When seeking to end - N bytes, first line may be partial +def read_tail(file_path, max_bytes=1_000_000): + with open(file_path, 'r') as f: + f.seek(0, 2) # End + size = f.tell() + + if size > max_bytes: + f.seek(size - max_bytes) + f.readline() # Discard partial first line + else: + f.seek(0) + + return f.readlines() +``` + +### 5. Non-Dict JSON Values + +**Scenario:** Line contains valid JSON but not an object. + +```python +# File might contain: 123, "string", [1,2,3], null +data = json.loads(line) +if not isinstance(data, dict): + continue # Skip non-object JSON +``` + +## Type Coercion Edge Cases + +### Integer Conversion + +```python +def safe_int(value): + """Convert to int, rejecting booleans.""" + # Python: isinstance(True, int) == True, so check explicitly + if isinstance(value, bool): + return None + if isinstance(value, int): + return value + if isinstance(value, float): + return int(value) + if isinstance(value, str): + try: + return int(value) + except ValueError: + return None + return None +``` + +### Token Summation + +```python +def sum_tokens(*values): + """Sum token counts, handling None/missing.""" + valid = [v for v in values if isinstance(v, (int, float)) and not isinstance(v, bool)] + return sum(valid) if valid else None +``` + +## Session State Edge Cases + +### 1. Orphan Sessions + +**Scenario:** Multiple sessions claim same Zellij pane (e.g., after --resume). + +**Resolution:** Keep session with: +1. Highest priority: Has `context_usage` (indicates real work) +2. Second priority: Latest `conversation_mtime_ns` + +```python +def dedupe_sessions(sessions): + by_pane = {} + for s in sessions: + key = (s['zellij_session'], s['zellij_pane']) + if key not in by_pane: + by_pane[key] = s + else: + existing = by_pane[key] + # Prefer session with context_usage + if s.get('context_usage') and not existing.get('context_usage'): + by_pane[key] = s + elif s.get('conversation_mtime_ns', 0) > existing.get('conversation_mtime_ns', 0): + by_pane[key] = s + return list(by_pane.values()) +``` + +### 2. Dead Session Detection + +**Claude:** Check Zellij session exists +```python +def is_claude_dead(session): + if session['status'] == 'starting': + return False # Benefit of doubt + + zellij = session.get('zellij_session') + if not zellij: + return True + + # Check if Zellij session exists + result = subprocess.run(['zellij', 'list-sessions'], capture_output=True) + return zellij not in result.stdout.decode() +``` + +**Codex:** Check if process has file open +```python +def is_codex_dead(session): + transcript = session.get('transcript_path') + if not transcript: + return True + + # Check if any process has file open + result = subprocess.run(['lsof', transcript], capture_output=True) + return result.returncode != 0 +``` + +### 3. Stale Session Cleanup + +```python +ORPHAN_AGE_HOURS = 24 +STARTING_AGE_HOURS = 1 + +def should_cleanup(session, now): + age = now - session['started_at'] + + if session['status'] == 'starting' and age > timedelta(hours=STARTING_AGE_HOURS): + return True # Stuck in starting + + if session.get('is_dead') and age > timedelta(hours=ORPHAN_AGE_HOURS): + return True # Dead and old + + return False +``` + +## Tool Call Edge Cases + +### 1. Missing Tool Results + +**Scenario:** Session interrupted between tool_use and tool_result. + +```python +def pair_tool_calls(messages): + pending = {} # tool_use_id -> tool_use + + for msg in messages: + if msg['type'] == 'assistant': + for block in msg['message'].get('content', []): + if block.get('type') == 'tool_use': + pending[block['id']] = block + + elif msg['type'] == 'user': + content = msg['message'].get('content', []) + if isinstance(content, list): + for block in content: + if block.get('type') == 'tool_result': + tool_id = block.get('tool_use_id') + if tool_id in pending: + pending[tool_id]['result'] = block + + # Any pending without result = interrupted + incomplete = [t for t in pending.values() if 'result' not in t] + return pending, incomplete +``` + +### 2. Parallel Tool Call Ordering + +**Scenario:** Multiple tool_use in one message, results may come in different order. + +```python +# Match by ID, not by position +tool_uses = [b for b in assistant_content if b['type'] == 'tool_use'] +tool_results = [b for b in user_content if b['type'] == 'tool_result'] + +paired = {} +for result in tool_results: + paired[result['tool_use_id']] = result + +for use in tool_uses: + result = paired.get(use['id']) + # result may be None if missing +``` + +### 3. Tool Error Results + +```python +def is_tool_error(result_block): + return result_block.get('is_error', False) + +def extract_error_message(result_block): + content = result_block.get('content', '') + if content.startswith('Error:'): + return content + return None +``` + +## Codex-Specific Edge Cases + +### 1. Content Injection Filtering + +Codex may include system context in messages that should be filtered: + +```python +SKIP_PREFIXES = [ + '', + '', + '', + '# AGENTS.md instructions' +] + +def should_skip_content(text): + return any(text.startswith(prefix) for prefix in SKIP_PREFIXES) +``` + +### 2. Developer Role Filtering + +```python +def parse_codex_message(payload): + role = payload.get('role') + if role == 'developer': + return None # Skip system/developer messages + return payload +``` + +### 3. Function Call Arguments Parsing + +```python +def parse_arguments(arguments): + if isinstance(arguments, dict): + return arguments + if isinstance(arguments, str): + try: + return json.loads(arguments) + except json.JSONDecodeError: + return {'raw': arguments} + return {} +``` + +### 4. Tool Call Buffering + +Codex tool calls need buffering until next message: + +```python +class CodexParser: + def __init__(self): + self.pending_tools = [] + + def process_entry(self, entry): + payload = entry.get('payload', {}) + ptype = payload.get('type') + + if ptype == 'function_call': + self.pending_tools.append({ + 'name': payload['name'], + 'input': self.parse_arguments(payload['arguments']) + }) + return None # Don't emit yet + + elif ptype == 'message' and payload.get('role') == 'assistant': + msg = self.create_message(payload) + if self.pending_tools: + msg['tool_calls'] = self.pending_tools + self.pending_tools = [] + return msg + + elif ptype == 'message' and payload.get('role') == 'user': + # Flush pending tools before user message + msgs = [] + if self.pending_tools: + msgs.append({'role': 'assistant', 'tool_calls': self.pending_tools}) + self.pending_tools = [] + msgs.append(self.create_message(payload)) + return msgs +``` + +## File System Edge Cases + +### 1. Path Traversal Prevention + +```python +import os + +def validate_session_id(session_id): + # Must be basename only + if os.path.basename(session_id) != session_id: + raise ValueError("Invalid session ID") + + # No special characters + if any(c in session_id for c in ['/', '\\', '..', '\x00']): + raise ValueError("Invalid session ID") + +def validate_project_path(project_path, base_dir): + resolved = os.path.realpath(project_path) + base = os.path.realpath(base_dir) + + if not resolved.startswith(base + os.sep): + raise ValueError("Path traversal detected") +``` + +### 2. File Not Found + +```python +def read_session_file(path): + try: + with open(path, 'r') as f: + return f.read() + except FileNotFoundError: + return None + except PermissionError: + return None + except OSError: + return None +``` + +### 3. Empty Files + +```python +def parse_jsonl(path): + with open(path, 'r') as f: + content = f.read() + + if not content.strip(): + return [] # Empty file + + return [json.loads(line) for line in content.strip().split('\n') if line.strip()] +``` + +## Subprocess Edge Cases + +### 1. Timeout Handling + +```python +import subprocess + +def run_with_timeout(cmd, timeout=5): + try: + result = subprocess.run( + cmd, + capture_output=True, + timeout=timeout, + text=True + ) + return result.stdout + except subprocess.TimeoutExpired: + return None + except FileNotFoundError: + return None + except OSError: + return None +``` + +### 2. ANSI Code Stripping + +```python +import re + +ANSI_PATTERN = re.compile(r'\x1b\[[0-9;]*m') + +def strip_ansi(text): + return ANSI_PATTERN.sub('', text) +``` + +## Cache Invalidation + +### Mtime-Based Cache + +```python +class FileCache: + def __init__(self, max_size=100): + self.cache = {} + self.max_size = max_size + + def get(self, path): + if path not in self.cache: + return None + + entry = self.cache[path] + stat = os.stat(path) + + # Invalidate if file changed + if stat.st_mtime_ns != entry['mtime_ns'] or stat.st_size != entry['size']: + del self.cache[path] + return None + + return entry['data'] + + def set(self, path, data): + # Evict oldest if full + if len(self.cache) >= self.max_size: + oldest = next(iter(self.cache)) + del self.cache[oldest] + + stat = os.stat(path) + self.cache[path] = { + 'mtime_ns': stat.st_mtime_ns, + 'size': stat.st_size, + 'data': data + } +``` + +## Testing Edge Cases Checklist + +- [ ] Empty JSONL file +- [ ] Single-line JSONL file +- [ ] Truncated JSON line +- [ ] Non-object JSON values (numbers, strings, arrays) +- [ ] Missing required fields +- [ ] Unknown message types +- [ ] Content as string vs array +- [ ] Boolean vs integer confusion +- [ ] Unicode in content +- [ ] Very long lines (>64KB) +- [ ] Concurrent file modifications +- [ ] Missing tool results +- [ ] Multiple tool calls in single message +- [ ] Session without Zellij pane +- [ ] Codex developer messages +- [ ] Path traversal attempts +- [ ] Symlink escape attempts diff --git a/docs/claude-jsonl-reference/06-quick-reference.md b/docs/claude-jsonl-reference/06-quick-reference.md new file mode 100644 index 0000000..52784da --- /dev/null +++ b/docs/claude-jsonl-reference/06-quick-reference.md @@ -0,0 +1,238 @@ +# Quick Reference + +Cheat sheet for common Claude JSONL operations. + +## File Locations + +```bash +# Claude sessions +~/.claude/projects/-Users-user-projects-myapp/*.jsonl + +# Codex sessions +~/.codex/sessions/**/*.jsonl + +# Subagent transcripts +~/.claude/projects/.../session-id/subagents/agent-*.jsonl + +# AMC session state +~/.local/share/amc/sessions/*.json +``` + +## Path Encoding + +```python +# Encode: /Users/dev/myproject -> -Users-dev-myproject +encoded = '-' + project_path.replace('/', '-') + +# Decode: -Users-dev-myproject -> /Users/dev/myproject +decoded = encoded[1:].replace('-', '/') +``` + +## Message Type Quick ID + +| If you see... | It's a... | +|---------------|-----------| +| `"type": "user"` + string content | User input | +| `"type": "user"` + array content | Tool results | +| `"type": "assistant"` | Claude response | +| `"type": "progress"` | Hook/tool execution | +| `"type": "summary"` | Session summary | +| `"type": "system"` | Metadata/commands | + +## Content Block Quick ID + +| Block Type | Key Fields | +|------------|------------| +| `text` | `text` | +| `thinking` | `thinking`, `signature` | +| `tool_use` | `id`, `name`, `input` | +| `tool_result` | `tool_use_id`, `content`, `is_error` | + +## jq Recipes + +```bash +# Count messages by type +jq -s 'group_by(.type) | map({type: .[0].type, count: length})' session.jsonl + +# Extract all tool calls +jq -c 'select(.type=="assistant") | .message.content[]? | select(.type=="tool_use")' session.jsonl + +# Get user messages only +jq -c 'select(.type=="user" and (.message.content | type)=="string")' session.jsonl + +# Sum tokens +jq -s '[.[].message.usage? | select(.) | .input_tokens + .output_tokens] | add' session.jsonl + +# List tools used +jq -c 'select(.type=="assistant") | .message.content[]? | select(.type=="tool_use") | .name' session.jsonl | sort | uniq -c + +# Find errors +jq -c 'select(.type=="user") | .message.content[]? | select(.type=="tool_result" and .is_error==true)' session.jsonl +``` + +## Python Snippets + +### Read JSONL +```python +import json + +def read_jsonl(path): + with open(path) as f: + for line in f: + if line.strip(): + try: + yield json.loads(line) + except json.JSONDecodeError: + continue +``` + +### Extract Conversation +```python +def extract_conversation(path): + messages = [] + for event in read_jsonl(path): + if event['type'] == 'user': + content = event['message']['content'] + if isinstance(content, str): + messages.append({'role': 'user', 'content': content}) + elif event['type'] == 'assistant': + for block in event['message'].get('content', []): + if block.get('type') == 'text': + messages.append({'role': 'assistant', 'content': block['text']}) + return messages +``` + +### Get Token Usage +```python +def get_token_usage(path): + total_input = 0 + total_output = 0 + + for event in read_jsonl(path): + if event['type'] == 'assistant': + usage = event.get('message', {}).get('usage', {}) + total_input += usage.get('input_tokens', 0) + total_output += usage.get('output_tokens', 0) + + return {'input': total_input, 'output': total_output} +``` + +### Find Tool Calls +```python +def find_tool_calls(path): + tools = [] + for event in read_jsonl(path): + if event['type'] == 'assistant': + for block in event['message'].get('content', []): + if block.get('type') == 'tool_use': + tools.append({ + 'name': block['name'], + 'id': block['id'], + 'input': block['input'] + }) + return tools +``` + +### Pair Tools with Results +```python +def pair_tools_results(path): + pending = {} + + for event in read_jsonl(path): + if event['type'] == 'assistant': + for block in event['message'].get('content', []): + if block.get('type') == 'tool_use': + pending[block['id']] = {'use': block, 'result': None} + + elif event['type'] == 'user': + content = event['message'].get('content', []) + if isinstance(content, list): + for block in content: + if block.get('type') == 'tool_result': + tool_id = block['tool_use_id'] + if tool_id in pending: + pending[tool_id]['result'] = block + + return pending +``` + +## Common Gotchas + +| Gotcha | Solution | +|--------|----------| +| `content` can be string or array | Check `isinstance(content, str)` first | +| `usage` may be missing | Use `.get('usage', {})` | +| Booleans are ints in Python | Check `isinstance(v, bool)` before `isinstance(v, int)` | +| First line may be partial after seek | Call `readline()` to discard | +| Tool results in user messages | Check for `tool_result` type in array | +| Codex `arguments` is JSON string | Parse with `json.loads()` | +| Agent ID vs session ID | Agent ID survives rewrites, session ID is per-run | + +## Status Values + +| Field | Values | +|-------|--------| +| `status` | `starting`, `active`, `done` | +| `stop_reason` | `end_turn`, `max_tokens`, `tool_use`, null | +| `is_error` | `true`, `false` (tool results) | + +## Token Fields + +```python +# All possible token fields to sum +token_fields = [ + 'input_tokens', + 'output_tokens', + 'cache_creation_input_tokens', + 'cache_read_input_tokens' +] + +# Context window by model +context_windows = { + 'claude-opus': 200_000, + 'claude-sonnet': 200_000, + 'claude-haiku': 200_000, + 'claude-2': 100_000 +} +``` + +## Useful Constants + +```python +# File locations +CLAUDE_BASE = os.path.expanduser('~/.claude/projects') +CODEX_BASE = os.path.expanduser('~/.codex/sessions') +AMC_BASE = os.path.expanduser('~/.local/share/amc') + +# Read limits +MAX_TAIL_BYTES = 1_000_000 # 1MB +MAX_LINES = 400 # For context extraction + +# Timeouts +SUBPROCESS_TIMEOUT = 5 # seconds +SPAWN_COOLDOWN = 30 # seconds + +# Session ages +ACTIVE_THRESHOLD_MINUTES = 2 +ORPHAN_CLEANUP_HOURS = 24 +STARTING_CLEANUP_HOURS = 1 +``` + +## Debugging Commands + +```bash +# Watch session file changes +tail -f ~/.claude/projects/-path-to-project/*.jsonl | jq -c + +# Find latest session +ls -t ~/.claude/projects/-path-to-project/*.jsonl | head -1 + +# Count lines in session +wc -l session.jsonl + +# Validate JSON +cat session.jsonl | while read line; do echo "$line" | jq . > /dev/null || echo "Invalid: $line"; done + +# Pretty print last message +tail -1 session.jsonl | jq . +``` diff --git a/docs/claude-jsonl-reference/README.md b/docs/claude-jsonl-reference/README.md new file mode 100644 index 0000000..b81ad9f --- /dev/null +++ b/docs/claude-jsonl-reference/README.md @@ -0,0 +1,57 @@ +# Claude JSONL Session Log Reference + +Comprehensive documentation for parsing and processing Claude Code JSONL session logs in the AMC project. + +## Overview + +Claude Code stores all conversations as JSONL (JSON Lines) files — one JSON object per line. This documentation provides authoritative specifications for: + +- Message envelope structure and common fields +- All message types (user, assistant, progress, system, summary, etc.) +- Content block types (text, tool_use, tool_result, thinking) +- Tool call lifecycle and result handling +- Subagent spawn and team coordination formats +- Edge cases, error handling, and recovery patterns + +## Documents + +| Document | Purpose | +|----------|---------| +| [01-format-specification.md](./01-format-specification.md) | Complete JSONL format spec with all fields | +| [02-message-types.md](./02-message-types.md) | Every message type with concrete examples | +| [03-tool-lifecycle.md](./03-tool-lifecycle.md) | Tool call flow from invocation to result | +| [04-subagent-teams.md](./04-subagent-teams.md) | Subagent and team message formats | +| [05-edge-cases.md](./05-edge-cases.md) | Error handling, malformed input, recovery | +| [06-quick-reference.md](./06-quick-reference.md) | Cheat sheet for common operations | + +## File Locations + +| Content | Location | +|---------|----------| +| Claude sessions | `~/.claude/projects//.jsonl` | +| Codex sessions | `~/.codex/sessions/**/.jsonl` | +| Subagent transcripts | `~/.claude/projects///subagents/agent-.jsonl` | +| AMC session state | `~/.local/share/amc/sessions/.json` | +| AMC event logs | `~/.local/share/amc/events/.jsonl` | + +## Path Encoding + +Project paths are encoded by replacing `/` with `-` and adding a leading dash: +- `/Users/taylor/projects/amc` → `-Users-taylor-projects-amc` + +## Key Principles + +1. **NDJSON format** — Each line is complete, parseable JSON +2. **Append-only** — Sessions are written incrementally +3. **UUID linking** — Messages link via `uuid` and `parentUuid` +4. **Graceful degradation** — Always handle missing/unknown fields +5. **Type safety** — Validate types before use (arrays vs strings, etc.) + +## Sources + +- [Claude Code Hooks Reference](https://code.claude.com/docs/en/hooks.md) +- [Claude Code Headless Documentation](https://code.claude.com/docs/en/headless.md) +- [Anthropic Messages API Reference](https://docs.anthropic.com/en/api/messages) +- [Inside Claude Code: Session File Format](https://medium.com/@databunny/inside-claude-code-the-session-file-format-and-how-to-inspect-it-b9998e66d56b) +- [Community: claude-code-log](https://github.com/daaain/claude-code-log) +- [Community: claude-JSONL-browser](https://github.com/withLinda/claude-JSONL-browser)