Skip to content

BLOCK_TAGGING_API.md

Latest commit

 

History

History
288 lines (232 loc) · 6.46 KB

BLOCK_TAGGING_API.md

File metadata and controls

288 lines (232 loc) · 6.46 KB
Raw

Block-Level Tagging API Reference

This document describes the codex-blocks.json index format and how to consume block-level tag data.

Index File: codex-blocks.json

The compiled block index is generated by scripts/build-index.mjs and published to the repository root.

Fetch Location

// Static exports (quarry.space)
const url = '/assets/codex-blocks.json'

// CDN/GitHub raw
const url = 'https://raw.githubusercontent.com/framerslab/codex/main/codex-blocks.json'

Index Structure

{
  "generatedAt": "2024-01-15T10:30:00.000Z",
  "version": "1.0.0",
  "stats": {
    "totalStrands": 150,
    "totalBlocks": 2340,
    "totalTags": 5670,
    "uniqueTags": 234,
    "worthyBlocks": 1890,
    "pendingSuggestions": 456,
    "tagsBySource": {
      "nlp": 3200,
      "llm": 890,
      "existing": 1200,
      "user": 380
    },
    "blocksByType": {
      "heading": 890,
      "paragraph": 1200,
      "code": 150,
      "list": 80,
      "blockquote": 20
    }
  },
  "tagIndex": {
    "javascript": [
      { "strandPath": "weaves/technology/js-basics.md", "blockId": "introduction", "confidence": 1.0 },
      { "strandPath": "weaves/technology/js-basics.md", "blockId": "block-45", "confidence": 0.85 }
    ]
  },
  "strands": {
    "weaves/technology/js-basics.md": {
      "path": "weaves/technology/js-basics.md",
      "title": "JavaScript Basics",
      "blockCount": 15,
      "tagCount": 42,
      "worthyBlockCount": 12,
      "blocks": [
        {
          "id": "introduction",
          "line": 5,
          "endLine": 12,
          "type": "heading",
          "headingLevel": 2,
          "headingText": "Introduction",
          "tags": ["javascript", "getting-started"],
          "suggestedTags": [],
          "worthiness": { "score": 0.72 }
        }
      ]
    }
  }
}

TypeScript Types

interface CodexBlocksIndex {
  generatedAt: string        // ISO 8601 timestamp
  version: string            // Semver version
  stats: BlockIndexStats
  tagIndex: Record<string, TagReference[]>
  strands: Record<string, StrandBlockData>
}

interface BlockIndexStats {
  totalStrands: number
  totalBlocks: number
  totalTags: number
  uniqueTags: number
  worthyBlocks: number
  pendingSuggestions: number
  tagsBySource: Record<'nlp' | 'llm' | 'existing' | 'user', number>
  blocksByType: Record<string, number>
}

interface TagReference {
  strandPath: string
  blockId: string
  confidence: number
}

interface StrandBlockData {
  path: string
  title: string
  blockCount: number
  tagCount: number
  worthyBlockCount: number
  blocks: BlockEntry[]
}

interface BlockEntry {
  id: string
  line: number
  endLine?: number
  type: 'heading' | 'paragraph' | 'code' | 'list' | 'blockquote' | 'table' | 'html'
  headingLevel?: number
  headingText?: string
  tags: string[]
  suggestedTags: SuggestedTag[]
  worthiness?: { score: number; signals?: Record<string, number> }
  extractiveSummary?: string
  warrantsIllustration?: boolean
}

interface SuggestedTag {
  tag: string
  confidence: number
  source: 'nlp' | 'llm' | 'existing' | 'user'
  reasoning?: string
}

React Hook Usage

Frame.dev provides a React hook for consuming block data:

import { useBlockTags, hasBlocksInIndex, getAllBlockTags } from '@/lib/hooks/useBlockTags'

function MyComponent({ strandPath }) {
  const { 
    blocks,           // Block array
    isLoading,        // Loading state
    error,            // Error object
    stats,            // { total, tagged, pending, worthy }
    getBlockById,     // (blockId) => BlockEntry
    refetch           // () => Promise<void>
  } = useBlockTags(strandPath)

  if (isLoading) return <Spinner />
  if (error) return <Error message={error.message} />

  return (
    <ul>
      {blocks.map(block => (
        <li key={block.blockId}>
          {block.headingText || block.blockId}
          {block.tags.map(tag => <Tag key={tag}>{tag}</Tag>)}
        </li>
      ))}
    </ul>
  )
}

Utility Functions

Check if strand has blocks

const hasBlocks = await hasBlocksInIndex('weaves/technology/js-basics.md')
// true | false

Get all unique block tags

const allTags = await getAllBlockTags()
// ['javascript', 'typescript', 'react', ...]

Search blocks by tag

const results = await searchBlocksByTag('javascript')
// [{ strandPath, blockId, confidence }, ...]

Get index stats

const stats = await getBlockIndexStats()
// { totalStrands, totalBlocks, ... }

Direct API (Server Mode)

When running with API routes enabled, these endpoints are available:

GET /api/blocks

Fetch blocks for a strand:

curl "/api/blocks?strandPath=weaves/technology/js-basics.md"

Response:

{
  "blocks": [
    { "blockId": "introduction", "tags": ["javascript"], ... }
  ]
}

POST /api/blocks/tags

Update block tags (requires authentication):

curl -X POST "/api/blocks/tags" \
  -H "Content-Type: application/json" \
  -d '{
    "action": "accept",
    "strandPath": "weaves/technology/js-basics.md",
    "blockId": "introduction",
    "tag": "es6"
  }'

Actions: accept, reject, add, remove

Tag Index Queries

The tagIndex object provides an inverted index for tag-based searches:

// Find all blocks tagged with "typescript"
const index = await fetchBlocksIndex()
const typescriptBlocks = index.tagIndex['typescript'] || []

for (const ref of typescriptBlocks) {
  console.log(`${ref.strandPath} / ${ref.blockId} (${ref.confidence})`)
}

Caching Strategy

The useBlockTags hook implements caching:

  1. Memory Cache: Index cached as singleton
  2. Request Deduplication: Concurrent fetches share same promise
  3. No IndexedDB: Source of truth is codex repo, not browser

To force refresh:

import { clearBlocksIndexCache } from '@/lib/hooks/useBlockTags'

clearBlocksIndexCache()
await refetch()

Error Handling

const { error } = useBlockTags(strandPath)

if (error) {
  if (error.message.includes('404')) {
    // Index not available - strand may not be processed yet
  } else if (error.message.includes('network')) {
    // Network error - show offline indicator
  }
}

See Also