Patterns
Open Knowledge Format
Google Cloud Β· Apache 2.0 Β· v0.1 Draft

Open Knowledge Format

A minimal, interoperable format for knowledge representation in agentic AI systems β€” the missing context layer between raw data and agent reasoning.

What it is: OKF is Google Cloud’s open-standard format for organizational knowledge β€” published June 2026 by Sam McVeety and Amir Hormati, licensed Apache 2.0. Source: GoogleCloudPlatform/knowledge-catalog.

Problem: Your agents don’t know your organization. They can reason, write code, and summarize text β€” but only with the right internal context. That context lives everywhere and belongs nowhere: schema docs in BigQuery, metric definitions in a Confluence wiki, join logic in a senior engineer’s head, runbooks in a shared drive nobody maintains.

Every AI agent builder has been solving this context-assembly problem from scratch. Every data catalog vendor has been reinventing the same data models. The result is a sea of bespoke, siloed, non-portable “LLM wikis” β€” your team’s Obsidian vault works for your agents, but nobody else’s.

Solution: Open Knowledge Format (OKF) β€” the fix isn’t a new platform, it’s a format. A small, stable, agreed-upon set of conventions that makes a knowledge corpus self-describing, so any producer can write it and any consumer can read it.


The Core Principle: Intentionally Minimal

OKF is exactly what it says on the tin β€” markdown files with YAML frontmatter, organized in a directory. No schema registry. No central authority. No required SDK. No proprietary API standing between you and your metadata.

The design bet is explicit: broad, low-friction adoption of a simpler format produces more organizational value than narrow, high-friction adoption of a richer one. One required field. Everything else is optional or convention.


The Missing Knowledge Layer

OKF formalizes the knowledge layer between raw data (data catalogs, databases) and agent reasoning (prompts, LLM execution):

β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚                   Agentic AI Client                    β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                            β–Ό
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚    OKF Context Layer (Self-Describing Memory Graph)    β”‚
β”‚  [index.md] --> [concepts/schema] --> [playbooks/run]  β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                            β–Ό
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚     Raw Data Infrastructure (Databases, API Docs)      β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

An OKF bundle functions as structured external memory β€” persisted, version-controlled, traversable, and agent-navigable via index.md hierarchies. It’s neither a vector store nor a graph database; it’s a knowledge corpus that any agent with file read access can traverse.


Spec Breakdown

Key Terms

TermDefinition
Knowledge BundleA self-contained directory tree of knowledge documents. The unit of distribution.
ConceptA single unit of knowledge β€” one markdown file. Can be a table, an API, a metric, a runbook, an idea.
Concept IDThe file path within the bundle, .md suffix stripped. e.g., tables/users.md β†’ tables/users
FrontmatterYAML metadata block delimited by --- at the top of the file.
BodyEverything after the frontmatter β€” free-form markdown.
LinkA standard markdown link between concepts β€” how the knowledge graph is built.

Bundle Structure

A bundle is a plain directory tree:

path/to/bundle/
β”œβ”€β”€ index.md            <- Optional. Directory listing for progressive disclosure.
β”œβ”€β”€ log.md              <- Optional. Chronological change history.
β”œβ”€β”€ <concept>.md        <- A concept at the root.
└── <subdirectory>/
    β”œβ”€β”€ index.md
    β”œβ”€β”€ <concept>.md
    └── <subdirectory>/
        └── ...

Distribution options (in descending preference):

  • Git repository β€” gives history, attribution, diffs, PR reviews for free
  • Tarball / zip archive
  • Subdirectory within a larger monorepo

Reserved filenames: index.md and log.md have defined meaning at any level and must not be used for concept documents.

Concept Documents

Every concept is a UTF-8 markdown file with two parts: a frontmatter block and a body.

Frontmatter:

---
type: <Type name>          # REQUIRED β€” the only required field
title: <display name>      # Recommended
description: <one-liner>   # Recommended β€” used for index snippets and search
resource: <URI>            # Recommended β€” canonical URI of the underlying asset
tags: [tag1, tag2]         # Optional
timestamp: 2026-05-28T14:30:00Z  # Optional β€” ISO 8601, last meaningful change
# ...any additional producer-defined keys are valid
---

The only required field is type. It’s a free-form string β€” values like BigQuery Table, API Endpoint, Metric, Playbook, Reference are illustrative, not registered. Consumers must tolerate unknown types gracefully.

Body β€” conventional headings:

HeadingPurpose
# SchemaStructured field/column descriptions
# ExamplesFenced code blocks showing usage
# CitationsNumbered external sources backing claims

Cross-Linking

Concepts link to each other via standard markdown links β€” this is how the knowledge graph is formed.

Absolute (bundle-relative) β€” preferred:

See the [customers table](/tables/customers.md) for the join key.

Relative:

See the [neighboring concept](./other.md).

Link semantics are deliberately loose β€” a link from A to B asserts a relationship. The specific kind (references, joins-with, depends-on) is expressed in the surrounding prose. Broken links are valid; a link to a not-yet-written concept is not malformed.

Index Files

index.md can appear at any directory level for progressive disclosure:

# Tables

* [Customer Orders](/tables/orders.md) - one row per completed order
* [Customers](/tables/customers.md) - one row per registered customer

# Datasets

* [Sales Dataset](/datasets/sales.md) - all sales-related tables

Log Files

log.md tracks changes chronologically. Newest entries first:

# Directory Update Log

## 2026-05-22
* **Update**: Added new BigQuery table reference for [Customer Metrics](/tables/customer-metrics.md).
* **Creation**: Established the [Dataplex Playbook](/playbooks/dataplex.md).

Conformance

A bundle is conformant with OKF v0.1 if:

  1. Every non-reserved .md file contains a parseable YAML frontmatter block.
  2. Every frontmatter block contains a non-empty type field.
  3. index.md and log.md, when present, follow the structures above.

Consumers must not reject a bundle because of: missing optional frontmatter fields, unknown type values, unknown additional keys, broken cross-links, or missing index.md files. This tolerance is intentional β€” brittle consumers would poison the ecosystem.


Enterprise Use Case: Customer Success & Data Alignment

Scenario: Align Customer Success runbooks with Data Engineering database schemas, enabling an on-call AI support agent to automatically triage alert escalations.

cs-eng-bundle/
β”œβ”€β”€ index.md
β”œβ”€β”€ log.md
β”œβ”€β”€ datasets/
β”‚   └── customer_behavior.md
β”œβ”€β”€ tables/
β”‚   β”œβ”€β”€ index.md
β”‚   └── subscription_events.md
└── playbooks/
    └── churn_triage.md

1. Root Directory Navigation (index.md)

---
okf_version: "0.1"
title: "Customer Success & Data Engineering Catalog"
description: "Cross-functional knowledge index mapping customer metrics to telemetry tables"
---

# Datasets
* [Customer Behavior Dataset](/datasets/customer_behavior.md) - Aggregates application usage and user states.

# Tables
* [Subscription Events Table](/tables/subscription_events.md) - Transactional logs for stripe and app store events.

# Playbooks
* [Churn Triage Runbook](/playbooks/churn_triage.md) - Actionable steps when customer churn rates spike.

2. Dataset Definition (/datasets/customer_behavior.md)

---
type: Dataset
title: Customer Behavior Data
description: Unified business view tracking user subscription cycles and behaviors.
resource: https://console.cloud.google.com/datacatalog/customer_behavior
tags: [marketing, analytics, customers]
timestamp: 2026-06-15T09:00:00Z
---

# Overview
This dataset contains cleaned, sessionized user events and transactional logs designed
to compute user engagement and retention. It is the authoritative source for the Churn Prediction Model.

# Tables Included
* [Subscription Events](/tables/subscription_events.md) - Captured transactional events for subscription state changes.

3. BigQuery Table Schema (/tables/subscription_events.md)

---
type: BigQuery Table
title: Subscription Events Raw
description: Transactional logs mapping payment state transitions.
resource: https://console.cloud.google.com/bigquery?p=prod-data&d=raw_events&t=subscription_events
tags: [finance, raw-telemetry]
timestamp: 2026-06-17T14:30:00Z
---

# Schema

| Column | Type | Description |
| :--- | :--- | :--- |
| `event_id` | STRING | Unique event identifier. |
| `customer_id` | STRING | Unique system user ID. |
| `event_type` | STRING | Event action: SUBSCRIBE, RENEW, CANCEL, or EXPIRE. |
| `amount_usd` | NUMERIC | Transaction value in USD. |
| `timestamp` | TIMESTAMP | Event creation time (UTC). |

# Examples

## Querying Cancellation Trends
```sql
SELECT
  DATE(timestamp) as event_date,
  COUNT(DISTINCT customer_id) as cancel_count
FROM `prod-data.raw_events.subscription_events`
WHERE event_type = 'CANCEL'
GROUP BY 1 ORDER BY 1 DESC;

Citations

[1] Stripe Webhook Event Mapping Reference


### 4. Incident Response Playbook (`/playbooks/churn_triage.md`)

```markdown
---
type: Playbook
title: Customer Churn Alert Triage
description: Standard operating procedure for investigating sudden increases in subscription cancellations.
tags: [on-call, operations]
timestamp: 2026-06-18T08:00:00Z
---

# Trigger
This playbook fires when hourly CANCEL event spikes exceed the standard 3-sigma threshold.

# Triage Steps

1. **Verify Telemetry Latency:**
   Compare event ingestion time against payload timestamp in [Subscription Events](/tables/subscription_events.md).

2. **Execute Diagnostic Analysis:**
   Run the SQL cancellation trends query from [Subscription Events Examples](/tables/subscription_events.md#examples).

3. **Cross-reference Dataset Metrics:**
   Check if affected cohorts concentrate in specific marketing channels via [Customer Behavior Dataset](/datasets/customer_behavior.md).

4. **Escalate:**
   If logs indicate API communication issues, escalate to the `#billing-eng` Slack channel.

Positioning & Trade-Offs

vs RAG (Retrieval-Augmented Generation)

RAG re-derives knowledge at query time from raw chunks of unstructured content. OKF stores curated, cross-linked, version-controlled concepts that agents read and update directly. Not mutually exclusive β€” OKF bundles can feed a RAG pipeline, but the bundle itself is the authoritative, navigable artifact. RAG misses structured relationships (mapping column customer_id directly to its parent table’s description); OKF provides the pre-structured map.

vs CLAUDE.md / AGENTS.md

CLAUDE.md and AGENTS.md are per-repository agent instruction files β€” they tell an agent how to behave in a specific codebase context. OKF is a knowledge corpus format β€” it captures what things are and how they relate. Complementary, not competing.

vs Data Catalogs (Alation, Collibra)

Enterprise catalogs are heavy databases running on proprietary APIs. OKF doesn’t replace these β€” it references them. A data catalog can generate an OKF bundle representation, check it into Git, and lightweight development agents can consume it easily.

vs Domain-Specific Schemas (Avro, Protobuf, OpenAPI)

OKF doesn’t replace these β€” it references them. An OKF concept for an API endpoint would link to the OpenAPI spec; it wouldn’t try to re-express it in markdown. OKF is the context layer, not the schema layer.


Honest Trade-Off Read

What OKF gets right:

  • Zero friction: cat to read, git clone to ship, any text editor to author.
  • One required field is the right call. Broad adoption of a minimal spec beats narrow adoption of a rich one.
  • Treating knowledge as version-controlled source code inherits diffs, blame, PRs, and review without ceremony.
  • The permissive consumer model avoids the fragility death-spiral that kills most interoperability specs.
  • Decoupling producers from consumers is architecturally sound. Any framework can produce; any consumer can read.

Open design space (v0.1 is a starting point):

  • Contradiction handling β€” Two OKF docs disagree on the same fact. No merge semantics defined yet.
  • Trust and provenance β€” Human-authored vs. agent-generated content looks identical. No trust tier signals.
  • Typed relationships β€” All cross-links are untyped. “Joins with” and “depends on” look the same to a graph consumer.
  • Stale content β€” No staleness signals beyond timestamp. Operational discipline is a process problem, not a format problem.
  • Access control β€” Permissions are filesystem / git permissions β€” nothing in the spec.
  • Faceted search β€” Tags are free-form strings. No controlled vocabulary or centralized tag registry.

Quick Reference

ThingRule
Only required frontmatter fieldtype
Reserved filenamesindex.md, log.md
Preferred link formAbsolute bundle-relative (/path/to/concept.md)
Conventional body headings# Schema, # Examples, # Citations
Conformance checkAll .md files have frontmatter with non-empty type
Consumer toleranceMust not reject on unknown types, missing optional fields, broken links
Version declarationokf_version: "0.1" in root index.md frontmatter
Date formatISO 8601 (YYYY-MM-DD for log headings, datetime for timestamp)
DistributionGit repo (recommended), tarball, or subdirectory
LicenseApache 2.0

Core Components

Bundle, Concept, Frontmatter, Cross-Links, and Index/Log Files β€” the building blocks of OKF

πŸ“¦

Knowledge Bundle

A self-contained directory tree of knowledge documents β€” the unit of distribution. Organizes concepts hierarchically. Preferred distribution is a Git repository (history, attribution, diffs, PR reviews for free).

πŸ“„

Concept

A single unit of knowledge β€” one markdown file. Can represent tables, APIs, metrics, runbooks, or any knowledge unit. File path within the bundle (minus .md) is the Concept ID.

🏷️

Frontmatter

YAML metadata block at the top of each file. Only required field is `type` β€” a free-form string. Consumers must tolerate unknown types gracefully. Additional custom fields are always valid.

πŸ”—

Cross-Linking

Standard markdown links between concepts build the knowledge graph. Prefer absolute bundle-relative paths (/path/to/concept.md) for stability. Broken links are valid β€” link to a not-yet-written concept is not malformed.

πŸ“‹

Index & Log Files

Reserved filenames with defined meaning. index.md provides progressive disclosure of available concepts. log.md tracks chronological change history (newest first, ISO 8601 dates). Both can appear at any directory level.

Key Benefits

Why OKF's intentional minimalism makes it the right bet for interoperable organizational knowledge

✨

Zero Friction

`cat` to read, `git clone` to ship, any text editor to author. No schema registry, no SDK, no central authority standing between you and your metadata.

🎯

One Required Field

Broad adoption of a minimal spec beats narrow adoption of a rich one. Just `type` is required β€” everything else is optional, letting bundles grow incrementally without breaking conformance.

πŸ“

Git-Native Knowledge

Treating knowledge as version-controlled source code inherits diffs, blame, PRs, and review without ceremony. Knowledge evolves with your codebase.

πŸ›‘οΈ

Permissive Consumer Model

Consumers must not reject bundles for unknown types, missing optional fields, or broken links. This tolerance prevents the fragility death-spiral that kills most interoperability specs.

πŸ”§

Producer-Consumer Decoupling

Any framework can produce OKF; any agent can consume it. Decoupling producers from consumers is the architecturally sound bet β€” matching how OpenAPI became the lingua franca for API description.

Common Use Cases

From data catalogs and runbooks to knowledge graph visualization and LLM wiki standardization

πŸ—„οΈ

Data Catalog Documentation

Document database schemas, tables, join paths, and column meanings with cross-references. The BigQuery enrichment agent auto-generates OKF concept docs for every table/view.

🌐

API Specifications

Capture API endpoints, authentication flows, error codes, rate limits, and related service docs. Cross-link to the canonical OpenAPI spec rather than re-expressing it in markdown.

πŸ“–

Operational Runbooks

Incident response procedures, churn triage playbooks, and troubleshooting guides that link directly to the database tables and datasets they reference.

πŸ“Š

Metric Definitions

Define business metrics (WAU, MRR, churn rate), calculation logic, source tables, and ownership. Cross-link to underlying datasets so agents can answer 'how is this computed?' reliably.

πŸ“

Knowledge Graph Visualization

The OKF Static HTML Visualizer renders any bundle as an interactive force-directed graph β€” colored nodes by type, directed edges from cross-links, detail panel with rendered markdown.

πŸ”—

LLM Wiki Standardization

Formalize the 'LLM wiki' pattern (Karpathy, April 2026) into an interoperable spec. A wiki written by one producer can be consumed by a different agent without translation.

Interactive Tools

Practical tools to help you think systematically, build better AI agents, and master prompt engineering.

Future References

Explore these resources for deeper learning on AI agent development, spec-driven development, and prompt engineering tools.

Spec-Driven Development

Comprehensive guide on Spec-Driven Development practices and methodologies.

Awesome Copilot

Curated list of GitHub Copilot resources, extensions, and best practices.

Promptfoo

Tool for testing, evaluating, and improving LLM prompts and applications.

Prompts.chat

Collection of prompt engineering resources and templates.

Agent Skills

Agent skills resources and documentation for building AI agent skills.

Awesome Skills

Curated list of awesome skill repositories and collections.

Agent Skills Topic

GitHub topic for discovering agent-related skills and repositories.

AI Agent Topic

Trendshift topic for discovering AI agents.

AI Skills Topic

Trendshift topic for discovering AI skills.

Agent Governance Toolkit

Agent governance toolkit.

Pattern Sources

Our patterns are curated from industry-leading sources with proper attribution and licensing compliance.

Refactoring.Guru

Classic GoF design patterns, code smells catalog, and refactoring techniques (https://refactoring.guru).

Enterprise Integration Patterns

65 messaging patterns for integrating enterprise applications by Gregor Hohpe and Bobby Woolf (CC BY 4.0).

Microservices.io

Comprehensive patterns for microservice architectures by Chris Richardson.

Agent Catalog Patterns

Patterns for agentic systems from agentpatternscatalog.org (CC BY 4.0).

OWASP Foundation

Security patterns from OWASP Top 10 for Web Applications, LLM Applications, and Agentic Applications (CC BY-SA 4.0).

Industry Research

ML/AI patterns from Microsoft, Google, Anthropic, and academic research.

AI Agent Patterns

Spec-driven development patterns from Claude, Gemini, OpenAI, and GitHub Copilot on github/spec-kit and OpenSpec.

Data Engineering Leaders

Data platform patterns from Martin Fowler (Data Mesh), Kimball Group (Dimensional Modeling), and cloud providers.

MLOps Best Practices

Data science patterns from MLflow, Great Expectations, and MLOps practitioners.

Streaming & Analytics

Real-time patterns from Confluent/Kafka, Apache projects, and serverless analytics platforms.

Academic Papers

Rigorous ML patterns from peer-reviewed research including data leakage prevention and active learning.

5-Day AI Agents Course

Intensive Vibe Coding Course With Google by Brenda Flynn et al. (2026) on Kaggle.

The Agent Loop

Foundational Agent Definition (Perceive + Act):
Russell, S. J., & Norvig, P. (1995). Artificial Intelligence: A Modern Approach. Prentice Hall. (Current edition: 4th Ed., Pearson, 2020)

Modern Iterative LLM Agent Loop:
Yao, S., Zhao, J., Yu, D., et al. (2022). ReAct: Synergizing Reasoning and Acting in Language Models. arXiv:2210.03629.

Historical Context:
Incorporating AIMA's perceive/act model, Classical robotics' Sense-Plan-Act loop (Brooks, 1986), and ReAct's Thought→Action→Observation cycle.