Graphspec

Graphspecs - Rich Data-Driven Diagrams

Graphspecs are dictim's high-level format for expressing complex systems as structured data with rich metadata. Unlike basic dictim syntax, graphspecs preserve complete data context throughout processing, enabling powerful template-based styling and sophisticated diagram generation.

When to Use Graphspecs

Both dictim and graphspecs are designed for programmatic data production, but graphspecs provide a higher level of abstraction for expressing a diagram as data.

Key Advantages of Graphspecs

Higher Level Abstraction

• Work directly with nodes and edges as data objects
• Natural graph representation without premature flattening to dictim elements
• Preserve rich data context throughout the processing pipeline

Better Separation of Concerns

• Data: Pure graph structure (nodes, edges, containers)
• Presentation: Templates control styling, labeling, and visual arrangement
• Organization: Cluster mappings independent of data structure

Flexible Diagram Arrangement

• Dynamically organize nodes into containers based on any data field
• Rearrange clustering without touching the underlying data
• Nest containers hierarchically with simple configuration changes
• Template-driven styling means the same data can produce different visual representations

When to Use Each

Use Graphspecs when:

• Working with rich node/edge data objects
• Need flexible arrangement and rearrangement of elements
• Want to separate data structure from visual presentation
• Require template-driven styling based on data properties

Use Basic Dictim when:

• Simple, direct control over diagram elements
• Fixed visual structure that doesn't need rearrangement
• Minimal template requirements

Table of Contents

• Graphspec Fundamentals
• Required Fields
• Key Indifference
• Optional Features
  • Node Templates
  • Edge Templates
  • Node to Container Mapping
  • Container Hierarchy
  • Container Templates
  • Container Attributes
  • String Interpolation
  • Global Directives
  • Default Template
• Templates in Graphspecs
• Container Hierarchies
• CLI Usage
• Clojure Library Usage
• Real-World Examples
• Best Practices

Graphspec Fundamentals

What is a Graphspec?

A graphspec is a structured data format that describes a diagram in terms of:

• Nodes: Rich data objects representing diagram elements
• Edges: Connections between nodes with metadata
• Templates: Rules for styling and labeling based on data properties
• Containers: Hierarchical grouping and organization
• Directives: Global diagram configuration

Basic Structure

{
  "node->key": "id",
  "nodes": [
    {"id": "app1", "name": "Web Service", "dept": "Engineering"},
    {"id": "db1", "name": "Database", "dept": "Infrastructure"}
  ],
  "edges": [
    {"src": "app1", "dest": "db1", "type": "sql", "frequency": "high"}
  ]
}

Graphspec vs Basic Dictim

Aspect	Basic Dictim	Graphspec
Data richness	Limited to keys, labels, attributes	Full object properties preserved
Template power	Structural tests only	Data-driven conditional logic
Use case	Simple, static diagrams	Complex, data-driven visualizations
Maintenance	Manual updates	Programmatic generation from data sources

Required Fields

Every graphspec must include these mandatory fields:

node->key (String)

Specifies which field in each node object serves as the unique identifier.

{
  "node->key": "id",  // Nodes must have an "id" field
  // ... or
  "node->key": "uuid", // Nodes must have a "uuid" field
}

nodes (Array)

Array of objects representing diagram elements. Objects can have any structure.

{
  "nodes": [
    {
      "id": "service-1",
      "name": "User Service", 
      "type": "microservice",
      "language": "java",
      "team": "user-management",
      "cpu_cores": 4,
      "memory_gb": 8,
      "status": "healthy"
    },
    {
      "id": "db-1",
      "name": "User Database",
      "type": "database",
      "engine": "postgresql",
      "size_gb": 500,
      "backup_enabled": true
    }
  ]
}

edges (Array)

Array of connection objects. Each must have src and dest fields matching node keys.

{
  "edges": [
    {
      "src": "service-1",
      "dest": "db-1", 
      "protocol": "tcp",
      "port": 5432,
      "data_type": "sql_queries",
      "frequency": "high",
      "latency_ms": 15
    }
  ]
}

Key Indifference

Graphspecs work with either string or keyword keys throughout — both are valid and can be used interchangeably. The only requirement is internal consistency: if your nodes use keyword keys, your templates and mappings should too.

// String keys (JSON-style)
{"node->key": "id", "nodes": [{"id": "a"}]}

;; Keyword keys (EDN/Clojure-style)
{:node->key :id, :nodes [{:id "a"}]}

Optional Features

Node Templates

Define conditional styling based on node properties. See Templates for complete template syntax.

{
  "node-template": [
    ["contains", "attrs", {"type": "microservice"}], {"style.fill": "lightblue"},
    ["contains", "attrs", {"status": "error"}], {"style.fill": "red"},
    [">", "cpu_cores", 2], {"style.border": "2px solid orange"}
  ]
}

Edge Templates

Style connections based on edge properties:

{
  "edge-template": [
    ["=", "protocol", "https"], {"style.stroke": "green"},
    ["=", "frequency", "high"], {"style.stroke-width": "3"},
    [">", "latency_ms", 100], {"style.stroke": "red", "style.stroke-dash": "5"}
  ]
}

Container Templates

Style containers based on their data. Requires both container->data and container-template to be present:

{
  "container->data": {
    "frontend": {"team_size": 5, "budget": 200000},
    "backend": {"team_size": 8, "budget": 400000}
  },
  "container-template": [
    [">", "budget", 300000], {"style.fill": "lightgreen"},
    ["<", "team_size", 3], {"style.opacity": "0.7"}
  ]
}

Container Attributes

A simpler alternative to container->data + container-template when you just want to assign fixed attributes to named containers directly, without data-driven logic:

{
  "container->attrs": {
    "frontend": {"style.fill": "lightblue"},
    "backend": {"style.fill": "lightgreen"},
    "infrastructure": {"style.fill": "lightyellow"}
  }
}

Note: container->attrs and container-template/container->data are mutually exclusive. If both are present, container->attrs wins and the template/data pair is ignored.

Node to Container Mapping

node->container specifies which field on each node determines which container (cluster) the node belongs to. The field's value becomes the container name:

{
  "node->container": "department",
  "nodes": [
    {"id": "app1", "department": "Engineering"},
    {"id": "db1",  "department": "Infrastructure"}
  ]
}

Nodes whose container field is absent or null are placed at the top level, outside any container.

Container Hierarchy

container->parent is a map from container name to its parent container name, used to build nested container hierarchies:

{
  "node->container": "team",
  "container->parent": {
    "frontend-team": "Engineering",
    "backend-team": "Engineering",
    "devops-team": "Infrastructure"
  }
}

This produces Engineering and Infrastructure as top-level containers, each containing their respective teams. Containers not listed in container->parent become top-level containers automatically.

String Interpolation

Embed node/edge data into labels and attributes:

{
  "node-template": [
    ["=", "type", "microservice"], 
    {"label": ["%s (%s)", "name", "language"]}
  ],
  "edge-template": [
    ["=", "protocol", "tcp"],
    {"label": ["Port %d", "port"]}
  ]
}

Result: Node becomes "User Service (java)", edge becomes "Port 5432"

Global Directives

Add top-level diagram configuration:

{
  "directives": {
    "direction": "right",
    "classes": {
      "critical": {"style": {"fill": "red", "stroke": "darkred"}}
    }
  }
}

Default Template

Apply base styling to all elements:

{
  "template": [
    ["=", "element-type", "shape"], {"style.border": "1px solid gray"},
    ["=", "element-type", "conn"], {"style.stroke": "black"}
  ]
}

Templates in Graphspecs

Graphspec templates are more powerful than basic dictim templates because they operate on rich data objects. See Templates - Testing Element Attributes for complete syntax.

Template Execution Order

1. Node templates applied to nodes
2. Edge templates applied to edges
3. Container templates applied to containers
4. Default template applied to final dictim elements

Common Template Patterns

Status-Based Styling

{
  "node-template": [
    ["=", "status", "healthy"], {"style.fill": "lightgreen"},
    ["=", "status", "warning"], {"style.fill": "yellow"}, 
    ["=", "status", "error"], {"style.fill": "red"},
    ["=", "status", "unknown"], {"style.fill": "gray"}
  ]
}

Performance-Based Sizing

{
  "node-template": [
    [">", "cpu_usage", 80], {"style.border": "3px solid red"},
    [">", "memory_usage", 90], {"style.fill": "orange"},
    ["and", [">", "cpu_usage", 50], ["<", "cpu_usage", 80]], {"style.fill": "yellow"}
  ]
}

Team-Based Organization

{
  "node->container": "team",
  "container-template": [
    ["=", "team", "frontend"], {"style.fill": "lightblue"},
    ["=", "team", "backend"], {"style.fill": "lightgreen"},
    ["=", "team", "devops"], {"style.fill": "orange"}
  ]
}

Default Styling with Catch-All Tests

When working with dynamic data, you may encounter values that don't match your specific conditions. Use catch-all tests (:else, "else", or "_") to provide default styling:

{
  "node-template": [
    ["=", "status", "healthy"], {"style.fill": "lightgreen"},
    ["=", "status", "warning"], {"style.fill": "yellow"},
    ["=", "status", "error"], {"style.fill": "red"},
    "else", {"style.fill": "lightgray", "style.opacity": "0.8"}
  ]
}

Why this matters in graphspecs:

In data-driven scenarios, your source data may include:

• New status values not yet in your template
• Missing or null fields
• Evolving data schemas

A catch-all ensures these elements still receive styling rather than being left unstyled.

Advanced pattern - type-based defaults:

{
  "node-template": [
    ["=", "type", "microservice"], {"shape": "rectangle", "style.fill": "lightblue"},
    ["=", "type", "database"], {"shape": "cylinder", "style.fill": "lightyellow"},
    ["=", "type", "cache"], {"shape": "diamond", "style.fill": "orange"},
    ["=", "type", "queue"], {"shape": "queue", "style.fill": "purple"},
    ":else", {"shape": "circle", "style.fill": "lightgray"}
  ],
  "edge-template": [
    ["=", "protocol", "https"], {"style.stroke": "green", "style.stroke-width": "2"},
    ["=", "protocol", "http"], {"style.stroke": "blue"},
    ["=", "protocol", "tcp"], {"style.stroke": "purple"},
    "else", {"style.stroke": "gray", "style.stroke-dash": "3"}
  ]
}

Note: You can use :else (keyword), "else" (string), or "_" (underscore) interchangeably. The keyword :else and string "else" are recommended for clarity.

Container Hierarchies

Graphspecs support sophisticated hierarchical organization:

Basic Containers

Group nodes by any field:

{
  "node->container": "department",
  "nodes": [
    {"id": "app1", "name": "Web App", "department": "Engineering"},
    {"id": "app2", "name": "Mobile App", "department": "Engineering"},
    {"id": "db1", "name": "Database", "department": "Infrastructure"}
  ]
}

Nested Containers

Create hierarchical groupings:

{
  "node->container": "team",
  "container->parent": {
    "frontend-team": "Engineering",
    "backend-team": "Engineering", 
    "devops-team": "Infrastructure"
  }
}

Container Data and Styling

Provide rich data for container templates:

{
  "container->data": {
    "Engineering": {
      "head": "Sarah Chen",
      "budget": 2000000,
      "headcount": 15,
      "projects": ["web-app", "mobile-app"]
    },
    "Infrastructure": {
      "head": "Mike Rodriguez", 
      "budget": 800000,
      "headcount": 8,
      "projects": ["database", "monitoring"]
    }
  },
  "container-template": [
    [">", "budget", 1500000], {"style.fill": "lightgreen"},
    [">", "headcount", 10], {"style.border": "2px solid blue"},
    ["contains", "projects", "mobile-app"], {"style.opacity": "0.9"}
  ]
}

CLI Usage

Reminder: Always use stdin redirection (<) to pass file contents to dict. Passing a filename directly without < will not read the file. Watch mode flags (-giw, -gw) are the exception — they take a filename directly as the file to watch.

Basic Graphspec Processing

# Convert graphspec to dictim
dict -g < system.json -o system.edn

# Direct graphspec to image (new feature!)
dict -g -i < system.json -o diagram.svg

# With templates
dict -g -i -t infrastructure.edn < system.json -o styled.svg

# With d2 options
dict -g -i --layout elk --theme 4 < system.json -o diagram.svg

Watch Mode Development

# Live development with browser serving
dict -giw system.json

# With templates  
dict -giw system.json -t infrastructure.edn

# Output to file
dict -g -i -w system.json -t styles.edn -o live.svg

# Custom d2 options
dict -giw system.json --layout tala --theme 105 --scale 1.5

Validation

# Validate graphspec structure
dict -g < system.json  # Shows validation errors

# Validate with templates
dict -g -t template.edn < system.json  # Validates both

Pipeline Integration

# Database to diagram
pg_dump --schema-only | schema-to-graphspec.py | dict -g -i -o schema.svg

# Kubernetes services
kubectl get services -o json | k8s-to-graphspec.js | dict -giw

# Monitoring data
curl metrics-api | metrics-to-graphspec | dict -g -i -t monitoring.edn -o dashboard.svg

Clojure Library Usage

Basic Processing

(require '[dictim.graphspec :as gs])

;; Convert graphspec to dictim
(def dictim-result 
  (gs/graph-spec->dictim graphspec-data))

;; With templates
(def styled-dictim
  (gs/graph-spec->dictim graphspec-data 
    {:validate? true :output-format :d2}))

Advanced Processing

;; Custom validation
(def result
  (gs/graph-spec->dictim graphspec
    {:validate? true
     :output-format :d2}))

;; Skip validation for performance
(def fast-result  
  (gs/graph-spec->dictim graphspec
    {:validate? false}))

Template Application

See Templates - Template Application and Directives for complete template usage.

(require '[dictim.template :as tmp])

;; Process graphspec then apply additional templates
(-> graphspec-data
    (gs/graph-spec->dictim)
    (tmp/apply-template {:template additional-styles}))

Real-World Examples

Microservices Architecture

{
  "node->key": "service_id",
  "node->container": "team",
  "nodes": [
    {
      "service_id": "user-service",
      "name": "User Management",
      "type": "microservice", 
      "language": "java",
      "team": "user-mgmt",
      "cpu_cores": 4,
      "memory_gb": 8,
      "requests_per_second": 1200,
      "error_rate": 0.02,
      "status": "healthy"
    },
    {
      "service_id": "order-service", 
      "name": "Order Processing",
      "type": "microservice",
      "language": "python",
      "team": "commerce",
      "cpu_cores": 8,
      "memory_gb": 16,
      "requests_per_second": 800,
      "error_rate": 0.05,
      "status": "warning"
    },
    {
      "service_id": "user-db",
      "name": "User Database",
      "type": "database",
      "engine": "postgresql",
      "team": "data",
      "size_gb": 500,
      "connections": 45,
      "status": "healthy"
    }
  ],
  "edges": [
    {
      "src": "user-service",
      "dest": "user-db",
      "protocol": "tcp",
      "port": 5432,
      "latency_ms": 15,
      "frequency": "high"
    },
    {
      "src": "order-service", 
      "dest": "user-service",
      "protocol": "http",
      "port": 8080,
      "latency_ms": 45,
      "frequency": "medium"
    }
  ],
  "node-template": [
    ["=", "type", "microservice"], {"shape": "rectangle"},
    ["=", "type", "database"], {"shape": "cylinder"},
    [">", "error_rate", 0.03], {"style.fill": "red"},
    ["=", "status", "healthy"], {"style.fill": "lightgreen"},
    ["=", "status", "warning"], {"style.fill": "yellow"}
  ],
  "edge-template": [
    ["=", "frequency", "high"], {"style.stroke-width": "3"},
    [">", "latency_ms", 30], {"style.stroke": "orange"},
    ["=", "protocol", "https"], {"style.stroke": "green"}
  ],
  "container->data": {
    "user-mgmt": {"budget": 500000, "headcount": 6},
    "commerce": {"budget": 800000, "headcount": 10},
    "data": {"budget": 300000, "headcount": 4}
  },
  "container-template": [
    [">", "budget", 600000], {"style.fill": "lightblue"},
    [">", "headcount", 8], {"style.border": "2px solid purple"}
  ],
  "directives": {
    "direction": "right",
    "classes": {
      "critical": {"style": {"border": "3px solid red"}}
    }
  }
}

Infrastructure Monitoring

{
  "node->key": "hostname",
  "node->container": "datacenter", 
  "nodes": [
    {
      "hostname": "web-01",
      "type": "web_server",
      "datacenter": "us-east-1",
      "cpu_percent": 75,
      "memory_percent": 60,
      "disk_percent": 45,
      "uptime_days": 42,
      "status": "healthy"
    },
    {
      "hostname": "db-01",
      "type": "database",
      "datacenter": "us-east-1", 
      "cpu_percent": 90,
      "memory_percent": 85,
      "disk_percent": 78,
      "uptime_days": 120,
      "status": "warning"
    },
    {
      "hostname": "cache-01",
      "type": "cache",
      "datacenter": "us-west-2",
      "cpu_percent": 45,
      "memory_percent": 70,
      "disk_percent": 30,
      "uptime_days": 5,
      "status": "healthy"
    }
  ],
  "edges": [
    {
      "src": "web-01",
      "dest": "db-01",
      "connection_type": "database_query",
      "bandwidth_mbps": 100,
      "latency_ms": 5
    },
    {
      "src": "web-01", 
      "dest": "cache-01",
      "connection_type": "cache_lookup",
      "bandwidth_mbps": 50,
      "latency_ms": 2
    }
  ],
  "node-template": [
    ["and", [">", "cpu_percent", 80], [">", "memory_percent", 80]], 
    {"style.fill": "red", "style.border": "2px solid darkred"},
    
    ["or", [">", "cpu_percent", 70], [">", "memory_percent", 70]], 
    {"style.fill": "orange"},
    
    ["<", "uptime_days", 7], 
    {"style.stroke-dash": "3", "label": ["%s (NEW)", "hostname"]},
    
    ["=", "type", "database"], 
    {"shape": "cylinder"},
    
    ["=", "type", "cache"], 
    {"shape": "diamond"}
  ],
  "edge-template": [
    [">", "latency_ms", 10], {"style.stroke": "red"},
    [">", "bandwidth_mbps", 75], {"style.stroke-width": "3"}
  ]
}

Business Process Flow

{
  "node->key": "process_id",
  "node->container": "department",
  "nodes": [
    {
      "process_id": "lead-gen",
      "name": "Lead Generation", 
      "department": "Marketing",
      "owner": "Sarah Johnson",
      "automation_level": "high",
      "monthly_volume": 5000,
      "cost_per_unit": 25,
      "satisfaction_score": 4.2
    },
    {
      "process_id": "qualification",
      "name": "Lead Qualification",
      "department": "Sales", 
      "owner": "Mike Chen",
      "automation_level": "medium", 
      "monthly_volume": 3000,
      "cost_per_unit": 45,
      "satisfaction_score": 3.8
    },
    {
      "process_id": "proposal",
      "name": "Proposal Generation",
      "department": "Sales",
      "owner": "Lisa Rodriguez",
      "automation_level": "low",
      "monthly_volume": 800,
      "cost_per_unit": 120,
      "satisfaction_score": 4.5
    }
  ],
  "edges": [
    {
      "src": "lead-gen",
      "dest": "qualification", 
      "conversion_rate": 0.6,
      "avg_time_days": 2,
      "handoff_method": "automated"
    },
    {
      "src": "qualification",
      "dest": "proposal",
      "conversion_rate": 0.27,
      "avg_time_days": 5, 
      "handoff_method": "manual"
    }
  ],
  "node-template": [
    ["=", "automation_level", "high"], {"style.fill": "lightgreen"},
    ["=", "automation_level", "medium"], {"style.fill": "yellow"},
    ["=", "automation_level", "low"], {"style.fill": "orange"},
    [">", "cost_per_unit", 100], {"style.border": "2px solid red"},
    ["<", "satisfaction_score", 4.0], {"style.opacity": "0.7"}
  ],
  "edge-template": [
    ["<", "conversion_rate", 0.3], {"style.stroke": "red", "style.stroke-dash": "5"},
    [">", "avg_time_days", 4], {"style.stroke-width": "3"},
    ["=", "handoff_method", "automated"], {"style.stroke": "green"}
  ],
  "container->data": {
    "Marketing": {"budget": 2000000, "headcount": 12},
    "Sales": {"budget": 1500000, "headcount": 18}
  },
  "container-template": [
    [">", "budget", 1800000], {"style.fill": "lightblue"}
  ]
}

Best Practices

Data Modeling

1. Rich node objects: Include all relevant properties for template logic
2. Consistent key naming: Use standard field names across similar objects
3. Normalized edge data: Keep connection metadata consistent
4. Meaningful containers: Group by logical business/technical boundaries

Template Design

1. Progressive specificity: Order templates from specific to general conditions
2. Performance awareness: Complex templates on large datasets may be slow
3. Readable conditions: Use clear, self-documenting template logic
4. Template composition: Build complex styling from simpler template pieces

Validation Strategy

1. Always validate in development: Catch schema issues early
2. Skip validation in production: For performance in high-volume scenarios
3. Test template logic: Verify templates work with representative data
4. Document template assumptions: What data fields are required?

CLI Workflows

1. Use -giw for development: Live reloading speeds iteration
2. Validate before processing: dict -g < graphspec.json
3. Pipeline-friendly: Integrate with data transformation tools
4. Version control graphspecs: Track diagram evolution with data

Library Integration

1. Process in stages: Validate → Transform → Style → Render
2. Error handling: Graceful degradation for invalid data
3. Caching: Cache processed graphspecs for repeated use
4. Monitoring: Track processing performance for large datasets

Summary

Graphspecs provide a powerful foundation for data-driven diagram generation. By preserving rich metadata throughout processing, they enable sophisticated template-based styling that would be impossible with basic dictim syntax.

Key advantages:

• Data preservation: Full context available for templates
• Flexible styling: Conditional formatting based on any data property
• Programmatic generation: Perfect for automated documentation and dashboards
• Hierarchical organization: Natural container grouping and nesting
• Validation: Built-in schema checking ensures data integrity

When to use graphspecs:

• Complex systems with rich metadata
• Data-driven diagram generation
• Dashboard and monitoring visualizations
• Automated documentation systems
• Any scenario where data drives visual representation

For simple, static diagrams, basic dictim syntax may be sufficient. For dynamic, data-rich visualizations, graphspecs provide the power and flexibility needed for sophisticated diagram generation.