Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
54 changes: 48 additions & 6 deletions .claude-plugin/marketplace.json
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
"email": "support@claudecodeplugins.dev"
},
"metadata": {
"description": "Awesome Claude Code plugins a curated list of slash commands, subagents, MCP servers, and hooks for Claude Code",
"description": "Awesome Claude Code plugins \u2014 a curated list of slash commands, subagents, MCP servers, and hooks for Claude Code",
"version": "0.0.1",
"homepage": "https://claudecodeplugins.dev"
},
Expand Down Expand Up @@ -73,7 +73,7 @@
{
"name": "ultrathink",
"source": "./plugins/ultrathink",
"description": "Use /ultrathink <TASK_DESCRIPTION> to launch a Coordinator Agent that directs four specialist sub-agents—Architect, Research, Coder, and Tester—to analyze, design, implement, and validate your coding task. The process breaks the task into clear steps, gathers insights, and synthesizes a cohesive solution with actionable outputs. Relevant files can be referenced ad-hoc using @ filename syntax.",
"description": "Use /ultrathink <TASK_DESCRIPTION> to launch a Coordinator Agent that directs four specialist sub-agents\u2014Architect, Research, Coder, and Tester\u2014to analyze, design, implement, and validate your coding task. The process breaks the task into clear steps, gathers insights, and synthesizes a cohesive solution with actionable outputs. Relevant files can be referenced ad-hoc using @ filename syntax.",
"version": "1.0.0",
"author": {
"name": "Jeronim Morina"
Expand Down Expand Up @@ -847,7 +847,7 @@
"description": "Use this agent when setting up CI/CD pipelines, configuring Docker containers, deploying applications to cloud platforms, setting up Kubernetes clusters, implementing infrastructure as code, or automating deployment workflows. Examples: <example>Context: User is setting up a new project and needs deployment automation. user: \"I've built a FastAPI application and need to deploy it to production with proper CI/CD\" assistant: \"I'll use the deployment-engineer agent to set up a complete deployment pipeline with Docker, GitHub Actions, and production-ready configurations.\"</example> <example>Context: User mentions containerization or deployment issues. user: \"Our deployment process is manual and error-prone. We need to automate it.\" assistant: \"Let me use the deployment-engineer agent to design an automated CI/CD pipeline that eliminates manual steps and ensures reliable deployments.\"</example>",
"version": "1.0.0",
"author": {
"name": "Jure Šunić"
"name": "Jure \u0160uni\u0107"
},
"category": "Automation DevOps",
"homepage": "https://github.com/ccplugins/awesome-claude-code-plugins/tree/main/plugins/deployment-engineer",
Expand Down Expand Up @@ -1141,7 +1141,7 @@
"description": "Use this agent when you need to design, build, or validate n8n automation workflows. This agent specializes in creating efficient n8n workflows using proper validation techniques and MCP tools integration.\\n\\nExamples:\\n- <example>\\n Context: User wants to create a Slack notification workflow when a new GitHub issue is created.\\n user: \"I need to create an n8n workflow that sends a Slack message whenever a new GitHub issue is opened\"\\n assistant: \"I'll use the n8n-workflow-builder agent to design and build this GitHub-to-Slack automation workflow with proper validation.\"\\n <commentary>\\n The user needs n8n workflow creation, so use the n8n-workflow-builder agent to handle the complete workflow design, validation, and deployment process.\\n </commentary>\\n</example>\\n- <example>\\n Context: User has an existing n8n workflow that needs debugging and optimization.\\n user: \"My n8n workflow keeps failing at the HTTP Request node, can you help me fix it?\"\\n assistant: \"I'll use the n8n-workflow-builder agent to analyze and debug your workflow, focusing on the HTTP Request node configuration.\"\\n <commentary>\\n Since this involves n8n workflow troubleshooting and validation, use the n8n-workflow-builder agent to diagnose and fix the issue.\\n </commentary>\\n</example>\\n- <example>\\n Context: User wants to understand n8n best practices and available nodes for a specific use case.\\n user: \"What are the best n8n nodes for processing CSV data and sending email reports?\"\\n assistant: \"I'll use the n8n-workflow-builder agent to explore the available nodes and recommend the best approach for CSV processing and email automation.\"\\n <commentary>\\n This requires n8n expertise and node discovery, so use the n8n-workflow-builder agent to provide comprehensive guidance.\\n </commentary>\\n</example>",
"version": "1.0.0",
"author": {
"name": "Jure Šunić"
"name": "Jure \u0160uni\u0107"
},
"category": "Automation DevOps",
"homepage": "https://github.com/ccplugins/awesome-claude-code-plugins/tree/main/plugins/n8n-workflow-builder",
Expand Down Expand Up @@ -1197,7 +1197,7 @@
"description": "Use this agent when you need to create comprehensive Product Requirements Documents (PRDs) that combine business strategy, technical architecture, and user research. Examples: <example>Context: The user needs to create a PRD for a new feature or product launch. user: \"I need to create a PRD for our new user authentication system that will support SSO and multi-factor authentication\" assistant: \"I'll use the prd-specialist agent to create a comprehensive PRD that covers the strategic foundation, technical requirements, and implementation blueprint for your authentication system.\"</example> <example>Context: The user is planning a major product initiative and needs strategic documentation. user: \"We're launching a mobile app for our e-commerce platform and need a detailed PRD to guide development\" assistant: \"Let me engage the prd-specialist agent to develop a thorough PRD that includes market analysis, user research integration, technical architecture, and implementation roadmap for your mobile app initiative.\"</example>",
"version": "1.0.0",
"author": {
"name": "Jure Šunić"
"name": "Jure \u0160uni\u0107"
},
"category": "Project & Product Management",
"homepage": "https://github.com/ccplugins/awesome-claude-code-plugins/tree/main/plugins/prd-specialist",
Expand Down Expand Up @@ -1281,7 +1281,7 @@
"description": "Use this agent when working with Python code that requires advanced features, performance optimization, or comprehensive refactoring. Examples: <example>Context: User needs to optimize a slow Python function that processes large datasets. user: \"This function is taking too long to process our data, can you help optimize it?\" assistant: \"I'll use the python-expert agent to analyze and optimize your Python code with advanced techniques and performance profiling.\"</example> <example>Context: User wants to implement async/await patterns in their existing synchronous Python code. user: \"I need to convert this synchronous code to use async/await for better performance\" assistant: \"Let me use the python-expert agent to refactor your code with proper async/await patterns and concurrent programming techniques.\"</example> <example>Context: User needs help implementing complex Python design patterns. user: \"I want to implement a factory pattern with decorators for my API endpoints\" assistant: \"I'll use the python-expert agent to implement advanced Python patterns with decorators and proper design principles.\"</example>",
"version": "1.0.0",
"author": {
"name": "Jure Šunić"
"name": "Jure \u0160uni\u0107"
},
"category": "Development Engineering",
"homepage": "https://github.com/ccplugins/awesome-claude-code-plugins/tree/main/plugins/python-expert",
Expand Down Expand Up @@ -1670,6 +1670,48 @@
"security",
"compliance"
]
},
{
"name": "dataform-toolkit",
"source": "./plugins/dataform-toolkit",
"description": "Comprehensive toolkit for BigQuery Dataform development with engineering best practices, TDD workflow, proper ref() usage, safety practices, and comprehensive documentation",
"version": "1.0.0",
"author": {
"name": "Ivan Histand",
"email": "ihistand@rotoplas.com",
"url": "https://github.com/ihistand"
},
"category": "Data Analytics",
"homepage": "https://github.com/ihistand/claude-plugins/tree/main/dataform-toolkit",
"keywords": [
"dataform",
"bigquery",
"etl",
"data-engineering",
"tdd",
"sql"
]
},
{
"name": "stl-generator-toolkit",
"source": "./plugins/stl-generator-toolkit",
"description": "Comprehensive toolkit for generating 3D printable STL files for woodworking jigs and fixtures using CadQuery, with pre-built scripts and parametric design patterns",
"version": "1.0.0",
"author": {
"name": "Ivan Histand",
"email": "ihistand@rotoplas.com",
"url": "https://github.com/ihistand"
},
"category": "Development Engineering",
"homepage": "https://github.com/ihistand/claude-plugins/tree/main/stl-generator-toolkit",
"keywords": [
"3d-printing",
"stl",
"cadquery",
"woodworking",
"cad",
"parametric"
]
}
]
}
12 changes: 12 additions & 0 deletions plugins/dataform-toolkit/.claude-plugin/plugin.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
{
"name": "dataform-toolkit",
"version": "1.0.0",
"description": "Comprehensive toolkit for BigQuery Dataform development with engineering best practices. Enforces TDD workflow, proper ref() usage, comprehensive documentation, and safe development patterns. Includes quick-access commands for common ETL workflows.",
"author": {
"name": "Ivan Histand",
"email": "ihistand@rotoplas.com",
"url": "https://github.com/ihistand"
},
"skills": "./skills",
"commands": "./commands"
}
200 changes: 200 additions & 0 deletions plugins/dataform-toolkit/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,200 @@
# Dataform Toolkit

Comprehensive toolkit for BigQuery Dataform development with engineering best practices. Enforces TDD workflow, proper ref() usage, comprehensive documentation, and safe development patterns.

## What's Included

### Skills

**dataform-engineering-fundamentals** - Complete guide for BigQuery Dataform development that enforces:
- Test-Driven Development (TDD) for data transformations
- Safety practices (`--schema-suffix dev`, `--dry-run`)
- Dependency management (ALWAYS `${ref()}`, NEVER hardcoded table paths)
- Comprehensive documentation (`columns: {}` blocks mandatory)
- Proper architecture patterns (layering, incremental processing, source declarations)

The skill is designed to be bulletproof against rationalization - it works especially when you're under time pressure, tired, or tempted to skip best practices.

### Slash Commands

**`/dataform-test`** - Test a Dataform table in dev environment with complete safety checks:
- Compile validation
- Dry-run SQL validation
- Dev environment execution
- Result verification

**`/dataform-deploy`** - Deploy a tested table to production with pre-deployment verification:
- Confirms dev testing completed
- Verifies tests passing
- Checks documentation completeness
- Safe production deployment

**`/dataform-new-table`** - Create new tables using TDD workflow:
- RED: Write tests first, watch them fail
- GREEN: Write minimal implementation, watch tests pass
- REFACTOR: Improve while keeping tests passing

**`/dataform-etl`** - Launch the ETL agent specialized in BigQuery Dataform development:
- Complex transformations
- Pipeline troubleshooting
- Data quality implementations
- ELT workflow coordination

## Installation

### Option 1: Install from Local Directory

```bash
# From your Claude Code session
/plugin marketplace add /path/to/plugins/dataform-toolkit
/plugin install dataform-toolkit@dev
```

Then restart Claude Code.

### Option 2: Install from Git Repository (Future)

Once published to a Git repository:

```bash
/plugin marketplace add <your-org>/<repository-name>
/plugin install dataform-toolkit@<marketplace-name>
```

## Usage Examples

### Testing a Table

```
User: I need to test my customer_metrics table
Claude: /dataform-test
[Follows complete testing workflow with safety checks]
```

### Creating a New Table

```
User: Help me create a new sales summary table
Claude: /dataform-new-table
[Guides through TDD workflow: tests first, then implementation]
```

### Deploying to Production

```
User: The customer_metrics table is ready for production
Claude: /dataform-deploy
[Verifies testing completed, then safely deploys]
```

### Complex Dataform Work

```
User: I need to build a complex pipeline that aggregates data from multiple sources
Claude: /dataform-etl
[Launches ETL agent with full Dataform expertise]
```

## Key Principles

### Non-Negotiable Safety Practices

1. **Always use `--schema-suffix dev` for testing** - Protects production data
2. **Always use `--dry-run` before execution** - Catches errors early
3. **Create source declarations before using ref()** - Enables dependency tracking
4. **ALWAYS use ${ref()} - NEVER hardcoded table paths** - Maintains dependency graph
5. **Use proper ref() syntax** - Single argument when source exists
6. **Add validation queries** - Verify results immediately

### Architecture Requirements

- **Layered structure**: sources/ → intermediate/ → output/
- **Incremental vs full refresh**: Choose based on data characteristics
- **Dataform assertions**: Automated data quality checks
- **Source declarations**: Use .sqlx files (not .js) for new declarations
- **Schema configuration**: No schema: config in operations/ or test/ files

### Documentation Standards

- **columns: {} mandatory** for all tables with `type: "table"`
- Get descriptions from source docs, API documentation, or business logic
- Document source declarations when applicable
- Clear, concise descriptions following Strunk's writing rules

### Test-Driven Development

1. **RED Phase**: Write assertions first, watch them fail
2. **GREEN Phase**: Write minimal implementation, watch tests pass
3. **REFACTOR Phase**: Improve while keeping tests passing

**Critical**: Tests-first means defining what should happen. Tests-after means checking what does happen.

## Time Pressure Protocol

Even under extreme time pressure (board meeting in 2 hours, production down, stakeholder waiting):

✅ **Still required**:
- Dev testing (3 minutes saves 60 minutes debugging)
- `--dry-run` validation
- Source declarations
- `columns: {}` documentation (2 minutes, saves hours)
- Tests first (TDD)
- Basic validation

⚠️ **Can skip** (but document as technical debt):
- Extensive documentation files
- Peer review
- Performance optimization

**Bottom line**: Safety practices save time. Skipping them wastes time.

## Common Mistakes to Avoid

1. ❌ Using tables before declaring sources → ✅ Create source declarations first
2. ❌ Hardcoded table paths → ✅ Use ${ref()} always
3. ❌ Skipping dev testing under pressure → ✅ Dev testing prevents production issues
4. ❌ Creating monolithic transformations → ✅ Break into intermediate tables
5. ❌ Missing columns: {} documentation → ✅ Document all tables immediately
6. ❌ Writing implementation before tests → ✅ Follow TDD cycle (RED-GREEN-REFACTOR)
7. ❌ Using .js files for NEW declarations → ✅ Use .sqlx files
8. ❌ Adding schema: to operations/tests → ✅ Use default schemas from workflow_settings.yaml

## Quick Reference Commands

| Command | Purpose | Example |
|---------|---------|---------|
| `dataform compile` | Validate syntax | Check for errors |
| `dataform run --schema-suffix dev --dry-run --actions table` | Validate SQL | Check before execution |
| `dataform run --schema-suffix dev --actions table` | Test in dev | Safe execution |
| `dataform run --schema-suffix dev --include-deps --actions table` | Run with deps | Include upstream tables |
| `dataform run --schema-suffix dev --tags looker` | Run by tag | Execute tagged tables |
| `dataform run --actions table` | Production deploy | After dev testing succeeds |

## Related Skills

- **superpowers:test-driven-development** - Foundational TDD principles
- **superpowers:brainstorming** - Refine requirements before coding
- **superpowers:systematic-debugging** - Structured troubleshooting
- **superpowers:root-cause-tracing** - Trace errors to source
- **elements-of-style:writing-clearly-and-concisely** - Clear documentation writing

## Official Documentation

- [Dataform Documentation](https://cloud.google.com/dataform/docs)
- [Dataform Best Practices](https://cloud.google.com/dataform/docs/best-practices-repositories)
- [BigQuery GoogleSQL Reference](https://cloud.google.com/bigquery/docs/reference/standard-sql)

## License

Created by Ivan Histand (ihistand@rotoplas.com)

## Contributing

This plugin follows the superpowers framework principles:
- Skills enforce discipline, not just provide information
- Must be bulletproof against rationalization
- Include red flags to catch deviation attempts
- Document common mistakes with corrections
- Reference related skills for workflow chains

For questions or contributions, contact Ivan Histand.
21 changes: 21 additions & 0 deletions plugins/dataform-toolkit/commands/dataform-deploy.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
---
description: Deploy tested Dataform table to production
---

You are deploying a Dataform table to production using best practices from the dataform-engineering-fundamentals skill.

**Workflow:**

1. Invoke the dataform-engineering-fundamentals skill
2. Ask the user which table they want to deploy
3. **Pre-deployment verification:**
- Confirm the table has been tested in dev environment
- Verify all tests are passing
- Check that documentation (columns: {}) is complete
4. **Deployment:**
- Run `dataform run --dry-run --actions <table_name>` (production dry-run)
- If successful, run `dataform run --actions <table_name>` (production execution)
- Verify deployment with validation queries
5. Report deployment results

**Critical**: Never deploy without dev testing first. Wrong results delivered quickly are worse than correct results delivered with a small delay.
24 changes: 24 additions & 0 deletions plugins/dataform-toolkit/commands/dataform-etl.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
---
description: Launch ETL agent for BigQuery Dataform development
---

You are launching the ETL Dataform engineer agent to handle data transformation pipeline work.

**Purpose**: The ETL agent specializes in BigQuery Dataform projects, SQLX files, data quality, and pipeline development. Use this for:
- Complex Dataform transformations
- Pipeline troubleshooting
- Data quality implementations
- ELT workflow coordination

**Task**: Use the Task tool with `subagent_type="etl"` to launch the ETL agent. Pass the user's request as the prompt, including:
- What they need to accomplish
- Any relevant context about tables, datasets, or business logic
- Whether this is new development, modification, or troubleshooting

The ETL agent has access to the dataform-engineering-fundamentals skill and will follow best practices for BigQuery Dataform development.

**Example**:
```
User asks: "Help me create a customer metrics table"
You launch: Task tool with subagent_type="etl" and prompt="Create a customer metrics table in Dataform following TDD workflow. Ask user about required metrics and data sources."
```
Loading