From e75a9a1d5acd624ae41a488a72e1e973aaffce3e Mon Sep 17 00:00:00 2001 From: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Date: Sat, 22 Nov 2025 17:58:48 +0000 Subject: [PATCH 1/2] docs: add comprehensive fixture testing guidance to instructions Co-Authored-By: Catherine Deskur --- CLAUDE.md | 105 ++++++++++++++++++++++++++++++++++++++++++++++++ CONTRIBUTING.md | 36 +++++++++++++++++ 2 files changed, 141 insertions(+) diff --git a/CLAUDE.md b/CLAUDE.md index df3de1a5dc0f..f4a35e4fe149 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -164,6 +164,7 @@ Multi-stage process: API Schema → IR Updates → Generator Updates → Release - **Testing**: Always run `pnpm test:ete` for end-to-end validation after CLI changes - **Monorepo**: Use Turbo filters for focused builds/tests on specific packages - **Docker**: Generators run in Docker containers for isolation and consistency +- **Fixture Testing**: Always include a fixture to test your changes (see "Testing Changes with Fixtures" section below) ## Generated Code Management @@ -181,6 +182,110 @@ Multi-stage process: API Schema → IR Updates → Generator Updates → Release **Update fixtures**: `pnpm seed:build --filter ` or `pnpm test:update` for snapshots +## Testing Changes with Fixtures + +**IMPORTANT**: Always include a fixture to test your changes. This is a critical requirement for all code changes in this repository. + +### When to Add/Update Fixtures + +You must include fixture testing when: +- **Adding new generator features**: Create a new test definition in `/test-definitions/fern/apis/` and reference it in the appropriate `seed//seed.yml` +- **Fixing bugs**: Add a fixture that reproduces the bug, verify the fix resolves it +- **Modifying existing behavior**: Update existing fixtures to reflect the changes +- **Changing IR structure**: Update fixtures for all affected generators +- **Adding CLI features**: Create fixtures that exercise the new functionality + +### How to Test with Fixtures + +**Step 1: Identify or Create Test Definition** +```bash +# For new features, create a test definition in test-definitions/fern/apis/ +# Example: test-definitions/fern/apis/my-new-feature/ + +# For existing features, identify the relevant test definition +ls test-definitions/fern/apis/ +``` + +**Step 2: Add to seed.yml (if new)** +```bash +# Edit seed//seed.yml to reference your test definition +# Example: seed/python-sdk/seed.yml +``` + +**Step 3: Run Seed Test** +```bash +# Test specific generator and fixture +pnpm seed test --generator --fixture + +# Example for Python SDK with a specific fixture +pnpm seed test --generator python-sdk --fixture my-new-feature + +# Skip compilation scripts for faster iteration during development +pnpm seed test --generator python-sdk --fixture my-new-feature --skip-scripts +``` + +**Step 4: Verify Changes** +```bash +# Check what changed in the generated code +git diff seed/// + +# Ensure changes are intentional and correct +# Review generated code for quality and correctness +``` + +**Step 5: Commit Generated Code** +```bash +# Always commit both your source changes AND the generated fixture code +git add generators// +git add seed/// +git commit -m "feat(): add new feature with fixture" +``` + +### Best Practices + +- **Test locally first**: Always run seed tests locally before pushing to ensure your changes work +- **Use appropriate fixtures**: Choose or create fixtures that specifically exercise your changes +- **Check diffs carefully**: Review generated code diffs to catch unintended changes +- **Update snapshots**: Run `pnpm test:update` if snapshot tests need updating +- **Document new fixtures**: Add comments in test definitions explaining what they test +- **Keep fixtures focused**: Each fixture should test a specific feature or scenario +- **Run full test suite**: Before creating a PR, run the full test suite for your generator + +### Common Fixture Testing Patterns + +**Testing a bug fix:** +```bash +# 1. Create a minimal test definition that reproduces the bug +# 2. Run seed test and verify it fails or produces incorrect output +pnpm seed test --generator python-sdk --fixture bug-reproduction + +# 3. Fix the bug in your generator code +# 4. Re-run seed test and verify the fix +pnpm seed test --generator python-sdk --fixture bug-reproduction + +# 5. Commit both the fix and the updated fixture +``` + +**Testing a new feature:** +```bash +# 1. Create a test definition with API schema using the new feature +# 2. Add to seed.yml +# 3. Run seed test to generate initial output +pnpm seed test --generator typescript-sdk --fixture new-feature + +# 4. Verify generated code is correct +# 5. Commit everything together +``` + +**Iterating on changes:** +```bash +# Use --skip-scripts for faster iteration during development +pnpm seed test --generator go-sdk --fixture my-feature --skip-scripts + +# Once satisfied, run full test with scripts to ensure compilation +pnpm seed test --generator go-sdk --fixture my-feature +``` + ## Pull Request Guidelines When creating pull requests in this repository: diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index e7edb65b44a8..1899df699f64 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -231,6 +231,42 @@ Below are some examples of using the command. - For a single generator, test definition, and skipping scripts: `pnpm seed test --generator python-sdk --fixture file-download --skip-scripts` - For running the generator locally (not on docker): `pnpm seed test --generator python-sdk --local` +#### Important: Always Include Fixtures to Test Changes + +**When contributing code changes, you must always include a fixture to test your changes.** This is a critical requirement for maintaining code quality and preventing regressions. + +**When to add/update fixtures:** +- Adding new generator features → Create a new test definition in `/test-definitions/fern/apis/` and reference it in `seed//seed.yml` +- Fixing bugs → Add a fixture that reproduces the bug and verify your fix resolves it +- Modifying existing behavior → Update existing fixtures to reflect the changes +- Changing IR structure → Update fixtures for all affected generators + +**Workflow for testing with fixtures:** +1. Identify or create a test definition in `/test-definitions/fern/apis/` +2. If new, add it to the appropriate `seed//seed.yml` +3. Run `pnpm seed test --generator --fixture ` to generate code +4. Review the generated code with `git diff seed///` +5. Commit both your source changes and the generated fixture code together + +**Example workflow:** +```sh +# Create or identify test definition +ls test-definitions/fern/apis/ + +# Run seed test for your changes +pnpm seed test --generator python-sdk --fixture my-feature + +# Review generated code changes +git diff seed/python-sdk/my-feature/ + +# Commit both source and generated code +git add generators/python/ +git add seed/python-sdk/my-feature/ +git commit -m "feat(python): add new feature with fixture" +``` + +For more detailed guidance on fixture testing, see the "Testing Changes with Fixtures" section in [CLAUDE.md](./CLAUDE.md). + ### Running seed against a custom fern definition It may be valuable to run seed on a particular Fern definition or OpenAPI spec. To do this, From be08855879fbd03154163fae96c5617cb652008f Mon Sep 17 00:00:00 2001 From: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Date: Sat, 22 Nov 2025 18:11:11 +0000 Subject: [PATCH 2/2] chore: trigger CI re-run Co-Authored-By: Catherine Deskur