feat(ask-fern): add system prompt caching for LLM providers #5475
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add extensible prompt caching support for Anthropic and Bedrock providers: - Create CacheConfig and ProviderCapabilities dataclasses - Anthropic: uses cache_control with ephemeral type - Bedrock: uses cachePoint with default type - Cohere: logs warning (no native caching support) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
Test coverage for: - CacheConfig and ProviderCapabilities dataclasses - Anthropic provider caching with ephemeral cache_control - Bedrock provider caching with cachePoint - Cohere warning log when caching unsupported - Factory cache_config passthrough 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
Contributor
|
The latest updates on your projects. Learn more about Vercel for GitHub.
1 Skipped Deployment
|
Contributor
🚀 FAI Chat Lambda Preview DeployedYour Lambda function has been deployed to a preview environment! 🔗 Preview URL: https://1k3721ejqg.execute-api.us-east-1.amazonaws.com/dev2 📝 Available Endpoints:
📋 Example Usage: # Test health endpoint
curl "https://1k3721ejqg.execute-api.us-east-1.amazonaws.com/dev2/health"
# Test chat endpoint (currently returns hardcoded response)
curl -X POST "https://1k3721ejqg.execute-api.us-east-1.amazonaws.com/dev2/chat"🏷️ Stack Name: ℹ️ Note: This preview will be automatically destroyed when the PR is closed or merged. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
CacheConfigandProviderCapabilitiesdataclasses for provider-agnostic configurationcache_controlwith ephemeral type per their API speccachePointwith default type per AWS Converse APITest plan
CacheConfigandProviderCapabilitiesdataclasses🤖 Generated with Claude Code