4 phút đọc
NVIDIA-NeMo/Switchyard: Switchyard lets LLM applications route traffic across models and providers while preservin – Repository NVIDIA-NeMo/Switchyard. Mô tả: Switchyard lets LLM applications route traffic across models and providers while preserving native OpenAI and Anthropic API compatibility – enabling flexible model selection, benchmarking, and cost/performance optimization.. 3478 stars, 349 forks, ngôn ngữ chính Rust, cập nhật 2026-11-02.
1. Repository làm gì?
Switchyard is an open-source library that helps an AI agent choose which model handles each request. It combines efficient models with more capable models so you can balance task success, cost, and latency on your workload.
Adopt Switchyard by embedding its Rust core, libsy, in your gateway or agent harness. You own the model clients, credentials, and service. Switchyard supplies the routing decision.
Quickstart · Documentation · Benchmarks · What’s new · Roadmap · Releases · Contributing
Evaluate the complete agent, model pool, and routing configuration against your single-model baseline. A cheaper model call does not guarantee a cheaper successfully completed task. The tuning loop gives the steps for that evaluation, and Troubleshooting lists the settings that disable routing without an error.
Relay 0.8.x and 0.9.0 can lose upstream error status and details when the plugin is enabled, even for models outside its routes. Review the upstream error compatibility note before enabling the plugin.
LiteLLM and NeMo Relay run in your own deployment. Follow each gateway's guidance for credentials and service operation.
2. Dữ liệu và cấu trúc đáng chú ý
Algorithm::run_stream hands model calls and the final routing outcome back to your application. In this development-API outline, comments mark the work your code supplies:
The standalone server is provided only for demos and local evaluation. It is not a supported production deployment path.
Evaluate routing locally with the standalone server and your own model pool, then verify routing is active.
The experimental desktop and terminal quickstart covers local installation, coding-tool connections, sessions, usage, recovery, and uninstall.
Agent-specific guides are available for pi and Oh My Pi.
Start with Auto. Choose Task, Execution, or Composite when you need more control. These names describe routing choices. The configuration keys are unchanged.
Auto is a fixed preset in v0.3.0. It does not compare strategies at runtime. The TOML runner supports type = "auto". In Rust, use StageRouterConfig::new(PickerMode::EfficientFirst, 0.5) for the same preset.
3. Khả năng ứng dụng và giới hạn
The routing overview retains the full catalogue, including Plan/Execute, escalation, advisor, sub-agent, and random strategies. Integration support varies: check the gateway guide before choosing an algorithm.
Prefill routing is designed for fast, efficient model selection using a model's prefill activations and a classifier trained on your own evaluation data.
Compare benchmark results and configs for Terminal-Bench 2.1 and DeepSWE. For latency and routing overhead testing, see Soak Testing.
Pre-1.0 software. APIs, configuration, and routing behavior can change between releases. Pin the version you integrate.
Features and integrations recently merged into main.
2026-10-16 · Route between Sonnet and Opus in Claude Code Use your Claude login for automatic routing on Linux x86_64 and see the selected model in the status line. Setup guide →
2026-10-15 · Use decision models in composite routing Use Mercury Decide to judge tasks and Stage routing to adjust model selection as tools run. Example configuration →
2026-10-13 · Use Jev to choose between models Configure Jev as a task judge with candidate descriptions and a probability cutoff. Configuration guide →
2026-10-12 · Bring Switchyard to TypeScript and Pi Select models in Node.js or try automatic routing through Pi’s virtual models. TypeScript bindings → · Pi example →
Số liệu repository có thể thay đổi; nên mở liên kết nguồn để kiểm tra README, giấy phép và trạng thái phát hành.
Nguồn tham khảo: Xem bài gốc
