Simulate throttling of the OpenAI API
This sample contains a preset and mocks to simulate the throttling of the OpenAI API. As more and more applications use OpenAI API to use AI, it's important that developers can verify that their apps can properly handle cases when the API is throttled. Especially when the app is using lower-tier API plans, throttling is more likely to happen so validating the app's behavior is crucial to ensure great user experience. Using this preset you can simulate throttling of the OpenAI API and see how your app will handle it. The preset includes 3 different throttling scenarios: exceeded tokens per minute, exceeded number of requests per minute and service overload, which includes dynamic simulation of handling retry-after. Proxy will simulate throttling the API using one of these modes at random.
Simulate throttling of the OpenAI API
Summary
This sample contains a preset and mocks to simulate the throttling of the OpenAI API.
As more and more applications use OpenAI API to use AI, it’s important that developers can verify that their apps can properly handle cases when the API is throttled. Especially when the app is using lower-tier API plans, throttling is more likely to happen so validating the app’s behavior is crucial to ensure great user experience.
Using this preset you can simulate throttling of the OpenAI API and see how your app will handle it.
Compatibility
Contributors
Version history
| Version | Date | Comments |
|---|---|---|
| 1.24 | July 1, 2026 | Updated to Dev Proxy v3.1.0 |
| 1.23 | June 17, 2026 | Updated to Dev Proxy v3.0.1 |
| 1.13 | May 30, 2026 | Updated to Dev Proxy v3.0.0 |
| 1.12 | March 26, 2026 | Updated to Dev Proxy v2.3.0 |
| 1.11 | March 11, 2026 | Updated to Dev Proxy v2.2.0 |
| 1.10 | February 4, 2026 | Updated to Dev Proxy v2.1.0 |
| 1.9 | January 18, 2026 | Moved config files to .devproxy folder |
| 1.8 | January 5, 2026 | Updated to Dev Proxy v2.0.0 |
| 1.7 | June 27, 2025 | Updated to Dev Proxy v0.29.2 |
| 1.6 | January 9, 2025 | Updated to Dev Proxy v0.23 |
| 1.5 | January 17, 2024 | Updated plugin path |
| 1.4 | January 11, 2024 | Updated to new format |
| 1.3 | January 7, 2024 | Added new throttling error |
| 1.2 | December 22, 2023 | Updated to new format |
| 1.1 | November 14, 2023 | Renamed to Dev Proxy |
| 1.0 | August 10, 2023 | Initial release |
Minimal path to awesome
- Get the sample:
-
Download just this sample:
npx gitload-cli https://github.com/pnp/proxy-samples/tree/main/samples/openai-throttlingor
-
Download as a .ZIP file and unzip it, or
-
Clone this repository
-
- Start Dev Proxy by running
devproxy
Features
This preset includes configuration for simulating 3 different throttling scenarios:
- exceeded tokens per minute
- exceeded number of requests per minute
- service overload, which includes dynamic simulation of handling retry-after
Proxy will simulate throttling the API using one of these modes at random.
Help
We do not support samples, but this community is always willing to help, and we want to improve these samples. We use GitHub to track issues, which makes it easy for community members to volunteer their time and help resolve issues.
You can try looking at issues related to this sample to see if anybody else is having the same issues.
If you encounter any issues using this sample, create a new issue.
Finally, if you have an idea for improvement, make a suggestion.
Disclaimer
THIS CODE IS PROVIDED AS IS WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING ANY IMPLIED WARRANTIES OF FITNESS FOR A PARTICULAR PURPOSE, MERCHANTABILITY, OR NON-INFRINGEMENT.