Updated: April 24, 2024
Introducing APIs for GPT-3.5 Turbo and Whisper
Developers can now integrate GPT-3.5 Turbo and Whisper models into their apps and products through our API.
Update on April 24, 2024: The ChatGPT API name has been discontinued. Mentions of the ChatGPT API in this blog refer to the GPT-3.5 Turbo API.
ChatGPT and Whisper models are now available on our API, giving developers access to cutting-edge language (not just chat!) and speech-to-text capabilities. Through a series of system-wide optimizations, weâve achieved 90% cost reduction for ChatGPT since December; weâre now passing through those savings to API users. Developers can now use our open-source Whisper large-v2 model in the API with much faster and cost-effective results. GPT-3.5 API users can expect continuous model improvements and the option to choose dedicated capacity for deeper control over the models. Weâve also listened closely to feedback from our developers and refined our API terms of service to better meet their needs.
Early users of GPT-3.5 Turbo and Whisper APIs
Snap Incâ (opens in a new window)., the creator of Snapchat, introduced My AI for Snapchat+ this week. The experimental feature is running on GPT-3.5 API. My AI offers Snapchatters a friendly, customizable chatbot at their fingertips that offers recommendations, and can even write a haiku for friends in seconds. Snapchat, where communication and messaging is a daily behavior, has 750 million monthly Snapchatters:
Quizletâ (opens in a new window) is a global learning platform with more than 60 million students using it to study, practice and master whatever theyâre learning. Quizlet has worked with OpenAI for the last three years, leveraging GPT-3 across multiple use cases, including vocabulary learning and practice tests. With the launch of GPT-3.5 API, Quizlet is introducing Q-Chat, a fully-adaptive AI tutor that engages students with adaptive questions based on relevant study materials delivered through a fun chat experience:
Instacartâ (opens in a new window) is augmenting the Instacart app to enable customers to ask about food and get inspirational, shoppable answers. This uses ChatGPT alongside Instacartâs own AI and product data from their 75,000+ retail partner store locations to help customers discover ideas for open-ended shopping goals, such as âHow do I make great fish tacos?â or âWhatâs a healthy lunch for my kids?â Instacart plans to launch âAsk Instacartâ later this year:
Shopâ (opens in a new window), Shopifyâs consumer app, is used by 100 million shoppers to find and engage with the products and brands they love. GPT-3.5 API is used to power Shopâs new shopping assistant. When shoppers search for products, the shopping assistant makes personalized recommendations based on their requests. Shopâs new AI-powered shopping assistant will streamline in-app shopping by scanning millions of products to quickly find what buyers are looking forâor help them discover something new:
Speakâ (opens in a new window) is an AI-powered language learning app focused on building the best path to spoken fluency. Theyâre the fastest-growing English app in South Korea, and are already using the Whisper API to power a new AI speaking companion product, and rapidly bring it to the rest of the globe. Whisperâs human-level accuracy for language learners of every level unlocks true open-ended conversational practice and highly accurate feedback:
GPT-3.5 API
Model: The ChatGPT model family we are releasing today, gpt-3.5-turbo, is the same model used in the ChatGPT product. It is priced at $0.002 per 1k tokens, which is 10x cheaper than our existing GPT-3.5 models. Itâs also our best model for many non-chat use casesâweâve seen early testers migrate from text-davinci-003 to gpt-3.5-turbo with only a small amount of adjustment needed to their prompts.
API: Traditionally, GPT models consume unstructured text, which is represented to the model as a sequence of âtokens.â ChatGPT models instead consume a sequence of messages together with metadata. (For the curious: under the hood, the input is still rendered to the model as a sequence of âtokensâ for the model to consume; the raw format used by the model is a new format called Chat Markup Languageâ (opens in a new window) (âChatMLâ).)
Weâve created a new endpoint to interact with our ChatGPT models:
ChatGPT upgrades
We are constantly improving our ChatGPT models, and want to make these enhancements available to developers as well. Developers who use the gpt-3.5-turbo model will always get our recommended stable model, while still having the flexibility to opt for a specific model version. For example, today weâre releasing gpt-3.5-turbo-0301, which will be supported through at least June 1st, and weâll update gpt-3.5-turbo to a new stable release in April. The models pageâ (opens in a new window) will provide switchover updates.
Dedicated instances
We are also now offering dedicated instances for users who want deeper control over the specific model version and system performance. By default, requests are run on compute infrastructure shared with other users, who pay per request. Our API runs on Azure, and with dedicated instances, developers will pay by time period for an allocation of compute infrastructure thatâs reserved for serving their requests.
Developers get full control over the instanceâs load (higher load improves throughput but makes each request slower), the option to enable features such as longer context limits, and the ability to pin the model snapshot.
Dedicated instances can make economic sense for developers running beyond ~450M tokens per day. Additionally, it enables directly optimizing a developerâs workload against hardware performance, which can dramatically reduce costs relative to shared infrastructure. For dedicated instance inquiries, contact usâ .
Whisper API
Whisperâ , the speech-to-text model we open-sourced in September 2022, has received immense praise from the developer community but can also be hard to run. Weâve now made the large-v2 model available through our API, which gives convenient on-demand access priced at $0.006 / minute. In addition, our highly-optimized serving stack ensures faster performance compared to other services.
Whisper API is available through our transcriptions (transcribes in source language) or translations (transcribes into English) endpoints, and accepts a variety of formats (m4a, mp3, mp4, mpeg, mpga, wav, webm):
Developer focus
Over the past six months, weâve been collecting feedback from our API customers to understand how we can better serve them. Weâve made concrete changes, such as:
- Data submitted through the API is no longer used for service improvements (including model training) unless the organization opts in
- Implementing a default 30-day data retention policy for API users, with options for stricter retention depending on user needs.
- Removing our pre-launch review (unlocked by improving our automated monitoring)
- Improving developer documentation
- Simplifying our Terms of Service and Usage Policiesâ (opens in a new window), including terms around data ownership: users own the input and output of the models.
For the past two months our uptime has not met our own expectations nor that of our users. Our engineering teamâs top priority is now stability of production use casesâwe know that ensuring AI benefits all of humanity requires being a reliable service provider. Please hold us accountable for improved uptime over the upcoming months!
We believe that AI can provide incredible opportunities and economic empowerment to everyone, and the best way to achieve that is to allow everyone to build with it. We hope that the changes we announced today will lead to numerous applications that everyone can benefit from. Start building next-generation apps powered by ChatGPT & Whisper.
Contributors
Jeff Belgum, Jake Berdine, Trevor Cai, Alexander Carney, Brooke Chan, Che Chang, Derek Chen, Ruby Chen, Aidan Clark, Thomas Degry, Steve Dowling, Sheila Dunning, Liam Fedus, Vik Goel, Scott Gray, Aurelia Guy, Jeff Harris, Peter Hoeschele, Angela Jiang, Denny Jin, Jong Wook Kim, Yongjik Kim, Michael Lampe, Daniel Levy, Brad Lightcap, Patricia Lue, Bianca Martin, Christine McLeavey, Luke Metz, Andrey Mishchenko, Vinnie Monaco, Evan Morikawa, Mira Murati, Rohan Nuttall, Alex Paino, Ashley Pantuliano, Mikhail Pavlov, Andrew Peng, Henrique Ponde de Oliveira Pinto, Alec Radford, Kendra Rimbach, Aliisa Rosenthal, Nick Ryder, Ted Sanders, Heather Schmidt, John Schulman, Zarina Stanik, Felipe Such, Nick Turley, Carroll Wainwright, Peter Welinder, Clemens Winter, Sherwin Wu, Tao Xu, Qiming Yuan, Barret Zoph