Users have been asking for plugins since we launched ChatGPT because they unlock a vast range of possible use cases. We’re starting with a small set of users and are planning to gradually roll out larger-scale access as we learn more (for plugin developers, ChatGPT users, and after an alpha period, API users who would like to integrate plugins into their products). We’re excited to build a community shaping the future of the human–AI interaction paradigm.
What happened if I told you ChatGPT plugins can now be created?
Plugin developers who have been invited off our waitlist can use our documentation to build a plugin for ChatGPT, which then lists the enabled plugins in the prompt shown to the language model as well as documentation to instruct the model how to use each. The first plugins have been created by Expedia, FiscalNote, Instacart, KAYAK, Klarna, Milo, OpenTable, Shopify, Slack, Speak, Wolfram, and Zapier.
OpenAi, the creator of GPT has made two plugins themselves, a web browser and code interpreter. How can you use these plugins, well sure, it’s easy!
You simply head to your GPT panel and edit the plugins and then you’re ready to rock and roll.
Browsing
Alpha
An experimental model that knows when and how to browse the internet
Motivated by past work (our own WebGPT, as well as GopherCite, BlenderBot2, LaMDA2 and others), allowing language models to read information from the internet strictly expands the amount of content they can discuss, going beyond the training corpus to fresh information from the present day.
Code interpreter
Alpha
An experimental ChatGPT model that can use Python, handle uploads and downloads
We provide our models with a working Python interpreter in a sandboxed, firewalled execution environment, along with some ephemeral disk space. Code run by our interpreter plugin is evaluated in a persistent session that is alive for the duration of a chat conversation (with an upper-bound timeout) and subsequent calls can build on top of each other. We support uploading files to the current conversation workspace and downloading the results of your work.
Here’s an example of the kind of experience that browsing opens up to ChatGPT users, that previously would have had the model politely point out that its training data didn’t include enough information to let it answer. This example, in which ChatGPT retrieves recent information about the latest Oscars, and then performs now-familiar ChatGPT poetry feats, is one way that browsing can be an additive experience.
Retrieval
The open-source retrieval plugin enables ChatGPT to access personal or organizational information sources (with permission). It allows users to obtain the most relevant document snippets from their data sources, such as files, notes, emails or public documentation, by asking questions or expressing needs in natural language.
As an open-source and self-hosted solution, developers can deploy their own version of the plugin and register it with ChatGPT. The plugin leverages OpenAI embeddings and allows developers to choose a vector database (Milvus, Pinecone, Qdrant, Redis, Weaviate or Zilliz) for indexing and searching documents. Information sources can be synchronized with the database using webhooks.
To begin, visit the retrieval plugin repository.
I can’t wait to see what comes of this and how it will change the GPT area, are you looking forward to these plugins and have you used them yet?