I recently sat down with Microsoft’s Jeremy Thake, Group Program Manager in charge of the Microsoft 365 Copilot developer experience, to catch up on recent announcements from the Microsoft Copilot Wave 2 event and the GA milestone. You can check out that conversation in episode 316 of the Microsoft 365 Developer Podcast.
Microsoft 365 Copilot GA with Andrew Connell
In this episode, #316 of the Microsoft 365 Developer Podcast, Jeremy Thake catches up with Andrew Connell to talk through the latest updates you can work with now in Declarative agents inside of Microsoft 365 Copilot. Jeremy walks through the renamed elements and all the options for building Declarative agents.
https://www.m365devpodcast.com/e/declarative-agents-update/
Now, if you’re like me, you’ve had difficulty keeping up with the evolution—and ebb and flow—of Copilot extensibility over the last year.
In this article, I want to give you my take on the current state of the extensibility story with Copilot for Microsoft 365 by putting the announcements in some historical context, and touch on what we can expect to come next!
Learn more about from the related announcement from Jeremy Thake, Group Program Manager in charge of the Microsoft 365 Copilot developer experience:
What’s new for Microsoft 365 Copilot developer tooling
Learn how declarative agents, now generally available, can quickly and easily integrate your enterprise data securely with Microsoft 365 Copilot.
https://devblogs.microsoft.com/microsoft365dev/whats-new-for-microsoft-365-copilot-developer-tooling/
Microsoft 365 Copilot Extensibility history
Let’s dive into a brief history of Copilot extensibility in Microsoft 365.
Ignite 2023 Conference (November 2023)
In November 2023 at the Ignite conference in Seattle, Microsoft announced Copilot Studio and introduced the first pieces of their developer story for Copilot. This included enhancing Microsoft Teams message extensions (ME) with additional properties, allowing them to become Copilot plugins for fetching real-time data when Copilot deduced a matching user’s intent from their prompt to what the ME could provide.
Build 2024 (May 2024)
Then in May 2024 at the Build conference in Seattle, Microsoft unveiled the Copilot extensibility story. This covered Microsoft Teams messaging extensions (in preview), OpenAPI plugins (not yet available at the time), declarative copilots (in preview), and custom engine copilots (generally available). The entire Copilot extensibility story was well covered in one breakout session, BRK150 - Developer’s Guide to Customizing Microsoft Copilot.
Learn more about all the news and updates for Microsoft 365 full-stack developers (including Copilot news) from Build 2024 in my article, Microsoft 365 Full-Stack Developer’s Recap to Build 2024.
The Microsoft 365 Full-Stack Developer's Recap to Build 2024
I spent over 40+ hours watching 30+ sessions, keynotes, & reading announcements to gather what Microsoft 365 full-stack developers should know from Build 2024.
https://www.andrewconnell.com/blog/microsoft-365-fullstack-developer-recap-microsoft-build-2024/
Copilot Wave 2 (September 2024)
In September 2024, at the Copilot Wave 2 announcement virtual event (watch it here), Microsoft introduced new user features for Copilot in various Office applications. For the developer story, they rebranded “Copilot extensions” to “Copilot agents” as the overarching term. This meant declarative Copilots became declarative agents, and custom engine Copilots became custom engine agents.
Microsoft 365 Copilot Wave 2: Pages, Python in Excel, and agents
Launching the next wave of Microsoft 365 Copilot, bringing together web, work, and pages as a complete new design system for work.
https://www.microsoft.com/microsoft-365/blog/2024/09/16/microsoft-365-copilot-wave-2-pages-python-in-excel-and-agents/
However, the Wave 2 event provided little information for developers beyond this renaming.
That’s what the rest of this article will unpack!
Summarizing Microsoft 365 Copilot GA news
Before we dive into the details from today’s GA announcements, here are the main points from today’s generally available (GA) announcement:
- Copilot agents: now the umbrella term, replacing Copilot extensions.
- BizChat: new name for Copilot for Microsoft 365 (also known as Microsoft 365 Copilot and M365 Copilot).
- Declarative agents (DA’s): GA and available in the Microsoft Teams right-hand rail experience.
- Custom engine agents (CEA’s): GA when published to Microsoft Teams.
- OpenAPI plugins (API plugins): GA in declarative agents.
These add to the existing parts of the Copilot story in Microsoft 365 that were already GA, including
- Copilot Studio - GA since August 2024
- Microsoft Graph connectors - GA for quite some time.
Now, let’s dive deeper into these announcements.
Renaming all things Copilot in Microsoft 365 - Why?
Why did Microsoft rename so many things in this generally available announcement? They noticed the industry was trending towards calling custom AI solutions “agents.” Initially, Microsoft was cautious about using this term because their offerings weren’t fully agent-full, or having the ability to act on our behalf automatically while it was something they were working on for the future. However, they ultimately decided to adopt the “agent” terminology now to avoid future renaming.
Long-time Microsoft 365 developers are familiar with such naming inconsistencies.
For instance, in Microsoft Teams, a tab for a personal app is called a staticTab in the app manifest, while a channel tab is a configurableTab. In SharePoint, we have “sites” and “site collections,” but in the code, they’re referred to as SPWebs and SPSites respectively.
By renaming everything at the generally available milestone, Microsoft aims to avoid such confusion. These new names are now consistent across documentation, user experience, code, and all manifests.
Copilot Agents
The first term you want to be aware of is Copilot agents. Consider this the overarching term for all Copilot extensibility that’s replacing Microsoft used to call Copilot extensions. Copilot extensions was the term heavily used at the Microsoft Build 2024 conference.
BizChat (was: Copilot for M365)
BizChat, also known as Microsoft 365 Copilot or M365 Copilot, was formerly known as Copilot for Microsoft 365. BizChat is the chat experience that leverages data from Microsoft 365 and is available across various Microsoft 365 experiences, including Microsoft Teams, all Office apps, and office.com/chat.
When you send a prompt to BizChat, it draws from two knowledge sources:
- Its Large Language Model (LLM) training, is similar to what you get with ChatGPT from OpenAI.
- Access to all your data within your Microsoft 365 tenant, which sets it apart from other AI chatbots.
This combination allows BizChat to provide unique, personalized responses based on your organization’s information.
While BizChat offers a great out-of-the-box (OOTB) experience, customers wanted to extend its capabilities with additional data sources. This would enhance its knowledge and make it more useful for different organizations.
Let’s explore this Copilot extensibility story and what’s covered by the October 2024 general availability announcement.
Microsoft Graph Connectors
The first extensibility option is Microsoft Graph connectors. These have been generally available for quite some time. Microsoft Graph connectors enable customers to upload unstructured content to Microsoft Graph, which then indexes and vectorizes the content.
Because this content is not immediately available since it needs to get indexed and vectorized, you should be focusing on content that doesn’t change often. For example, manuals, standard operating procedures, checklists, and policy documents… all good examples.
This adds a third knowledge source to Copilot. Copilot can now use this data provided by a Microsoft Graph connector to expand its knowledge beyond the other two sources I previously mentioned.
All of the content uploaded through Microsoft Graph connectors is available within the mainline BizChat experience.
This effectively serves as a third knowledge source for Copilot.
Plugins (Message Extension & Open API)
The next extensibility option is a plugin. In Copilot, developers can provide plugins that allow Copilot to implement the Retrieval Augmented Generation (RAG) pattern. This pattern fetches structured, real-time information that Copilot can use to enhance its response. Copilot can deduce from the user’s intent which plugins it can use to gather additional information when the user submits their prompt.
A plugin will add a fourth knowledge source to Copilot.
There are two types of plugins that exist today.
- Microsoft Teams Message Extension (ME) plugins, were announced at Ignite 2023 and have been in preview since then. These plugins are still currently in preview. More on these & their future later in this article.
- OpenAPI plugin (aka API plugin). These were announced at Build 2024 but are not yet available. API plugins are now generally available in the Copilot experience if they are included within a declarative agent.
As part of the GA announcement, we previously had the ability to use plugins by @mentioning them in the main BizChat experience. However, these now belong as part of declarative agents. You can no longer @mention them as we saw prior to the Wave 2 announcements in September 2024.
Declarative agents (DA’s)
Part of today’s announcement includes the general availability of declarative agents, formerly known as declarative copilots.
Under the hood, declarative agents run on the BizChat stack. This allows them to leverage the three knowledge sources I previously mentioned (LLM training, your Microsoft 365 tenant content, and content from Microsoft Graph Connectors), and use Microsoft’s models and orchestrator.
You’ll see declarative agents in the right-hand rail of Microsoft Teams if you have a Microsoft 365 Copilot license (formerly called “Copilot for Microsoft 365” license). You can also @mention them in various Copilot experiences, including Office clients and the office.com/chat experience.
Declarative agents can include plugins, but they’re currently limited to OpenAPI plugins. Microsoft Teams message extension plugins are not supported in declarative agents at general availability. Microsoft is working on addressing the investments developers made during the message extensions preview.
Microsoft’s official recommendation is to create API plugins. Why?
During private and public preview periods over the past year, customers & Microsoft found that simply adding a message extension plugin to what we now call BizChat didn’t work as well as expected. Developers had to invest much more work to train Copilot to effectively use and call message extensions, as the orchestrator couldn’t pick them up as easily as anticipated. Most customers in the private preview requested a way to directly call APIs, which is why API-based plugins became the preferred option. More on message extension plugins & their future later in this article.c
Custom engine agents (CEA’s)
Custom engine agents, formerly called custom engine copilots, are now generally available when published to Microsoft Teams. However, they are not yet generally available in BizChat.
These agents allow developers to implement their own Copilot using any AI stack, such as using their models and orchestrators, even on their platforms. For example, developers can leverage the Teams AI Library, Copilot Studio, or choose models from Hugging Face, Meta, Google Gemini, Amazon Web Services (AWS), or other sources.
The key advantage of custom engine agents is that developers have control over the model and orchestrator. For example, they can perform the fine-tuning of their models, but it’s worth noting that declarative agents also offer significant customization options through developer-defined instructions.
Looking ahead in custom engine agents
What’s not yet generally available or in preview for custom engine agents? Microsoft is currently working on enabling handoff from a plugin to a custom engine agent.
In the future, custom engine agents will appear in the right-hand rail of the Microsoft Teams experience. Users won’t be able to distinguish between declarative agents and custom engine agents, and they’ll have the ability to @mention custom engine agents just as they do with declarative agents.
Handoff
You might recall the handoff feature demonstrated at Build 2024. This feature allowed us to transition from an immersive chat experience to a custom engine agent. However, with recent investments in custom engine agents—such as the ability to @mention them and list them in the right-hand rail—handoff will no longer be necessary.
In other words, you can think of handoff as a stopgap option until custom engine agents can be pinned in the right-hand rail and @mentioned.
Copilot Studio
As I mentioned earlier, Copilot Studio has been generally available since August 2024.
Before today’s Copilot extensibility GA announcement, you could create actions in Copilot Studio. These actions could be prompts using the AI prompt action, conversation plugins, or REST API plugin. However, this approach is no longer available.
With today’s GA announcement, you now build agents in Copilot Studio. You can add different action types to these agents, such as the AI prompt action, conversational plugins, or API plugins. You can then publish the agent to BizChat. Additionally, you can create an API plugin in Copilot Studio, which is called a REST API action.
Agent Builder
While not part of the GA announcement, it’s worth calling out the agent builder shown during the Wave 2 announcements in September. When you’re inside an office.com/chat experience or Microsoft Teams where declarative agents are listed, you’ll find a new “Create Agent” link.
The create agent link gives users a mini Copilot Studio where they can provide instructions, select SharePoint knowledge either within the site or from specific files, and pick Microsoft Graph connectors that they’ve previously created. As a user, you can publish it yourself or use the share link to share the agent you built with others.
This raises the question: why use Copilot Studio or Microsoft Teams toolkit to build a declarative agent if you can do this using the agent builder?
One key limitation is that the agent builder can’t take advantage of API plugins or any other actions that require calling external APIs. These more complex tasks remain in the realm of developers and are not available through the Agent Builder.
What’s next for Copilot extensibility?
This article has covered the general availability announcements for Copilot extensibility. Now, let’s highlight a few upcoming developments in this area that you can look forward to.
Addressing Microsoft Teams Message Extension-based Copilot Plugins
During the preview period, many customers created message extension plugins for use within Microsoft Teams and Outlook, following the guidance on how to make them available as plugins for Copilot. However, as mentioned earlier, these are not available for use within declarative agents at GA.
Microsoft isn’t leaving us hanging, though; Microsoft is actively working on options for developers to bring these message extension plugin investments forward for use within Copilot agents.
Creating OpenAPI Specification Documents
One of the key insights Microsoft gained during the public and private preview period over the last year is that most organizations don’t have OpenAPI documents (formerly known as Swagger files) for their APIs. Furthermore, existing API documents often aren’t well-suited for use by LLMs. These LLMs struggle to interpret and utilize the information in these OpenAPI documents effectively.
What’s needed in OpenAPI docs and specifications are clear, detailed descriptions for the various endpoints and their parameters. These descriptions help guide and train the LLM on what the API can do and how to use it properly.
Recognizing this challenge, Microsoft is working on streamlining this process through improved developer experiences in Visual Studio Code. Stay tuned for upcoming enhancements in this area.