Generative AI: Where Does It Fit In
& What Myths Need to Be Debunked?

Kai Andrews
Kai Andrews

Azure AI, Data, & Power Platform Technology Practice Leader

In recent years, the rise of ChatGPT and other AI-powered chatbots has propelled artificial intelligence into the spotlight of mainstream attention. With their ability to generate responses to a myriad of questions, AI platforms have fostered a perception that they possess boundless capabilities. This perception has led many to believe that AI can tackle virtually any challenge, prompting business leaders to eagerly embrace the idea that generative AI holds the key to solving a wide array of business problems.

Custom Copilots Are Good For:

1. Generating New Content Based on a Library or Pool of Similar Content

Do you have a library full of similar documents that you wish you could summarize and reuse? For example, many organizations have contracts or service agreements; these documents contain a wealth of information around how, where, when, and with whom work has been done. These contracts typically include assumptions and prerequisites, many of which will be rewritten repeatedly because it is too difficult to remember where certain language was used in the past. Generative AI can excel at “reading” these documents and uncovering relevant content needed for future documents. This is arguably the most powerful advantage of using generative AI—overcoming the challenge of the “blank page.” Let Generative AI jump-start your creativity with a few draft paragraphs that you can manipulate into your desired final output.

2. Qualitative Content Comparisons

Product-centric organizations publish vast catalogs of their offerings. Most of the time, this content is published in descriptive text instead of structured tables with rows and columns. Written prose is great for a consumer who wants to read about a particular product, but the same verbose text is not ideal when the same consumer wants to pivot and compare two or more products. Instead of having to maintain both descriptive text as well as a table of features, Generative AI can extract the comparisons from unstructured text sources and provide the consumer with the desired comparisons. While this may not be suitable for a detailed, technical product comparison, this does allow for natural language comparisons of form and function.

3. Q & A Based on Curated Unstructured Content

Too many hours are spent designing and building intranet pages that users must navigate to ultimately offer up an HR or IT policy, or other standard operating procedures. Some users will hope to find the information they need by searching for it, or entering search keyword combinations that they hope will retrieve the desired document. Generative AI is ideally positioned to transform this legacy experience. Users can ask natural questions and the AI will return the favor by providing users with the information they seek. The only thing the content author needs to do is ensure that the source documentation is up to date.

4. Repeatable Calculations Based on User-Provided Data Input

Generative AI doing math? Sure! If users can provide the AI with the needed input parameters, a chatbot can easily pass this information to a workflow that calculates the output based on structured rules. For example, if a worker wants to know their prorated commission, they could provide the bot with a date range and projected hours/sales, and the system can use inherent data such as the user’s location and role to calculate the commission amount. While not exactly “generative” in the sense that the bot is creating net new content, this is a use case where users would have had to historically interact with a separate app or system and can now have a seamless integration with a bot.

5. Data Searches (Lookup) Where Users Provide Specific Parameters

This last one also skirts the concept of “generative” AI. The convenience of being able to do quick data lookups from a central chat interface cannot be dismissed. Like the repeatable calculations above, the bot would ask the user for input parameters but then, instead of performing a calculation, the bot would pass the parameters to a lookup function that filters a data table and returns the desired result. This works best when the input parameters are unique numbers or text that are not prone to misspellings. For example, entering a SKU or product ID is less error-prone than typing in someone’s last name. Ideal scenarios for data lookups could include finding shipping rates based on product IDs and zip codes or warranty information based on a product number. What doesn’t work as well are broad, undefined data searches… more on that below.

Custom Copilots Are NOT Good For:

1. Data Analysis Where You Expect Precise Numeric Answers 

Many believe that a custom copilot can be pointed to a library of structured data sources (insert your favorite Excel spreadsheet here) and have it return with data-centric answers. What do we mean by “data-centric?” We are referring to answers that are precise and repetitive. This means that the answers are accurate and identical when asked multiple times. For example, having the AI ingest a product catalog and then asking it, “How many red widgets do we sell?” will likely result in less-than-ideal answers such as, “We sell a variety of red widgets, some of which include the square widget and the round widget.” The user was hoping for, “We sell 174 red widgets.” The answer is not wrong, per se, it is just not of the quality that the user is hoping for. The user/AI interaction can be improved by providing more information to the copilot as outlined in item #5 above. Just remember that a custom copilot is not a data summarization or extraction tool. Do note that there are other copilots, such as the one built into the Fabric platform, that excel at this type of data querying… it’s just not our custom copilots, which are the main topic of this article.

2. Random Comprehensive Content Extraction and/or Manipulation 

Like the case above, if a user wants to extract or identify all occurrences of specific text in a document, then a custom copilot is likely not the right solution. Note that we underlined the term “all.” This is an important distinction. A Generative AI tool will reply well to a prompt such as, “Show me assumptions that were made when delivering our product overseas,” however, it will likely fail when prompted to, “Show me all of the assumptions made when making an international sale.” It will surely list some, but a user will not get a reliably consistent and comprehensive answer. Generative AI, at least today, will struggle with absolute expectations.

3. Content Associations Based on Company or Industry-Specific Vernacular

The large language models that are available when building a custom copilot continue to improve. However, today, they do struggle with identifying contextual relationships if all that they can refer to is company-specific or industry-specific information. Let’s walk through an example. Let’s assume that there are metal and wooden bed frames in a product catalog. The metal ones are referred to as “Luxor,” while the wooden frames are referred to as “Rustic.” These names are unique to the company selling these products and are simply known to sellers as brand identifiers. The same company may have a document that outlines the pros and cons of metal and wooden frames, but that document does not use the “Luxor” and “Rustic” identifiers. Unless this name-to-type association exists somewhere else that the copilot has access to, users asking to compare the “Luxor” and “Rustic” frame with regards to durability will likely get the dreaded, “I cannot help you with that” response. It is certainly possible to create the contextual relationship using other AI mechanisms but to expect an out-of-box association will likely lead to a disappointing experience.

4. Content Generation in the Absence of Historical Context

While this last scenario appears self-evident, it does need to be included in the list. There is a certain perceived “magic” surrounding many of the AI experiences today. It seems that we can ask any question and receive an answer. But this illusion is a result of incredible investments in teaching and grounding the AI in a vast amount of information. It is necessary to remember that many organizations have proprietary data and information, and that this information must be provided to the custom copilot for it to learn and generate new answers. Depending on where and how an organization has historically stored information, it may require a focused effort to migrate and secure the content in a cloud-based repository. While not a trivial exercise, a pragmatic and iterative approach can yield quick and evolving experiences. Return on investment should be calculated for such initiatives… but that is another story for another day.

We Can Help!

Custom copilots are a revolutionary tool that extends the benefits of Generative AI and natural language interaction to any organization. Let us help you explore when and where these benefits can be employed for your workforce and customers.

Sign up for an Action Accelerator Workshop to learn the best use cases and benefits that Custom Copilots can provide to your organization.
Contact us at info@eGroup-us.com or complete the form below!

Need Help Navigating Custom Copilots?

Contact our team to schedule a call with one of our Copilot experts.