I will share my early experiences with Microsoft Copilot, the positives and negatives, clear up some false expectations, and explain why I think of Generative AI as a digital intern.
What is Generative AI?
The name gives it away. Generative AI generates or creates something from other known things. Examples are:
- DALL-E: Creating images, such as Bing Create
- Chat GPT: A text-based interface for finding things and generating text, such as the Copilot brand from Microsoft.
There are lots of brands out there but the one that’s grabbing most of the headlines is Open AI because of ChatGPT, which is only on of their products. Like millions of others, I’ve played with ChatGPT. I’ve used it to create Terraform code. It was “OK” but I found:
- Some of the code was out of date.
- The structure wasn’t great.
I had to clean up that code to make it usable. But ChatGPT saved me time. I didn’t have to go googling. I was able to create a baseline and use my knowledge and ability to troubleshoot/edit to make the code usable.
I also “ChatGPTd” myself – don’t do it too often or you’ll go blind! Most of what ChatGPT wrote about me was correct. But there were some factual errors. Apparently, I’ve written two books on Azure. Factcheck: I have not published any books on Azure.
Some of the facts were also out of date. I have been “an Azure MVP for 2 years”. That was probably pulled from some online source. ChatGPT didn’t understand the fact (it’s just a calculated set of numbers) and therefore hadn’t the logic to use “2 years” and the publication date to recalculate – or maybe put a date in brackets with the fact.
Microsoft has just launched Microsoft 365 Copilot and there is a lot of hoopla and hype which is helping Microsoft shares, even with a bit of a slump in the stock market in general.
I’ve been playing with it and trying things out. First up was PowerPoint. Yes, I can quickly create a presentation. I can add slides. I can change images. But the logic is limited. For example, I cannot change the theme after creating the slides.
The usual fact-checking issues are there too. I used Copilot to create a presentation for my wife on company X in Ireland. The name of company X is also used by companies in the UK and the USA. Even with precise instructions, Copilot tried to inject facts from the UK/USA companies.
However, Copilot did create a skeleton presentation and that saved some time. I played around with it in Word, and it’ll generate a doc nicely. For example, it will write a sales proposal in the style of Yoda. Copilot in Teams is handy – ask it to summarize a chat that you’ve just been added to. Outlook too does a nice job at drafting an email.
Drafting is a good choice of words. Because the text is often just mumbo jumbo that is nothing to do with your or your organisation. It’s filler. In the end, it’s up to you to put in the real information that you want to push.
Bing Enterprise Chat is an option too. You can go into Bing Chat and select the M365 option. You can interrogate facts from “the graph” and M365. You can ask your agenda for the day.
Don’t ask Copilot to tell you how many vacation days are in your calendar. It will search your chat/email history for discussions of vacation time. It does not look at items in your calendar. It will not do maths – more on this next.
Go into Bing Create and ask it to create an image of a countryside scene. Expand the prompt in different ways:
- Add a run-down building
- Change the time of day
- Alter the viewing point
- Add a background
- Place some birds in the sky
- Add a person into the scene
- Make the foreground more interesting
- Change the style of image
The image changes gradually as you expand or change the prompt. This is called prompt engineering. Eventually, the final image is nothing like the first image from the basic prompt. What you ask for changes things. Think of the AI as lacking in the “I” part and be as clear and precise as you can be – like how one might instruct a toddler.
I decided to do a mini-recreation of something that I saw the folks from Prodata do with Power BI years ago for presentations. I downloaded publicly available residential property sale information for the Irish market and supplied it to Copilot.
“Tell me how many properties were sold in Dublin in 2023”. No answer because that information was not in the data. Each property sale including address, county, value, and description was in the data, but the “Y properties were sold” fact was not in the data. One would assume that an artificial intelligence would understand the question and know to list/count the items that match the search filter but that is not what happens.
I also found other logic issues. “What was the most expensive property sold in 2023” resulted in a house in Dublin for €1.55 million. I then asked it to list all houses costing more than €1 million. The €1.55m house was not included. I tried other prompts and then returned to my list question – and I got a different answer!
Don’t ask Copilot to do any maths – it won’t tell you averages, differences or sums – because that information was not in the “table” of supplied data.
You cannot expect to just throw your data at Copilot and for magic to happen. Copilot needs data to be prepared, especially custom (non-Office) data. It needs to be in consumable chunks. You also need to understand what people might ask for – and include that information in the data.
I’m wandering outside of my expertise now, but let’s take my property example. I wanted to analyze property values, do summations, averages, and comparisons. The act of preparing this data for Copilot needs to do these calculations in advance and include the results in the data that is shared with Copilot.
I am not writing off ChatGPT/Copilot. There are problems but it is still very early days and things will be improved.
Right now, we need to understand what Copilot can do, and what it is good at/not good at, and match it up with what will assist the organization.
The most important thing is how we consider Copilot. The name choice by Microsoft was deliberate. They did not call it “Pilot”.
Generative AI is an assistant. It will handle repetitive tasks based on existing data. It has no intelligence to infer new data. It cannot connect two facts that we know are logically connected but are not written down as connected. And Generative AI makes mistakes.
Microsoft called it Copilot because the pilot is responsible for the plane. The user is the pilot. The intention is that Generative AI handles the dull stuff but we add the creativity (prompt engineering/editing) and fact-checking (review/editing).
If you think about it, Copilot is acting like a Digital Intern. How are interns used? You ask them to do the simple things: get lunch, research X and write a short report, write a draft document, and so on. Does the intern produce the final product for a customer/boss? No. Is the intern responsible for what comes out of your team/department? No.
The intern is fresh out of school and knows almost nothing. They will produce exactly what you tell them – if the prompt is too general they get lost in the possibilities. You take what the intern gives you and review/edit/improve it. Their work saves you time, but your knowledge, expertise, and creativity are still required.
I might sound like a downer – I’m not. I’m just not on board the hype train. I’m saying that the train is useful to get from A to B right now, but the line doesn’t go all the way to Z yet. It is still valuable but you have to understand that value and don’t get lost in the hype and the Hollywood-ing of IT.