IT’s lovefest with GPT-3 needs to meet reality now

There are few pieces of technology — ever — that have enjoyed a deeper lovefest from the media and the consumer public than ChatGPT and other efforts built atop vanilla GPT-3. And enterprise IT executives have been stampeding to develop homegrown apps based on GPT-3.

So far, so good.

But as we’ve seen before — think of the internet rush of the mid-1990s or blockchain more recently — companies can easily get ahead of themselves by making big investments on things other than strategic goals. 

I remember in the very early days of the World Wide Web, I would be talking with an executive about efforts to launch a website and asking, “Why? What for? What are you trying to accomplish?” Instead of getting a litany of concrete goals and objectives, I heard variations of, “One of our board members read about it and insists that we get one,” or, “Our CEO’s son can’t stop talking about it” or the lamest: “Everyone else seems to be doing it.”

Those are the exact kind of comments I’m hearing today about GPT-3. To be clear, many of the industry’s most hyped technologies ultimately proved to be strategically important. Not all, but many.

ChatGPT has an impressive set of capabilities, but it’s really simply a massive database with an interface that effectively mimics human communications. Think of it as a hyper-powered intranet. 

The information presented in most ChatGPT exchanges is nothing that couldn’t be found with a decent Google search. “Found” is the key point. A user might have to review dozens of Google search results to find the one answer that ChatGPT finds. 

Another advantage — and this is arguably where the most IT value can be found — is in its human-like interactions. In theory, this could ultimately allow a lot of coding projects to forego the more elementary programming efforts. Most programming projects start with some line-of-business executive or manager saying, “We need the system to be able to XYZ. Go make that happen” to technical talent.

What if ChatGPT could bypass some of that coding talent and create code directly based on line-of-business instructions? Some coding is highly creative and imaginative and will continue to need a human touch. But, candidly, a lot of programming is painstaking and repetitive. Could GPT take over that portion?

On the downside, we have all seen the ridiculous errors and flat-out fabrications that GPT-3 systems have delivered. Until that’s fixed, GPT-3 uses will be limited. As tempting as the natural language interface is, letting a GPT-3 chat program speak on your behalf with customers is asking for a disaster.

So how can it be used? There are two ways to explore that critical question: prescriptive and open-ended. Depending on your business and objectives — not to mention budget — both approaches can be very attractive.

The prescriptive approach is easier and is likely to deliver more near-term results. What are you trying to accomplish? What can GPT-3 do today to help your business and perhaps make viable some product/service rollouts you’ve wanted for awhile but couldn’t make happen.

The open-ended approach is more interesting. That is where you give your team wide latitude to play with GPT-3 and get creative and see what it can do. But that approach needs to have some limits.

CIOs need to figure out what they want to do with this or developers will spin their wheels on wacky ideas without end, said Scott Castle, the chief strategy officer at analytics firm Sisense. “CIOs need to strategically filter or else you are just Willy Wonka in the chocolate factory,” he said.

One of the top analytics experts in the industry is Roy Ben-Alta. Ben-Alta just last month left Meta/Facebook as director of AI to launch his own company. Before Meta, he served 11 years with Amazon, ending with the title of director, analytics/machine learning, data streaming and NoSQL databases. 

CIOs “need to ask, ‘What is this going to do to my business?’” Ben-Alta said. “The best way to approach that is to work backwards from the customer. What problem are we trying to solve? Here is the catch: in order to jump in, you need to spend a lot of money. Training requires a lot of GPUs. Every use case requires specific data sources and if they don’t have the data availability, they need to determine how much will it cost them to acquire that data.”

The most powerful element of GPT-3 is its coding, its interface. But for businesses trying to build atop all of that, the issue won’t be coding. It will absolutely be data.

“The Achilles heel of every analytics system is data quality,” Ben-Alta said. “Most of the work involves the data. Data integration is always the problem and it is the most challenging element. The format of the data and the type of data to be used is evolving. The analytics model only gets good when the data gets good.”

The data concern is important, but a lot of the analytics complexities materialize because of data interactions. Waqaas Al-Siddiq, the CEO of medical analytics firm Biotricity, offers a powerful example of how interactions can undermine the best of large language models. 

“Anything that is a spike or an anomaly — as in three or four standard deviations from the mean — it will have a lot of trouble. The more variables the more challenging it gets because you will need more data,” Al-Siddiq said. “But because they are anomalies, you won’t be able to supply enough data.”

Al-Siddiq offered an example of inventory logistics: “Let’s say there is a construction project that happens causing people to divert and during that same two weeks there is a heat spell. That caused people to stop and grab a drink. Now there are multiple variables. You will never have enough data to handle that anomaly in an autonomous or predictable way unless you make sure you are tracking those variables. The more variables you track, the more complex your AI model.”

There is a tremendous amount of potential value in leveraging these large language models, but it’s obviously a good idea to not let emotions take hold. 

“This whole buzz is because of one product from one company: OpenAI. This society runs a lot on the bandwagon effect, the fear of missing out,” said Jay Chakraborty, a partner at PwC (formerly Price-Waterhouse-Coopers). “This is another version of the California gold rush, the dotcom euphoria, that whole Y2K ‘the whole world is going to crash’ situation.”

Chakraborty encourages CIOs to simply do some sandbox experiments and “push the business to come up with ideas and use-cases. If I’m a hedge fund, why would I not think about what I can automate? It could easily knock out investment letters a lot more efficiently. It generates the analysis automatically and that is one more crucial step to the finish line. It’s writing the end piece.”

Forrester analyst Rowan Curran, who specializes in data science, machine learning, artificial intelligence and computer vision, agrees GPT-3 has great potential, but said  executives must look at it as just another strategic effort. 

“The first thing to do is take a step back from the public attention and ask, ‘Where can we actually apply these where we can take advantage of their strengths and downplay their weaknesses? How can we use it?’” Curran said.

Although GPT-3 “is potentially a fantastic way to innovate, it is also really important to concentrate on what is practical in the short term. There absolutely is a need to educate yourself about what is even possible. This is a new and dynamic space,” Curran said, adding that he sees serious limits. For example, he considers the idea of  using it for chat in a customer-facing application is “deeply irresponsible.”

Large language models are nothing new, but the human-mimicking frontend that GPT-3 has crafted has awakened the IT world to the possibilities and allowed many to dream. That is great, as long as they wake up just before investment decisions are made and project directions are decided.