AI Confidential: Notes From An Off-The-Record AI Expert Roundtable

If you want to get past a lot of the hype around generative AI, here’s a trick: Get a bunch of really, really smart folks who have been spending most of the last few years wrangling with it. Put them in a room, close the door and tell them that we’re all going to talk/vent about their experiences and none of what they say will be attributed to them.

You’ll be astounded at what you hear—at least I was when we pulled together exactly this kind of group at our recent Chief Executive of the Year pre-gala festivities. Who was in the room? Can’t say exactly as I promised anonymity in exchange for true candor (I will say Lisa Su from AMD, our CEO of the Year, was not there). What I can say is that the group included working CEOs in various industries and company sizes who were deep into AI rollouts or had it as a central part of their operations already, as well as advisors and consultants focused on AI from some sizable, name-brand firms.

Once behind that door and shield of anonymity, they were quickly game for sharing some opinions about the technology that their titles and companies would likely not have allowed them to utter publicly—or at least not comfortably.

For the sake of brevity and utility, I’ve boiled the conversation down to some key takeaways, I hope you find them useful—or at least provocative—as you dig into your AI journey. Maybe they will resonate. They may not surprise you:

• The small opportunity. The consultants and AI CEOs in the room all expressed some frustration that their clients and customers invariably looked at generative AI primarily for productivity and cost cutting. Most agreed that “productivity and cost cutting” were among the least interesting things you could do with generative AI.

• The big opportunity. Not enough clients were using it to think about “the big problems” and imagine whole new business opportunities and models that weren’t there before.

• Public data is a problem. Data, of course, is the lifeblood of AI. But too many companies are trying to put the generative AI engines that are trained on public data (aka: the whole Internet) to work on real business problems—and getting ulcers as a result. Because the “big models” are fed the whole Internet, and the Internet is filled with so much garbage, what you get will invariably be unreliable and hallucination-filled (ie: the Cow Egg vs. Chicken Egg problem). That may not ever change.

• Your data is the play. Instead, everyone in the room agreed that the better play was creating “closed systems” based on smaller, more directed AI models trained on vetted, clean, tremendously accurate, proprietary data.

• Redefine AI “productivity.” As a result, you face a choice on to spend your AI resources/money: Cleaning up “slop” and error created by the big public models after the fact, to insure quality as you also try to ramp productivity or spend dear at the beginning to build your own expensive proprietary closed systems that may actually generate more useful, original insights and products.

• Don’t focus on the “chat” interface. Chat, they reminded was/is only one way of engaging with generative AI, but its ease of use and its accessibility has already narrowed executive thinking. The little prompt window is too often the proverbial hammer that turns all problems into nails, and lures you into missing big opportunities as your teams seek “the perfect prompt” to do something.

• Rigorous vetting for now. No matter what you’re doing with generative AI, from drug discovery to marketing copy, you have to review what it is doing very, very closely, and likely will for a lot longer than most experts are now letting on. The way to go is to build a framework/process for doing so at the same time as you’re bringing it online in your shop, so it scales with your efforts.

• A great win right away. While everyone knows or has learned that generative AI is good at writing code, one tech CEO said his big win—and one lots of other CEOs should tune into but haven’t yet—is how great AI is at re-doing OLD CODE to retire tech debt and update systems far faster than humans. Build a small team, have them dive in.

• Artificial general intelligence? This is potentially either the biggest nightmare or most thrilling achievement in human history. And the folks in the room—including at least a couple folks with PhDs in computer science and machine learning—couldn’t and didn’t agree on when, how or even if it would ever come about. But it was fun to watch them argue about it since this topic almost never comes up anymore at most business gatherings.

For me, the most startling thing to come out of the conversation was how much bigger some of those in in the room thought the societal impact would likely be than anyone in politics or business is now discussing publicly—and how much faster the change might arrive. For a topic that has not—as far as I can tell—ever come up in this election cycle, those in the room thought it would lead to very large-scale job losses and socioeconomic displacement within the decade.

One AI company CEO focused on backstopping retail said he actually had to downplay how much better his tools were than his clients’ employees at most commonplace efforts—to keep from unnerving them. He said he had to keep telling the workers that the AI was just a humble, obedient servant that was not going to replace them, when the truth is that it was already starting to do exactly that.