Beyond Pilots: How Organizations Can Turn AI Into Real Results

Produced by Glean and co-hosted by Atlantic CEO Nicholas Thompson this was an enjoyable and insighful afternoon hearing from a wide range of leaders and subject matter experts.

Hosted in NYC on 19 Nov 2025 what follows are my notes on what I found interesting from the event - the full recording is available here.

The Adoption Problem: Why AI Only Works If People Trust It

with Chris Duffy, Connie Noonan Hadley, and Stephen Wunker

Obstacles

• Resistance to change (from workers)

• Loneliness - using AI tends to make us more competitive with each other and less likely to ask colleagues for help and build connections

Suggestions

• HR should be involved in the change

• AI is not a tool, it's a paradigm shift

• Focus on the business objective, communicate this to employees so they are less afraid and more invested

• Consider the possibility of building a skills bank of employees so that your in-house AI interface can suggest work colleagues to talk to on a particular subject. This can increase human connections and decrease loneliness & siloes

How can you measure the ROI or effectiveness of your AI usage?

• Don't just focus on Engagement and Productivity metrics

Rebuilding Enterprise AI: What Comes After the Pilot Era

with Ruchir Puri and Nicholas Thompson

• The use of agents is not evenly distributed across the org, but they are already prevalent in software engineering

• He has agents debugging & documenting legacy code written in Cobalt to understand it better

• We have to rethink how we work, not just use AI as a tool to automate tasks

• Although new AI companies will be quicker to rethink how we work, legacy enterprises have horizontal reach which will help with implementing AI everywhere, as a platform feature

• AI does not have self-improvement yet, whereas humans do. Humans know what we don't know, Models do not.

  • "Knowing what you don't know is intelligence" - Ruchir

Requirements for Intelligence

• Continuous learning

• Knowing what you don't know

• Humans + Agents will likely be around forever - there will always be checkpoints where agents need to confirm the next step

The most successful agents will be the ones best calibrated for knowing when to ask for human intervention

What are the differences between AI and Human intelligence?

• You can audit the bias of AI and try to make it unbiased. Humans are much more opaque!

How do you measure intelligence?

1. IQ - Intelligence

2. EQ - Emotional

3. RQ - Relationships

AI's will struggle with 2 + 3 (they can't hug someone!)

Five Shifts Every Organization Must Make for the Age of Superintelligence

with Rebecca Hinds and Nicholas Thompson

1. Psychological (employee buy-in)

   1. Just 9% employees see AI as a team mate rather than just a tool - but these people tend to be more successful in using AI

   2. Don't gamify AI clicks per employee!

2. Top down change + Bottom up change

   1. Identify supporters and people within each function to identify use cases for AI acceleration

   2. Establish a firm wide low code platform to reduce tool sprawl and security risks

3. Ask "How can this make the team/org better - not just the individual"

4. Toggle Tax is real - the number of times employees have to switch between tabs, apps, slack etc. Don't make this worse with you AI implementation!

5. Org chart needs rewiring, we're seeing HR & IT become more closely aligned in the rollout of AI org wide.

We may see more flattened org structure between product, engineering, design & data as AI helps each discipline cross boundaries and understand the other areas. However, be careful to mind the difference between heads-down and heads-up style work where this approach could be detrimental.

What should organisations do tomorrow?

• Understand what departments need to be better connected

• Moderna merged HR & IT (for example)

• Collaboration will be key

Making AI Real in Complex Enterprises

with Conor Grennan, John Borthwick, and Rohan Sharma

How to be successful with AI?

• Rohan Sharma - suggested Blackstone are a good example and are doing well with their AI pilots

• John Borthwick - It's a behavioral shift. We all need to consider how we work with AI. We know well how to interact with people and software but this is a new interface

• Conor Grennan - ChatGPT adoption is through the roof as it's a better Search Engine. You don't have to leave the page.