# Beyond Pilots: How Organizations Can Turn AI Into Real Results

Produced by [Glean](https://www.glean.com) and co-hosted by *Atlantic* CEO [Nicholas Thompson](https://www.linkedin.com/in/nicholasxthompson) this was an enjoyable and insighful afternoon hearing from a wide range of leaders and subject matter experts.

Hosted in NYC on 19 Nov 2025 what follows are my notes on what I found interesting from the event - the full recording is available [here](https://www.theatlantic.com/sponsored/glean-2025/beyond-pilots-apply/4036/).

## The Adoption Problem: Why AI Only Works If People Trust It

*with Chris Duffy, Connie Noonan Hadley, and Stephen Wunker*

### Obstacles

* Resistance to change (from workers)
    
* Loneliness - using AI tends to make us more competitive with each other and less likely to ask colleagues for help and build connections
    

### Suggestions

* HR should be involved in the change
    
* AI is not a tool, it's a paradigm shift
    
* Focus on the business objective, communicate this to employees so they are less afraid and more invested
    
* Consider the possibility of building a skills bank of employees so that your in-house AI interface can suggest work colleagues to talk to on a particular subject. This can increase human connections and decrease loneliness & siloes
    

### How can you measure the ROI or effectiveness of your AI usage?

* Don't just focus on Engagement and Productivity metrics
    

## Rebuilding Enterprise AI: What Comes After the Pilot Era

*with Ruchir Puri and Nicholas Thompson*

* The use of agents is not evenly distributed across the org, but they are already prevalent in software engineering
    
* He has agents debugging & documenting legacy code written in Cobalt to understand it better
    
* <mark>We have to rethink how we work, not just use AI as a tool to automate tasks</mark>
    
* Although new AI companies will be quicker to rethink how we work, legacy enterprises have horizontal reach which will help with implementing AI everywhere, as a platform feature
    
* AI does not have self-improvement yet, whereas humans do. Humans know what we don't know, Models do not.
    
    * "Knowing what you don't know is intelligence" - Ruchir
        

**Requirements for Intelligence**

* Continuous learning
    
* Knowing what you don't know
    
* Humans + Agents will likely be around forever - there will always be checkpoints where agents need to confirm the next step
    

<mark>The most successful agents will be the ones best calibrated for knowing when to ask for human intervention</mark>

### What are the differences between AI and Human intelligence?

* You can audit the bias of AI and try to make it unbiased. Humans are much more opaque!
    

### How do you measure intelligence?

1. IQ - Intelligence
    
2. EQ - Emotional
    
3. RQ - Relationships
    

AI's will struggle with 2 + 3 (they can't hug someone!)

## Five Shifts Every Organization Must Make for the Age of Superintelligence

*with Rebecca Hinds and Nicholas Thompson*

1. Psychological (employee buy-in)
    
    1. Just 9% employees see AI as a team mate rather than just a tool - but these people tend to be more successful in using AI
        
    2. Don't gamify AI clicks per employee!
        
2. Top down change + Bottom up change
    
    1. Identify supporters and people within each function to identify use cases for AI acceleration
        
    2. Establish a firm wide low code platform to reduce tool sprawl and security risks
        
3. Ask "How can this make the team/org better - not just the individual"
    
4. Toggle Tax is real - the number of times employees have to switch between tabs, apps, slack etc. Don't make this worse with you AI implementation!
    
5. Org chart needs rewiring, we're seeing HR & IT become more closely aligned in the rollout of AI org wide.
    

We may see more flattened org structure between product, engineering, design & data as AI helps each discipline cross boundaries and understand the other areas. However, be careful to mind the difference between heads-down and heads-up style work where this approach could be detrimental.

### What should organisations do tomorrow?

* Understand what departments need to be better connected
    
* Moderna merged HR & IT (for example)
    
* Collaboration will be key
    

## Making AI Real in Complex Enterprises

*with Conor Grennan, John Borthwick, and Rohan Sharma*

### How to be successful with AI?

* **Rohan Sharma** - suggested Blackstone are a good example and are doing well with their AI pilots
    
* **John Borthwick** - It's a behavioral shift. We all need to consider how we work with AI. We know well how to interact with people and software but this is a new interface
    
* **Conor Grennan** - ChatGPT adoption is through the roof as it's a better Search Engine. You don't have to leave the page.