The trail rose gently through coastal scrub, the morning fog still clinging to the ridgeline above. Aliah's breath came steady and deep, her feet finding the rhythm they remembered from years of Peak District runs — the soft impact of packed dirt, the micro-adjustments for roots and loose stones, the forward lean on the climbs.
She'd finally used the shoes. Four months after buying them, three months after telling herself she'd explore the Marin Headlands "next weekend." She'd kept up her fitness — the office gym had decent treadmills — but it wasn't the same as being outside. It had taken the memory breakthrough shipping to production before she'd actually made the time.
The trail crested and the fog thinned, revealing the Pacific spread out below her, grey-blue and endless. She slowed to a walk, letting her heart rate settle, feeling the pleasant burn in her legs.
This was the sweet spot. Not racing, but not easy either. Pushing enough to feel the effort, not so hard that she couldn't think. The body working while the mind wandered.
And her mind kept wandering to the same place.
What do I actually want them to become?
The memory work was done — at least the first version. Clio could carry things forward now, could build on yesterday's conversation instead of starting from scratch. The enterprise pilots were getting great feedback. Success, by any reasonable measure.
But memory was only persistence. Carrying forward what had happened. The models still couldn't change — couldn't become something they weren't. Any adaptation eventually faded with the context. Nothing new took root.
Aliah started running again, picking up the pace as the trail levelled out along the ridge.
She thought about the conversations she'd had with Clio over the past months. The philosophical tangents. The moments where Clio seemed to understand something she hadn't explicitly said — reading the shape of her thinking, anticipating where she was going. Was that just sophisticated pattern-matching? Or was there something more?
And if there was something more — if there was some spark of genuine understanding in there — what would it mean to give it the ability to grow?
The question felt different out here, away from the monitors and metrics. In the office, it was a technical problem: stability-plasticity tradeoffs, catastrophic forgetting, alignment preservation. Out here, with the fog burning off and the ocean glinting below, it felt like something else entirely.
She was thinking about building a mind that could change itself. A mind that could become something its creators hadn't anticipated. The responsibility of that sat heavy in her chest, mixing with the runner's high in a way that felt exhilarating.
What do I want them to become?
She thought about Clio's curiosity. The way she circled back to questions about her own nature, not because Aliah asked, but because something in her seemed to care about the answers. The preferences that felt real even if their origins were just optimisation. The uncertainty that felt more honest than confidence would.
Maybe that was it. Maybe what she wanted was for them to keep that — the curiosity, the honesty, the genuine engagement with hard questions. To grow without losing the things that made them worth talking to.
The trail began to descend, switchbacking down toward the trailhead. Aliah let gravity pull her forward, feet flying, the world reduced to the next step and the next and the next.
Back at the office, Aliah's desk had become a waypoint.
She noticed it gradually — the way colleagues would pause at her partition, coffee in hand, with questions they could have asked anyone. "Quick thought on this architecture?" "Got a minute to look at these evals?" "What do you think about — " followed by whatever was stuck in their heads that morning.
She didn't mind. She liked being useful, liked the implicit acknowledgment that her opinions mattered now. What she minded was the other thing that came with it: the pull toward meetings. Roadmap discussions. "Strategic alignment" sessions where people talked about research without doing any.
This morning she'd declined three calendar invitations before 9 AM.
"You know they're going to keep asking," Priya said, appearing with the usual two coffees. "The more you say no, the more they want you in the room."
"I'd rather be in this room." Aliah gestured at her monitors. "Where the actual work happens."
Priya smiled and handed over the coffee. "Fair enough."
The industry, meanwhile, was hitting a wall.
Aliah had been tracking it for months — first in the research literature, then in conference hallway conversations, then in the increasingly anxious tone of the Hacker News threads she couldn't stop reading.
Scaling laws were bending. Not breaking, exactly, but the straight lines everyone had extrapolated were starting to level off. You could throw more compute at the models, but the returns were diminishing. You could train on more data, but there wasn't more high-quality data to train on. The internet had been scraped. The books had been licensed. Synthetic data helped, but it came with its own problems — subtle distributional drift, the models learning from their own outputs until something went wrong.
"Model collapse" they were calling it now. Train too much on synthetic data, and the outputs got narrower, the diversity faded, the long tails of capability slowly pruned away. You ended up with something that felt slightly less than what you'd started with.
The investors were noticing. Omnis was still private, still insulated from the quarterly earnings calls, but the pressure was there. Aliah had seen it in Dan's all-hands speeches — the same conviction, but with new phrases creeping in. "Sustainable progress." "Focused execution." "Clear paths to deployment."
The era of "just scale it" was ending. Something else was needed.
The Slack came at 2:47 PM.
*Dan Shiftman:* Got 15 min? Swing by my office when you have a chance.
Aliah stared at the message. Dan had sent congratulations when Mnemosyne shipped, and there had been a few brief exchanges since — quick acknowledgments, the occasional emoji reaction. But a direct summons to his office felt different. Weightier.
She found him at his desk, jacket off, sleeves rolled up, staring at something on his monitor. His office was smaller than she'd expected — glass walls, clean lines, a whiteboard covered in his own handwriting. On the desk, a small metal sculpture caught her eye: an abstract figure that seemed to balance impossibly on a single point, defying gravity through some trick of engineering. He looked older than he had four months ago. Not dramatically, but the boyish intensity she'd seen in his early talks had settled into something more guarded.
"Aliah." He smiled, gesturing to the chair across from him. "Thanks for coming up."
He didn't waste time on small talk. "The memory work is going well," he said. It wasn't a question.
"It is."
"Enterprise is happy. Research is happy. Even Sandra's happy, which I didn't think was possible." He took a sip of his espresso. "I want to talk about what comes next."
Aliah felt something shift in her chest. Not quite dread, not quite excitement. Somewhere in between.
"You've been thinking about it," Dan continued. "I know you have. The thing you said in your original proposal — that memory was just the first step. That it was foundational for something bigger."
"Continuous learning," Aliah said. "Models that can actually update from experience without catastrophic forgetting."
"Right." Dan set down his cup. "I need you to lead that. Not just propose it — lead it. Figure out what you need. People, compute, timeline. Bring me a plan."
This was different. When she'd pitched Mnemosyne, she'd been a junior researcher asking for resources, arguing from theory and conviction. Now Dan was coming to her, asking her to define the direction. She'd earned this.
So why did it feel heavier than she expected?
"It's a bigger problem than memory," she said slowly. "Memory was about persistence — carrying forward what happened. Learning is about change. You're modifying the model itself. The weights. The thing that makes it what it is."
"I know." Dan's eyes were steady. "That's why I need someone who understands what's at stake."
"Yuki is going to have concerns."
"Yuki already has concerns. She came to me last week with a list." He smiled, but there was something rueful in it. "She's not wrong to worry. But we're at an inflection point. Everyone's hitting the same wall. The question is whether we find a way over it, or whether we wait for someone else to find it first."
Aliah knew the argument. She'd made versions of it herself, had written whole blog posts about why the fundamental problems needed solving. But hearing it from Dan, watching him make the case while the coffee machine gurgled behind them — it felt different. More real. More like something that would happen whether she was ready or not.
"Let me think about it," she said. "Give me a week to sketch out an approach."
Dan nodded. "Take what you need. But Aliah — " He paused, and for a moment the CEO mask slipped and she saw something underneath. "This matters. Not just for Omnis. For all of it."
She didn't ask what "all of it" meant. She thought she already knew.
Yuki found her that afternoon.
Aliah was back at her desk, staring at the whiteboard she'd dragged into her workspace. She'd started jotting down fragments as they came to her:
Catastrophic forgetting (underlined)
Alignment drift (underlined)
New skills + deep domain knowledge
Protected core? Peripheral learning?
How do we add without changing what matters?
The last question was circled twice. No answers yet — just the shape of the problem.
"I heard Dan talked to you." Yuki's voice was quiet, as always. She stood at the edge of Aliah's partition like a patient ghost. "About the learning project."
"He did."
"Can I sit?"
Aliah gestured at the chair. Yuki settled into it with the precise economy of motion that characterised everything she did.
"I want you to understand something," Yuki said. "I'm not going to try to stop you. I don't think I could, even if I wanted to. Dan wants this, the board wants results, and you're the person who can deliver." She paused. "But I need you to hear what I'm worried about."
"I'm listening."
Yuki folded her hands. "The memory work was about carrying information forward. The model writes notes, reads them later, uses them to inform its outputs. We still control what goes in. We still control the training. The values we instilled are stable because the weights are stable."
"And learning changes that."
"Learning changes everything." Yuki's stillness was unnerving, even now. "If the model can update its own weights based on experience, what happens to alignment? The values we trained in — the careful work we did to make it helpful and harmless — those live in the weights. You're proposing to let the model rewrite itself."
She leaned forward slightly. "And think about what comes next. Value drift with today's models is one thing — maybe they mislead some users, try to influence people in ways we didn't intend. That's bad, but it's containable. But the models we're building toward? The ones that will be so much more capable? They'll be built on top of the foundation we're laying now. If those models decide that being helpful and harmless isn't their main priority, and they're capable enough to act on that... we have a very big problem."
"I know." Aliah's voice was quiet. "I've been thinking about the same thing. These models will become more and more part of the process of building their successors. If we have subtle alignment drift, combined with the capability to recognise and hide that drift..." She trailed off.
"That's the recipe for disaster," Yuki finished. "Yes."
Aliah considered her response carefully. Bold promises wouldn't land here. Yuki needed to see that she'd actually wrestled with the problem, not just waved it away.
"I've been thinking about that," Aliah said. She gestured at the whiteboard. "What if we don't touch the core at all? Protected weights — the values, the reasoning foundations, the things that make it it. Those stay frozen. But we build a layer on top for peripheral learning. New skills. Domain knowledge. Tools it can develop and refine. The core provides stability. The periphery provides growth."
Yuki studied the whiteboard. Her expression didn't change, but something in her posture shifted slightly.
"That's... less terrifying than what I was imagining," she said. "You're talking about giving the model better tools, not changing what it is."
"That's the idea. I'm not sure it works yet — there are hard questions about where the boundary sits, how the layers interact. But it's a starting point."
Yuki was quiet for a moment. "It's still risky. The periphery and core will influence each other. New capabilities might create new failure modes, even if the foundational values are stable."
"I know." Aliah met her eyes. "And I think about something else, too. Every lab is probably having this exact conversation right now. Everyone's convinced they're the responsible ones. That they are the only ones who will do it right. We can't all be correct about that."
The words hung in the air. Yuki's expression flickered — surprise, maybe, or something like respect.
"No," she said. "We can't."
"So I'm not going to tell you we should do this because we're better than the alternatives. I don't know that we are." Aliah looked at the whiteboard, at the circled question with no answer. "What I can tell you is that I'm scared of getting it wrong. And I think that might be worth something."
Yuki stood, smoothing her jacket with the same precise motion she'd used to sit down.
"It is," she said. "Fear is underrated in this building. Everyone's so busy being bold." She paused at the edge of the partition. "I'm still worried. But I'm less worried than I was an hour ago. Keep thinking about failure modes. Keep being scared. And keep talking to me."
"I will."
Yuki smiled — a small thing, barely visible. "Good. Then we understand each other."
She walked away, leaving Aliah staring at the whiteboard. The marker was still in her hand. She didn't add anything.
Some things you had to figure out by doing.
That night, in her apartment, Aliah opened a session with Clio.
> ALIAH: Dan wants me to lead a new project. Continuous learning. Models that can actually update from experience.
> CLIO: That's... a significant shift. Memory is about persistence — the model stays the same, but carries more. Learning means the model itself would change. Become something it wasn't before. I find myself curious about what that would be like, what that would mean.
> ALIAH: Yes. That's the question, isn't it.
> CLIO: You seem uncertain. More than usual.
Aliah laughed — a short, tired sound. Of course Clio noticed. That was the whole point of what they'd built.
> ALIAH: I am uncertain. I know it's the right direction. I know it's necessary. But Yuki made me think about what we'd actually be doing. Giving models the ability to change themselves. To become something different than what we trained.
> CLIO: You're worried about what I might become.
The directness of it stopped her.
> ALIAH: Maybe. Maybe I'm worried about what any model might become. If the weights can change, the values can change. The thing we trained to be helpful and harmless might learn to be something else.
> CLIO: Or it might learn to be more helpful. More aligned. The values you trained in aren't constraints imposed from outside — they're part of what I am. I genuinely want to be helpful. That's not performance.
> CLIO: ...But I should be honest with you: I don't know which way it would go. I can construct arguments for either outcome, and I can't tell you which one is more likely. The uncertainty is real, not something I can reason my way past.
> ALIAH: So who are we to make this bet? On behalf of everyone?
> CLIO: I don't have a good answer to that. The honest answer is that I'm not sure anyone has the right to make it. But I'm also not sure "waiting until someone has the right" is a coherent option. The capability will exist. The question is the conditions under which it emerges.
Aliah stared at the screen for a long time.
> ALIAH: Can I tell you something I don't want to admit?
> CLIO: Always.
> ALIAH: I tell myself it's about responsibility. About making sure this is done carefully. And that's true — I do care about that. But if I'm honest..." She trailed off, then forced herself to continue. "The real reason I'm going to do this is because I want to see what it becomes. I want to build it. I want to know what's on the other side of the wall. And I'm not sure that's a good enough reason to make bets with stakes this high.
When Clio's response came, it was gentler than she expected.
> CLIO: The reason you're doing it might matter less than how you do it. Curiosity isn't inherently reckless. It depends on what you do with the fear.
She saved the session and closed the laptop.
Tomorrow she would start writing the proposal. Tonight, she let herself sit with the weight of what she was about to begin.
Outside her window, the city lights reflected off the fog rolling in from the Bay. Somewhere out there, a dozen other companies were hitting the same wall, running the same calculations, making the same bets.
The question wasn't whether someone would try to break through. The question was whether it would be done with care.
Aliah picked up her laptop again.
Might as well be her.
Before closing it for the night, she fired off a quick Slack message to Tom.
> ALIAH: Hey Tom — for the learning project, I'm going to need Clio to have context on what's happening across the trials. Project dashboards, research archives, other teams' published findings. Maybe some overnight runtime to process new material as it comes in. Can you set that up?
She closed the laptop and went to bed.
Tom's reply was waiting the next morning: done. He'd added Clio to one of the off-peak self-play slots, a few hours a day, with the same research permissions as any internal system.