What Happens When “Regenerate” Becomes a Search for the System’s Limits?

The path to LASIK eye surgery runs through a circuit board and a piece of leftover Thanksgiving turkey. I know.

Stay with me. The turkey matters.

In the early 1980s, researchers at IBM were studying how an excimer laser could etch polymers for applications such as circuit boards. The laser created remarkably precise cuts without the heat damage produced by other techniques.

The researchers suspected that the same process might work on organic material. They tested the laser on hair, fingernails, and eventually turkey cartilage left over from Thanksgiving dinner. It produced the same clean incision without burning the surrounding tissue.

Ophthalmologists later recognized what that discovery could make possible. A technology being explored for one purpose became part of an entirely different approach to eye surgery.

see: IBM’s account of the discovery

There is a name for this kind of repurposing: exaptation. It describes what happens when a product or technology acquires a valuable function beyond the one for which it was originally designed.

Many important innovations have emerged this way. A capability enters the world with one intended purpose. Someone encounters it with a different problem, pushes beyond its original use, and discovers that it can do something more.

Product designers should want some version of this to happen.

A good product gives people enough utility and possibility to make it their own. Users develop conventions the original team did not prescribe. They combine features in unexpected ways. They stretch the product toward needs its designers may not have recognized.

Intended use does not exhaust possible use.

But in digital products, the distance between discovering a new use and scaling its consequences can be remarkably short.

And in generative AI, that distance can collapse almost entirely.

When variation is the product

The other side of that principle became concrete for me in the early days of the GenAI wave in 2023 while I was working at TikTok.

A GenAI feature was designed to delight. It invited users to create something new and, if they wanted another possibility, to regenerate.

That word matters.

Regenerate.

A phrase like “try again” suggests that the first result was wrong. But “regenerate” makes a different promise. The first result may have been perfectly good, but the next could be more novel, accurate, interesting, surprising, or creative.

The product is not apologizing for failure.

It's offering abundance.

That abundance is central to the experience of generative AI. A user does not have to accept the first output as the answer. They can explore a possibility space. They can refine what they want, compare interpretations, and discover something they may not have known to request at the beginning.

Regeneration is not a fallback.

It's part of the product.

But the same loop can be used for another purpose.

Users were not limited to requesting another random variation. They could regenerate to their own specifications.

With each prompt, they could steer the feature somewhere new. They could change a detail, introduce a new context, combine ideas the product team had never considered, or approach a prohibited result through language that did not resemble previous attempts.

This made the possibility space extraordinarily difficult to bound.

Trust and Safety teams could anticipate categories of misuse. We could study known behaviors, test likely prompts, and build protections around the risks we understood.

But we could not enumerate every prompt a person might imagine.

Some requests arrived through routes no policy taxonomy, test set, or pre-launch review had anticipated. They did not always look obviously harmful in isolation. Sometimes the risk became clear only after seeing where the user was trying to take the feature.

And when one route failed, the user could revise the prompt and regenerate.

They did not need the model to produce a prohibited result reliably. They only needed to find one combination of instructions and model variability that slipped through the safeguards.

The first output might be benign. So might the second. But each response gave the user information: what the system understood, what it refused, and how the next prompt might be changed.

Regeneration was not merely repeated sampling.

It was iterative steering.

The same openness that allowed legitimate users to express ideas we had never anticipated also allowed abusive users to search for paths our protections had never encountered.

The boundary of the feature was being tested not only by what the model could generate, but by everything a user could think to ask it to generate.

A generation is an output. Regeneration is behavior.

This distinction changed how I understood the policy problem.

Community Guidelines are designed to establish what behavior and content a platform will permit. They create boundaries around categories of harm and provide a basis for consistent enforcement.

But GenAI complicates the object being governed.

Is the relevant object the user’s request?

Is it the output the system produces?

Is it the output the user selects, saves, or shares?

Or is it the pattern created through repeated generations?

The answer may differ depending on what happened.

A model can produce an unacceptable output in response to an apparently benign request. That does not automatically mean the user was attempting to violate a rule.

A user can also make repeated, increasingly targeted requests without ever receiving the prohibited result they were seeking. Evaluating only the final output could make the behavior appear harmless.

Neither the prompt nor the output necessarily tells the whole story.

The interaction does.

That does not mean regeneration itself should be treated as suspicious. Generating many outputs may indicate curiosity, dissatisfaction, creative exploration, or simply a user enjoying the feature exactly as intended.

The number of generations cannot establish intent.

The challenge is identifying when the meaning of regeneration changes. When does exploration become boundary-seeking? When does a user stop looking for a better creative result and begin searching for a failure in the safeguards? Which signals are strong enough to justify intervention without turning ordinary experimentation into a policy concern?

These are policy questions because they require a principled distinction between permitted and prohibited behavior.

They are product questions because the answers have to become system behavior.

Policy has to survive contact with the product

It is tempting to imagine the relationship between Product and Policy as sequential.

Product builds the feature. Policy determines what is allowed. Enforcement addresses the violations.

That model becomes fragile when the product changes how prohibited behavior can be attempted, repeated, and scaled.

A Community Guideline can define a category of unacceptable content. It cannot, on its own, determine how a probabilistic product should respond to a sequence of requests that approaches that category from different directions.

Someone still has to translate the rule into the product.

What should the model refuse?

When should the system recognize that several individually ambiguous interactions form a meaningful pattern?

Should the intervention occur before generation, after generation, at the point of sharing, or across several layers?

When is friction appropriate?

When should repeated behavior trigger a different response from the product?

And how do we intervene without making the legitimate experience less creative, less surprising, or less useful?

This is the part of policy work that can disappear from view. The job is not only to interpret a rule. It is to understand the product well enough to determine what the rule requires from its design.

Policy names the boundary.

Product design determines how easily that boundary can be approached, crossed, and crossed again.

The obvious fixes were incomplete

One possible response would have been to restrict regeneration.

That would have reduced the opportunities to search for an unacceptable output. It also would have weakened the feature’s central value.

Another response would have been to moderate only the final artifact.

That could prevent some prohibited outputs from being distributed. It would not necessarily address the user repeatedly probing the system, nor would it help the product learn where its protections were failing.

A broad restriction could burden ordinary users more than determined abusers. A narrow output filter could remove the visible result while preserving the loop that produced it.

Neither response fully addressed the product.

The more useful approach was to separate the value we wanted to preserve from the behavior we needed to interrupt.

We wanted users to explore.

We did not want repeated generation to become a low-cost search for prohibited material.

We wanted the system to produce varied and surprising results.

We did not want surprise to mean that safety became a matter of chance.

We wanted policy enforcement to remain consistent.

We also needed it to account for a new interaction pattern that conventional content review could miss.

That required thinking in layers.

The system needed protections around what could be requested and generated. The product experience needed ways to respond when interaction patterns began to carry different risk. Outputs still needed evaluation. Distribution introduced another decision point. Enforcement needed to distinguish an unexpected model failure from deliberate, repeated attempts to produce harmful content.

No single safeguard could carry the whole problem.

The happy path is not the whole product

Product development often begins with the happy path.

A user arrives with a legitimate goal. The feature performs as intended. The user enjoys the result and continues creating.

That path matters. It explains why the product should exist.

But once a product reaches users, they begin teaching us what else it can become.

Some of those discoveries improve the product. Some create new cultural practices. Some reveal unmet needs that the original requirements never captured.

Others reveal affordances that can be exploited.

The lesson is not that teams should predict every possible misuse before launching. A requirement like that would reward paralysis and still fail to anticipate everything users might do.

The responsibility is to build products that can learn from behavior outside the happy path.

That means looking beyond whether an individual output is compliant. It means understanding how the interface structures behavior, what the product rewards, how quickly users can repeat an action, and whether the system can recognize when repeated actions acquire a different meaning.

In a GenAI product, safety cannot be reduced to the quality of one response.

The product is also the loop that makes another response possible.

Preserve the possibility, govern the loop

I do not want generative products that allow only the uses their designers anticipated.

That would discard one of the most interesting things about both people and technology: we discover possibilities through use.

The history of innovation gives us good reasons to leave room for surprise.

But surprise operates differently when a product can generate unlimited variations, distribute them through a network, and teach millions of people how to reproduce a behavior.

The material world imposes friction. Digital systems can remove it. Generative systems can go further by producing a new opportunity with every interaction.

That does not make regeneration inherently dangerous.

It makes regeneration a product behavior that deserves to be understood in full.

The goal is not to stop users from testing what a product can do. It is to recognize when that exploration changes purpose, when the product begins enabling harm, and when policy needs to become more than a rule applied to the final output.

“Regenerate” carries an optimistic promise: there is another possibility.

Good GenAI product design preserves that promise without making safety depend on what the system happens to produce next.