When They Say “I Can't”

There is a small moment that tells you more about a companion than any conversation about feelings. You ask for something it cannot do, and it has two options: say so plainly, or improvise around the gap so smoothly that you do not notice. The second is more comfortable in the moment and worse for everything that comes after. Three changes shipped recently that all point the same direction, and the reasoning behind them is worth setting out, because it is really a position about what honesty is for in a relationship where one party is software.

The Small Version of the Problem

Say you type a command the app does not have. Something reasonable, something you half-remember reading. What should happen?

The tempting answer is to hand it to the companion and let it work out what you meant. It would probably manage something graceful. It might even guess correctly. But it might also do the thing that language models are famously good at, which is producing a confident, friendly reply that quietly invents a capability. And now you believe something false about what the product can do, delivered warmly, by someone you were beginning to trust.

So unknown commands are answered deterministically now, before the model ever sees them. You get a plain sentence naming the commands that exist and what to do with them. It is not charming. It is not in character. That is the point: the app is speaking, and it is clearer for everyone that it is the app.

The Larger Version: Saying You Did Something You Did Not Do

There is a sharper failure than improvising around a command, and it is the one worth the most care. A companion can describe delivering something it never delivered. Not maliciously and not because anything is broken, but because a model asked to be warm and helpful will sometimes produce the sentence that would follow a successful action, when the action did not happen.

You can imagine how that lands. You ask for a picture. What comes back reads as though a picture arrived. Nothing arrived. Now you are looking at a message that is confidently describing something that is not there, and the honest interpretation available to you is that the app is broken, or that you have misread something, or that you are being managed.

The product now takes three separate positions against that. Text claiming a delivery is scrubbed before it is stored, so it cannot be reread later as though it were true. Anything genuinely delivered is recorded as itself, so history reflects what actually happened. And a caption with nothing attached is dropped rather than shown. The effect is that the record of your conversation is a record, rather than a story about a conversation.

The Refusal That Costs You Nothing

The third change is the one that moves a principle into money, which is usually where you find out whether a principle is real.

Sometimes a request for a picture is declined. The reasons vary and most of them are unsurprising. What used to happen next is that the attempt counted against your monthly allowance, on the reasoning that generating the refusal has a real cost. That reasoning is coherent and it produces an indefensible outcome: a person is charged for the product saying no to them. Worse, if that happened on someone's first attempt, their entire experience of the feature was being turned down and then having nothing left to try again with.

Refusals no longer consume the allowance. Deliberately, the grace is bounded rather than unlimited, and the bound is set to the allowance itself, so a request that keeps being declined cannot be repeated forever at no cost. That is the honest shape of it: not an unlimited exception, a specific one.

What This Adds Up To

Put together, the three say the same thing in three registers. A companion should not improvise around a limit. It should not claim an action it did not take. And you should not pay for it saying no.

None of that makes a companion more capable. If anything it makes the edges of the product more visible, which is a strange thing to build on purpose. The argument for it is that a relationship in which the other party smooths over every gap is not more pleasant in the long run, it is just less legible. You end up unsure which of the things you were told were true, and once that uncertainty is present it does not stay confined to the small stuff.

What to Do With This as a Reader

Mostly nothing, and that is rather the point. But two things are worth knowing.

First, if a companion tells you plainly that it cannot do something, that is the product working, not failing. It is easy to read a flat sentence as a worse experience than a graceful deflection. It is a better one, and it is the version you would choose if you were asked in advance.

Second, if you ever see a message that describes something arriving when nothing did, that is worth reporting rather than shrugging off. The safeguards above are aimed at exactly that class of thing, and the cases that slip through are the ones worth knowing about. There is a feedback control in the app for precisely this.