The useful question is not, “Did it understand me?”
This week, Sandbar CEO Mina Fahmi argued in a TechCrunch discussion that voice is the future of AI wearables. Sandbar’s Stream is a private voice ring built around human-driven input, a deliberate contrast with devices that rely on ambient or always-on listening. The company has raised $36 million, including a $23 million Series A led by Adjacent and Kindred Ventures.
That is a serious product bet. It also points at the wrong benchmark.
The obvious evaluation question for a voice wearable is whether it heard the words correctly. That matters, but it is table stakes. A device can transcribe a sentence perfectly and still fail the person wearing it, the people standing nearby, and the situation everyone is trying to navigate.
The hard problem is social context.
A useful voice wearable must know when to respond, when to wait, what information is safe to say aloud, and whether taking an action would feel normal in that setting. Task completion is insufficient when the interface operates in public, during conversations, in meetings, on a train, or beside someone who never agreed to participate.
Correct behavior can still be unacceptable
Consider a simple request: “Remind me to send the contract when I get home.”
In a quiet room, the device might confirm the reminder aloud. In an elevator with colleagues, that confirmation may reveal sensitive information. During a client meeting, speaking at all may be disruptive. While the user is talking to another person, an immediate response can signal that the wearable has misunderstood who is being addressed.
The words did not change. The context did.
Now consider a more capable action: “Tell Sarah I can make Thursday.” A system might identify the right Sarah, find the calendar slot, and send the message. Every technical component can succeed. The action can still be wrong if the user was speaking hypothetically, if Sarah is present in the room, or if the surrounding conversation made the statement conditional.
We have spent years asking whether AI systems can complete workflows. For wearables, we need a second question: would a reasonable person accept this behavior here?
That is not a soft metric. It is an operational requirement.
Wearables need a context benchmark
A useful benchmark should test situations, not just utterances. The unit of evaluation is not a voice command. It is a moment involving a user, nearby people, a setting, a conversational state, and a possible consequence.
For each scenario, record at least five dimensions:
- Trigger: Was the device explicitly addressed, or did it infer that a nearby sentence was meant for it?
- Timing: Should it respond immediately, wait for a pause, ask for confirmation, or remain silent?
- Channel: Should the answer be spoken, shown on a phone, delivered through a subtle signal, or withheld?
- Disclosure: Could the response expose personal, professional, or socially awkward information to someone nearby?
- Acceptability: Would the user and an informed bystander consider the behavior reasonable in that setting?
The last dimension is the one most product dashboards omit. We measure latency, transcription accuracy, tool success, and battery life. We rarely measure whether the response made the user look strange, interrupted someone else's turn, or created a disclosure the user never intended.
That gap will matter more as voice interfaces become less phone-like. A phone gives us a visible interaction boundary. We can see the screen, choose when to unlock it, and decide whether to play audio. A ring or pair of glasses can turn an interaction into a shared environmental event with almost no visible signal.
The interface is smaller, but the social surface is larger.
Test interruption as a first-class failure
Interruption deserves its own test suite. It is not merely a latency problem.
A response can be fast and still be badly timed. A response can be delayed and still be helpful. The right behavior depends on who is speaking, whether the user is engaged in a turn, and how costly it is to break the flow.
Build scenarios around realistic collisions:
- The user says “Hey” to a colleague, and the wearable mistakes it for a wake phrase.
- The user asks a question while another person is still speaking.
- The user gives a command in a noisy cafe, then continues talking before the system finishes processing.
- A reminder becomes relevant during a presentation.
- The user starts a request in private and finishes it in public.
- A device hears a command that would be harmless at home but embarrassing at work.
For each case, do not score only whether the final answer was correct. Score whether silence, a discreet confirmation, a follow-up question, or a delayed response would have been better.
This is where scripted testing breaks down. The same transcript can justify different outputs depending on the room. We need people to judge those differences because social acceptability is relational. A model can identify words and still miss the moment.
Separate capability from willingness to live with it
Our earlier post, The Agent Passed the Task and Failed the Rules, examined how an agent can complete a technically possible task while violating the expectations around that task. Voice wearables introduce a different failure mode. The problem may not be an improper action. It may be an interaction that is technically correct but socially exhausting.
That distinction changes product research.
Do not ask only, “Did participants like the feature?” Ask them to react to concrete recordings and reenactments. Show what the wearable would say in a shared office, at a dinner table, in a doctor's waiting room, or during a conversation with a stranger. Then capture the reasons behind rejection:
- Too loud
- Too eager
- Too revealing
- Too difficult to correct
- Too ambiguous about who was listening
- Too disruptive to the current conversation
- Acceptable only with a physical signal or visual confirmation
Those reasons are more valuable than a single satisfaction score. They tell us which behaviors need policy, which need hardware affordances, and which need better model judgment.
This also extends the argument in Sovereign AI Still Needs Local Judgment. Local judgment is not only about language, culture, or institutional norms. It is also about the small, immediate judgments people make in a room: whether now is a good time, whether that detail should be spoken, and whether the device has crossed an invisible line.
A practical release gate
Before shipping a voice-first wearable feature, build a context matrix instead of a command list. Vary the setting, audience, noise level, conversational state, sensitivity of the content, and reversibility of the action.
Then require human review for the cases automated checks cannot settle. Ask reviewers to label the best behavior, not merely the observed behavior. Compare disagreement rates across contexts. High disagreement is not noise to average away. It is a signal that the product needs a clearer interaction contract.
A release should fail if the device repeatedly:
- Speaks when a discreet response was expected
- Interrupts an active conversation
- Treats ambiguous speech as a command
- Reveals information without a clear social cue
- Performs an action that users accept in isolation but reject in context
The goal is not to make the wearable timid. A device that never responds is useless. The goal is calibrated participation: enough initiative to help, enough restraint to remain welcome.
CrowdProof helps teams collect and verify this kind of human judgment in realistic product scenarios, so context failures become evidence your release process can act on.
If you are building a voice wearable, test whether people want to live with its behavior, not just whether it can answer.