Skip to content

Reading Past the Feedback Summary

In a small study of employee comments, brief AI summaries kept topics mentioned repeatedly much more often than topics mentioned once.

Before a team talks about its survey results, a manager can ask AI for a short account of the comments. Microsoft’s instructions for Viva Glint make the sequence concrete: open a Team Summary report, select a comment count to reveal the All Comments panel, then choose Summarize. The software groups feedback into themes to explore before a conversation with the team.

The list offers places to begin. But the headings alone cannot tell you whether they cover every subject in the comments.

In a controlled study of employee comments, researchers generated briefings from 15 teams at one company. They reported that briefings capped at 3 bullets kept 14% of topics mentioned once, compared with 74% of topics mentioned in at least 3 comments.

Those percentages count topics from the source, not employees. The same model wrote the summaries and judged which topics reappeared. The study did not measure how much detail survived, topics’ importance or managerial decisions, and its private comments are not publicly available for independent checking.

Glint was not tested. Before preparing a discussion, are you looking for recurring subjects, or trying to learn what else people have raised?

What the headings tell you

Repetition can be a sensible guide to a brief overview. If the immediate question is what comes up most often, recurring subjects deserve space. A summary would not become more useful simply by giving every passing remark equal prominence.

But deciding what to discuss asks something more of the reader. A subject does not become important just because it appears once. Nor does the absence of repetition make it unimportant. Deciding that requires more than counting appearances.

Changing the instructions also changed the result. A short-paragraph version retained more topics than the 3-bullet version. Format and instructions changed together, so this comparison does not isolate the effect of length.

A different summary can therefore offer a broader starting point. It still cannot tell its reader the contents of everything left outside it. To find that material, the reader has to go somewhere the headings do not point.

Checking what was never suggested

Interview-analysis software makes this reading work visible. In Nielsen Norman Group’s account of testing AI research tools, Dovetail displays a summary beside an interview transcript. Timestamps connect the transcript to the recording. The software also suggests tags for passages, which a researcher can accept or reject. The authors report that these suggestions missed large sections that should have been tagged.

Their example concerns interview tagging, rather than employee-survey summaries. But it shows why reviewing a tool’s suggestions and finding its omissions are different jobs. A proposed tag is waiting for a yes or no. An overlooked passage offers no such prompt.

Following a theme back to a comment helps answer whether the heading fits. It can recover context and deepen understanding of the subject already on the list. Reading beyond those themes asks whether the list itself needs to change. Neither step guarantees completeness, but checking every proposed item does not amount to checking everything in the source.

There are legitimate limits to that source, too. Microsoft’s comment-summary documentation says Glint uses the current report’s filters and only data the user is allowed to view. Confidentiality thresholds can prevent a summary altogether. Those protections define what a reader may inspect; they are different from a topic disappearing as text is condensed.

When preparing the team conversation, the task is to understand the feedback available within those limits. Checking examples under the generated headings can make those subjects clearer. Looking through the remaining comments leaves the agenda open to a subject the summary never named.

Sources

Understand How People and Agents Work Together.

Practical essays, experiments, and lessons from building software for people and agents.

Occasional emails, sent when there is something worth reading. Free to subscribe. Unsubscribe at any time.