User-testing the user-tester: synthetic user feedback driven self improvemnt
The article discusses an innovative approach to user testing where a coding agent autonomously runs user studies on its own product. This self-referential loop allows the agent to iterate and improve based on participant feedback without human intervention. The experiment highlights the complexities and potential of using coding agents in user research.
- ▪The product noemica was designed to autonomously conduct user studies.
- ▪A coding agent iterated on noemica by analyzing user feedback and making changes without human input.
- ▪The experiment involved multiple iterations and adjustments based on participant responses.
Opening excerpt (first ~120 words) tap to expand
field study · the loop on itself · part 1User-testing the user-tester.An agent in the loop, the product on the table. What happens when the thing you built to run user studies is the thing being user-studied, and the participant deciding what to fix next is a coding agent that can’t stop to ask.setup noemica on noemicaiters 25+phases 3agent Claude CodeI built noemica to autonomously run user studies. The natural way to find out if it works is to put it in front of users. I wanted to run user studies against the product that runs user studies, so I pointed the product at itself, then put a coding agent in front of the result and let it iterate.The agent could edit the code. Redeploy. Launch a new run. Read the participants’ verdicts. Decide what to change next.
…
Excerpt limited to ~120 words for fair-use compliance. The full article is at noemica.
Discussion
0 commentsMore from noemica
