An enterprise recently deployed their highest-benchmarked AI for customer communications. It had scored 94% on reasoning tests, placing it among the top performers on every leaderboard. Within three weeks, they started getting complaints. Nothing was technically wrong with the responses—grammar impeccable, facts accurate, tone professionally calibrated. But customers kept saying something felt “off.” The AI was solving problems that didn’t exist while missing the actual human need behind each inquiry.
The disconnect wasn’t a bug in the deployment. It revealed something more fundamental about how we’re measuring these systems. When AI writes emails, generates marketing copy, or assists with strategic decisions, we’re not asking it to solve math problems. We’re asking it to participate in meaning-making—the messy, contextual work of human communication. And our evaluation methods haven’t caught up to that reality.
The Cultural Blindspot in Current Testing
Current AI evaluation treats culture like mathematics: one correct answer, universally true, objectively scorable. This approach works brilliantly for calculating trajectories or parsing syntax. It catastrophically fails for the work most organizations actually deploy these systems to do.
Research from the Alan Turing Institute identifies three reasons why culture breaks traditional benchmarking models. Context determines meaning completely. The phrase “I need to talk” from your manager at 4:59 PM on Friday carries radically different weight than the same words from your partner over breakfast. Same vocabulary, entirely different implications. No universal “correct response” exists—appropriateness emerges from the situation itself.
Multiple valid interpretations coexist without requiring resolution. What reads as professional formality in one organizational culture feels cold and distant in another. Recent research shows that AI systems exhibit strong pro-Western cultural bias in how they explain decisions, privileging certain communication styles while marginalizing others. The AI serves users who fundamentally disagree about what’s appropriate, and flattening these perspectives into a single “objective” output doesn’t eliminate bias—it just makes it invisible.1
Productive ambiguity gets systematically optimized away. Effective business communication often maintains interpretive richness. Strategic vagueness allows stakeholders to align around shared language while preserving legitimate differences in implementation. But benchmarks that reward clarity and specificity above all else train systems that can’t navigate this reality. They resolve ambiguity that should remain productively open.
The philosophical tradition of hermeneutics—the study of interpretation and meaning-making—has grappled with these challenges for centuries. Hermeneutics recognizes that understanding cultural artifacts requires examining the historical and social context in which they’re created, used, and perceived. When the frame of reference shifts, so does the meaning. This insight applies directly to how AI systems generate cultural content.2
What We’re Actually Building
Here’s where the technical architecture becomes relevant. The transformer models powering these systems already operate through self-attention mechanisms that iteratively update understanding of each token based on its relationship to the broader sequence. This mirrors what philosophers call the hermeneutic circle—using specific parts to refine understanding of the whole, and the whole to reinterpret the parts.3
The self-attention mechanism allows models to weigh the importance of different words in context. When processing “The animal didn’t cross the street because it was too tired,” the attention mechanism helps determine whether “it” refers to the animal or the street by examining relationships across the entire sequence. This is interpretation happening at the architectural level.
Vector space embeddings encode sophisticated contextual co-occurrence patterns—essentially capturing which words appear together and in what circumstances. Since context is central to cultural meaning, these systems have the technical substrate for contextual interpretation. What’s missing is evaluation frameworks that recognize and measure this capability appropriately.
IBM’s research on attention mechanisms explains that these systems “learn to focus on the most relevant parts of the input when generating each part of the output”. The probabilistic decoding process maintains multiple possible interpretations rather than collapsing to a single answer. This isn’t a limitation—it’s a feature that accommodates the plurality inherent in cultural communication.
The Business Consequences of Evaluation Mismatch
Organizations deploying AI for cultural work face a measurement problem with real costs. Procurement decisions get made based on benchmark leaderboards that show aggregate scores with almost no predictive power for whether a model will produce appropriate tone, understand organizational context, or navigate cultural nuances in your specific domain.4
The European Commission’s AI Watch identified nine fundamental challenges in current AI benchmarking practices. Benchmarks often fail to capture real-world utility because they test in controlled conditions that don’t reflect deployment contexts. Models optimize for standardized test performance rather than contextual appropriateness.
When vendors compete on benchmark scores, they optimize toward the wrong goals—universal correctness rather than contextual appropriateness. This creates what researchers call “epistemic monoculture,” where diverse approaches get replaced by narrow optimization toward a single metric that doesn’t measure what actually matters.
A 2024 analysis found that LLM benchmarks consistently fail to predict AI success in business contexts. The three primary reasons: benchmarks don’t account for domain-specific requirements, they measure capabilities in isolation rather than in collaborative workflows, and they ignore the cultural context that shapes whether outputs are actually useful.5
Current approaches try to “solve” cultural challenges through ever-larger training datasets and better benchmarks, assuming neutrality is achievable. But there’s no view from nowhere, no disembodied perspective that has transcended cultural particularity. Every interpretation comes from somewhere, reflects particular values, serves specific purposes. Evaluation frameworks that ignore this don’t eliminate bias—they just obscure whose values are being served.
Research published in PNAS Nexus demonstrates that large language models exhibit systematic cultural biases in their outputs, with values and perspectives skewing toward Western, educated, industrialized, rich, and democratic societies. These biases aren’t bugs to be debugged—they’re inevitable consequences of training data and design choices that reflect particular cultural perspectives.6
A Better Framework for Evaluation
Recent research proposes a fundamental shift in how we evaluate AI for cultural work, drawing on hermeneutics to reimagine evaluation from the ground up. Three principles translate abstract philosophy into actionable evaluation strategies.
Make Evaluation Iterative
Cultural meaning emerges through conversation, not single prompts. How often do your teams accept the first AI output without refinement? Rarely. They clarify, push back, provide additional context, iterate toward something that works. That iterative process is where the actual work happens, and it’s where meaning gets made.
Test AI on its ability to participate in ongoing dialogue—updating understanding through exchange rather than generating one-shot responses. Design evaluation scenarios that mirror actual use cases with multi-turn interactions. Assess not just final outputs but the AI’s ability to refine understanding through clarification, incorporate feedback, and maintain coherent context across exchanges.
Codecademy’s implementation guide for context engineering recommends building evaluation frameworks that test context retention across conversation turns, the system’s ability to disambiguate based on conversational history, and how effectively it adapts responses as new information emerges. These capabilities matter far more for real-world performance than aggregate scores on standardized tests.
Evaluation must account for both the model’s general architecture and the specific dialogic frame in which outputs are generated. Aggregate metrics indicating average performance often fail to predict instance-by-instance behavior where contextual nuances matter most.
Include People in Assessment
Testing AI in isolation misses the collaboration between human and machine that defines real-world use. We don’t just prompt these systems—we shape them through how we ask questions, what we accept or reject, how we incorporate their outputs into our thinking. They shape us too, influencing creative processes and decision-making patterns.
Build evaluation frameworks that assess human-AI collaboration quality rather than solo AI performance. Measure how effectively the AI supports human decision-making, how transparently it acknowledges its perspective, and how well it adapts to feedback from users with different backgrounds and needs.
Anthropic’s research on evaluation challenges emphasizes that robust assessments must account for the interactive nature of deployment contexts. Evaluations conducted in isolation systematically miss failure modes that only emerge through human-AI interaction over time.7
Current approaches to assessing creativity in AI-generated content range from automated metrics to expert human judgment, but these often treat creativity as a model property rather than a relational phenomenon. A hermeneutic approach evaluates how human-AI collaboration produces interpretations, examining not just outputs but the interpretive dialogue that generates them.
Center Cultural Context
Instead of asking “Is this objectively correct?” ask “How and why does this achieve appropriateness within its specific framework?” That’s a harder question. It requires acknowledging which values are being served, whose perspectives are centered, what tradeoffs are being made.
Develop domain-specific evaluation scenarios that reflect the cultural contexts where your AI will actually operate. Assess not just output quality but the AI’s ability to recognize when context matters, acknowledge which perspective it’s operating from, and adapt to different situational requirements.
The Conversation’s research demonstrates how cultural assumptions embed invisibly in AI systems, with Western communication norms treated as universal defaults while other approaches are framed as deviations. Evaluation must surface these embedded values rather than accepting them as neutral baselines.
This is intellectually uncomfortable. It means confronting decisions that were always being made but remained invisible under the fiction of objective evaluation. Standard evaluation practices treat context as secondary to model performance metrics, but “thin” signals like positive/negative ratings cannot provide the contextual grounding needed to assess cultural appropriateness.
Rather than scrubbing away context to create universal tests, embrace context as the medium through which performance emerges. Frameworks like HELM (Holistic Evaluation of Language Models) recognize the need for contextually dependent approaches beyond simple accuracy metrics.
Practical Next Steps
Organizations serious about deploying AI for cultural work need evaluation strategies that match the actual challenge. Start by auditing current evaluation practices. What metrics drive your AI procurement and deployment decisions? Are you optimizing for benchmark scores that don’t predict real-world performance in your specific context?
Build contextual test scenarios that mirror your organization’s actual use patterns, cultural context, and stakeholder diversity. Test iteratively, include human collaboration, assess contextual appropriateness rather than universal correctness.
Demand transparency from vendors. Ask AI providers not just what benchmark scores their models achieve, but how they evaluate cultural appropriateness, contextual adaptation, and interpretive collaboration. Request performance data on scenarios similar to your use cases rather than standardized tests that may have little relevance to your domain.
Create feedback loops with actual users. Deploy with clear mechanisms for people to report when AI outputs miss the mark culturally, even if they’re technically correct. Use this feedback to continuously refine both the systems and your evaluation approaches. This grounds evaluation in real deployment contexts rather than hypothetical test cases.
The AI Watch report emphasizes that effective benchmarking requires moving beyond one-size-fits-all assessments toward domain-specific evaluation that accounts for the particular requirements and constraints of different deployment contexts. Generic benchmarks may satisfy academic curiosity but provide limited guidance for business decisions.
AI as Cultural Participant
The shift from treating AI as information processors to understanding them as cultural participants is simultaneously more modest and more ambitious than current discourse suggests.
More modest because it abandons the fantasy of a universal, objective AI that has transcended cultural particularity. There’s no disembodied perspective that has synthesized all human knowledge into universal truth. Every interpretation comes from somewhere, reflects particular values, serves specific purposes.
More ambitious because it recognizes that these systems actively shape culture, not just reflect or optimize it. They don’t just read context—they help create it through how they respond, what they emphasize, which interpretations they make available. This influence extends to affecting human metacognition, shaping assumptions about relational norms, and enabling novel creative practices.
The messy reality is that culture involves genuine disagreement, competing values, and interpretations that can’t all be simultaneously satisfied. Designing AI systems that work productively within that complexity—rather than pretending to resolve it through technical optimization—is the actual challenge.
Getting evaluation right is the first step toward building AI systems that acknowledge their situated perspective, operate transparently within specific cultural contexts, and collaborate effectively with humans who bring different but equally legitimate viewpoints. This requires evaluation frameworks sophisticated enough to measure what matters: not whether an AI can match predetermined answers, but whether it can navigate the interpretive challenges inherent in human meaning-making.
The organizations that figure this out won’t just deploy better AI. They’ll gain clearer understanding of the cultural work they’re asking these systems to do, why it matters, and how to measure success in ways that actually predict real-world performance. That clarity—about what we’re building and why—may be the most valuable outcome of rethinking evaluation altogether.
Connect the Dots
Consider the power dynamic at play. Every AI vendor presenting leaderboard positions understands that standardized test performance doesn’t translate to specific cultural contexts. The research documenting these limitations is public and peer-reviewed. Yet purchasing decisions continue as if those scores matter, because they provide the appearance of objective comparison. Organizations collectively maintain a fiction everyone privately knows is false. The question isn’t just about better evaluation—it’s about what strategic advantage accrues to the first competitor in any industry who stops playing this particular game. Who demands evaluation methods that actually predict success in their domain rather than success on someone else’s standardized test? That clarity about what matters becomes a competitive moat.
When these systems learn contextual patterns from massive corpora—which words appear together, in what circumstances, weighted by attention across relationships—they’re absorbing interpretive frameworks embedded in their training data, not discovering universal truths. The self-attention mechanism deciding that “professional” correlates more strongly with certain communication patterns than others isn’t finding objective reality. It’s operationalizing cultural judgments about what professionalism means. Those judgments vary wildly across contexts but get compressed into model weights and deployed as if they were neutral. Technical sophistication doesn’t transcend culture—it scales particular cultural perspectives with industrial efficiency. Which surfaces an uncomfortable question: What happens to organizations and individuals whose cultural frameworks aren’t well-represented in training data? Are these tools augmenting human capability across contexts, or rewarding conformity to a narrow cultural model while framing it as optimization?
This pattern should sound familiar: teaching to standardized tests in education, optimizing for quarterly earnings in business strategy, measuring healthcare quality through billing codes. Create scalable metrics because they’re comparable. Watch as optimization toward those metrics produces systems that score well while failing at their actual purpose. The difference with AI is velocity and reach. An educational system takes decades to reveal the consequences of teaching to tests. An AI trained on benchmark optimization gets deployed across millions of interactions in months. The pattern repeats—just faster, wider, harder to reverse once embedded in infrastructure. Which raises a practical question for technical and business leaders: If the pattern is predictable, what would it actually take to break it rather than accelerate down the same trajectory? Not rhetorical—what specific organizational practices would need to change?
Building evaluation frameworks that assess iterative dialogue quality, human-AI collaborative effectiveness, and contextual appropriateness requires something rare: articulating what cultural work actually looks like in a specific domain. Who it serves, how success manifests beyond surface metrics, which dimensions matter most and why. Most organizations can’t clearly define the cultural aspects of their communication, decision-making, or stakeholder relationships because those patterns have always been tacit—just “how things work here.” Developing contextual AI evaluation forces that implicit knowledge into explicit frameworks. The strategic value isn’t just better AI deployment. It’s organizational self-awareness about cultural capabilities that differentiate in the market, and competitive clarity about which of those capabilities actually create value versus which are just legacy patterns. While competitors compare benchmark scores, that clarity compounds. Worth considering which organizations in any given industry are developing it right now.