Why mechanical prominence metrics miss perceived importance—and why LLM judges cannot be the only evaluator.
Mechanical visibility does not capture perceived importance
Equal word share and position do not guarantee that two citations feel equally important. Unique evidence, relevance to the question, and the role a source plays in the conclusion can change how a reader perceives it.
Subjective Impression evaluates several perceived dimensions
Aggarwal and colleagues used an LLM-based evaluator for relevance, answer influence, uniqueness, subjective position and amount, click likelihood, and information diversity. This recognized that prominence has objective and perceived components.
LLM judges introduce their own measurement risks
Evaluation can vary by model, prompt, style preference, and calibration. Circularity becomes especially concerning when related model families generate and judge the same content.
A judge score is a measurement instrument with assumptions—not an independent ground truth.
Triangulate perception with observable and behavioral data
Pair subjective evaluation with citation presence, position, answer share, repetition, human ratings, recall, and referral clicks. Agreement across methods is more informative than precision from a single judge score.
Questions about this topic
What is Subjective Impression?+
It is an evaluator-based estimate of how visible, relevant, unique, influential, or clickable a source appears within an answer.
Why use an LLM judge?+
It can evaluate qualitative aspects that word counts and positions do not capture at scale.
What are the risks of LLM evaluation?+
Model dependence, stylistic bias, calibration problems, and circularity when similar models generate and evaluate content.
Should subjective impression be used alone?+
No. It should be paired with observable answer metrics and, where possible, human or behavioral validation.
References
Sources are listed in APA 7 style. Preprints are identified as such and should not be treated as peer-reviewed findings unless separately published.
- Aggarwal, P., Murahari, V., Rajpurohit, T., Kalyan, A., & Narasimhan, K. (2024). GEO: Generative engine optimization. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (pp. 5–16). Association for Computing Machinery. https://doi.org/10.1145/3637528.3671900
- Martinez, O. (2026). Optimizing visibility in generative engines: A critical survey of generative engine optimization (2023–2026) [Preprint]. arXiv. https://doi.org/10.48550/arXiv.2607.14035