How to measure consumer’s inconsistency in sensory testing?
Consumer sensory testing plays a pivotal role in guiding food product development. While expert assessors undergo rigorous training to ensure consistency in their evaluations, the variability among consumers remains largely unexplored. This study addresses this gap by introducing a statistical framework to assess consumer inconsistency in sensory evaluations, with a focus on biscuit samples enriched with different levels of pollen.
The Need for Measuring Consumer Inconsistency
In sensory research, consumer preferences drive product improvements. However, inconsistent responses can distort findings, leading to suboptimal product modifications. This study aims to identify inconsistency in consumer evaluations, ensuring more reliable data for food developers. By employing a dual-scale approach—hedonic response (liking) and Just-About-Right (JAR) intensity scales—the researchers analyze the alignment between perceived intensity and overall liking.
Methodology: A Dual-Scale Approach
The study involved 100 consumers evaluating biscuit samples enriched with three types of pollen (rapeseed, sunflower, and phacelia) at different concentrations (2%, 5%, 10%). Participants rated the biscuits using a nine-point hedonic scale for liking and a five-point JAR scale for intensity. The data were then analyzed using Kendall’s rank correlation coefficient to determine consistency.
Key Findings: Identifying Inconsistent Assessors
The results revealed that 88 consumers provided consistent responses, while 12 exhibited significant inconsistencies. The latter group showed weak or even positive Kendall’s tau correlation values, indicating potential misunderstanding or random scoring. Interestingly, inconsistent assessors demonstrated lower variability in their responses, suggesting a conservative use of the rating scales.
Attribute-Based Consistency Analysis
Among the sensory attributes assessed, color and overall taste were the most consistently rated, whereas margarine odor and margarine taste exhibited the highest inconsistency. This suggests that certain attributes may be more challenging for consumers to evaluate, potentially due to lower perceptual sensitivity or ambiguity in the attribute definition.
Impact of Inconsistency on Sensory Data
The study further examined how inconsistent evaluations affected overall product ratings. Regression analysis indicated that global taste had the strongest influence on liking among consistent assessors, whereas inconsistent assessors exhibited more variability in how different attributes impacted their ratings. This highlights the need to filter out unreliable responses to enhance the accuracy of sensory testing data.
Implications for Future Sensory Testing
The study emphasizes the importance of incorporating multiple evaluation scales in consumer research. By identifying and mitigating inconsistencies, researchers can refine data quality, leading to more precise product development insights. Future studies should consider integrating pre-screening mechanisms or training sessions for consumers to enhance their understanding of rating scales.
Conclusion
Consumer inconsistency in sensory evaluations is a critical challenge that can influence the outcomes of food product research. By leveraging statistical frameworks such as Kendall’s tau correlation, researchers can systematically identify inconsistent assessors and refine data collection methodologies. This approach paves the way for more reliable consumer insights, ultimately leading to better-informed product development strategies in the food industry.
Sipos, L., Ágoston, K. C., Biró, P., Bozóki, S., & Csató, L. (2025). How to measure consumer’s inconsistency in sensory testing? Current Research in Food Science, 100982.
