Two online studies compared psychometric properties, demonstrating similar reliability and validity for AI-generated personality scales, highlighting implications for AI integration.
Artificial intelligence (AI) techniques such as natural language processing can be used to construct personality scales. The present study compares the psychometric properties of scales in the domains of Emotional Stability and Conscientiousness developed by experts with those of scales in these domains that were generated by the AI chatbot ChatGPT. We conducted two online studies ( N = 155 and N = 459) in which respondents completed inter alia the expert-developed and the AI-generated scales. Results show that all four scales demonstrated similar reliability and similar content, construct, and criterion validity, but differences regarding fit with a latent measurement model and measurement invariance across genders. We discuss the implications of these findings for the integration of ChatGPT into the process of scale construction and give general recommendations for prompting ChatGPT for automatic item generation.
No takes yet. Share an insight, caveat, or question.
Oeljeklaus et al. (2025) studied this question.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: