南方科技大学知识苑(SUSTech KC): A feasibility study for the application of AI-generated conversations in pragmatic analysis

题名	A feasibility study for the application of AI-generated conversations in pragmatic analysis
作者	Chen，Xi 1; Li，Jun 2 ; Ye，Yuting 3
通讯作者	Chen，Xi
发表日期	2024-04-01
DOI	10.1016/j.pragma.2024.01.003
发表期刊	Journal of Pragmatics 影响因子和分区
ISSN	0378-2166
卷号	223页码:14-30
摘要	This study explores the potential of including AI-generated language in pragmatic analysis– a field that has primarily been conducted on human language use. With the rapid growth of large language models, AI-generated texts and AI-human interactions constitute a growing field where pragmatics research is expanding to. Language data that humans used to hold a full authorship may also involve modifications made by AI. The foremost concern is thus the pragmatic qualities of AI-generated language, such as whether and to which extent AI data mirror the pragmatic patterns we have found in human speech behaviours. In this study, we compare 148 ChatGPT-generated conversations with 82 human-written ones and 354 human evaluations of these conversations. The data are analysed using various methods, including traditional speech strategy coding, four computational methods developed in NLP, and four statistical tests. The findings reveal that ChatGPT performs equally well as human participants in four out of the five tested pragmalinguistic features and five out of six sociopragmatic features. Additionally, the conversations generated by ChatGPT exhibit higher syntactic diversity and a greater sense of formality compared to those written by humans. As a result, our participants are unable to distinguish ChatGPT-generated conversations from human-written ones.
关键词	ChatGPT Pragmalinguistic Pragmatic competence Sociopragmatic Speech act
相关链接	[Scopus记录]
收录类别	SSCI ; AHCI
语种	英语
学校署名	其他
ESI学科分类	SOCIAL SCIENCES, GENERAL
Scopus记录号	2-s2.0-85185904497
来源库	Scopus
引用统计
成果类型	期刊论文
条目标识符	http://sustech.caswiz.com/handle/2SGJ60CL/729105
专题	南方科技大学
作者单位	1.University of Central Lancashire,Preston,United Kingdom 2.Southern University of Science and Technology,Shenzhen,China 3.Southern University of Science and Technology,Business School 335,SUSTech,Shenzhen,China
推荐引用方式 GB/T 7714	Chen，Xi,Li，Jun,Ye，Yuting. A feasibility study for the application of AI-generated conversations in pragmatic analysis[J]. Journal of Pragmatics,2024,223:14-30.
APA	Chen，Xi,Li，Jun,&Ye，Yuting.(2024).A feasibility study for the application of AI-generated conversations in pragmatic analysis.Journal of Pragmatics,223,14-30.
MLA	Chen，Xi,et al."A feasibility study for the application of AI-generated conversations in pragmatic analysis".Journal of Pragmatics 223(2024):14-30.