Logo image
Assessing the capabilities of ChatGPT in recognising customer intent in a small training data scenario
Conference paper

Assessing the capabilities of ChatGPT in recognising customer intent in a small training data scenario

Chinedu Pascal Ezenkwu, Celestine Iwendi and Ebuka Ibeke
3rd International Conference on Advanced Communication and Intelligent Systems (ICACIS 2024), Vol.May
3rd International Conference on Advanced Communication and Intelligent Systems (ICACIS 2024) (Jawaharlal Nehru University, New Delhi, India, 16/05/2024–17/05/2024)
05/2024

Abstract

ChatGPT, customer intent, machine learning, prediction, GPT, small data, few-shot learning
This study addresses the issue of recognising customer intent when only limited training data is available. The performance of ChatGPT was evaluated in this scenario, and it was found to be better than traditional machine learning algorithms and the Bidirectional Encoder Representations from Transformers (BERT) model, which performed the worst in this case. While Random Forest with PCA was objectively the best among traditional models when the training examples were randomly selected, a qualitative evaluation showed that ChatGPT had better generalisation ability and could produce contextually correct outputs. Our research found that to improve ChatGPT?s performance on small data classification tasks, it is essential to utilise stratified sampling to select representative examples for few-shot learning. This research provides valuable insights into using ChatGPT in customer-facing applications with limited training data. Knowing the strengths and limitations of ChatGPT can enhance response accuracy, customer satisfaction, and loyalty
url
https://icacis.org/View

Metrics

33 Record Views

Details

Logo image

Usage Policy