This study addresses the issue of recognising customer intent when only limited training data is available. The performance of ChatGPT was evaluated in this scenario, and it was found to be better than traditional machine learning algorithms and the Bidirectional Encoder Representations from Transformers (BERT) model, which performed the worst in this case. While Random Forest with PCA was objectively the best among traditional models when the training examples were randomly selected, a qualitative evaluation showed that ChatGPT had better generalisation ability and could produce contextually correct outputs. Our research found that to improve ChatGPT?s performance on small data classification tasks, it is essential to utilise stratified sampling to select representative examples for few-shot learning. This research provides valuable insights into using ChatGPT in customer-facing applications with limited training data. Knowing the strengths and limitations of ChatGPT can enhance response accuracy, customer satisfaction, and loyalty
- Assessing the capabilities of ChatGPT in recognising customer intent in a small training data scenario
- Chinedu Pascal EzenkwuCelestine IwendiEbuka Ibeke - Robert Gordon University
- 3rd International Conference on Advanced Communication and Intelligent Systems (ICACIS 2024), Vol.May
- 3rd International Conference on Advanced Communication and Intelligent Systems (ICACIS 2024) (Jawaharlal Nehru University, New Delhi, India, 16/05/2024–17/05/2024)
- 9911106308841
- Computing
- Conference paper
- Paper accepted for Oral Presentation at ICACIS 2024.