Center for Machine Learning and Intelligent Systems
About  Citation Policy  Donate a Data Set  Contact


Repository Web            Google
View ALL Data Sets

CLINC150 Data Set
Download: Data Folder, Data Set Description

Abstract: This is a intent classification (text classification) dataset with 150 in-domain intent classes. The main purpose of this dataset is to evaluate various classifiers on out-of-domain performance.

Data Set Characteristics:  

Text

Number of Instances:

23700

Area:

N/A

Attribute Characteristics:

N/A

Number of Attributes:

N/A

Date Donated

2020-05-08

Associated Tasks:

Classification

Missing Values?

N/A

Number of Web Hits:

930


Source:

Please cite as: 'Stefan Larson, Anish Mahendran, Joseph J. Peper, Christopher Clarke, Andrew Lee, Parker Hill, Jonathan K. Kummerfeld, Kevin Leach, Michael A. Laurenzano, Lingjia Tang, and Jason Mars. 2019. An evaluation dataset for intent classification and out-of-scope prediction. In Proceedings of EMNLP-IJCNLP.'

Bibtex link: https://www.aclweb.org/anthology/D19-1131.bib

Paper link: https://www.aclweb.org/anthology/D19-1131/

Email contact: stefan.dataset '@' gmail.com


Data Set Information:

There are 4 versions of the dataset:
- data_full.json: each of the 150 in-domain intent classes have 100 train, 20 val, and 30 test samples. The out-of-domain class has 100 train, 100 val, and 1,000 test samples. Note that the out-of-domain class does not necessarily need to be used at training time. This is the main version of the dataset.
- data_small.json: each of the 150 in-domain intent classes have 50 train, 20 val, and 30 test samples. The out-of-domain class has 100 train, 100 val, and 1,000 test samples. Note that the out-of-domain class does not necessarily need to be used at training time.
- data_imbalanced.json: each of the 150 in-domain intent classes have either 25, 50, 75, or 100 train, 20 val, and 30 samples. The out-of-domain class has 100 train, 100 val, and 1,000 test samples. Note that the out-of-domain class does not necessarily need to be used at training time.
- data_oos_plus.json: same as data_full.json except there are 250 out-of-domain training samples.


Attribute Information:

All samples are in text format. No tokenization has been applied. Users of this dataset are free to use whatever sentence representation (e.g. bag-of-words, sentence embeddings) they choose.


Relevant Papers:

This dataset is from this paper: [Web Link]

'Stefan Larson, Anish Mahendran, Joseph J. Peper, Christopher Clarke, Andrew Lee, Parker Hill, Jonathan K. Kummerfeld, Kevin Leach, Michael A. Laurenzano, Lingjia Tang, and Jason Mars. 2019. An evaluation dataset for intent classification and out-of-scope prediction. In Proceedings of EMNLP-IJCNLP.'



Citation Request:

Please cite as: 'Stefan Larson, Anish Mahendran, Joseph J. Peper, Christopher Clarke, Andrew Lee, Parker Hill, Jonathan K. Kummerfeld, Kevin Leach, Michael A. Laurenzano, Lingjia Tang, and Jason Mars. 2019. An evaluation dataset for intent classification and out-of-scope prediction. In Proceedings of EMNLP-IJCNLP.'

Bibtex link: [Web Link]

Paper link: [Web Link]

Email contact: stefan.dataset '@' gmail.com


Supported By:

 In Collaboration With:

About  ||  Citation Policy  ||  Donation Policy  ||  Contact  ||  CML