KDC-4007 dataset test Collection 
-------------------------

1. DESCRIPTION
--------------

KDC-4007 dataset Collection is the Kurdish Documents Classification text for the Sorani dialect.
     It is a set of corpus contains about 4,007 articles talking about the eight following topics "categories":
     Style, Religion, Economy, Health, Education, Art, socialite and sport for text mining research.
      

1.1. Compilation
----------------
 Since there are no standard test collections available for the Kurdish Sorani dialect. 
This corpus has been collected from free or free for research sources at the Web,especially from three sources, 
these are namely; Rudaw, which is a Kurdish daily online news and information website about Kurdistan (http://www. rudaw.net/Sorani),
and Nrttv, which is another Kurdish daily online news (http://www.nrttv.com), Sulaimany News Network (http://www.snnc.co), and other websites

The most important feature of this dataset is its simplicity to use and its being well-documented, which can be widely used in various studies of text analysis regarding Kurdish Sorani news and articles.   

The documents consist of eight categories, which are Sport, Religion, Art, Economic, Education, Social, Style, and Health. Each of them consisted of 500 text documents, where the total size of the corpus is 4,007 text files. 
The dataset and documents have become freely accessible in order to have repeatable outcomes for experimental assessment.


1.2. Statistics
---------------

There is four collection:
 
- ST-Ds datasets, just stop words elimination is performed by using Kurdish preprocessing-step approach. 
- Pre-Ds dataset, Kurdish preprocessing-step approach is used. 
- Pre+TW-Ds dataset, TF×IDF term weighting on Pre-Ds dataset is performed. 
- Orig-Ds datasets, no process is used which is original dataset.



1.3. Format
-----------
The files in ARFF format that describes a list of instances is sharing a set of features.




2. USAGE
--------
I have prepared this corpus in order to achieve experiments on text classification for Kurdish Sorani language.
It has been extracted from web site of articles.The collection may also be of interest to researchers in machine learning, as it provides a classification task with challenging properties.
There are multiple categories, the categories are overlapping and nonexhaustive, and there are relationships among the categories. 
There are interesting possibilities for the use of domain knowledge.  There are many possible feature sets that can be extracted from the text, and most plausible feature/example matrices are large and sparse.

This work presents a number of statistics, studies and baseline results for several machine learning methods.
[1] Arazo M. Mustafa and Tarik A. Rashid,“Kurdish Stemmer Pre-processing Steps for Improving Information Retrieval”, Journal of Information Science, First published date: january-01-2017, 10.1177/0165551516683617

[2] Tarik A. Rashid, Arazo M. Mustafa and Ari M. Saeed, 2017."A Robust Categorization System for Kurdish Sorani Text Documents". Information Technology Journal, 16: 27-34.

[3] Tarik A. Rashid, Arazu M. Mustafa, Ari M. Saeed Automatic Kurdish Text Classification Using KDC 4007 Dataset, accepted in Springer book, Series Title: Lecture Notes on Data Engineering and Communications Technologies: Book title: Advances in Internetworking, Data & Web Technologies, Indexing: The books of this series are submitted to ISI Proceedings, EI, Scopus, MetaPress, Springerlink, 2017.



3. ABOUT
--------

The corpus has been collected by Arazo M. Mustafa https://www.researchgate.net/profile/Arazo_Mustafa    Date: April, 2016

We would like to thank Dr. Tarik A. Rashid  https://www.researchgate.net/profile/Tarik_Rashid for his suggestion to make KDC-4007 dataset available for free  



4. LICENSE/DISCLAIMER
---------------------

We would appreciate if:

- In case you find this corpus useful, please make a reference to previous papers and the web page:https://github.com/arazom/KDC-4007-Dataset in your papers, research, etc.
- The KDC-4007 dataset Collection is provided for free and with no limitations excepting
- Arazo Muhamed Mustafa and Tarik Ahmed Rashid hold the copyrigth (c) for the KDC-4007 dataset Collection.