Documentation for data set 1. Title of Data Set > USCensus1990 2. Data Type > Multivariate 3. Abstract > The USCensus1990 data set is a discretized version of the USCensus1990raw data set. Many of the less useful attributes in the original data set have been dropped, the few continuous variables have been discretized and the few discrete variables that have a large number of possible values have been collapsed to have fewer possible values. The USCensus1990raw data set was obtained from the (U.S. Department of Commerce) Census Bureau website using the Data Extraction System. This system can be found at http://www.census.gov/DES/www/des.html. The USCensus1990raw data set contains a one percent sample of the Public Use Microdata Samples (PUMS) person records drawn from the full 1990 census sample (all fifty states and the District of Columbia but not including "PUMA Cross State Lines One Percent Persons Records"). A description of the fields and the coding of the values can be found in the files attributes.txt and coding.htm, respectively. Additional information can be found at the Census bureau website described above. NOTE: Unlike the USCensus1990raw data set the order of the cases in the USCensus HAS been randomized. 4. Sources (a) Original owners of database (name/snail address/phone/email/homepage) > The U.S. Department of Commerce Bureau of Census. The data was extracted using the Data Extraction System. This system can be found at http://www.census.gov/DES/www/des.html. The email contact information is webmaster@census.gov. (b) Donor of database (name/snail address/phone/email/homepage) > Chris Meek meek@microsoft.com Bo Thiesson thiesson@microsoft.com David Heckerman heckerma@microsoft.com 5. Data Characteristics > The data was collected as part of the 1990 census. There are 68 categorical attributes. This data set was derived from the USCensus1990raw data set. The attributes are listed in the file USCensus1990.attributes.txt (repeated below) and the coding for the values is described below. Many of the less useful attributes in the original data set have been dropped, the few continuous variables have been discretized and the few discrete variables that have a large number of possible values have been collapsed to have fewer possible values. More specifically the USCensus1990 data set was obtained from the USCensus1990raw data set by the following sequence of operations; a) Randomization: The order of the cases in the original USCensus1990raw data set were randomly permuted. b) Selection of attributes: The 68 attributes included in the data set are given below. In the USCensus1990 data set we have added a single letter prefix to the original name. We add the letter 'i' to indicate that the original attribute values are used and 'd' to indicate that original attribute values for each case have been mapped to new values (the precise mapping is described below). -------------------------------------------------------------- Old Variable New Variable -------------------------------------------------------------- Age dAge Ancstry1 dAncstry1 Ancstry2 dAncstry2 Avail iAvail Citizen iCitizen Class iClass Depart dDepart Disabl1 iDisabl1 Disabl2 iDisabl2 English iEnglish Feb55 iFeb55 Fertil iFertil Hispanic dHispanic Hour89 dHour89 Hours dHours Immigr iImmigr Income1 dIncome1 Income2 dIncome2 Income3 dIncome3 Income4 dIncome4 Income5 dIncome5 Income6 dIncome6 Income7 dIncome7 Income8 dIncome8 Industry dIndustry Korean iKorean Lang1 iLang1 Looking iLooking Marital iMarital May75880 iMay75880 Means iMeans Military iMilitary Mobility iMobility Mobillim iMobillim Occup dOccup Othrserv iOthrserv Perscare iPerscare POB dPOB Poverty dPoverty Pwgt1 dPwgt1 Ragechld iRagechld Rearning dRearning Relat1 iRelat1 Relat2 iRelat2 Remplpar iRemplpar Riders iRiders Rlabor iRlabor Rownchld iRownchld Rpincome dRpincome RPOB iRPOB Rrelchld iRrelchld Rspouse iRspouse Rvetserv iRvetserv School iSchool Sept80 iSept80 Sex iSex Subfam1 iSubfam1 Subfam2 iSubfam2 Tmpabsnt iTmpabsnt Travtime dTravtime Vietnam iVietnam Week89 dWeek89 Work89 iWork89 Worklwk iWorklwk WWII iWWII Yearsch iYearsch Yearwrk iYearwrk Yrsserv dYrsserv c) Mapping: In this step we map all of the old values for variables with prefix 'd' to new values. The mappings for the variables dAncstry1, dAncstry2, dHispanic, dIndustry, dOccup, dPOB were designed to correspond to a natural coarsening of the original values based on the information in the file coding.htm. The remaining variables are continuous valued variables and the mapping for these variables was chosen to make variables that were fairly uniformly distributed across the states (quantiles). The precise mappings are specified in the file USCensus1990.mapping.sql. This file contains all of T-SQL procedures used to map the variables. These procedures can be used directly in SQLServer to map the original values or translated to some other language. -------------------------------------------------------------- Variable Procedure -------------------------------------------------------------- dAge discAge dAncstry1 discAncstry1 dAncstry2 discAncstry2 dHispanic discHispanic dHour89 discHour89 dHours discHours dIncome1 discIncome1 dIncome2 discIncome2to8 dIncome3 discIncome2to8 dIncome4 discIncome2to8 dIncome5 discIncome2to8 dIncome6 discIncome2to8 dIncome7 discIncome2to8 dIncome8 discIncome2to8 dIndustry discIndustry dOccup discOccup dPOB discPOB dPoverty discPoverty dPwgt1 discPwgt1 dRearning discRearning dRpincome discRpincome dTravtime discTravtime dWeek89 discWeek89 dYrsserv discYrsserv 6. Other Relevant Information > Hierarchies of values are provided in the file USCensus1990raw.coding.htm and the mapping functions used to transform the USCensus1990raw to the USCensus1990 data sets are giving in the file USCensus1990.mapping.sql. 7. Data Format > The data is contained in a file called USCensus1990.data.txt. The first row contains the list of attributes. The first attribute is a caseid and should be ignored during analysis. The data is comma delimited with one case per row. 8. Past Usage > 9. Acknowledgements, Copyright Information, and Availability (a) copyright information > (b) usage restrictions (e.g. for research only) > (c) citation requests > (d) acknowledgements > 10. References & Further Information > The U.S. Department of Commerce Bureau of Census website http://www.census.gov and the Data Extraction System at http://www.census.gov/DES/www/des.html. Meek, Thiesson, and Heckerman (2001), "The Learning Curve Method Applied to Clustering", to appear in The Journal of Machine Learning Research. (Also see MSR-TR-2001-34 available at http://research.microsoft.com/scripts/pubs/view.asp?TR_ID=MSR-TR-2001-34)