Data Mining by Mehmed Kantardzic (good book recommendations TXT) 📕

with the d.f. for an m × n dimensional table computed as

In general, the hypothesis H0 is rejected at the level of significance α if

where T(α) is the threshold value from the χ2 distribution table usually given in textbooks on statistics. For our example, selecting α = 0.05 we obtain the threshold

A simple comparison shows that

and therefore, we can conclude that hypothesis H0 is rejected; the attributes analyzed in the survey have a high level of dependency. In other words, the attitude about abortion shows differences between the male and the female populations.

The same procedure may be generalized and applied to contingency tables where the categorical attributes have more than two values. The next example shows how the previously explained procedure can be applied without modifications to the contingency table 3 × 3. The values given in Table 5.7a are compared with the estimated values given in Table 5.7b, and the corresponding test is calculated as χ2 = 3.229. Note that in this case parameter

TABLE 5.7. Contingency Tables for Categorical Attributes with Three Values

We have to be very careful about drawing additional conclusions and further analyzing the given data set. It is quite obvious that the sample size is not large. The number of observations in many cells of the table is small. This is a serious problem and additional statistical analysis is necessary to check if the sample is a good representation of the total population or not. We do not cover this analysis here because in most real-world data-mining problems the data set is enough large to eliminate the possibility of occurrence of these deficiencies.

That was one level of generalization for an analysis of contingency tables with categorical data. The other direction of generalization is inclusion into analysis of more than two categorical attributes. The methods for three- and high-dimensional contingency table analysis are described in many books on advanced statistics; they explain the procedure of discovered dependencies between several attributes that are analyzed simultaneously.

5.8 LDA

LDA is concerned with classification problems where the dependent variable is categorical (nominal or ordinal) and the independent variables are metric. The objective of LDA is to construct a discriminant function that yields different scores when computed with data from different output classes. A linear discriminant function has the following form:

where x1, x2, … , xk are independent variables. The quantity z is called the discriminant score, and w1, w2, … ,wk are called weights. A geometric interpretation of the discriminant score is shown in Figure 5.5. As the figure shows, the discriminant score for a data sample represents its projection onto a line defined by the set of weight parameters.

Figure 5.5. Geometric interpretation of the discriminant score.

The construction of a discriminant function z involves finding a set of weight values wi that maximizes the ratio of the between-class to the within-class variance of the discriminant score for a preclassified set of samples. Once constructed, the discriminant function z is used to predict the class of a new nonclassified sample. Cutting scores serve as the criteria against which each individual discriminant score is judged. The choice of cutting scores depends upon a distribution of samples in classes. Letting za and zb be the mean discriminant scores of preclassified samples from class A and B, respectively, the optimal choice for the cutting score zcut-ab is given as

when the two classes of samples are of equal size and are distributed with uniform variance. A new sample will be classified to one or another class depending on its score z > zcut-ab or z < zcut-ab. A weighted average of mean discriminant scores is used as an optimal cutting score when the set of samples for each of the classes are not of equal size:

The quantities na and nb represent the number of samples in each class. Although a single discriminant function z with several discriminant cuts can separate samples into several classes, multiple discriminant analysis is used for more complex problems. The term multiple discriminant analysis is used in situations when separate discriminant functions are constructed for each class. The classification rule in such situations takes the following form: Decide in favor of the class whose discriminant score is the highest. This is illustrated in Figure 5.6.

Figure 5.6. Classification process in multiple-discriminant analysis.

5.9 REVIEW QUESTIONS AND PROBLEMS

1. What are the differences between statistical testing and estimation as basic areas in statistical inference theory?

2. A data set for analysis includes only one attribute X:

X = {7, 12, 5, 18, 5, 9, 13, 12, 19, 7, 12, 12, 13, 3, 4, 5, 13, 8, 7, 6}.

(a) What is the mean of the data set X?

(b) What is the median?

(c) What is the mode, and what is the modality of the data set X?

(d) Find the standard deviation for X.

(e) Give a graphical summarization of the data set X using a boxplot representation.

(f) Find outliers in the data set X. Discuss the results.

3. For the training set given in Table 5.1, predict the classification of the following samples using simple Bayesian classifier.

(a) {2, 1, 1}

(b) {0, 1, 1}

4. Given a data set with two dimensions X and Y:XY1542.753352.5

(a) Use a linear regression method to calculate the parameters α and β where y = α + β x.

(b) Estimate the quality of the model obtained in (a) using the correlation coefficient r.

(c) Use an appropriate nonlinear transformation (one of those represented in Table 5.3) to improve regression results. What is the equation for a new, improved, and nonlinear model? Discuss a reduction of the correlation coefficient value.

5. A logit function, obtained through logistic regression, has the form:

Find the probability of output values 0 and 1 for the following samples:

(a) { 1, −1, −1 }

(b) { −1, 1, 0 }

(c) { 0, 0, 0 }

6. Analyze the dependency between categorical attributes X and Y