Decision Tree Classification

Decision Tree

Decision Tree Classifier Explained Simply¶

What is a Decision Tree Classifier?

A Decision Tree Classifier is a method used in machine learning to categorize data into different groups. It works like a flowchart, where each question helps you narrow down the possible outcomes until you arrive at a final decision.

How Does It Work?¶

Tree Structure:
- The decision tree looks like a tree:
  - Root Node: This is where the decision-making starts. It represents the first question about the data.
  - Branches: Each branch represents an answer to the question (like “Yes” or “No”).
  - Leaf Nodes: These are the final outcomes or categories at the end of the branches.
Asking Questions:
- At each node, the tree asks a question based on one of the features of the data (e.g., “Is the temperature above 70 degrees?”).
- Depending on the answer, the data splits into two or more groups.
Making Splits:
- The goal is to split the data in a way that groups similar items together. The best splits are made using measures like Gini Index or Entropy to see how mixed or pure the groups are.
- For example, if most people in a group like ice cream, that group is more “pure” for ice cream lovers.
Stopping Criteria:
- The tree stops growing when:
  - All items in a group belong to the same category (like all being “yes” or “no”).
  - There are no more questions to ask (features to split).
  - A certain limit is reached, like the maximum depth of the tree.

Steps to Use a Decision Tree Classifier¶

Collect Your Data:
- Get a dataset with features (like age, income) and labels (like “will buy” or “won’t buy”).
Prepare the Data:
- Clean the data to fix any errors or missing values.

Train the Model:

Use a library to create and train the decision tree with your data.

from sklearn.tree import DecisionTreeClassifier

model = DecisionTreeClassifier()  # Create the model
model.fit(X_train, Y_train)  # Train the model with training data

Make Predictions:
- Once trained, the model can predict categories for new data.
```
Y_pred = model.predict(X_test)  # Predict the outcomes for test data
```

Evaluate the Model:

Check how well the model performed by comparing predictions with actual results.

from sklearn.metrics import accuracy_score, confusion_matrix

accuracy = accuracy_score(Y_test, Y_pred)  # Measure how accurate the predictions are
cm = confusion_matrix(Y_test, Y_pred)  # Compare predicted vs actual results

print("Accuracy:", accuracy)
print("Confusion Matrix:\n", cm)

Why Use Decision Tree Classifier?¶

Easy to Understand: The tree format makes it simple to follow the decision-making process.
No Need for Scaling: You don’t have to worry about changing the scale of your features.
Handles Complex Data: It can deal with various types of data and relationships.

Things to Keep in Mind¶

Overfitting: Decision trees can sometimes learn too much from the training data, making them less effective on new data.
Sensitive to Changes: Small changes in the data can lead to different tree structures, which may affect predictions.

Conclusion¶

A Decision Tree Classifier is a straightforward and powerful tool for categorizing data. It helps you make decisions based on a series of questions, making it easy to understand and use. If you have more questions or need further details, just let me know!

Let’s review example step by step.¶

1. Importing Libraries¶

In [1]:

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt

NumPy: For numerical operations.
Pandas: For data manipulation and analysis.
Matplotlib: For data visualization.

2. Loading the Dataset¶

In [3]:

data = pd.read_csv('Social_Network_Ads.csv')
data

Out[3]:

	User ID	Gender	Age	EstimatedSalary	Purchased
0	15624510	Male	19	19000	0
1	15810944	Male	35	20000	0
2	15668575	Female	26	43000	0
3	15603246	Female	27	57000	0
4	15804002	Male	19	76000	0
…	…	…	…	…	…
395	15691863	Female	46	41000	1
396	15706071	Male	51	23000	1
397	15654296	Female	50	20000	1
398	15755018	Male	36	33000	0
399	15594041	Female	49	36000	1

400 rows × 5 columns

This line reads the dataset from a CSV file into a Pandas DataFrame named data.

3. Preparing Features and Target Variables¶

In [4]:

X = data.iloc[:, 2:4].values  # Features: Age and Salary
y = data.iloc[:, 4].values      # Target variable: Purchased (Yes/No)

X: Contains the feature columns (Age and Salary).
y: Contains the target variable, indicating whether the user purchased the product.

4. Visualizing the Data¶

In [5]:

plt.title("Social Network Ads by Age and Salary")
plt.xlabel("Age")
plt.ylabel("Salary")

plt.scatter(X[y==0, 0], X[y==0, 1], label='No')  # Users who did not purchase
plt.scatter(X[y==1, 0], X[y==1, 1], label='Yes')  # Users who purchased

plt.legend()
plt.show()

No description has been provided for this image

This section visualizes the data points by plotting Age against Salary.
Different colors represent users who did or did not purchase.

5. Splitting the Dataset into Training and Test Sets¶

In [6]:

from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=0)

The data is split into training (75%) and testing (25%) sets to evaluate the model’s performance.

6. Feature Scaling¶

In [7]:

from sklearn.preprocessing import StandardScaler
sc_x = StandardScaler()

In [8]:

X_train = sc_x.fit_transform(X_train)  # Fit and transform the training data
X_test = sc_x.transform(X_test)        # Transform the test data

StandardScaler is used to standardize the features, ensuring they have a mean of 0 and a standard deviation of 1. This is important for many machine learning algorithms.

7. Creating and Training the Decision Tree Classifier¶

In [9]:

from sklearn.tree import DecisionTreeClassifier
classifier = DecisionTreeClassifier(random_state=0)  # Initialize the classifier
classifier.fit(X_train, y_train)  # Train the classifier

Out[9]:

DecisionTreeClassifier(random_state=0)

In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.

The Decision Tree Classifier is initialized and trained using the training data.

8. Making Predictions¶

In [10]:

y_pred = classifier.predict(X_test)  # Predicting the test set results

Predictions are made for the test set using the trained model.

9. Evaluating the Model¶

In [11]:

classifier.score(X_test, y_test) * 100  # Model accuracy in percentage

Out[11]:

90.0

In [12]:

from sklearn.metrics import accuracy_score
accuracy_score(y_test, y_pred) * 100  # Accuracy score calculation

Out[12]:

90.0

In [13]:

from sklearn.metrics import confusion_matrix
cm = confusion_matrix(y_test, y_pred)  # Confusion matrix
cm

Out[13]:

array([[62,  6],
       [ 4, 28]], dtype=int64)

Model Accuracy: The overall accuracy of the model on the test set is calculated.
Confusion Matrix: This matrix shows the performance of the classifier by comparing predicted and actual values.

10. Visualizing the Decision Boundary¶

In [14]:

X_set, y_set = X_test, y_test  # Using the test set for visualization

In [15]:

plt.title("Social Network Ads by Age and Salary")
plt.xlabel("Age")
plt.ylabel("Salary")

Out[15]:

Text(0, 0.5, 'Salary')

In [16]:

X1 = np.arange(X_set[:, 0].min()-1, X_set[:, 0].max()+1, 0.01)  # Range for Age
X2 = np.arange(X_set[:, 1].min()-1, X_set[:, 1].max()+1, 0.01)  # Range for Salary

In [17]:

xx, yy = np.meshgrid(X1, X2)  # Create a grid of values
X3 = np.array([xx.ravel(), yy.ravel()]).T  # Combine grid values

In [18]:

zz = classifier.predict(X3).reshape(xx.shape)  # Predictions for the grid

In [19]:

plt.contourf(xx, yy, zz)  # Plotting the decision boundary

plt.scatter(X_set[y_set == 0, 0], X_set[y_set == 0, 1], label='No')  # Users who did not purchase
plt.scatter(X_set[y_set == 1, 0], X_set[y_set == 1, 1], label='Yes')  # Users who purchased

plt.legend()
plt.show()

This section visualizes the decision boundary of the classifier.
A grid is created to show areas where the model predicts different outcomes (purchased vs. not purchased).

Conclusion¶

This code effectively demonstrates the process of using a Decision Tree Classifier to predict user behavior based on age and salary. It includes data loading, preprocessing, model training, prediction, evaluation, and visualization of results.

If you have any questions or need further clarification on any part, feel free to ask!