Machine Learning Tutorials, Courses and Certifications
What is Grid Search?¶
- Think of a Recipe: Imagine you want to make a perfect cake. You have different ingredients (like flour, sugar, eggs) and different amounts for each ingredient. Grid search is like trying out all possible combinations of these ingredients to find the best cake recipe.
- In Machine Learning: In machine learning, you have hyperparameters (settings that affect how your model works). Grid search tests all combinations of these hyperparameters to see which one makes the model perform best.
What is K-Fold Cross-Validation?¶
- Sharing Pizza: Suppose you have a pizza to share with friends. Instead of letting just one friend eat the whole pizza and give their opinion, you cut it into equal slices. Each friend takes a slice and gives feedback.
- In Machine Learning: K-fold cross-validation splits your data into “k” equal parts (or folds). You train your model on “k-1” parts and test it on the remaining part. You repeat this for each part. This way, every part of your data is used for both training and testing, helping to ensure your model is good.
Combining Them¶
When you put grid search and k-fold cross-validation together:
- Try Every Combination: You start by defining a range of hyperparameters and their values to test (like different amounts of flour or sugar).
- Test with Cross-Validation: For each combination, you use k-fold cross-validation to check how well the model performs.
- Find the Best Combination: After testing all combinations, you look for the one that gives the best performance across all folds.
Why Use This Method?¶
- Better Results: By trying many combinations and validating them properly, you are more likely to find a model that performs well.
- Reliable Evaluation: It ensures that the model’s performance is consistent and not just a lucky guess based on one specific split of the data.
Simple Example¶
Imagine you’re trying to find the best way to make lemonade. You have:
- Sugar levels (low, medium, high)
- Lemon juice levels (low, medium, high)
- Grid Search: You try every combination of sugar and lemon juice.
- K-Fold Cross-Validation: For each combination, you taste it multiple times using different batches of lemonade to make sure your taste isn’t biased by one batch.
At the end, you find the combination that makes the best lemonade!
In machine learning, this process helps you build a model that works really well on new data, not just the data you trained it on.
Let’s review Practically¶
1. Importing Required Libraries¶
In [1]:
import numpy as np
import matplotlib.pyplot as plt
import pandas as pd
2. Importing the Dataset¶
In [2]:
dataset = pd.read_csv('Social_Network_Ads.csv')
X = dataset.iloc[:, [2, 3]].values # Features (Age and Estimated Salary)
y = dataset.iloc[:, 4].values # Target variable (Purchased)
dataset.head() # Displaying the first few rows of the dataset
Out[2]:
| User ID | Gender | Age | EstimatedSalary | Purchased | |
|---|---|---|---|---|---|
| 0 | 15624510 | Male | 19.0 | 19000.0 | 0 |
| 1 | 15810944 | Male | 35.0 | 20000.0 | 0 |
| 2 | 15668575 | Female | 26.0 | 43000.0 | 0 |
| 3 | 15603246 | Female | 27.0 | 57000.0 | 0 |
| 4 | 15804002 | Male | 19.0 | 76000.0 | 0 |
3. Splitting the Dataset into Training and Test Sets¶
In [3]:
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=0)
4. Feature Scaling¶
In [4]:
from sklearn.preprocessing import StandardScaler
sc = StandardScaler()
X_train = sc.fit_transform(X_train) # Scaling the training set
X_test = sc.transform(X_test) # Scaling the test set
5. Fitting SVM to the Training Set¶
In [5]:
from sklearn.svm import SVC
classifier = SVC(kernel='rbf', random_state=0, C=1, gamma=0.7)
classifier.fit(X_train, y_train)
Out[5]:
SVC(C=1, gamma=0.7, random_state=0)In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.
SVC(C=1, gamma=0.7, random_state=0)
6. Predicting the Test Set Results¶
In [6]:
y_pred = classifier.predict(X_test) # Making predictions on the test set
y_pred # Displaying the predicted values
Out[6]:
array([0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 1,
0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0,
1, 0, 0, 1, 0, 1, 1, 0, 0, 1, 1, 1, 0, 0, 1, 0, 0, 1, 0, 1, 0, 1,
0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 1, 1, 1, 1, 0, 0, 1, 0, 0, 1,
1, 0, 0, 1, 0, 0, 0, 0, 0, 1, 1, 1], dtype=int64)
7. Making the Confusion Matrix¶
In [7]:
from sklearn.metrics import confusion_matrix
cm = confusion_matrix(y_test, y_pred) # Creating the confusion matrix
cm # Displaying the confusion matrix
Out[7]:
array([[64, 4],
[ 3, 29]], dtype=int64)
8. Calculating Accuracy Score¶
In [8]:
from sklearn.metrics import accuracy_score
accuracy = accuracy_score(y_test, y_pred) # Calculating accuracy score
accuracy # Displaying accuracy
Out[8]:
0.93
9. Performing Cross-Validation¶
In [9]:
from sklearn.model_selection import cross_val_score
accuracies = cross_val_score(estimator=classifier, X=X_train, y=y_train, cv=10) # Cross-validation
print(accuracies.mean()) # Mean accuracy
print(accuracies.std()) # Standard deviation of accuracy
0.9066666666666666 0.06798692684790378
Note: The mean accuracy is 90%, and the standard deviation of 0.06 indicates that accuracy may vary by ±6%.
10. Hyperparameter Tuning with Grid Search¶
In [10]:
from sklearn.model_selection import GridSearchCV
parameters = [
{'C': [1, 10, 100, 1000], 'kernel': ['linear']},
{'C': [1, 10, 100, 1000], 'kernel': ['rbf'], 'gamma': [0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7]}
]
grid_search = GridSearchCV(estimator=classifier, param_grid=parameters, scoring='accuracy', cv=10, n_jobs=1)
grid_search = grid_search.fit(X_train, y_train) # Fitting the grid search
best_accuracy = grid_search.best_score_ # Best accuracy from grid search
best_parameters = grid_search.best_params_ # Best hyperparameters
print(best_accuracy) # Displaying the best accuracy
print(best_parameters) # Displaying the best hyperparameters
0.9066666666666666
{'C': 1, 'gamma': 0.7, 'kernel': 'rbf'}
11. Visualizing the Training Set Results¶
In [11]:
X_set, y_set = X_train, y_train
plt.scatter(X_set[y_set == 0, 0], X_set[y_set == 0, 1], label=0, color='k') # Class 0
plt.scatter(X_set[y_set == 1, 0], X_set[y_set == 1, 1], label=1, color='red') # Class 1
X1, X2 = np.meshgrid(np.arange(start=X_set[:, 0].min() - 1, stop=X_set[:, 0].max() + 1, step=0.01),
np.arange(start=X_set[:, 1].min() - 1, stop=X_set[:, 1].max() + 1, step=0.01))
Z = classifier.predict(np.array([X1.ravel(), X2.ravel()]).T).reshape(X1.shape) # Predicting for the grid
plt.contourf(X1, X2, Z, alpha=0.2) # Contour plot
plt.title('Support Vector Training Set')
plt.xlabel('Age')
plt.ylabel('Estimated Salary')
plt.legend()
plt.show() # Displaying the plot
12. Visualizing the Test Set Results¶
In [13]:
X_set, y_set = X_test, y_test
plt.scatter(X_set[y_set == 0, 0], X_set[y_set == 0, 1], label=0, color='k') # Class 0
plt.scatter(X_set[y_set == 1, 0], X_set[y_set == 1, 1], label=1, color='red') # Class 1
X1, X2 = np.meshgrid(np.arange(start=X_set[:, 0].min() - 1, stop=X_set[:, 0].max() + 1, step=0.01),
np.arange(start=X_set[:, 1].min() - 1, stop=X_set[:, 1].max() + 1, step=0.01))
Z = classifier.predict(np.array([X1.ravel(), X2.ravel()]).T).reshape(X1.shape) # Predicting for the grid
plt.contourf(X1, X2, Z, alpha=0.2) # Contour plot
plt.title('Support Vector Test Set')
plt.xlabel('Age')
plt.ylabel('Estimated Salary')
plt.legend()
plt.show() # Displaying the plot
Summary¶
- Dataset: Social Network Ads dataset is loaded and prepared for analysis.
- Model: A Support Vector Machine (SVM) classifier is built and trained.
- Evaluation: The model’s performance is evaluated using accuracy, confusion matrix, and cross-validation.
- Hyperparameter Tuning: Grid search is performed to find the best hyperparameters.
- Visualization: The results for both the training and test sets are visualized using scatter plots and contour plots.
This complete breakdown provides clarity on each step of the process in the SVM classification code. If you have any specific questions or need further explanation on any part, feel free to ask!
In [ ]:
