Your Perfect Assignment is Just a Click Away

We Write Custom Academic Papers

100% Original, Plagiarism Free, Customized to your instructions!


Model Selection And Validation

Model Selection And Validation

Model Selection And Validation

For this assignment, you are to determine which model is best for prediction, report the

right hyperparameters, and the resulting accuracy for the Digit Recognition data set that

was used in the previous assignment.

As before, create a PDF of your notebook showing your steps and include the table

below as mentioned. Attach both a jupyter notebook and the PDF version. Also, be sure

to include your name at the start of the notebook and at the top of the PDF.

Specifically, you are to test the following models

Model Hyperparameter Testing range Notes

Support Vector Machine

Gamma – size of the kernel C – slack variable

10-x for x = -5 to 5 use the ‘rbf’: radial basis function kernel

K-nearest neighbors

k – number of neighbors

1,3,5,7,9 use the sklearn function

Decision Trees min_samples_split 3,5,7,9 (1 was removed)

use the defaults for the other hyperparameters.

Logistic Regression C – inverse of the regularization strength (smaller = more regularization)

10-x for x = -5 to 5 with the L1 penalty (Lasso)

Steps are as follows:

1. Separate your data into training and testing. We will use cross-validation over the

training set to select the right parameters

a. Use train_test_split to create a separate training and test set.

X_train, X_test, y_train, y_test = train_test_split(X,

y, stratify=True, test_size=0.20)

b. For the training set, you have two choices to perform hyperparameter


i. Use cross-validation to evaluate each model variant and select the

best hyperparameters (standard practice, most recommended)

ii. Create a hold-out validation set and train on one portion of the data

and use the accuracy on the hold-out validation set to pick the right

hyperparameters (also valid)

2. Steps to turn in for the assignment

a. Train the four models with their default parameters. Report the resulting

accuracy of each model using the default parameters.

b. For each of the four models, find the hyperparameters giving the highest

accuracy on the validation set by performing an exhaustive grid search.

Report the hyperparameter values and accuracy on the validation


i. Consider using sklearn.model_selection.GridSearchCV

ii. For the models with two hyperparameters, you will need to search

both simultaneously to find the optimum combination

c. Now apply the highest accuracy trained models to the test set. Report the

accuracy of each model.

Fill the following table with the information.

Model Default validation accuracy

Tuned validation accuracy

Selected hyperparameter s

Final test set accuracy



Decision Trees

Logistic Regression

Order Solution Now

Our Service Charter

1. Professional & Expert Writers: Writers Hero only hires the best. Our writers are specially selected and recruited, after which they undergo further training to perfect their skills for specialization purposes. Moreover, our writers are holders of masters and Ph.D. degrees. They have impressive academic records, besides being native English speakers.

2. Top Quality Papers: Our customers are always guaranteed papers that exceed their expectations. All our writers have +5 years of experience. This implies that all papers are written by individuals who are experts in their fields. In addition, the quality team reviews all the papers before sending them to the customers.

3. Plagiarism-Free Papers: All papers provided by Writers Hero are written from scratch. Appropriate referencing and citation of key information are followed. Plagiarism checkers are used by the Quality assurance team and our editors just to double-check that there are no instances of plagiarism.

4. Timely Delivery: Time wasted is equivalent to a failed dedication and commitment. Writers Hero is known for timely delivery of any pending customer orders. Customers are well informed of the progress of their papers to ensure they keep track of what the writer is providing before the final draft is sent for grading.

5. Affordable Prices: Our prices are fairly structured to fit all groups. Any customer willing to place their assignments with us can do so at very affordable prices. In addition, our customers enjoy regular discounts and bonuses.

6. 24/7 Customer Support: At Writers Hero, we have put in place a team of experts who answer all customer inquiries promptly. The best part is the ever-availability of the team. Customers can make inquiries anytime.