Quentin | Data Scientist @QuentinData

Implementing Cross Validation in Machine Learning using sklearn !!! If you missed the first thread with the theory, check it out ⬇️ x.com/_jaydeepkarale… `sklearn` provides two methdods - cross_val_score - cross_validate To simplify cross validation i.e. determining if it's actually worth building the model by training the model over various combinations of the data set ▶️ Difference between cross_val_score & cross_validate `cross_val_score` does not return the trained models & can only run a single metric against each fold `cross_validate` on the other hand accepts a list of metrics and also returns some other information such as `fit_time, test_score, test_time, estimator(the trained model)` ▶️ Details of the dataset used ▶️ Using the `DecisionTreeClassifier` & `cross_val_score` the model yields an accuracy of roughly 62% which isn't all that bad. The problem is there is no way to retrieve this trained model `cross_val_score` enables Data Scientists and Machine Learning Engineers to know if it is worth implementing Cross Validation` Let's now implement actual CROSS Validation using `cross_validate` ▶️ We first split the dataset into training and test, reserving 10% to test our models. We run the Cross Validation with 10 sub groups and display scores of each of the 10 trained models ▶️ Finally we go over the models stored in 'estimator' and check how they do over the 10% test data we kept aside We do get a decent 61.8% here as well, not the best model but the importance of cross validation is clear to see ▶️How to use CV models for predictions? There are different approaches depending on the practitioner: > Take the best of the N models and use it directly > Take the best of the N models and re-train it on the whole data set > Keep the N models and rely on the opinion of the majority

keyboard_arrow_left Previous keyboard_arrow_right Next

Jaydeep @_jaydeepkarale

3 years ago

Cross Validation in Machine Learning clearly explained !!! 1⃣ What Is Cross Validation ? Imagine you have a limited dataset, and you want to know how well your machine learning model will perform on new, unseen data. Cross-validation is like a smart technique that helps you get

keyboard_arrow_left Previous keyboard_arrow_right Next

8 40 194 45K 196

@Fixyouk_ @_jaydeepkarale You can compute the average performance of a model on each fold, it gives a great estimation of the model performance with all the data. You can then compare this between all your models and select the best one

@_jaydeepkarale Awesome post. Cross-Validation is such an important method. We want more posts like this one !

Cross Validation in Machine Learning clearly explained !!! 1⃣ What Is Cross Validation ? Imagine you have a limited dataset, and you want to know how well your machine learning model will perform on new, unseen data. Cross-validation is like a smart technique that helps you get a more reliable estimate of your model's performance. Here's how it works: Instead of just splitting your data into two parts (training and testing sets), cross-validation breaks your data into several subsets. It trains your model multiple times, using different combinations of these subsets for training and testing. This way, you get a better sense of how well your model generalizes to new data, making your predictions more trustworthy. 2⃣ Different types of cross validation ? - Hold Out Method - Leave One Out Cross-Validation - K-Fold Cross Validation - Stratified k-Fold Cross Validation Let us now see each in detail along with implementation 3⃣ Hold Out Method Simplest method where the data is split into two sets based on % i.e. 70-30 or 60-40 and so on. 4⃣ Leave One Out Cross-Validation Same as Hold Out Method but a single sample is taken as test data and rest all as training data. This process is repeated over entire data set so that each data acts as test data at least once. As seen in the output, the test data contains only a single sample 5⃣ K-Fold Cross Validation The whole data is divided into k sets of almost equal sizes called `fold`. The first set is selected as the test set and the model is trained on the remaining k-1 sets. The test error rate is then calculated after fitting the model to the test data. The process is repeated by using each of the `fold/s` as test at least once. 6⃣ Stratified k-Fold Cross Validation Slight variation from K-Fold Cross Validation, which uses ‘stratified sampling’ instead of ‘random sampling.’ This helps further reduce bias by carefully choosing the `fold/s` instead of randomly forming them.

keyboard_arrow_left Previous keyboard_arrow_right Next

The 2 building blocks of a Machine Learning model. Parameters and Hyperparameters 101. 1⃣ Hyperparameters Any quantity you choose, twist, change, and set before the training process is a hyperparameter. They control the learning process and determine the structure and behavior of the model. They are external since the model cannot change these values. Hyperparameters need to be tuned to find the best configuration for the specific problem and dataset. This process is often called "hyperparameter tuning" or "hyperparameter optimization." It involves trying different combinations of values to find the best-performing combination. Examples: - Batch size - Learning rate - Train-test split ratio - Number of trees in random forest - Number of layers in a neural network 2⃣Parameters Any quantity that the model creates or modifies during the training process is a parameter. They are continuously changing and adapting based on the data to minimize the error or loss function during training. They are estimated from the data. They are internal to the model. Examples: - Decision tree's split points - Linear regression model's coefficients - Weights and biases of a neural network To conclude: You set the hyperparameters before training and the algorithm will use the setup to learn the parameters. ___ That's it for today. I hope you've found this Tweet helpful. Like/Retweet for support and follow @levikul09 for more Data Science content. Thanks 😉