Train Your First Machine Learning Model with scikit-learn

What does "training a model" actually mean?

Think of teaching a new colleague to sort post: no rulebook, just a pile of envelopes, "this one goes to Lahore, this one to Karachi", until they spot the pattern themselves. Then you test them on envelopes they have never seen and count how many they get right.

A machine learning model is the same arrangement, in code. The training data is the pile of labelled examples, training is finding the pattern, the test data is the fresh pile, and accuracy is how many it gets right.

Three words you will meet constantly: features. The measurements the model may look at (here, four numbers per flower); labels, the correct answers attached to the training examples (here, the species); prediction. The model's answer for a flower it has never seen.

Because the answer is one of three categories, this is a classification problem. The model assigns each record to a category, rather than predicting a number (that would be regression). Classification is the gentlest first model to build, because the result is easy to check: right or wrong, flower by flower.

What is the iris dataset: and why start with flowers?

The iris dataset is the oldest teaching dataset in machine learning: 150 iris flowers, 50 of each of three species, each described by four measurements in centimetres. It ships inside scikit-learn itself, so there is nothing to download and no file to find. Load_iris() just works, offline.

That is exactly why it is the right first dataset. Your first model should teach you the workflow, not fight you over data. Real datasets come with missing values, wrong types and duplicates. You already learned to handle those in Cleaning Data with pandas. Here the data is clean on purpose, so every surprise in the output is about the model, not the mess. One honest warning: iris is almost too easy. The species separate cleanly on petal measurements, so a simple model scores perfectly. Learn the moves here, load, split, train, evaluate, predict, and expect real data to be harder. The moves are the same; only the score changes.

What are the five steps of training a model?

Every classification project, from this flower example to a spam filter, follows the same five steps: load the features and labels; split off a test set the model never trains on (usually 80% train, 20% test); train by calling fit() on the training data; evaluate by predicting the test labels and counting how many are right; predict on genuinely new examples.

Step 2 is the one beginners most often skip, and it matters most. A model tested on the data it trained on will look brilliant and tell you nothing. It may simply have memorised the answers. The test set is the only honest judge.

A decision tree is a good first algorithm because you can picture it: a sequence of yes/no questions about the measurements ("is the petal length less than 2.5 cm?") leading down to a species. You do not need its internals yet. DecisionTreeClassifier does the work; your job is the workflow around it.

Worked example: train and test a flower classifier

Install scikit-learn first, if you have not already:

pip install scikit-learn

Then save this as first_model.py and run it with python first_model.py:

from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.tree import DecisionTreeClassifier
from sklearn.metrics import accuracy_score

# 1. Load the data
iris = load_iris()
print("Flowers in dataset:", iris.data.shape[0])
print("Measurements per flower:", iris.data.shape[1])
print("Classes:", ", ".join(iris.target_names))

# 2. Split into training and test sets
X_train, X_test, y_train, y_test = train_test_split(
    iris.data, iris.target, test_size=0.2, random_state=42
)
print("Train on", X_train.shape[0], "| test on", X_test.shape[0])

# 3. Train the model
model = DecisionTreeClassifier(random_state=42)
model.fit(X_train, y_train)

# 4. Evaluate on data the model has never seen
accuracy = accuracy_score(y_test, model.predict(X_test))
print("Test accuracy:", accuracy)

# 5. Predict one new flower
new_flower = [[5.1, 3.5, 1.4, 0.2]]
predicted = model.predict(new_flower)[0]
print("Prediction for", new_flower[0], "->", iris.target_names[predicted])

Exact output, as run for this tutorial:

Flowers in dataset: 150
Measurements per flower: 4
Classes: setosa, versicolor, virginica
Train on 120 | test on 30
Test accuracy: 1.0
Prediction for [5.1, 3.5, 1.4, 0.2] -> setosa

Read the code line by line. load_iris() returns the dataset; .data is the 150-by-4 table of measurements and .target is the list of 150 labels. The ", ".join(iris.target_names) is deliberate. Printing the raw array shows NumPy string wrappers in current versions, which is confusing in a beginner tutorial.

train_test_split with test_size=0.2 holds back exactly 30 flowers for testing, and random_state=42 fixes the shuffle so the split is identical on every run. The naming is a convention worth learning: X for features, y for labels, _train and _test for the two halves.

DecisionTreeClassifier(random_state=42) creates the model; model.fit(X_train, y_train) trains it. model.predict(X_test) produces 30 guesses, and accuracy_score compares them with the truth: 30 out of 30 correct is 1.0. Finally, one new flower, [5.1, 3.5, 1.4, 0.2]. Goes in as a list inside a list, because predict always expects a table of rows, even for one row. The model answers setosa: short petals, wide sepals, a classic setosa.

How do I read the accuracy score: and why is 1.0 not the whole story?

Accuracy is the share of test examples the model labelled correctly: 1.0 means all 30, 0.9 would mean 27 of 30. It is the simplest honest number in machine learning, and the right first metric to learn.

But a perfect score deserves suspicion, not celebration. Ask three questions whenever accuracy looks too good:

• Did the test data leak into training? Here it did not, the split happened before training, and fit only ever saw X_train. Fit on the full dataset and "test" on the same rows, and the score is meaningless.
• Is the dataset too easy? Iris is. The species separate almost perfectly on petal length alone, so a 1.0 here says more about the dataset than about your skill.
• Is the test set big enough? Thirty flowers is small: one wrong answer drops the score to 0.97. Real projects use larger test sets and more metrics. Precision, recall and the confusion matrix, covered in the course lesson "Is My Model Any Good? Accuracy and Confusion Matrix".

There is also overfitting: a model that memorises the training data so closely that it performs worse on new data. The defence is the test set you already have. If training accuracy is perfect but test accuracy drops, the model memorised instead of learning. The course lesson "Overfitting: Why Perfect Scores Can Lie" takes this further.

What should I try next?

Change one thing at a time and re-run:

1. Change random_state to 7 and watch the accuracy. The split changes; the score usually stays 1.0 on iris, which tells you the result is stable, not a lucky split.
2. Change test_size to 0.3 and see the train/test counts move to 105 and 45.
3. Try a different new flower, [6.0, 2.2, 4.0, 1.0] should come back versicolor.
4. Print the training accuracy too (accuracy_score(y_train, model.predict(X_train))). If it ever disagrees sharply with the test accuracy, you have found overfitting in the wild.

When you are ready for a real dataset, go back to Cleaning Data with pandas and clean something messy first, then train on it. Clean data first, model second: that order is the whole discipline of applied machine learning.

Common mistakes

• Testing on the training data. Calling model.predict(X_train) and celebrating a perfect score proves nothing. The model has already seen those rows. Always evaluate on X_test.
• Forgetting the double brackets on a single prediction. model.predict([5.1, 3.5, 1.4, 0.2]) raises an error because predict expects a table of rows. One flower is a table with one row: [[5.1, 3.5, 1.4, 0.2]].
• Skipping random_state and wondering why results change. Without it, every run shuffles differently. Set it while you are learning so your results are repeatable.
• Reading 1.0 as "my model is perfect". On iris, 1.0 is the dataset being easy. The honest reading is "the workflow works and the model learned the pattern", nothing more.
• Jumping to a model before cleaning the data. Missing values will stop most scikit-learn models with an error, and duplicates will quietly bias the result. Clean first, train second.

Frequently asked questions