Machine Learning: The Honest First Lesson

Posted on Oct 15 • Originally published at iqraa.tech

TL;DR: Machine learning lets software work out its own rules from examples instead of following instructions a programmer typed, and every model in the field trains through one repeating cycle of prediction, penalty, and update.

Most introductions to machine learning tell a story that collapses under a fact-check: dates shifted a decade, a funding collapse nobody mentions, and driverless claims that are not driverless. The video below is the checked version, drawn board by board; this article walks the same ground. Watch the full episode on YouTube, or read on.

The episode this article accompanies: 15 boards, 7.5 minutes, every date and number checked against the sources cited below.

The first board is the promise itself: three claims crossed out and replaced.

Board 1: the three claims that do not survive a fact-check.

The definition is smaller than the hype suggests. Machine learning covers the AI techniques whose behavior comes from training on examples rather than from instructions a developer typed: the algorithm scans data, extracts patterns, and applies them to inputs it was never shown, with nobody encoding the decision rules directly ( Wikipedia summary ). Training produces a model: a large set of numbers mapping fresh inputs to outputs.

A traditional spam filter is a hand-written list of tests; a trained one is a model fitted to millions of labeled messages, catching campaigns nobody wrote rules for.

Training runs on one cycle, whatever the family. The model produces an output for an example; a loss function converts the gap to the correct answer into a number; an optimizer nudges the model’s parameters in whatever direction lowers the number. Then the next example, and the next, until the errors stop shrinking.

The cycle is identical whether the model sorts mail, recommends products, or drafts chat replies; what varies is what counts as an example and as a wrong answer. Our model training glossary covers the vocabulary in more depth.

What does a loss function actually measure? A loss function puts a number on the distance between what the model predicted and what was true, averaged over examples. For a house-price model it might be squared error in dollars: predict 310,000 for a house worth 300,000 and the loss is the square of the gap. For a classifier it might be how much probability the model put on the wrong answer.

The optimizer never sees right or wrong; it only sees loss going down, which is why the choice of loss function decides what the model optimizes for. Get that choice wrong and the model dutifully learns the wrong thing. The loss function reference walks the common families if you want the math.

The origin story usually compresses two dates into one. Work on a program that learned to play checkers began at IBM in 1949, under Arthur Samuel; its earliest version executed on the IBM 701, scored each board by estimating who was ahead, and improved by rote learning, per his biography. A decade on, his 1959 paper introduced the phrase “machine learning”. The research is a 1949 artifact; the name arrived in 1959.

Each row is verified on the page linked for it: Samuel’s biography (1949), the perceptron article (1969), the deep learning page (October 2012), and the language model overview (2018 to 2023). Most tutorials stop at 2012 and treat the field as a decade old; the transformer half of the timeline is newer than the tutorials describing it.

Board 6: the deep-learning revolution, and deep learning nested inside machine learning where it belongs.

Deep learning, despite the branding, is the multilayer interior of machine learning: the layers themselves work out which features matter rather than engineers specifying them by hand. One branch of the tree, not a different tree.

Why couldn’t a perceptron solve XOR? A single-layer perceptron draws one straight line through the input space and puts a yes on one side and a no on the other. XOR is the logical function that is true when its two inputs differ: of the four possible input pairs, the two true pairs sit diagonally across from each other, and no single line can separate the two true corners from the two false corners. That is all linear separability means, and XOR famously lacks it.

Minsky and Papert’s 1969 book Perceptrons contained the proof that a one-layer network cannot crack the function, and research money evaporated for more than a decade afterward. The fix was already waiting: stack multiple layers between input and output, and the network can bend the boundary. Training those stacks only became practical in the 1980s, when backpropagation supplied a way to assign each layer its share of the error.

The standard taxonomy counts four families, separated by the supervision signal available during training, not by any notion of intelligence.

Supervised learning covers two prediction jobs: naming an input’s category, or putting a number on it, like a house price. Unsupervised learning receives no answer key and must discover whatever structure the data holds. Reinforcement learning is trial and error: an agent acts inside an environment, is scored by rewards, and settles on behavior that earns more of them. That third family gets its own treatment in our reinforcement learning walkthrough.

Where do large language models fit in the four families? Pretraining a large language model is supervised learning at extreme scale. The examples come from massive text corpora, and each one carries its own label for free: predict the next token, and the actual next token in the text is the answer key. No human labels anything; the corpus labels itself. That is why the transformer paper of June 2017 mattered so much: attention-only processing parallelizes next-token prediction across entire sequences, which is what made that scale trainable.

The lineage runs from BERT and GPT-1 (2018) through ChatGPT’s late-2022 breakout to today’s multimodal models. When a model is then tuned toward helpfulness with human feedback on its own outputs, a reinforcement layer joins the stack. For the full mechanics, our LLM concepts deep dive continues from here.

Board 10: the ML already running in products you open daily.

Since 1998, Amazon’s recommendations have used item-to-item collaborative filtering, described in a 2003 IEEE paper by its engineers ( method overview ); it is still the reference design for production recommenders. The trick is comparing products to products instead of people to people, which scales far better.

Google’s documentation for Gmail names two learning systems doing the visible work: the spam and phishing filters, credited with 99.9 percent detection in 2017, and Smart Reply. The five inbox tabs, added in 2013, organize the mail that survives; the catching happens upstream ( per the Gmail article ). Diagnostics, forecasting, speech, and vision round out the board’s point: trained models already sit inside products you opened today.

Why does item-to-item filtering scale better than comparing users? User-to-user collaborative filtering has to answer an expensive question continuously: which of millions of customers resemble you right now? Those similarity rows go stale as tastes change and grow more expensive as the customer base grows. Item-to-item flips the question: which products resemble this product, judged by which customers bought both?

A retail catalog changes far more slowly than a customer base and is much smaller, so the item-similarity table stays manageable, stable, and computable in advance. That is the engineering argument Amazon made in its 2003 IEEE paper on the system it had been running since 1998, and it is why the method became the default shape for production recommenders. A shopper with only a handful of purchases still gets recommendations, because the item table already knew the answers before that shopper arrived.

This is the correction to hold on to. The Full Self-Driving package Tesla sells to car owners is Level 2 assistance: it steers, accelerates, and brakes while the human behind the wheel stays responsible and must be able to intervene at any second ( per the Autopilot article ). Whatever the branding implies, no consumer Tesla drives itself.

Actual driverlessness starts at Level 4. Waymo has carried paying riders with nobody in the driver’s seat since 2020, and by June 2026 its service covered ten US metro areas ( per the Waymo article ). Tesla runs a separate Robotaxi service. The consumer package still depends on a human supervisor; the robotaxis have none.

Is Tesla’s Robotaxi the same product as consumer Full Self-Driving? No, and conflating them is exactly the mistake the fact-check exists to catch. Consumer Full Self-Driving is the Level 2 option Tesla sells to car owners: the person in the driver’s seat is the supervisor and remains responsible throughout. The Robotaxi service is a separate commercial operation, in the same category as Waymo’s driverless rides, with no customer sitting in a driver’s seat at all.

The two can share underlying technology and still sit on different rungs of the autonomy ladder, because the levels describe who is responsible for supervision, not which sensors or neural networks are installed. When you read a claim about driverless Teslas, check which product the sentence means before repeating it.

Models inherit their raw material. In one cited case, a healthcare system’s model judged how much care patients needed by how much their care cost; because access is unequal, Black patients looked healthier than they were and lost extra programs. In a 2018 facial-analysis study, error rates reached 35 percent among darker-skinned women, against below 1 percent for lighter-skinned men ( both cases documented on the algorithmic bias page ). The bias lived in the data; training locked it in and scaled it up.

Keep the map straight. Deep learning sits inside machine learning as its multilayer branch, and the generative wing of the family produces new content where the classic wing predicts labels or numbers. Same training cycle underneath, different outputs.

Watch the tense. Multimodal and generative systems have already shipped; the actual research frontiers are quantum approaches to learning and models small enough to run on the device itself. Any article presenting ChatGPT as tomorrow’s technology has stamped its own publication date on the page.

Start with Python (R is the traditional alternative), then a first course’s libraries: NumPy and SciPy for numerical machinery, pandas for loading and reshaping data, Matplotlib for charts, and scikit-learn wrapping the classic algorithms in one consistent API. The whole first lesson fits in seven lines:

from sklearn.datasets import load_iris from sklearn.model_selection import train_test_split from sklearn.neighbors import KNeighborsClassifier X, y = load_iris ( return_X_y = True ) X_train, X_test, y_train, y_test = train_test_split ( X, y, random_state = 0 ) clf = KNeighborsClassifier ( n_neighbors = 3 ). fit ( X_train, y_train ) print ( " test accuracy: ", clf. score ( X_test, y_test )) Enter fullscreen mode Exit fullscreen mode That snippet is the whole cycle: labeled data in, a split into train and test sets, a fit, a score on unseen data. Three areas of mathematics do the underlying work:

For practice, real datasets beat toy ones: Kaggle has hosted competitions since 2010, and the long-lived UCI repository is the other standard starting point.

Why split the data before scoring the model? A model evaluated on its own training data can win by memorization: it has already seen every answer, so a perfect training score proves nothing about tomorrow’s inputs. The split holds a portion back, trains on the rest, and scores on the part the model never saw.

In the snippet above that held-out portion is a quarter of the iris measurements, the default of scikit-learn’s train_test_split. The resulting score is your only honest estimate of generalization, and generalization is the entire promise of machine learning: handling data the model has never seen. The cross-validation guide extends the same idea to multiple splits, so no single lucky partition gets to flatter the result.

Board 15: the recap, one line per chapter.

Strip everything away and one idea remains: a machine learning system works out its own rules from the examples you show it, cycling through a prediction, a penalty, and an update until the penalties shrink. The discipline predates its own 1959 name, weathered a decade in the wilderness after 1969, and now runs on transformer architectures younger than many posts about it. Four families share that engine; the wheel still belongs to the human in a Level 2 car, and whatever skew the training data carries comes back out through the predictions, magnified.

This lesson first appeared on iqraa.tech.

More whiteboard lessons on the Whiteboard Architect YouTube channel.