Introduction to Machine Learning

info

In this chapter you'll be introduced to:

• Machine Learning (ML): Understanding what machine learning is and how it differs from traditional programming.
• Learning from Experience vs. Instructions: The distinction between learning from data and following explicit procedures.
• Components of a Machine Learning System: The stages involved in building and using a machine learning model.
• Types of Machine Learning: An overview of supervised, unsupervised, and reinforcement learning.
• Other Categorizations: Discussions on batch vs. online learning, passive vs. active learning, and structured prediction.

What is Machine Learning?

Machine Learning (ML) is a field of artificial intelligence that focuses on enabling machines to learn from experience rather than being explicitly programmed. In essence, it's about developing algorithms that can recognize patterns in data and make informed decisions or predictions based on that data.

Unlike traditional programming, where a computer follows a set of predefined instructions to produce an output, machine learning allows computers to learn and adapt from data inputs without being explicitly programmed for specific tasks.

Learning from Instructions (Programming)

In traditional programming, humans provide explicit instructions or procedures for the computer to follow. For example, learning to perform multiplication involves following a specific algorithm taught in school:

1. Align the numbers based on place value.
2. Multiply each digit accordingly.
3. Sum the results to get the final product.

This method relies on learning from instructions, where the steps are clearly defined and must be followed precisely.

Learning from Experience (Machine Learning)

In contrast, machine learning focuses on learning from experience or data. For example, recognizing handwritten letters involves exposure to many examples of each letter in various styles and fonts. The learning algorithm identifies patterns and features that distinguish one letter from another without explicit programming for each variation.

Example

An Example of MNIST dataset of numbers.

This approach is essential when:

• The desired function is too complex to model with explicit instructions.
• The rules governing the task are unknown or hard to articulate.
• The system needs to adapt to new patterns or data over time.

Components of a Machine Learning System

A machine learning system typically involves the following components:

1. Experience (Data): The dataset comprising examples the system learns from.
2. Machine Learning Algorithm: The method or procedure used to learn from data.
3. Machine Learning Model: The resulting function or representation after learning.
4. New Data (Possibly Unseen): Fresh inputs the model will make predictions on.
5. Prediction: The output or decision made by the model based on new data.

Machine learning involves two primary phases:

1. Training (Learning): The model learns from the training data using a machine learning algorithm to create a predictive model.
2. Prediction (Inference): The trained model makes predictions or decisions based on new, unseen data.

Types of Machine Learning

Machine learning can be broadly categorized into three types:

Supervised Learning

Supervised Learning involves learning a function that maps an input to an output based on example input-output pairs. It requires:

• Specific Goal of Prediction: The target variable or outcome is known and defined.
• Known Target During Training: The correct output (labels) for each training example is provided.

Example

Imagine you have a box containing an animal you cannot see. You can feel its size and the softness of its fur. The goal is to predict whether it's a cat or a dog.

• Features (Inputs):
  • Size: A continuous value between 0 (small) and 1 (large).
  • Fur Softness: A continuous value between 0 (hard) and 1 (soft).
• Target (Output): 'Cat' or 'Dog'.

During training, you have labeled examples:

Size	Fur Softness	Animal
0.2	0.9	Cat
0.8	0.3	Dog
0.5	0.8	Cat
0.7	0.4	Dog

The model learns to classify new animals based on these features.

Tasks in Supervised Learning

Among the supervised learning category, we can broadly distinguish two kinds of task:

• Classification: The target variable is categorical (e.g., animal type, spam detection).
• Regression: The target variable is continuous (e.g., predicting stock prices).

Example

Imagine you have data about various houses:

• Features (Inputs):
  • Size: Total square footage of the house.
  • Number of Bedrooms: Total bedrooms in the house.
  • Location: Neighborhood or area code.
  • Age: Number of years since the house was built.

Regression Task

• Target (Output): Market price of the house (a continuous value in dollars).

In the regression task, the goal is to predict the exact market price based on the house's features.

Size (sq ft)	Bedrooms	Location	Age (years)	Price ($)
2,000	3	A	5	350,000
1,500	2	B	10	250,000
2,500	4	A	2	500,000
1,800	3	C	20	200,000

The model learns to predict prices like 375,000 or 410,000 for new houses.

Classification Task

• Target (Output): Price tier category (e.g., 'Affordable', 'Mid-range', 'Luxury').

In the classification task, the goal is to assign each house to a price tier based on its features.

Size (sq ft)	Bedrooms	Location	Age (years)	Price Tier
2,000	3	A	5	Mid-range
1,500	2	B	10	Affordable
2,500	4	A	2	Luxury
1,800	3	C	20	Affordable

The model learns to classify new houses into categories like 'Affordable' or 'Luxury'.

Unsupervised Learning

Unsupervised Learning deals with unlabeled data, aiming to find patterns or intrinsic structures within the data without predefined labels.

Clustering

Grouping similar data points together based on feature similarity.

Example

Grouping pacients into clusters based on severity and number of symptoms per week.

An Example of Clustering¹.

Outlier Detection

Identifying data points that deviate significantly from the majority of the data.

Example

A Comparison of Outlier Detection methods².

Representation Learning

Learning useful representations or features from the data that can be used for various tasks.

Example

Extracting features from images that capture important information, which can then be used for tasks like image classification or captioning.

Representation transformation to perform image classification³.

Reinforcement Learning

Reinforcement Learning (RL) involves learning to make decisions by taking actions in an environment to maximize cumulative rewards.

• Key Characteristics:
  • Learning through trial and error.
  • No explicit target during training.
  • Receiving feedback in the form of rewards or penalties.

Loading...

Note: RL is covered extensively in the Intro to RL documentation.

Other Categorizations of Machine Learning

Machine learning algorithms can also be categorized based on different criteria:

Batch vs. Online Learning

• Batch Learning: The model is trained on the entire dataset at once. Data is static and doesn't change over time.
• Online Learning: The model learns incrementally as new data comes in, often one instance at a time.

Passive vs. Active Learning

• Passive Learning: The model learns from the dataset provided without influencing data collection.
• Active Learning: The model can query or request additional data points or labels to improve learning.

Structured Prediction

• Structured Prediction: Involves predicting multiple interrelated outputs simultaneously, often with dependencies among them.

Example

Image Segmentation

Assigning a label to every pixel in an image to identify objects or regions, considering the spatial relationships between pixels.

An Example of Image Segmentation¹.

Recap

In this chapter, we've covered:

• Definition of Machine Learning: Understanding ML as learning from experience to make predictions.
• Learning from Experience vs. Instructions: The difference between machine learning and traditional programming.
• Components of a Machine Learning System: The stages of training and prediction.
• Types of Machine Learning: Supervised, unsupervised, and reinforcement learning.
• Other Categorizations: Batch vs. online learning, passive vs. active learning, and structured prediction.

---

Footnotes

1. Portions of this page are reproduced from work created and shared by Google and used according to terms described in the Creative Commons 4.0 Attribution License. ↩ ↩²

2. Portions of this page are reproduced from work created and shared by Scikit-learn and used according to terms described in the BSD 3-Clause License. ↩

3. Source: Spot Intelligence's blog post by Neri Van Otten. ↩