This chapter is from the book
This chapter is from the book
This chapter is from the book
2.7 Building a Machine Learning Model
Having outlined the basic components and objectives of a machine learning model, we are now ready to present a typical workflow of building such a model (see Figure 2.10). The process generally involves the following steps:
)
Figure 2.10: The main stages in building a supervised machine learning model. The initial phase involves using a learning algorithm to develop a model from a set of labeled training examples. The trained model is then applied to a test set of new samples to make predictions, and these predictions are compared to their actual labels in order to evaluate the model’s ability to generalize to new, unseen data. In practice, each of these stages consists of multiple steps; refer to the main text for more details.
Problem Definition:
Define the problem you want to solve. Is it a classification problem, regression, clustering, or other?
Clearly specify the inputs and desired outputs. For example, in a handwritten zip code recognition task, the input might be a single digit, the entire zip code, or even an image of the envelope containing the zip code.
Choose an appropriate performance metric to evaluate the model, such as accuracy, F1 score, mean squared error, etc.
Data Collection:
Acquire data relevant to the problem—e.g., from databases, web scraping, or third-party sources. The dataset needs to be representative of the real-world domain of the problem.
For supervised learning tasks, you also need to collect labels for the data, either from human experts or derived from measurements.
Data Cleaning and Preprocessing:
Handle missing values by imputation or removal.
Encode categorical data into numerical format, using techniques like one-hot encoding or ordinal encoding.
Detect and remove outliers, or determine how to handle them.
Normalize or standardize features when required by the algorithm.
Data Exploration:
Compute descriptive statistics to summarize the distribution of each feature.
Visualize the data to understand patterns, relationships, and anomalies.
Feature Engineering:
Create new features that could improve model performance.
Reduce dimensionality if needed using techniques like PCA or t-SNE.
Train-Test Split:
Divide your data into training and test sets. Typically, 70%-80% of the data is used for training and the rest for testing.
Consider using a validation set or cross-validation for model selection and hyperparameter tuning.
Model Selection:
Choose a suitable algorithm for the task. Different algorithms are suited for different types of data and problems:
For structured (tabular) data, gradient boosting methods such as XGBoost and LightGBM are often the go-to choice.
For text data, models such as naive Bayes and, more recently, transformer-based architectures are commonly used.
For image data, convolutional neural networks (CNNs) and vision transformers usually give the best results.
Start with a simple model as a baseline to quickly assess the problem’s difficulty and identify any issues in the data:
For regression tasks, you might start with linear regression (Chapter 4).
For classification tasks, logistic regression (Chapter 5) or a simple decision tree (Chapter 8) can be a good starting point.
If the baseline underfits (performs poorly on the training set), consider switching to more complex algorithms. These algorithms typically require more data and computation.
Be cautious of overfitting as model complexity increases. Techniques such as regularization or early stopping can help mitigate this.
Model selection is typically iterative: train a model, evaluate it on the validation set (or via cross-validation), and revise based on the results or new insights about the data.
Model Training:
Train the selected model on the training set.
Model Evaluation and Hyperparameter Tuning:
Evaluate the trained model on the validation set (if available) or using cross-validation.
Tune the hyperparameters using techniques such as grid search, random search, or Bayesian optimization (see Section 3.8).
Check for overfitting: if the model performs well on the training set but poorly on the validation set, consider applying regularization, choosing a simpler model, or adjusting the hyperparameters.
This step is typically iterative: tune the hyperparameters, train a new model, evaluate it on the validation set (or via cross-validation), and revise based on the results.
Final Model Training and Evaluation:
Once the best model and hyperparameters are selected, retrain the model on the full training data (including the validation set).
Evaluate the final model on the test set to obtain an unbiased estimate of its performance. Report the result as is, and avoid further tuning of the model.
Model Deployment:
Deploy the model to a production environment or make it accessible via APIs.
Continuously monitor the model’s performance in production. Due to concept drift— changes in the statistical properties of the data over time—the model may become outdated, requiring retraining or other adjustments.
Implementation of the entire workflow in Python is demonstrated in Section 3.4.
