Training Your First Model with SageMaker Built-in Algorithms

This is the second post in our Amazon SageMaker series, building on notebooks, training jobs, and endpoints.

For genuinely standard, well-understood problems — binary classification, regression, ranking — writing custom training code from scratch is often more work than the problem actually requires. SageMaker's built-in algorithms are pre-implemented, optimized, well-tested implementations of common algorithms, requiring only your data and a set of hyperparameters — the same "reach for a well-maintained standard solution before writing something custom" instinct covered for Terraform's Registry and Ansible Galaxy earlier in this blog, applied here to ML algorithms specifically.

Preparing data in the format SageMaker expects

import pandas as pd
from sklearn.model_selection import train_test_split
 
df = pd.read_csv("customer_churn.csv")
 
# XGBoost's built-in algorithm expects the target column first, no header row
train_df, val_df = train_test_split(df, test_size=0.2, random_state=42)
train_df.to_csv("train.csv", index=False, header=False)
val_df.to_csv("validation.csv", index=False, header=False)

session.upload_data("train.csv", bucket=bucket, key_prefix="training-data")
session.upload_data("validation.csv", bucket=bucket, key_prefix="training-data")

Each built-in algorithm expects data in a specific format — here, XGBoost's expectation of the target column first with no header row. Getting this wrong doesn't usually produce an obviously-labeled error; it produces a model that trains "successfully" while learning something subtly different from what you intended, because it's reading the wrong column as the target. Checking the specific algorithm's documented data format requirement before training, not after a confusing result, is worth the extra five minutes.

Configuring and running training

from sagemaker.estimator import Estimator
from sagemaker.inputs import TrainingInput
 
xgb_image = sagemaker.image_uris.retrieve("xgboost", session.boto_region_name, "1.7-1")
 
estimator = Estimator(
    image_uri=xgb_image,
    role=role,
    instance_count=1,
    instance_type="ml.m5.xlarge",
    output_path=f"s3://{bucket}/model-output",
)
 
estimator.set_hyperparameters(
    objective="binary:logistic",
    num_round=100,
    max_depth=5,
    eta=0.2,
    eval_metric="auc",
)
 
train_input = TrainingInput(f"s3://{bucket}/training-data/train.csv", content_type="text/csv")
val_input = TrainingInput(f"s3://{bucket}/training-data/validation.csv", content_type="text/csv")
 
estimator.fit({"train": train_input, "validation": val_input})

sagemaker.image_uris.retrieve() resolves the correct pre-built container image for the algorithm and region — you're not writing training code at all here, only configuring hyperparameters and pointing the job at your prepared data. The container image itself contains the actual algorithm implementation.

Hyperparameter tuning: automating the search, not just the training

from sagemaker.tuner import HyperparameterTuner, ContinuousParameter, IntegerParameter
 
tuner = HyperparameterTuner(
    estimator=estimator,
    objective_metric_name="validation:auc",
    hyperparameter_ranges={
        "eta": ContinuousParameter(0.01, 0.3),
        "max_depth": IntegerParameter(3, 10),
        "num_round": IntegerParameter(50, 200),
    },
    max_jobs=20,
    max_parallel_jobs=4,
)
 
tuner.fit({"train": train_input, "validation": val_input})

Rather than manually guessing hyperparameter values and re-running training repeatedly by hand, a tuning job runs many training jobs automatically — 20 here, 4 running in parallel at a time — searching the defined ranges for the combination that best optimizes the chosen metric (validation AUC, here). This is a real, non-trivial compute cost (20 training jobs, not one), worth weighing against how much a better hyperparameter combination is actually likely to matter for a given problem, rather than defaulting to an exhaustive search for every model regardless of expected payoff.

Monitoring training progress

import boto3
 
client = boto3.client("sagemaker")
response = client.describe_training_job(TrainingJobName=estimator.latest_training_job.name)
print(response["TrainingJobStatus"], response.get("FinalMetricDataList"))

Training jobs run asynchronously — fit() blocks by default waiting for completion, but the job itself, and its logs (available directly in CloudWatch Logs), can be inspected independently, useful for a long-running job you don't want to keep a notebook connection open and blocking for the entire duration.

What to actually remember from this post

Built-in algorithms are pre-implemented and well-tested — the right starting point for standard problems, the same "use a maintained standard before writing custom" instinct as reaching for the Terraform Registry or Ansible Galaxy.
Each algorithm expects a specific data format — verify it before training, not after a confusing, silently-wrong result.
image_uris.retrieve() resolves the correct pre-built container for a given algorithm and region — you configure hyperparameters, you don't write the algorithm's implementation.
Hyperparameter tuning automates the search across many training jobs — a real compute cost, worth weighing against the problem's actual sensitivity to hyperparameter choice.