This is the fifth and final post in our Amazon SageMaker series, building on everything from notebooks and endpoints through training, deployment, and Pipelines.
The two most common, most expensive SageMaker mistakes are rarely a single bad decision — they're a forgotten always-on resource and an oversized instance choice, both accruing cost quietly, for weeks or months, before anyone happens to check a bill closely enough to notice. This closing post is a practical rundown of where SageMaker cost actually accumulates, and the concrete, specific levers for controlling each source.
The endpoint you forgot about
We flagged this directly in the deployment post earlier in this series: a real-time endpoint stays running, and billing, continuously until explicitly deleted. This is, in practice, the single most common source of real, avoidable SageMaker spend.
import boto3
client = boto3.client("sagemaker")
endpoints = client.list_endpoints()
for ep in endpoints["Endpoints"]:
print(ep["EndpointName"], ep["CreationTime"], ep["EndpointStatus"])Running this kind of audit on a schedule — not just once, reactively, after noticing an unexpectedly large bill — is the actual fix, the same "check this deliberately and regularly, not just when something already looks wrong" instinct behind rebuilding Docker images on a schedule and re-evaluating model choice periodically, both covered earlier in this blog.
Matching instance type to actual workload
estimator = Estimator(
image_uri=xgb_image,
role=role,
instance_count=1,
instance_type="ml.m5.xlarge", # is this actually the right size?
)The exact same instinct behind setting accurate resource requests in Kubernetes, covered earlier in this blog, applies directly here: an oversized training or inference instance wastes money on unused capacity; an undersized one risks failure or unacceptably slow performance. SageMaker's instance-type recommendation tooling, and simply testing a workload against 2-3 candidate instance types before committing to one for ongoing use, both meaningfully beat defaulting to whichever instance type a tutorial happened to use as its example.
Spot instances for training: substantial savings, with a real trade-off
estimator = Estimator(
image_uri=xgb_image,
role=role,
instance_count=1,
instance_type="ml.m5.xlarge",
use_spot_instances=True,
max_wait=3600,
max_run=1800,
checkpoint_s3_uri=f"s3://{bucket}/checkpoints/",
)Spot instances offer meaningfully discounted compute — often a substantial reduction versus on-demand pricing — in exchange for AWS being able to reclaim the instance with short notice if capacity is needed elsewhere. For training jobs specifically (as opposed to always-on inference endpoints, where an unexpected interruption is a much more serious problem), this is often a genuinely favorable trade: checkpoint_s3_uri lets an interrupted training job resume from its last checkpoint rather than restarting entirely from scratch, meaningfully limiting the actual cost of an interruption when one occurs.
Automatic scaling for real-time endpoints
client = boto3.client("application-autoscaling")
client.register_scalable_target(
ServiceNamespace="sagemaker",
ResourceId=f"endpoint/churn-prediction-endpoint/variant/AllTraffic",
ScalableDimension="sagemaker:variant:DesiredInstanceCount",
MinCapacity=1,
MaxCapacity=4,
)
client.put_scaling_policy(
PolicyName="cpu-scaling",
ServiceNamespace="sagemaker",
ResourceId=f"endpoint/churn-prediction-endpoint/variant/AllTraffic",
ScalableDimension="sagemaker:variant:DesiredInstanceCount",
PolicyType="TargetTrackingScaling",
TargetTrackingScalingPolicyConfiguration={
"TargetValue": 70.0,
"PredefinedMetricSpecification": {"PredefinedMetricType": "SageMakerVariantInvocationsPerInstance"},
},
)This is directly analogous to a Kubernetes Horizontal Pod Autoscaler, covered earlier in this blog — an endpoint scales its instance count based on actual traffic, rather than being permanently provisioned for peak load it experiences only occasionally, or permanently under-provisioned for load spikes it can't actually handle. For traffic with genuine daily or seasonal variation, this is frequently a larger cost saving than instance-type selection alone.
Choosing the right deployment shape, revisited
The previous post's framework — real-time for continuous, latency-sensitive traffic; serverless for intermittent, latency-tolerant traffic; asynchronous for genuinely long-running requests; multi-model for many similar, individually lower-traffic models — is itself the single highest-leverage cost decision covered in this entire series. An always-on real-time endpoint provisioned for a model called a handful of times a day is a structural cost mismatch no amount of instance-type tuning fully corrects; the right fix is choosing serverless (or a different shape entirely) for that traffic pattern in the first place, not right-sizing the wrong deployment shape.
What to actually remember from this series
- A forgotten, always-on endpoint is the single most common source of avoidable SageMaker cost — audit running endpoints on a schedule, not just reactively after noticing a large bill.
- Match instance type to actual measured workload needs, the same instinct as setting accurate Kubernetes resource requests — don't default to whatever a tutorial happened to use.
- Spot instances offer substantial training-cost savings, with checkpointing as the concrete mitigation for the interruption risk that trade-off carries.
- Choosing the right deployment shape for actual traffic patterns is a bigger cost lever than instance-level tuning within the wrong shape.
That closes out our Amazon SageMaker series — from notebooks and endpoints through training, deployment, Pipelines, and now cost optimization — and with it, our full 50-post series spanning Terraform, Ansible, Docker, Kubernetes, GitHub Actions, OpenAI, LangChain, LangGraph, Bedrock, and now SageMaker. If your team is building real production infrastructure or AI systems on any of these tools, this is exactly the kind of hands-on work we build our training programs around.