Taming Amazon Sagemaker

I expected SageMaker to be one of the easier AWS services to get started with. I was mostly interested in using it to experiment with machine learning models and see how ML workflows fit into AWS.

Getting to that point was a different story.

The first hurdle was SageMaker Studio. Setting it up turned into a lot more troubleshooting than I expected. I kept running into problems while creating the Studio environment and, especially, while trying to create the SageMaker domain.

At one point, AWS was giving me an error about not having enough domains available to create the Studio environment. That was particularly confusing because I was trying to create my first one. I spent quite a bit of time checking settings, permissions and configuration options before I could even get into the environment I actually wanted to use.

This was probably the most frustrating part of the experience. There are a lot of settings involved in creating a SageMaker domain, and when something fails, it isn't always immediately obvious what needs to be changed.

Once I finally got past the setup issues, SageMaker started making more sense.

The platform brings together many parts of the machine learning workflow: preparing data, developing models, training them, deploying them and monitoring them. Instead of having separate tools for every stage, SageMaker provides an environment where these pieces can work together.

But my biggest takeaway wasn't really about the ML features.

It was about how much AWS infrastructure sits underneath a seemingly simple ML environment.

Something that looks like opening a notebook can involve domains, users, permissions, networking and other AWS resources in the background. If you're new to AWS, that can make the first experience pretty confusing.

I also realised that errors during setup aren't necessarily a sign that you're doing something completely wrong. Sometimes the problem is a quota, a permission, a region-specific limitation or a resource that has to be configured first.

For anyone trying SageMaker for the first time, I'd recommend taking the setup slowly and reading the error messages carefully before changing random settings. It is also worth checking your AWS quotas and IAM permissions early.

SageMaker ended up being a good reminder that learning cloud-based ML isn't just about learning machine learning.

You're also learning how cloud infrastructure works.

And sometimes, before you get to train your first model, you have to spend a while figuring out why AWS won't let you create the thing you're supposed to train it in.