Data storage, ingestion and Feature Store
Most ML engineering time goes into data. The exam tests whether you can put data in the right place, move it cheaply and reuse features consistently.
Choosing storage
| Service | Best for | Notes |
|---|---|---|
| Amazon S3 | Default data lake, training input | Cheap, durable. Use File mode, Fast File mode or Pipe mode |
| Amazon EFS | Shared POSIX file system, many readers and writers | File locking, reachable from on-premises over Direct Connect or VPN |
| Amazon FSx for Lustre | Fastest training I/O | Can be linked to S3, good for repeated epochs |
| Amazon EBS | Single-instance block storage | Not shared across instances |
SageMaker AI training reads S3 by default. For large datasets, Pipe mode or Fast File mode streams data instead of copying everything before training starts, which shortens startup.
Ingestion and ETL
- Amazon Data Firehose (old name: Kinesis Data Firehose) for streaming into S3 with Parquet conversion.
- Amazon Kinesis Data Streams when consumers need replay or custom processing.
- AWS Glue for serverless ETL (Spark jobs) and the Data Catalog. A crawler discovers schemas. Glue DataBrew offers visual preparation.
- Amazon Athena for SQL on S3 using the Glue Data Catalog.
- Amazon EMR when you need your own Spark or Hadoop cluster and fine control.
Columnar formats such as Parquet reduce scan cost and speed up queries. For built-in algorithms, RecordIO-protobuf is the efficient format, while many algorithms accept CSV.
SageMaker Data Wrangler
Data Wrangler is the low-code tool for import, profiling, transformation and export. It offers built-in transforms (encoding, scaling, imputing), a Data Quality and Insights report, and exports to a SageMaker AI Pipelines step, a processing job, a notebook or Feature Store.
SageMaker Feature Store
A feature group has a record identifier and an event time.
- Online store: low-latency single-record reads for real-time inference.
- Offline store: S3-based history, queried with Athena for training and point-in-time correct datasets.
It prevents training-serving skew, because both phases use the same feature definitions.
SageMaker Ground Truth
Use Ground Truth for labeling, with private workforces, vendors or Amazon Mechanical Turk. Automated data labeling reduces cost by labeling easy items with a model.
Exam traps
- EFS is the answer when on-premises servers and notebooks need a shared file system with locking. S3 with Mountpoint has no file locking.
- FSx for Lustre is a performance layer, not durable primary storage. S3 stays the source of truth.
- Online store is for inference, offline store is for training. Do not swap them.
- Glue crawlers find schemas and do not transform data. Glue jobs transform.
- Firehose is near real time, not true real time, because it buffers.
Practice questions
Choose an answer, then review the explanation to check your understanding.
1. Training data on Amazon S3 must be read at very high throughput by many GPU instances, and the same files are loaded repeatedly across jobs. Which storage option fits best?
Show answer
Answer: B.FSx for Lustre gives sub-millisecond latency and high throughput, and can be linked to an S3 bucket so training jobs read S3 data as a file system.
2. Two teams want to reuse the same customer features for training and for low-latency predictions, without computing them twice. What should they use?
Show answer
Answer: A.The offline store (S3) serves training and the online store serves low-latency lookups, with one definition of each feature.
3. A data analyst wants a visual, low-code way to import data, profile it, apply transformations and export a reusable flow. Which tool is intended for this?
Show answer
Answer: A.Data Wrangler (in SageMaker Canvas and Studio) imports, analyzes and transforms data, and exports to Pipelines, Feature Store or a processing job.
4. Streaming clickstream events must be landed in S3 in near real time with optional format conversion to Parquet. Which service is the simplest fit?
Show answer
Answer: A.Amazon Data Firehose (formerly Kinesis Data Firehose) delivers streams to S3 and can convert JSON to Parquet or ORC.