AWS AI/ML Certifications (SageMaker, Bedrock)

AIF-C01 is a foundational credential (GA October 2024; 90 min, 65 questions) aimed at people who use but do not build AI/ML on AWS — business analysts, product/project managers, IT and sales staff — testing AI/ML and generative-AI concepts, AWS AI services, and responsible AI; no coding. MLA-C01 is an associate-level, role-based cert (130 min, 65 questions) for ML/MLOps engineers with ~1 year of SageMaker experience; it tests building, operationalizing, deploying, monitoring, and securing ML pipelines hands-on. As of 2026 the older ML Specialty (MLS-C01) is retiring (last exam March 31, 2026), so MLA-C01 is the current engineer-track exam.

Amazon Textract. It is purpose-built for document text extraction including handwriting, and its FORMS and TABLES features return key-value pairs and structured table cells from scanned documents — exactly the requirement. Rekognition does image/video analysis (objects, faces, moderation, some scene text) but not document forms/tables. Comprehend does NLP on already-extracted text (entities, sentiment, PII, topics) and cannot read a scanned PDF. A common pattern chains Textract (extract) then Comprehend (analyze), e.g., Comprehend Medical for clinical entities.

It is a centralized, governed repository for curated ML features, eliminating duplicated feature engineering and reducing training/serving skew. The online store is low-latency, backed for real-time inference lookups by record key; the offline store is S3-backed, append-only history used for batch training, backfills, and point-in-time-correct joins. Writing features once and reading them from both stores guarantees the same definitions and values are used at training and inference, the core defense against the offline/online skew that silently degrades production models.

Real-time endpoints: persistent instances for low, steady latency on small payloads (live recommendation API). Serverless inference: auto-scales to zero, pay-per-use, tolerates cold starts — good for spiky/intermittent low-volume traffic (an internal tool used a few times an hour). Asynchronous inference: queues requests, handles large payloads (up to ~1 GB) and long processing, can scale to zero — good for large documents or long-running models where seconds-to-minutes latency is acceptable. Batch transform: no endpoint, processes a whole S3 dataset in one job — good for periodic offline scoring of millions of records, e.g., nightly churn predictions.

It is Bedrock's managed Retrieval-Augmented Generation layer: it ingests documents from S3, chunks them, generates embeddings via a chosen embeddings model (e.g., Titan/Cohere), stores vectors in a vector store (OpenSearch Serverless, Aurora pgvector, Pinecone, Redis Enterprise, etc.), and at query time retrieves relevant chunks to augment the prompt to a foundation model — optionally returning source citations. It removes the need to hand-build the embed/index/retrieve pipeline. You still choose chunking strategy, embeddings model, vector store, and generation FM, and can use RetrieveAndGenerate or just Retrieve for custom orchestration.

Guardrails set policy independent of the FM: denied topics, content filters (hate, violence, sexual, insults, misconduct, prompt-attack) at configurable strengths, word/profanity blocklists, sensitive-information filters that block or mask PII/regex patterns, and contextual grounding/relevance checks to curb hallucination and off-topic answers. They apply to both input prompts and model outputs and work across models. Key limitation: they are probabilistic safety filters, not a guarantee — they reduce but do not eliminate jailbreaks or leakage, so you still need least-privilege IAM, input sanitization, output validation, and human review for high-stakes flows. Never treat a guardrail as your only control.

Beyond multiple-choice (one correct of four) and multiple-response (two or more correct of five-plus), AWS added ordering, matching, and case-study types. Ordering asks you to arrange 3-5 steps in the correct sequence; matching asks you to pair items between two lists; case studies present a scenario followed by several questions. The trap: these are scored all-or-nothing — you must order every step or match every pair correctly to earn the point; partial credit is zero, and unanswered questions count as incorrect with no guessing penalty, so never leave one blank. MLA-C01 has 65 questions (50 scored + 15 unscored), scaled 100-1000, passing at 720, with a compensatory model so you need only pass overall, not each domain.

File downloads the whole dataset to the instance EBS volume before training starts: high startup cost, needs disk >= dataset, but fastest steady-state and simplest — best for small/medium (<100 GB) sets read many epochs. Pipe streams ordered bytes via a FIFO with a multi-threaded S3 prefetcher: near-zero startup, tiny disk, highest sequential throughput, but no random access and best with recordIO — best for multi-TB sequential. FastFile mounts S3 as a FUSE filesystem and lazy-loads on first read: fast startup, low disk, allows random file access. Rule: <100 GB sequential → File; ~100 GB needing random/file access → FastFile; multi-TB streaming recordIO → Pipe.

Give the job a dedicated SageMaker execution role (trusted by sagemaker.amazonaws.com), not user credentials, scoped to only the specific S3 prefixes/buckets it needs (read input, write artifacts) and the KMS key — least privilege. Encrypt at rest with SSE-KMS and in transit with TLS; optionally enable inter-container traffic encryption. To avoid NAT/internet egress cost and exposure, run the job in a VPC with an S3 gateway endpoint (free, keeps S3 traffic on the AWS backbone) plus interface endpoints for the SageMaker APIs, and set the bucket policy to require the endpoint. Use bucket policies and the role policy together as defense in depth.

Clarify focuses on bias and explainability: pre-training it computes dataset bias metrics (e.g., class imbalance, DPL); post-training it measures model bias (disparate predictions across groups) and produces SHAP feature-attribution explanations. It runs as a processing job during development and can feed bias-drift jobs. Model Monitor watches a deployed endpoint in production for four problem types: data-quality drift (schema/stats vs a baseline), model-quality drift (live accuracy vs labels), bias drift, and feature-attribution drift — capturing inference data and alerting via CloudWatch. Clarify answers 'is the model fair/explainable now'; Model Monitor answers 'has production drifted from the baseline'.

Use an Agent when a request requires taking actions and multi-step reasoning, not just retrieving text — e.g., 'find my order, check stock, and start a return'. The agent uses the FM to plan, invokes action groups (Lambda functions or OpenAPI-defined APIs) and/or Knowledge Bases, observes results, and iterates until done (ReAct-style orchestration), optionally with session memory. Risk concentrates in excessive agency (OWASP LLM06): action groups are real privileged calls, so each Lambda/API must be least-privilege and idempotent, model output must be validated before triggering side effects, and you should avoid the lethal trifecta — untrusted input plus private data plus the ability to act externally — in one agent.

A typical pipeline chains: Processing step (data prep/feature engineering, optionally Clarify) -> Training step (estimator) -> Evaluation Processing step producing a metrics file -> Condition step gating on a threshold (e.g., AUC >= 0.8) -> RegisterModel step recording the model in the Model Registry with PendingManualApproval status and lineage. Approval (manual or automated) then triggers deployment. This beats a notebook because it is a reproducible, versioned, parameterized DAG with automatic lineage and CI/CD integration (EventBridge/CodePipeline triggers), giving auditable, repeatable retraining instead of imperative, untracked notebook runs.

Use a Bedrock Knowledge Base over the PDFs in S3 (managed chunk/embed/retrieve into a vector store) fronted by a chosen FM via RetrieveAndGenerate, with a Bedrock Guardrail attached for denied topics, PII masking, prompt-attack filtering, and contextual grounding to limit hallucination. It is fully managed — no SageMaker training, no embedding pipeline to build — which fits a team without ML engineers. Rationale: the requirement is retrieval-grounded generation plus safety, not custom model training; SageMaker would be unnecessary effort, and a bare FM call would lack grounding and the PII/injection controls.

Built-in XGBoost historically did NOT support Pipe mode for arbitrary CSV/libsvm the way the protobuf-recordIO algorithms (Linear Learner, PCA, K-Means, Factorization Machines) do; XGBoost loads the full DMatrix into memory, so streaming via Pipe doesn't help and isn't the natural path. The right fix is FastFile mode: it presents S3 objects as a POSIX filesystem and streams on first access (no full pre-download, no EBS-sized local copy), giving fast startup for large datasets while keeping File-like access semantics. Alternatively scale instances or use distributed XGBoost across nodes.

Both stem from missing checkpointing. Spot interruptions reclaim the instance; without checkpoints SageMaker restarts the job from epoch 0 on the next acquired capacity. And built-in/marketplace algorithms (or containers) that don't checkpoint are capped at MaxWaitTimeInSeconds=3600s, which is why they die at 60 minutes. Fix: implement checkpointing — write to the local /opt/ml/checkpoints dir; SageMaker auto-syncs that path to/from the configured S3 CheckpointConfig URI, restoring on restart. Also ensure MaxWaitTimeInSeconds > MaxRuntimeInSeconds (wait must cover runtime plus interruption gaps). Savings = $(1 - BillableTime/TrainingTime)\times100$, up to ~90%.

Data-quality drift compares each input feature's marginal distribution to a baseline; it fires only when an input's distribution shifts. But a model can degrade while inputs look stationary if the feature-to-target relationship changes (concept drift) or the model starts relying on different features. Feature-attribution drift baselines the ranked global feature importances (an NDCG-style comparison of attribution rankings) and alerts when the ordering shifts. Since attributions reflect how the model uses inputs, a change there signals the decision logic has moved even when every input marginal is unchanged — capturing concept drift that input-only monitoring is blind to.

Order by cost/complexity. Try prompt engineering + RAG first — cheapest, no training, updates instantly as documents change, ideal when the need is fresh/proprietary knowledge or formatting; it adds retrieval latency. Fine-tune (Bedrock custom models or JumpStart) only when you must change behavior/style/domain skill prompting can't reach, you have enough high-quality labeled examples, and you accept training cost, a new model version to host (provisioned throughput on Bedrock), and a retrain cycle when data changes. Choose JumpStart/SageMaker for full control, custom architectures, and your own instances/VPC; choose Bedrock customization for managed, serverless adaptation of supported FMs. RAG and fine-tuning compose, not compete.

Autopilot optimizes an objective metric on a provided split using automated feature engineering and a candidate search; it cannot know about leakage, temporal ordering, business cost asymmetry, or train/serve skew you did not encode. Common failures: target leakage from features unavailable at inference, or a random split on time-series data inflating offline metrics that collapse in production. Safeguards: use point-in-time-correct features (Feature Store offline store), time-based splits, audit the generated feature pipeline and candidate notebook, pick the objective metric to match business cost (e.g., recall-weighted), validate with Clarify for bias, and deploy behind Model Monitor with a shadow/canary stage before full traffic.

(1) Auto scaling on InvocationsPerInstance/latency tracks load and scales in during troughs — always helps for variable traffic. (2) Multi-model endpoints host many models on shared instances, loading on demand — great for many low-traffic models, but adds cold-load latency and hurts if every model is hot. (3) Right-size the instance and consider AWS Inferentia (inf2) for transformer inference — far better price/performance than general GPUs, but requires Neuron compilation and supported ops. (4) Compile/optimize with SageMaker Neo or TensorRT to cut latency and use smaller/cheaper instances. (5) Consider serverless or async only if latency/payload allow. The principal move: match traffic shape to endpoint type, then compile and right-size before scaling out.

The 30B model is memory-bound, not throughput-bound: it must be sharded across GPUs — model parallelism. SMP v2 wraps PyTorch FSDP (plus tensor/pipeline parallel and activation checkpointing) over EFA-connected nodes. Memory math: with mixed-precision Adam, ~20 bytes/param (2 BF16 param + 2 BF16 grad + 8 FP32 optimizer state + 4 FP32 param copy + 4 FP32 grad copy), so 30B ≈ 600 GB of state — far beyond one 80 GB GPU, forcing sharding. The CNN fits in memory, so it only needs more throughput: data parallelism (SMDDP), replicating the model and doing optimized AllReduce over AWS topology/EFA for near-linear scaling. Wrong choice = OOM or wasted GPUs.