AI Engineer vs ML Engineer: The Job and the Interview

An AI engineer builds products on top of foundation models that someone else trained: calling them through an API, feeding them the right context through retrieval, giving them tools, wrapping them in agents, and measuring the result with evals. An ML engineer builds the models: preparing training data, engineering features, training and tuning, and serving the model in production. The interviews follow the same split. AI engineer loops test whether you can build reliable systems around a model you treat as a component; ML engineer loops test whether you understand how models learn and how to train and ship one.

Job titles are inconsistent across teams, so read the job description before you prepare. The rest of this post describes the two roles as they are most commonly defined.

The day-to-day work

An AI engineer’s week might include writing the prompt and output schema for an extraction feature, deciding how to chunk a document set for retrieval, adding a tool so an assistant can look up order status, building an eval set of 100 labeled cases and running it before every prompt change, reading traces to find why an agent looped, and cutting cost by routing easy requests to a smaller model. The model’s weights are fixed; everything the engineer changes lives around them. Most of the work is software engineering with a probabilistic component in the middle, which is why so much of it is about validation, testing and failure handling.

An ML engineer’s week might include building a data pipeline that produces training examples, defining features, training a model and comparing it against the current one on a held-out set, tuning hyperparameters, investigating why validation loss stopped improving, and putting the model behind a serving endpoint with monitoring for drift. The model is the product of their work. Their tools are datasets, loss functions, training runs and serving infrastructure.

The roles overlap at the edges. An AI engineer may fine-tune a model when behavior (format, style, task patterns) cannot be fixed with prompting, or self-host an open-weight model for privacy or cost. An ML engineer may build an LLM feature. The center of gravity is still different: one starts from a trained model and builds outward; the other produces the model.

The skills

Both roles need solid software engineering: clean code, testing, version control, and comfort with production systems. Past that, the emphasis splits.

AI engineer core skills, in the order worth learning them:

• A working mental model of LLMs: tokens, context windows, why they hallucinate, what pretraining and post-training each contribute.
• Building blocks: calling a model API, prompting, structured outputs, tool calling.
• Retrieval: chunking, embeddings, hybrid search, reranking, and RAG end to end.
• Agents: the loop, stopping conditions, permissions, and defending against prompt injection.
• Production: evaluating AI systems, tracing, cost, latency, caching.

ML engineer core skills:

• ML fundamentals: supervised and unsupervised learning, overfitting, train/validation/test splits, choosing metrics.
• How training works: loss functions and gradient descent and backpropagation.
• Data: pipelines, feature engineering, labeling, data quality and leakage.
• Training infrastructure and frameworks, experiment tracking, distributed training.
• Serving: model packaging, batching, GPU memory, monitoring for drift.

Side by side

How the interviews differ

Both loops usually include a coding round and a behavioral round. The technical rounds diverge.

AI engineer interviews ask conceptual questions about LLMs (what a token is, why long context does not guarantee recall, how RAG differs from fine-tuning), then move to building questions: “How do you get reliable structured output?”, “How would you evaluate this RAG system?”, “When would you use an agent instead of a fixed workflow?” The system design round asks you to design an LLM-backed product, and you are graded on retrieval design, evaluation plan, failure handling, security and cost. A project deep-dive asks how you measured quality on something you built. Practice sets: LLM interview questions, AI engineer interview questions, RAG interview questions, agentic AI interview questions, and the AI system design interview.

ML engineer interviews ask ML fundamentals (bias and variance, regularization, picking a metric for an imbalanced dataset, how gradient descent works), sometimes with a whiteboard derivation or a coding exercise that implements part of a training loop. The ML system design round is usually a classic modeling problem such as ranking, recommendation or fraud detection, graded on problem framing, training data, features, offline and online metrics, and serving. A project deep-dive asks about a model you trained and how you validated it.

The two loops overlap at the edges. An AI engineer loop may include a few training questions (how a model is pretrained, what preference tuning does), and an ML engineer loop may include an LLM question or two. Prepare for the role in the job title, and keep a short answer ready for the other side.

Which one fits you

• You are a software engineer who likes shipping product features: AI engineer is the shorter path. Your existing skills (APIs, testing, distributed systems) carry over directly, and the new material is learnable in weeks to months. The roadmap post lays out the order.
• You enjoy statistics, experiments and working with data: ML engineer fits better, and expect to invest more time in math and training.
• You are a new grad: pick based on which kind of problem you would rather debug at 5 p.m.: “the retrieval returned the wrong passage” or “validation loss went up.”

When you have a role in mind, start with the free 50 LLM interview questions and the two-week preparation plan.