The basics
Inference is the stage where a trained AI model is actually used: it takes a new input and produces a prediction or an answer.
If training is studying for an exam, inference is sitting the exam. The model's weights are frozen. You give it something new, such as a photo, a sentence, or a sensor reading, and it runs the input through its layers and gives back an output. No learning happens during inference.
Inference is usually fast and cheap compared with training, but at scale it adds up. Every chat message, every voice command, every recommended video is one inference. Companies spend heavily on chips and data centers just to serve inference to millions of users at the same time.
The word comes from 'inferring,' which means working something out from evidence. The model infers the most likely output from the patterns it learned. When you hear that an AI service is slow or costly per request, that is inference cost.
When you ask a voice assistant for the weather and it replies in under a second, that instant reply is inference; the training happened months earlier.