AI Inference
- [Cornell University]
- Overview
AI inference is the process of using a trained model to make predictions on new data. It involves running live data through a trained AI model to make a prediction or solve a task.
During training, an AI model learns patterns and relationships that enable it to generalize on new data. Once an algorithm is in the “inference stage”, it's no longer learning.
AI inference is a vital part of AI and has a vast significance. It can revolutionize the way we approach problem-solving and decision-making. For example, if you're running a business, understanding how inference works can help you make better decisions about how to use AI to improve your products and services.
To deploy machine learning (ML) inference, you need three main components:
- Data sources
- A system to host the ML model
- Data destinations
- AI Inference
Inference is the critical moment for an AI model, a test of its ability to apply information learned during training to predict or solve a task. Can it accurately mark incoming emails as spam, transcribe conversations, or summarize reports?
During inference, the AI model begins processing live data, comparing the user's query to information processed during training and stored in its weights or parameters.
The response the model sends back depends on the task, whether it's identifying spam, converting speech to text, or distilling a long document into its key takeaways. The goal of AI inference is to calculate and output actionable results.
- AI Training vs. Inference
AI training is the process of teaching an AI system to learn from data. During training, a developer feeds the model a curated dataset so it can learn about the type of data it will analyze. The training process is typically computationally intensive.
AI inference is the process of using a trained model to make predictions on new data. During inference, the model goes to work on real-time data. Inference is usually faster and less resource-intensive than training.
Here are some other differences between AI training and inference:
- Iterative learning: Training requires iterative learning, while inference focuses on applying the learned knowledge quickly and efficiently.
- Adjusting parameters: Inference is typically much faster than training, as it does not require the model to adjust its parameters based on new data.
- Retraining: The training and inference phases are an iterative process, and the model may need to be retrained with new data as it becomes available.
The training phase - involves creating a machine learning (ML) model, training it by running the model on labeled data examples, then testing and validating the model by running it on unseen examples. Machine learning inference - involves putting the model to work on live data to produce an actionable output.
There are four types of machine learning algorithms: supervised, semi-supervised, unsupervised and reinforcement.
- Examples of AI Inference
Here are some examples of AI inference:
- Model inference: A ML model that can identify animals in pictures can also identify animals in new pictures.
- Computer vision: Inference is used to recognize objects and scenes in images or videos. For example, convolutional neural networks (CNNs) can identify patterns, shapes, and features in images to infer their content.
- Bayesian network: A Bayesian network can infer the probability of a patient having a disease based on a set of symptoms.
Other examples of AI inference include:
- Affirming the antecedent: A deductive inference rule that states that if A implies B and A is true, then B must also be true. For example, "If it's raining, then the ground is wet" (A implies B), "It's raining" (A is true), therefore "The ground is wet" (B is true).
- Modus ponens: A rule that states that if P and P → Q is true, then Q will be true. For example, "If I am sleepy then I go to bed" ==> P→ Q, "I am sleepy" ==> P.
Other types of inference engines in AI include:
- Rule-based inference engines
- Bayesian inference engines
- Fuzzy logic inference engines
- Neural network inference engines
- Genetic algorithm inference engines
- Decision tree inference engines
