Back to Curriculum
Advanced•Modern AI
Agentic AI
Reasoning loops, tools, and multi-agent systems.
Interactive Playground
Initializing Interactive Playground...
Research-Level Deep Dive & Equations
In classical machine learning, LLMs are used as static sequence-to-sequence mapping functions. Agentic AI transforms the model into an active controller. This behavior can be mathematically framed as a **Markov Decision Process (MDP)**, denoted by the tuple .
Let denote the state space, where a state represents the complete current context of the agent at execution step . This context includes the user prompt, the chat history, execution logs, and observations returned by tools. Let denote the action space, which is the union of textual generation actions (thought generation) and tool invocation calls: .
The transition probability matrix dictates the probability of transitioning to state given action . When the agent executes a reasoning thought (), the transition is deterministic and represents appending the generated tokens to the internal context window. When the agent invokes a tool (), the transition is stochastic, determined by the external environment's response (e.g., database output, API response latency, or execution errors).
The agent's objective is to construct an optimal policy that maximizes the expected cumulative discounted reward:
Here, represents the alignment reward evaluating the correctness of the execution trajectory, and represents the discount factor prioritizing shorter reasoning paths.
Test Your Knowledge
Check whether you have mastered this concept with a quick quiz.
Was this lesson helpful?
Your feedback helps us continuously improve the curriculum and interactive visualizations.