While traditional AI can sometimes be perceived as a ‘black box’, metacognitive AI enhances trust and transparency by making the decision-making process in black-box AI more understandable to users. This is achieved through the function $g(f(x),\theta )$ or the function $|gf$. The function $g(f(x),\theta )$ represents the process of generating explanations based on both the input $x$ and the parameters $\theta$ of the model $f$, while the function $|gf$ represents the function $f$ with a series of $g$. This function allows metacognitive AI to explain its decisions in terms of both the input data and its internal parameters, catering to different user expectations and motivations for seeking explanations.

On one hand, the nature of the explanation can vary significantly depending on whether it’s intended for an expert with technical knowledge or a layperson. If they are looking to understand why certain outputs come from a global perspective, then the focus is to have $g(\theta )$ to make the $\theta$ transparent; if users are looking to understand certain cases, then the focus is to have $g(f(x))$ to understand a certain prediction $f(x)$ on $x$. On the other hand, the purpose behind an explanation necessitates a different approach to how explanations are formulated and presented: is the explanation for enhancing the performance of the system, reducing bias, increasing fairness, or simply deriving a clearer understanding of the AI’s decision-making process? Enhancing the performance of the system through transparency could involve using the explanations to correct predictions or induce actions $|gf$ where the understanding outputted from $g$ is then processed by the AI system to better learn $f$.  [] argued building cognitive models of both the AI and the human user that could be introspected upon to adapt explanations according to the discrepancy between the two models, e.g., when the AI decision does not conform to the human model’s expectations. [] applied explainable AI framework to a real-world clinical machine learning (ML) use case, i.e., an explainable diagnostic tool for intensive care phenotyping. Co-designing with 14 clinicians, they provided five explanation strategies to mitigate decision biases and moderate trust. They implemented an early decision aid system to diagnose patients in an Intensive Care Unit (ICU) and found that users employed a diverse range of explainable AI facilities to reason.

Traditional AI systems $f$ often rely on predefined algorithms and data sets for reasoning or decision-making, which can limit their effectiveness in dynamic or unfamiliar scenarios. In contrast, metacognitive AI incorporates self-reflection and self-awareness into its logic, represented by $f(x;g(\theta ))$. This indicates how the AI’s self-reflection (through $g$) informs its decision-making process (through $f$), enhancing its reasoning capabilities. For instance, a metacognitive AI in healthcare could use this approach to evaluate and refine its diagnostic criteria over time, learning to differentiate between complex cases and refining its diagnostic criteria based on outcomes. This leads to decisions that are not only based on data but also enriched by the AI’s growing experiential knowledge, resulting in more accurate and reliable outcomes.

In [], the authors showed that model-based reflection may guide reinforcement learning with two benefits: The first is a reduction in learning time as compared to an agent that learns the task via pure reinforcement learning. Secondly, the reflection-guided RL agent shows benefits over the pure model-based reflection agent, matching the performance of that agent in the metrics measured in addition to converging to a solution in fewer trials. In addition, the augmented agent eliminates the need for an explicit adaptation library such as is used in the pure-model-based agent and thus reduces the knowledge engineering burden on the designer significantly. In [], a novel technique called Hindsight Experience Replay was introduced, whose intuition is to re-examine the trajectories with a different goal — while a trajectory may not help learn how to achieve the desired goal, it tells us something about how to achieve the state in the actual trajectory. They demonstrated this approach on the task of manipulating objects with a robotic arm on three different tasks: pushing, sliding, and pick-and-place, while the vanilla RL algorithm fails to solve these tasks.

Adaptation in metacognitive AI encapsulates the system’s ability to detect and correct errors of internal conditions and to flexibly adapt its behavior and strategies. This is represented by $f\prime (x;g(f(x),\theta ))$, where $f\prime$ is the adapted model based on the metacognitive assessment $g$ of the original model’s output $f(x)$ and parameters $\theta$. This notation reflects how metacognitive AI adapts by reassessing its outputs and parameters, allowing for more effective decision-making in the face of uncertainty and changing environments []. Additional adaptations could also be implemented with $:g(x)?f(x)h(x)$, where the metacognitive process $g$ decides whether to use the main function $f$ or an alternative function $h$ based on its analysis of the input $x$. This could model AI systems that choose different processing paths based on metacognitive assessment without modifying $f$. Such systems can adapt to new environments and tasks by understanding their learning process and limitations.  [] showed that uncertainty-informed decision referral can improve diagnostic performance. More recently, another metacognitive approach allowing for adaptability known as error detection and correction rules (EDCR) was introduced []. In this framework, function $g$ results in a set of learned rules that characterize the failure modes of $f$ and how to correct on those failure modes while $f\prime$ is an inference process conducted using these rules to erase or change the results of the underlying model $f$. In [] the authors applied this technique to the classification of geospatial movement trajectories and examined performance improvement on $f$, where the current state-of-the-art is neural architecture known as LRCN []. EDCR was able to both improve over the state-of-the-art as well as exhibit the ability to improve performance when exposed to out-of-distribution data.

Perception refers to the ability to interpret sensory information to understand the environment. Perception in metacognitive AI involves the system’s ability to interpret and understand sensory information, such as visual and auditory data, in a context-aware manner, represented by $f(g(x),x)$, where context is a metacognitive assessment of the AI’s own capacity rather than an external context. Here, $f$ represents the primary perceptual processing function, interpreting sensory data like visuals or audio, while the metacognitive function $g$ specifically evaluates the accuracy and limitations of the AI’s own sensory processing. The primary perceptual function $f$ then uses both the original sensory input $x$ and the metacognitive assessment $g(x)$ to refine its interpretation. This dual-input model allows the AI to recognize and compensate for any inherent biases or weaknesses in its perception. This could include AI in autonomous vehicles that must interpret complex visual environments, in medical imaging distinguishing subtle diagnostic details, or in environmental monitoring systems that detect and analyze changes through sensory data.