Author – Zeynep Kabadere
When the Terminator films first depicted Skynet becoming self-aware, the moment was presented as a turning point not only for humanity, but for the idea of intelligence itself. In the film, this “self-awareness” meant more than simply processing information or adapting to input. It signified the recognition of one’s own existence, the ability to formulate independent goals and the emergence of a self-preservative instinct. In other terms, this corresponds to an awareness of awareness that allows an entity to treat its own cognitive processes as objects of reflection. Skynet did not merely perceive the world; it perceived itself perceiving the world.
What kind of mechanism could make a system aware of itself? Today, self-awareness has become a serious research topic in many areas of computing, including artificial intelligence, machine learning, autonomous computing and self-adaptive systems. Each research community has defined self-awareness according to its own theoretical or technical priorities. Upon closer examination, it becomes clear that the definitions of self-awareness in computing literature actually refer to two distinct capacities. Many computational frameworks describe “self-aware systems” not only as systems that represent or have access to their own internal states and identify those states as belonging to themselves, but also as systems capable of evaluating and regulating their own reasoning processes (Kounev et al., 2017). This approach treats the representation expressing the state of awareness and the regulatory process governing this awareness together, without clearly distinguishing between them. However, when focusing on the regulation process, another structure called meta-reasoning becomes apparent.
The process of meta-reasoning is defined as an agent’s capacity to monitor, evaluate and act upon their own reasoning processes. To understand meta-reasoning, it is helpful to first consider what kind of cognitive ability reasoning represents. Reasoning is the cognitive process that allows us to connect the simplest pieces of information we possess and create new information as a result of these connections. It is relatively easy to see how reasoning works in classical logic. To illustrate this, let’s assume we have two different premises. Let’s say:
Premise A: “The author of this text is a human being.”
Premise B: “All humans are mortal.”
What happens if I want to combine these two premises? Can these two premises help me produce new information that I did not know before?
For example, a conclusion could be:
“If the author of this text is a human being and all humans are mortal, then the author of this text is also mortal.”
This is basic reasoning. I have connected the two different premises I possess, and this connection has produced new premises. Then, what is meta-reasoning? Now imagine that, at some point in the reasoning process, I realise that one of my premises is incorrect. Let’s say I discover that premise A—“The author of this text is a human being”—is actually false. At this moment, I step back from my reasoning and begin to examine the steps that led me to my conclusion. I follow the chain of reasoning I performed, locate exactly where the incorrect premise is, and replace it with the correct one. In this sense, meta-reasoning is the ability to observe, evaluate and intervene in one’s own reasoning process when necessary.
At the same time, both reasoning and meta-reasoning produce action. Acquiring information, combining it, generating new knowledge and acting on this knowledge require application; that is, the process involves both action and production. On the contrary, self-awareness is more of a state than a process. It is a state of “knowing”, referring to a condition of awareness rather than one of change or action (Gertler, 2010). In this sense, self-awareness refers to a state of awareness regarding mental states rather than a process performed on those states. Knowing what one thinks, feels or which cognitive state, points to an epistemic position. In this respect, self-awareness has a knowing-state rather than a dynamic structure involving change or action. What is at stake here is not the production or organisation of mental content itself, but the fact that this content is accessible and recognisable to the agent. Therefore, while one is more stable and state-based, the other is a cognitive activity in motion. In fact, the difference between them begins at this most fundamental level. Therefore, meta-reasoning refers to the individual’s ability to monitor, evaluate and intervene in their own reasoning processes when necessary. Self-awareness, in its most basic sense, is the knowledge of one’s own internal states and identify those states as belonging to themselves. (de Vignemont & Fourneret, 2004, Kounev et al., 2017).* I argue that the process of meta-reasoning is possible not only through the objective observation of internal states, but also through knowing “to whom” these processes belong.
However, when examining approaches in computational fields that explicitly or implicitly employ the concept of “self-awareness”, it is frequently observed that meta-reasoning mechanisms are also addressed together under this heading. This practice results in the grouping of two mechanisms with conceptually distinct functions under a single umbrella concept. While it is certainly possible to use self-awareness in such an inclusive manner, this approach also carries the risk of blurring important conceptual distinctions. Indeed, when these concepts are not clearly distinguished, references to “self-aware artificial intelligence” in literature often denote systems that exhibit little in the way of self-awareness but rather display meta-reasoning capabilities. As a result, the meaning of self-awareness becomes reduced to higher-level operations performed in reasoning processes. This conceptual shift not only obscures which capacities should genuinely fall within the scope of self-awareness at the theoretical level, but also weakens the conceptual foundations of empirical studies aimed at evaluating the cognitive capacities of artificial intelligence systems.
In the early 2000s, self-aware computing had not yet been defined as an explicit and commonly accepted concept within computing literature (Kounev et al., 2017). Instead, the capacity of systems to monitor and adapt their own states was addressed under self-attributes (self-monitoring, self-configuration, self-healing, self-optimisation). However, autonomic computing was among the first computational approaches to treat self-awareness as a systematic design principle. For this reason, I consider it useful to begin the analysis with this line of research (Kephart & Chess, 2003).
Autonomic computing primarily defines self-awareness as operational self-management; that is, a system’s ability to monitor and maintain its own internal state without human intervention (Kephart & Chess, 2003). A close everyday analogy is a self-regulating thermostat: it monitors the environmental temperature and automatically adjusts heating or cooling when the temperature deviates from a target value. Similarly, many digital systems today monitor their own performance and automatically correct problems when they arise. For example, in a cloud-based digital service, if an online application begins to slow down due to heavy usage, the system detects this condition and automatically activates additional resources in the background to ensure service continuity. This approach assumes the system carries an internal model of its own performance and operation. In this respect, it captures a minimal functional analogue of the knowing state defended in this study: the system maintains and uses information about its own operational state and functioning. However, autonomic computing does not leave self-awareness at this representational level. The system not only monitors its own state but also decides which self-management strategy to apply (Kounev et al., 2017). At this point, the system evaluates alternative action policies, anticipates their outcomes and modifies its strategies based on performance. These processes are clear examples of meta-reasoning since the system evaluates the effectiveness of its own self-management mechanisms and reconfigures them accordingly. Despite this, in literature on autonomous computation, these regulatory and reflective processes are generally not referred to as meta-reasoning; instead, they are treated as a natural extension of self-awareness. Consequently, self-awareness here becomes a composite mechanism of change and action that brings together monitoring, evaluation and control, rather than a state of knowing. This situation blurs the conceptual distinction between awareness and awareness-based regulation. Autonomic computing makes the overlap between self-awareness and meta-reasoning visible at the operational level, while a similar overlap also emerges in learning-based approaches. For this reason, I now turn to data-driven adaptive systems, where a similar overlap can be observed.
In the context of data-driven adaptive systems, self-awareness is generally considered to be the capacity to learn and infer from data; it refers to a system’s ability to learn from experience and adapt its behaviour based on feedback (Kounev et al., 2017). Within this framework, a self-aware system is defined not merely as a structure that processes inputs and produces outputs, but as a system capable of making inferences, evaluating its performance and modifying its behaviour according to context (Kounev et al., 2017). Here, self-awareness is understood not as a singular trait but rather as a capacity that emerges from the integration of learning, perception, planning and reasoning processes. The closest everyday example of this is digital platforms that improve their recommendations as users interact with them. For instance, a music or movie app tracks a user’s preferences, evaluates which recommendations are effective and adjusts its recommendation strategy over time. At this point, the system does not merely have information about “what it does”; it also evaluates which strategy produces better results. This evaluation indicates that the system has a minimal form of self-awareness with respect to its own performance and decision-making processes. The definition used in this field shows that the system generates information about its own performance, decision-making processes and updates this information over time. When the system decides which algorithm or decision strategy to use, compares performance metrics or changes its strategies, it begins to reflect on its own reasoning processes. Nevertheless, this regulatory and reflective part is often treated as part of self-awareness, and thus, no clear conceptual distinction is drawn between awareness and the regulation performed based on awareness.
Reflective computation paradigms directly link self-awareness to meta-reasoning itself. The origins of reflective computation can be traced back to the concept of computational reflection proposed by Maes in 1987. Maes defines reflection as a system’s ability to reason about itself and to intervene in its own operation when necessary. In this context, reflection involves a system evaluating not only its environment, but also its own resources, capabilities and limitations within its operating conditions (Maes, 1987). This reflective capacity can manifest at very basic levels or take more complex forms. At its most fundamental level, it may involve a system adjusting its own parameters or behaviours; at more advanced levels, the system may examine its own reasoning, planning and decision-making processes, recognise its current approach is not working, and revise that approach. In this sense, reflection is not limited to data perception or situation monitoring; rather, it necessarily involves the capacity to reason about the information obtained and to act based on reasoning (Kounev et al., 2017). In this context, the reflective computation directly corresponds to the structure defined in this study as meta-reasoning. In this respect, the reflective computation paradigm focuses on regulatory and reflective processes based on awareness, rather than on self-awareness, under the name of “self-awareness”. Consequently, this field offers perhaps the most striking example of the overlap between self-awareness and meta-reasoning: the concepts are so intertwined that meta-reasoning processes have gradually come to conceptually overshadow self-awareness itself.
The fragmented definition of self-awareness across different computing domains has highlighted the need for a common and consistent framework. To establish a shared definition, a seminar titled Model-Driven Algorithms and Architectures for Self-Aware Computing Systems was organised in 2015 as part of the Dagstuhl Seminar series. This seminar aimed to bring together different approaches to self-awareness from fields such as autonomic computing, artificial intelligence, adaptive systems, control theory and software architecture, to discuss and integrate these scattered uses on a common conceptual ground. As a result of this process, the following definition of self-aware computing systems was agreed upon (Kounev et al., 2017):
Self-aware computing systems are computing systems that:
1. Learn models capturing knowledge about themselves and their environment (such as their structure, design, state, possible actions and runtime behaviour) on an ongoing basis.
2. Reason using the models (e.g., predict, analyse, consider and plan) enabling them to act based on their knowledge and reasoning (e.g., explore, explain, report, suggest, self-adapt or impact their environment) in accordance with higher-level goals, which may also be subject to change. (Kounev et al., 2017)
The definition of self-awareness agreed upon at the Dagstuhl Seminar consists of two subcomponents: learning and reasoning. This structural distinction supports my claim that there is a difference between a representational state of knowledge and the use of that knowledge. However, combining both components under the single heading of a “self-aware computational system” remains conceptually problematic. The main reason for this is that the concept of reasoning is inherently difficult to define, multi-layered and structurally complex, encompassing sub-distinctions such as meta-reasoning. Intertwining two extremely complex concepts in order to define each other makes it difficult to reach a clear explanatory conclusion at the conceptual level. Therefore, while it is evident that self-awareness requires reasoning processes, incorporating additional concepts such as learning and reasoning into the definition of self-awareness renders the situation more complex rather than conceptually clearer. The resulting definition undoubtedly specifies the prerequisites for achieving self-awareness at the engineering level, however, this cannot always be regarded as the appropriate method for defining a concept. Although this definition represents significant progress, it complicates the philosophical explanation of self-awareness.
In this post, I have tried to draw attention to a conceptual confusion in the way “self-awareness” is used in computational research. In many cases, what is described as self-awareness actually includes mechanisms that look much closer to meta-reasoning. When these two capacities are grouped together without distinction, it becomes harder to understand what we are really attributing to artificial systems. This can create theoretical problems, especially when we evaluate claims about “self-aware AI”.
At the same time, this overlap may not be accidental. The persistent tendency to treat self-awareness and meta-reasoning together might reflect a deeper connection between them. It is possible that the field, even unintentionally, is pointing to a structural relationship: perhaps meta-reasoning presupposes some form of self-awareness.
For this reason, rather than simply rejecting this conceptual blending as a mistake, it may be more productive to examine it more carefully. Future work can explore whether the confusion in the literature actually reveals something important about the underlying relationship between these two mechanisms. Clarifying this issue could help us develop more precise theoretical frameworks and make more careful claims about what current AI systems can, and cannot, genuinely do.
*This definition is intended as a general conceptual characterisation of self-awareness, rather than as one restricted to human psychology; whether and to what extent computational systems satisfy this characterisation is a separate question.
References
de Vignemont, F., & Fourneret, P. (2004). The sense of agency: A philosophical and empirical review of the “Who” system. Consciousness and Cognition, 13(1), 1–19.
Gertler, B. (2010). Self-knowledge. Routledge.
Kephart, J. O., & Chess, D. M. (2003). The vision of autonomic computing. Computer, 36(1), 41–50. https://doi.org/10.1109/MC.2003.1160055.
Kounev, S., Lewis, P., Bellman, K. L., Bencomo, N., Camara, J., Diaconescu, A., … Zisman, A. (2017). The notion of self-aware computing. In Model-Driven Algorithms and Architectures for Self-Aware Computing Systems (Dagstuhl Seminar Report 17091). Schloss Dagstuhl Leibniz Center for Informatics.
Maes, P. (1987). Concepts and experiments in computational reflection. ACM Sigplan Notices, 22(12), 147-155.
Further Reading:
Cox, M. T., & Raja, A. (2011). Metareasoning: Thinking about Thinking. MIT Press.
Daniel Dennett (1991). Consciousness Explained.
Gallagher, S. (2000). Philosophical conceptions of the self: Implications for cognitive science. Trends in Cognitive Sciences.
Thomas Metzinger (2009). The Ego Tunnel.
Further Watching/Listening:
Closer to Truth–Episodes on Self and Consciousness. https://www.youtube.com/@CloserToTruthTV.
The Terminator (1984) & Terminator 2: Judgment Day (1991).
Sean Carroll’s Mindscape Podcast (episodes on consciousness & AI). https://preposterousuniverse.com/podcast/.
Lex Fridman episodes on self-awareness & AI
https://www.youtube.com/watch?v=e8qJsk1j2zE
Image Attribution
Generated by: ChatGPT 5.2
Date: 03 March 2026
Prompt: “Generate an image with a mirror and AI self-awareness theme. There should be no text in the image.”