Structural Causal World Models for Evolving Safety Assurance
Posted on Wednesday 23 September 2026
In our previous blog, we introduced what the re-emergence of this theme means for AI safety. We know that safety depends on more than the AI model itself. It also depends on the assumptions made about the operating environment, the requirements placed on the system, the evidence used to support those requirements, the conditions monitored during operation and any changes made after deployment. These elements need to remain connected as the system evolves.
For example, before such a system is released, engineers may test its software, run simulations, analyse hazards and collect evidence to support a safety argument. However,roads become wet or icy. Sensors degrade. Software is updated. New traffic patterns appear. An AI model may encounter situations that were rare or absent in its training data.
This leads to a fundamental question:
How can we continue to justify that an autonomous system is safe when the system, its environment and the available evidence are all changing?
Our research on Structural Causal World Models (SCWMs) addresses this question by treating safety assurance as an ongoing process rather than something completed once before release.
What is a Structural Causal World Model?
A world model represents a system and the part of its environment that matters for a particular purpose. For an autonomous vehicle, this might include the road, nearby vehicles, pedestrians, weather, speed, distance and braking behaviour. For a warehouse robot, it might include people, shelves, obstacles, loads and permitted working areas.
Many AI world models are designed mainly for prediction, planning or simulation. An SCWM has a different purpose: it is designed to support safety assurance. It represents the safety-relevant part of the world together with the assumptions, causal relationships, uncertainty and conditions on which safety conclusions depend. It is not intended to reproduce every detail of reality. It contains only the knowledge needed to support a particular safety decision. An SCWM acts as both an intermediate assurance representation and a living assurance artefact. It does not replace the underlying evidence obtained through testing, simulation, verification or operation, and it cannot establish a final safety claim on its own. Instead, it makes clear how evidence supports a claim, which assumptions are being made and where the reasoning stops being valid.
For example, a test may show that an emergency-braking system stopped successfully in a particular scenario. A broader safety claim may state that the vehicle can maintain an acceptable stopping margin throughout a defined operating domain. The SCWM connects the test result to the broader claim. It records which assumptions allow the result to be used, which causal relationships support the conclusion, how uncertainty has been handled and where the evidence no longer applies.
The SCWM is itself an assurance artefact. Evidence that the model is internally coherent, that its causal relationships are well founded and that its uncertainty bounds are appropriately conservative can therefore contribute to the wider assurance case. However, a well-structured model is not, by itself, proof that the deployed system is safe.
World models and safety decisions
No world model is complete. A model may describe some parts of a system well while leaving out others. Its assumptions may conflict. Its causal relationships may appear reasonable but have little supporting evidence. Its uncertainty limits may also be too optimistic. A model is therefore not simply “valid” or “invalid” in the abstract. It may be good enough to support one safety claim under one set of operating conditions, but not another.
That leaves us with a question which has been at the core of our research in this area;
Under what conditions is a Structural Causal World Model sufficiently valid, internally coherent and grounded in evidence to support a specific safety claim, or to derive a verifiable safety requirement, within a declared operational context?
For example, a model may be sufficient to support an emergency-braking requirement when speed, road friction, sensing uncertainty and response time stay within defined limits. The same model may no longer be sufficient if the vehicle is later allowed to travel faster, operate in severe weather or use a different sensing system.
A model-centred approach to assurance
Figure 1. A model-centred assurance framework supported by a Structural Causal World Model. The upper part shows the continuing relationship between specification, operation, models of the system and environment, and assurance. The lower part shows the four main layers of an SCWM: ontology, quantification, causality and contracts.
Figure 1 shows that specification, operation and assurance are not isolated stages. Operational data may reveal that an earlier assumption was unrealistic. Hazard analysis may identify a new requirement. A software update may invalidate part of the existing evidence. A runtime monitor may detect that the system is approaching the limit of its justified operating conditions. The SCWM provides a common point of reference across these activities. It keeps the system model, operating context, assumptions, evidence and safety reasoning connected.
This approach is guided by four broad principles:
- First, a safety argument should be based on a sufficiently accurate model of the system and its operating context. The model does not need to describe everything, but its level of detail should be appropriate to the safety risk being addressed.
- Second, the safety claim must be stated clearly. The SCWM helps connect high-level safety goals to more specific and verifiable requirements without losing their original intent.
- Third, the reasoning should be open to challenge. Models can be used to derive test conditions, identify rare but important combinations of circumstances and produce counterexamples showing where a requirement fails.
- Fourth, assurance must continue after deployment. Runtime observations and field data should be used to review assumptions, update the model and reassess whether the safety argument still holds.
The four layers of an SCWM
The lower part of Figure 1 shows how an SCWM organises safety-relevant knowledge. Information enters the model through observation and estimation of the operational context. The model then addresses four related questions: What are we talking about? What do we know with uncertainty? What affects what? Under what conditions does a safety promise hold?
The ontological layer: defining the terms
The ontological layer defines the concepts, attributes and rules used in the model. In an emergency-braking example, it may distinguish the true distance to an obstacle from the distance estimated by a sensor. It may also define road friction, vehicle speed, response time and braking capability, including their units and permitted ranges.
This may appear basic, but safety problems often arise when different documents or engineering teams use the same term in different ways, or when an important assumption is left unstated.
The quantification layer: representing uncertainty
Real systems do not observe the world perfectly. A distance measurement contains error. Road friction may only be estimated. Response time may vary. The quantification layer represents these uncertainties and, where appropriate, introduces conservative bounds.
Rather than pretending that a value is exact, the model records how uncertain it is and what additional safety margin may be needed because of that uncertainty.
The causal layer: explaining how factors interact
The causal layer describes how changes in one factor affect another. Lower road friction increases stopping distance. A longer sensing delay reduces the time available to brake. Higher speed increases the distance required to stop.
These relationships allow engineers to ask structured “what if?” questions. What happens if friction falls? What if the sensor becomes slower? Which assumption has the greatest effect on the safety margin?
The contract layer: stating the promise and its limits
The contract layer expresses safety responsibilities in an assume-and-guarantee form. In simple terms, it says:
If these stated conditions hold, the system is expected to provide this stated behaviour.
An emergency-braking contract might state that if speed, road friction, sensing error and response delay remain within defined limits, the vehicle will maintain a required stopping margin. This makes the limit of the claim visible. When an assumption is violated, the guarantee cannot simply be treated as though nothing has changed.
In our next blog, we will use a simplified example to provide an intuitive explanation of how the SCWM can serve as a living assurance artifact. We will explore why seemingly simple factors can become so powerful, discuss the challenges that SCWMs do not address, and conclude by looking toward the vision of continuously justified safety.
Find out more
This blog introduces ideas developed across two of our papers. For the technical foundations, methods and evaluation behind the discussion, scan the QR codes below to read the full papers.
This is only the beginning of the SCWM research series. Further papers will show how they can be applied across the design, deployment, operation and evolution of AI-enabled autonomous systems.