Testing, Trust, and Fielding Cognitive EW

Part of the Field Guide to Cognitive Electronic Warfare. All information is derived from unclassified, publicly releasable sources.

The fourth hard problem stops the most programs, and it is not a technical failure. A cognitive EW system produces value by changing its own behaviour, which is what makes a commander reluctant to let it act. Fixed software is tested against a list and certified once, because it behaves tomorrow as it did today. A system that revises its own logic offers no such guarantee, and no one delegates a consequential decision to a system whose behaviour cannot be bounded. This page covers what replaces the one-time certificate: validating how the system learns and confronting it with surprise in emulated battlespaces, bounding its autonomy under a human commander, fitting the decisive loop onto platform power at the edge, and the gap that separates the many demonstrations from the few fielded capabilities.

Why one-time certification fails

A pattern recurs in the field: a system performs well in a controlled demonstration against known threats and never deploys, because no one could establish that it was safe to trust with a decision that matters, under conditions the demonstration did not cover. The reason is structural. Conventional assurance assumes the thing being certified holds still, and a learning system breaks that assumption deliberately. Test a fixed program thoroughly and the certificate is meaningful, because the program that ships is the program that was tested. A cognitive system updates itself, so the version that operates next week is not the version tested last week, and a certificate attached to one version says little about the next. The more a system relies on learning, the less a one-time test can establish.

If a system cannot be certified once and set aside, trust has to be built in other ways, and they are concrete. The system's autonomy can be bounded, so it is fast where speed is safe and defers where judgment is required, with a human setting its goals and limits and able to intervene. Its confidence can be made honest, so it reports what it does not know rather than acting on a guess. And its behaviour can be exercised against surprise, so its response to the unfamiliar is observed before combat. Trust is not a property a system has or lacks. It is accumulated through design choices that keep the system's behaviour bounded and legible as it learns.

For that reason, assurance is not a compliance step added after the capability works. It is a design driver from the start, on the same footing as the capability, because a cognitive system that cannot be trusted is not a capability. The organizations that field cognitive EW treat earning trust as part of building the system, and the sections that follow are the functions that do the earning.

Testing a system that learns

Traditional testing checks a fixed system against a fixed list of cases, and running a fixed battery of known cases against a system whose purpose is to handle the unknown produces a certificate that is reassuring and nearly meaningless. The methods have to change to match the system being tested, and they change in two ways.

The first change is to certify how the system learns rather than any single thing it has learned. Instead of establishing that a particular model is correct, the assurance establishes that the learning is sound: that it improves in the right direction, does not forget what it knew, stays within its bounds, and fails safely when it meets something new. The object of trust becomes the learning process and its guardrails, not a snapshot of the model, because the snapshot will not last. This is a different discipline from testing fixed software.

The second change is to test against the unfamiliar deliberately. A system that handles only known cases can be tested against a list of them. A system that claims to handle the unknown has to be confronted with the unknown, which means generating inputs it was never trained on and confirming that it responds safely and sensibly rather than confidently wrong. This is harder than running a checklist, because the surprises must be produced and the correct behaviour has no precomputed answer, only the requirement that the system report its uncertainty and fail gracefully. It is also unavoidable, because the reason the system exists is the case that was not anticipated.

Both changes require volume and variety of engagement that the real world cannot safely supply. A cognitive system needs to face many varied and novel threats to become capable and to be shown safe, and live ranges are limited while real operations are slow and high-consequence. The practical answer is emulation: a high-fidelity, real-time model of the electromagnetic battlespace that a real system connects to and cannot distinguish from the world. Large-scale, real-time emulators can run a real system under test against hundreds of simultaneous synthetic emitters, and the party that operates the better proving ground can practice more, and more safely, than the rest, generating the scarce experience that fielding requires.

The limiting factor is fidelity. A system trained and tested in a synthetic environment performs in the real one only to the extent the synthetic environment is accurate, and the difference between them is the simulation gap. Measuring that gap and reducing it, through faithful physics, careful variation, and a small amount of well-chosen real data, is a discipline in its own right, and an emulator's value is set as much by how well its gap to reality is characterized as by its scale. A proving ground whose gap is unknown provides confidence that is itself unmeasured.

Battle management and human control

Above the individual engagement is the management of the whole electromagnetic fight, where machine autonomy and human command meet. A single cognitive jammer optimizes its own effect. Electronic battle management sits above that and asks a harder question: given many threats, many assets, one shared spectrum, and a mission to accomplish, what is the best use of effects over time? This does not decompose into independent decisions, because committing an asset in one place denies it elsewhere, and denying a frequency to the adversary can deny it to friendly systems as well. It is a planning and coordination problem, and it is the most plan-intensive of the EW functions because it reasons across a mission rather than a moment.

The planning works from an incomplete and changing picture. It does not know the full threat situation, cannot perfectly predict how threats will react, and holds uncertain assessments. It also has to treat the spectrum as the scarce, shared resource it is, deconflicting friendly use, concentrating effects where they matter, and avoiding denying the spectrum to the systems that depend on it, including its own communications and navigation.

Because these decisions carry mission and legal weight, autonomy is limited deliberately. The machine plans and coordinates at machine speed within an envelope the commander sets: rules of engagement, geographic and spectral limits, and thresholds that return control to a human. The engineering problem is to specify that envelope so the system is fast where speed is safe and returns control where judgment is required. Bounded autonomy is what allows a commander to accept a fast machine at all, because it makes the machine's freedom legible and its limits enforceable.

A recent proposal is to place a large language model in the planning loop as an agent that proposes courses of action, weighs options, and coordinates. Planning and the weighing of options are tasks these models handle well, so there is genuine use in it. There is also genuine risk, because a language model can produce a fluent, confident plan that is wrong, and battle management is the last place a confident wrong answer should be accepted. The appropriate use is to widen what a commander can consider, not to replace the judgment that decides, and to hold the model to the same assurance standard as any other part of the system. Every thread of the trust problem converges here, on a commander deciding to let the machine operate, and that decision rests on bounded behaviour, honest confidence, and demonstrated safety under surprise.

Cognition at the edge

The decisive part of a cognitive EW system runs on the platform, on hardware that flies, and that fact shapes how it is built. The inner loop runs in seconds, and it runs while an adversary attempts to jam the link a rear connection would use. A system that sends its observations away and waits for an answer has already lost the timing, and lost it when the engagement is hardest. The fast, decisive cognition therefore runs on the platform. The slow cognition, the training and reprogramming, can run off the platform, but the part that wins the engagement cannot, and that division is the first architectural decision.

An edge platform has a fixed budget of space, weight, power, and cooling, and every part of the system draws from it. A larger model is more capable and costs more power and heat. A faster loop needs more computation in the same envelope. These are first-order constraints, not details to optimize later, and they are why a method that runs comfortably in a data center can be impossible on a platform, and why reducing models to run within the budget is a core skill. The computation runs on a mix of processors matched to different tasks: general-purpose processors for control, programmable logic for fast fixed signal processing near the antenna, and parallel processors for the arithmetic of machine learning. Event-driven processors, which compute only when a signal changes rather than on a fixed clock, are being added to that mix; for a domain whose binding constraint is power on a platform, an approach that expends energy only when something changes is a direct fit.

The clean way to organize the architecture is to divide the loops by where they run. The inner loop, fast and decisive, runs on the platform, autonomously, within the edge budget. The outer loop, slow and data-intensive, runs off the platform, with people and with the proving ground described above. Getting that division right, and designing the interface so the platform degrades to full autonomy when the link is gone, is the core of a cognitive EW architecture. None of it fields quickly without a way to integrate cognitive payloads onto platforms affordably, which is what modular open systems architecture provides, and cognitive EW is among the most demanding payloads those standards carry. Spectral Autonomy's Field Guide to Open Architecture covers that foundation. The two subjects are linked: open architecture is how cognition reaches the platform in time to matter.

The fielding gap

Understanding cognitive electronic warfare is a small part of possessing it. The theory conceals little. The cognition loop, the idea of learning in the field, the reinforcement-learning methods, and the mapping of artificial intelligence onto the electronic-warfare functions are all published, much of it by the same small community across national boundaries. A newcomer can learn the shape of the field from open sources in a few weeks, and a competitor can reproduce an architecture in an afternoon. If the contest were about understanding, it would be a draw. It is not about understanding.

A reproduced architecture leaves a competitor years behind because the difficulty was never the architecture. A fielded system has to observe what its actions did, learn from very little data, outpace an adversary that is also learning, and earn enough trust to be allowed to act. Each is a deep problem with no clean solution, and each has to be solved at once, on real hardware, against a real opponent. The gap between the diagram and the capability is the four hard problems, together. This is why so many programs produce a demonstration and never a fielded system, and why the demonstration is the easy part.

If the idea is free, the advantage is in what the idea does not contain. It is in curated, real-world data that cannot be downloaded, especially the rare record of what actually happened when an effect was applied. It is in the proving grounds where systems are trained and earn trust. It is in the industrial base that turns a design into a fielded, open-architecture module, and in the assurance processes that let a commander rely on a system that learns. These capabilities are expensive and slow to build, and they decide the contest.

The same difficulty is what makes a real capability durable. A capability that is easy to field is easy for an adversary to field. Cognitive electronic warfare cannot be copied from a paper or a demonstration; it has to be built, through the accumulation of data, proving grounds, architecture, and trust, and that accumulation is a barrier that understanding alone cannot cross. The field rewards depth and the willingness to solve four hard problems well rather than one easy problem quickly.