Verifying Actions That Span Multiple Systems or Produce Delayed Effects

Single-system verification is tractable. Cross-system verification, where an action in system A produces effects in systems B, C, and D with varying delays, requires a fundamentally different design.

B

Balagei G Nagarajan

4 MIN READ


MINJA (arXiv:2503.03704, 2025), which evaluated agents on multi-agent injection attack scenarios, found that cross-agent verification - confirming that intended effects propagated correctly across agent boundaries - was the most commonly missing safety component. When agent A instructed agent B to take an action, the outcome in agent B's system was verified in only 22% of evaluated deployments. The other 78% verified only that the instruction was sent, not that the effect was produced.
Distributed verification challenges hero
Effect latency is the simplest to address but the most commonly missed.
— from “Verifying Actions That Span Multiple Systems or Produce Delayed Effects”

Key facts.

  • MINJA (arXiv:2503.03704, 2025): cross-agent effect verification present in only 22% of evaluated deployments; 78% verify only instruction delivery, not effect production.
  • Three distributed verification challenges: effect latency (action effects appear seconds to hours after the action), system boundary opacity (agent can't read the state of another system directly), and partial effects (some downstream systems receive the effect, others don't).
  • The average delay between action in system A and observable effect in system B across enterprise workflow integrations is 3-90 seconds for sync integrations and 10 minutes to 4 hours for async integrations.
  • Distributed transactions without saga patterns produce partial application states that are invisible to single-point verification - a commit in the billing system may not propagate to the provisioning system before the verification window closes.

The three distributed verification challenges

Effect latency is the simplest to address but the most commonly missed. An action in system A produces an effect in system B after a delay - from milliseconds for synchronous integrations to hours for batch processing integrations. A verification check that runs immediately after the action in system A and reads system B's state before the effect propagates will see the pre-action state and report no effect. The correct verification window depends on the integration pattern: sync checks should run after the sync response; async checks should run after the expected processing window plus a safety margin.

System boundary opacity is the hardest challenge. When agent A sends an instruction to agent B, agent A may not have read access to system B's internal state. It can verify that the instruction was sent (instruction delivery). It can't directly verify that system B's state changed as expected (effect production). Solutions include: designing system B to emit verification events that system A can subscribe to, creating a shared verification oracle that both systems can write to and read from, or using a saga coordinator that tracks cross-system state transitions explicitly.

Partial effects in distributed systems occur when an action is meant to update multiple systems simultaneously but only some updates complete before the verification window. The canonical case is a financial workflow where the payment system updates but the entitlements system doesn't. Point-in-time verification sees a consistent state in neither system individually and a correct state in neither system jointly. Distributed verification must check all affected systems within a single verification window, or use compensating transactions to restore consistency when partial application is detected.

Distributed verification patterns

Event-sourced verification subscribes to state-change events from all affected systems and confirms that the expected events appear within the expected window. this is the most scalable pattern for async integrations: the verification layer is a consumer of the event stream, not a polling caller of each system's state API.

Saga coordinator verification tracks the distributed transaction state in a dedicated coordinator and queries that coordinator for the completion state. The saga coordinator is the single point of truth for the multi-system transaction, which makes it the correct point for verification queries. Any system that doesn't confirm completion to the coordinator within its SLA window triggers a compensating transaction automatically.

# Distributed verification with saga pattern
class SagaVerifier:
 def __init__(self, saga_id, systems, expected_effects, timeout_s):
 self.saga_id = saga_id
 self.systems = systems
 self.expected_effects = expected_effects
 self.deadline = time.time() + timeout_s

def verify(self) -> VerificationResult: while time.time() < self.deadline: effects_seen = { sys: sys.read_effect(self.saga_id) for sys in self.systems } if all(self.expected_effects[sys].matches(effects_seen[sys]) for sys in self.systems): return VerificationResult.PASS time.sleep(2)

missing = [s for s in self.systems if not self.expected_effects[s].matches( self.systems[s].read_effect(self.saga_id))] return VerificationResult.trigger_compensating_transactions(missing)

Distributed verification saga pattern diagram

Verification approach by integration type

Integration typeEffect latencyVerification approachFailure response
Synchronous (REST, gRPC)MillisecondsImmediate read-back after responseRetry or escalate
Async message queueSeconds to minutesEvent subscription + timeout windowDead-letter queue alert
Batch processingMinutes to hoursBatch completion event + state queryCompensating transaction
Multi-agent handoffVariableSaga coordinator querySaga rollback

The Pattern Intelligence Layer classifies integration types in each pattern and specifies the corresponding verification approach. Patterns involving multi-system actions include saga coordinator specifications as required components. The 78% of deployments that verify only instruction delivery do so because they didn't have a pattern specification that required more - the Pattern Intelligence Layer closes that gap by design.

Frequently asked questions

Does upgrading the model fix unverified cross-system effects?
Cross-agent effects go unchecked as a class; a bigger model inherits it, and the retries land late. (arXiv:2503.03704)


Share this post

Join the discussion

Have a take, a war story, or a question? Sign in with GitHub to comment and react. Comments are powered by GitHub Discussions, ad-free and yours to moderate.

Continue Reading

Find where your agent breaks, before you build it

Faultmap maps where your agent will fail from the goal and your data, then hands you the first test suite it has to pass.