Continuous AI Agent Risk Monitoring After Deployment

AI agents are not static. After your agent passes underwriting and goes into production, its configuration will change — models get updated, new tools get added, permissions get adjusted, system prompts get edited. Each of these changes can materially alter the risk profile that your policy was priced against. A point-in-time assessment at underwriting cannot capture this drift, which is why Redberry requires the SDK to remain active on all insured agents throughout the policy period.

Monitored signals

The Redberry SDK watches for the following configuration changes after deployment:

Signal	Why it matters
Tool additions and removals	Adding a new tool — especially one that performs financial actions or accesses sensitive data — expands the agent’s action surface and may increase multiple risk dimensions
Model changes	Switching to a different model version can change reasoning behaviour, context window size, and output characteristics in ways that affect misrepresentation and operational failure risk
System prompt changes	The system prompt governs how the agent behaves; edits that haven’t gone through review can introduce unintended behaviour or remove important guardrails
Permission changes	Broadened permissions increase data exposure and financial error risk; narrowed permissions may indicate a misconfiguration rather than an improvement
Usage volume changes	A significant increase in action volume amplifies the consequence of any single failure and may push the agent into a different risk tier
Context window changes	A larger context window allows the agent to process more sensitive data per task, increasing data exposure risk if data minimisation controls are not updated accordingly

How monitoring works

Install the SDK

Connect the Redberry SDK to your agent platform via app.redberrylabs.com. The SDK integrates with your existing deployment environment and begins monitoring immediately after setup.

Change detection

The SDK continuously inspects your agent’s live configuration. When it detects a change to any monitored signal, it logs the change with a precise timestamp and produces an encrypted, structured log entry.

Risk profile update

Redberry recalculates the agent’s risk profile against the new configuration. Changes that affect risk dimensions are flagged automatically, so you can review them before they become a coverage issue.

Policy update if needed

For changes that materially alter your agent’s risk profile, Redberry will notify you and may update your policy terms. Routine changes within the bounds of your existing coverage require no action on your part beyond keeping the SDK active.

Example log output

The SDK produces encrypted structured logs for every detected change. Here is an example showing a model change, a tool addition, and a permission update detected within a single session:

14:32:06  model=claude-opus-4    [stable]
14:32:14  model_change → claude-opus-4-5
          ↳ risk flag: context window increased
14:32:31  tool_added: stripe_refund
          ↳ risk flag: financial action surface expanded
14:32:45  permissions_updated: read_customer_pii=true
          ↳ risk flag: data exposure dimension updated

Each log entry records the timestamp, the nature of the change, and the risk flag it triggers. These logs are tamper-evident and form part of the audit trail available to both you and Redberry in the event of a claim.

If you make significant changes to an insured agent — adding new tools, modifying permissions, changing the underlying model, or editing the system prompt — notify Redberry through the app or ensure the SDK is active and will detect the change automatically. Coverage terms require that Redberry’s risk profile for your agent reflects its actual configuration. Undisclosed changes that materially increase risk may affect your coverage. You can review flagged changes and their impact on your policy at any time in app.redberrylabs.com.

Get Started

Coverage

Risk Evaluation

For AI Companies

Legal

Continuous AI Agent Risk Monitoring After Deployment

Monitored signals

How monitoring works

Example log output

​Monitored signals

​How monitoring works

​Example log output

Monitored signals

How monitoring works

Example log output