5 posts tagged with "Security and governance"

How do you actually know your agent is production-ready? "It passed my last three tests" is a sample size, not a score. Actually know, with a statistically meaningful number behind it. That's what Pass@1 / Pass^k (from our last post), the reliability benchmark for moving an AI agent to autonomous action, is for.

Agents are trustworthy when they are highly unlikely to make errors while performing lots of tasks. Pass@1 determines the "not likely to make lots of errors" part of the benchmark. Pass^k determines the "reliability while performing lots of tasks" part. Together, these metrics provide a systematic way to establish how reliable an agent actually is at doing meaningful work.

But here is the harsh reality: manually testing an agent to prove it hits a 95% Pass@1 score is a monumental task. If a developer has to manually run 100 permutations of a prompt just to "verify" an agent, you haven't actually automated anything — you've just moved the bottleneck.

In this post, we'll walk you through exactly how to close that gap: turning a loosely defined agent into a hardened, stress-tested one using the Boomi Agent Trust Scorecard, step by step.

A customer wires Boomi Companion into their account using their own admin login, granting it complete access to every environment. Three days later, while debugging a failed run, it quietly edits a live production schedule. Nothing in its permissions had ever told it not to, or stopped it from trying.

That's not an AI problem. It's an access problem we'd never accept from a new hire. In this post, we'll walk through how to draw that boundary to properly scope Companion's access, step by step.

Building a Data Flow in Boomi Data Integration is easy: drag, drop, connect, done. But here's the question nobody thinks about until it's urgent — who changed that PostgreSQL-to-Snowflake schedule at 2 AM, and why? Or when did a source mapping shift?

When your governance framework or compliance audit asks, "What changed and when?" you need a real answer. Without a change history, you're stuck piecing things together from UI session logs or asking around until someone remembers.

That's exactly what this version control system solves. It treats your Data Flow configurations as version-controlled artifacts: a CI/CD pipeline automatically snapshots your Data Flows, detects changes, and opens a pull request for every Data Flow that drifts. Your team gets a living audit trail with zero manual effort.

Creating AI agent skills containing detailed knowledge of how to interact with remote systems on a user's behalf is a new area of product development for most companies. Much of the tooling that exists in the software development world for ensuring code quality doesn't fully extend or apply to developing AI agent skills, which are primarily natural language text files.

The approachability of this format may entice users to ask an AI agent to naively create skills based on questionable/incomplete documentation, with very few guardrails to spot inefficient, insecure, or even blatantly incorrect instructions. The Boomi Companion team is attempting to elevate skill development by incorporating hard-won lessons from traditional software development, including not only being careful and considered in our approach to security, but also being transparent about the decisions we've made and how we've implemented our design choices. Enabling users and their agents with the equivalent of years of experience using the Boomi Enterprise Platform is extremely powerful, but convenience can (and usually does) come with security caveats and tradeoffs. This blog will demonstrate how the Boomi Companion development team has been considering those implications, the mitigations we've implemented, and what might still be desirable for "hardening" an agent-based build system.

Primary data is not static. It is constantly evolving as systems update records, contributors make changes, and models get revised. What starts as a simple dataset can quickly turn into a complex web of pipelines, models, and dependencies.

Managing this evolution requires visibility, not just into the data itself, but into how it changes, who modifies it, and how it flows across systems.

This is where the Boomi Data Hub Platform API comes into play.