Building the Rules of the Road for Agentic AI

Enabling a future, where everyone owns multiple agents acting on their behalf, whether they are software agents or physical AI devices like robot lawnmowers or household robots, requires a new risk framework. The current agentic communications protocols – A2A, MCP, AP2, UCP – need to be extended to allow passing additional parameters about an agent’s risk posture and shutdown procedure.

Secondly, additional “rules of the road” are required for agents as they are no longer passive systems. They act, decide, and interact with external environments. That raises new questions:

• What happens if a personal shopping agent unintentionally overwhelms an e-commerce platform, causing downtime and lost revenue?

• What if a household robot malfunctions and damages a neighbor’s property?

Transportation Systems as an Analogy

A useful way to think about this is through the lens of transportation systems. A car has an owner and operator, is manufactured by a regulated car company and operated/driven on roads and highways according to a set of traffic rules. On the road, we encounter different vehicle types (bicycles, cars, trucks) that carry different risk levels and regulatory requirements. Bicycles pose relatively low risk and require minimal regulation. Trucks pose higher risk and require licensing, insurance, and stricter controls. Bicycles are allowed on roads, but not highways to ensure smooth traffic and safety. Emergency vehicles are granted priority access, with all other traffic yielding to ensure a clear path during emergencies.

By analogy, an agentic AI system has an owner, is programmed by that owner or its developer and executes tasks over the internet. Depending on their skills, system access, and whether they interact with the physical world, agents can carry low, medium or high risk. An agent running on a local laptop, accessing only the owner’s files or reading data from the internet poses low risk. On the other hand, an agent that interfaces with an external system, modifies data, or operates as physical AI introduces a higher level of risk. An agent accessing and updating critical systems such as banking, healthcare or governmental systems or operating as a robot in industrial and public environments represents the highest level of risk. 

Operating Requirements

Just as with transportation, each risk tier should carry corresponding licensing, safety and operating requirements:

For medium and high-risk agents, registration should include verified ownership (legal guardian for minor), the agent’s skills, issue date and proof of insurance. Medium- and high-risk agents should be subject to mandatory safety testing and certification, similar to vehicle licensing and independent safety inspections. They should also be required to implement kill switches and safe-stop mechanisms, analogous to safeguards in autonomous vehicles.

In case of resource constraints, such as limited compute capacity or bandwidth, high-risk agents should be prioritized over lower-risk ones. In emergency situations, affected parties and agent hosting platforms must have the authority to block medium- and high-risk agents from accessing their systems.

Conclusion

To evolve agentic AI from isolated tools into interconnected, autonomous systems, widespread adoption will depend on establishing a robust risk framework. If we want a future where agents can reliably interact across organizations and domains, we need to establish clear rules of the road - before traffic gets too heavy.