Building AI Agent Harnesses for Apache Magpie

When software teams adopt AI coding assistants, they quickly run into a major roadblock: every developer uses a different tool.

Some engineers work inside Cursor. Others live in the terminal using Aider or GitHub Copilot CLI. In high-security environments, developers are not even allowed to use cloud-based models; they run local models via Ollama or vLLM on private workstations.

For open source ecosystems like the Apache Software Foundation, this fragmentation creates a serious dilemma. If you build AI workflows for only one specific agent harness, you alienate contributors and lock your project into a proprietary vendor. If you rewrite every workflow for every tool, you end up with duplicated, brittle code that becomes impossible to maintain.

In the Apache Magpie project, an open source framework designed for security automation across Apache projects, I tackled this problem by designing and landing swappable runtime adapters for five major AI agent harnesses.

Here is how the system was built, the technical hurdles solved, and the measurable impact on open source security operations.

[1] The Fragmentation Problem in Open Source Security

The Apache Software Foundation oversees hundreds of critical open source projects, including Airflow, Kafka, Spark, and Tomcat. Companies around the world depend on these projects daily.

When a critical security vulnerability is reported in an Apache project, maintainers cannot handle it like a standard bug:

Strict confidentiality: Vulnerability discussions cannot happen in public GitHub issues before a patch is ready. If exploit details leak prematurely, attackers can target production systems before a fix is released.
Complex release governance: Handling a vulnerability requires requesting CVE numbers, preparing private patches, coordinating voting across project management committees, and drafting public security advisories.
High manual effort: Managing a single security advisory can take up to twenty manual review cycles.

To assist maintainers, Apache Magpie provides automated agentic workflows called skills. However, maintainers have different setups. Some wanted to run triage from Cursor. Others wanted terminal pair programming in Aider or Copilot. Security teams handling highly sensitive embargoed bugs required completely air-gapped local machines without any network traffic leaving the building.

The project faced an immediate challenge: how do we empower maintainers to use their preferred AI agent harnesses while guaranteeing absolute data confidentiality and zero vendor lock-in?

[2] The Architectural Challenge

The objective was straightforward but technically demanding:

Maintain one canonical source of truth: Write every security skill once in clean, vendor-neutral markdown, without mentioning proprietary vendor names or hardcoding API calls.
Support five diverse agent harnesses: Connect IDE assistants, terminal CLI tools, declarative recipe runners, and offline local inference engines.
Guarantee strict human oversight: Ensure that no AI model can autonomously push code to protected branches, merge pull requests, or send outbound messages without explicit human approval.
Enforce air-gapped data boundaries: Guarantee that confidential vulnerability data and private repository tokens never leak through cloud telemetry.

To solve this, we decoupled the workflow definition from the execution engine. Skills define what needs to be done. Capabilities define abstract operations like sending mail or querying advisories. Runtime adapters bind those capabilities to concrete developer harnesses.

My objective was to engineer the runtime adapters and operational guides for five distinct AI agent harnesses to achieve 100% harness neutrality.

[3] Engineering the Five Agent Harnesses

Each agent harness has its own execution model, configuration layout, and permission mechanics. I implemented custom runtime adapters tailored to each environment.

1. GitHub Copilot CLI and Coding Agent (PR #1287)

GitHub Copilot serves developers in two distinct ways: interactive command line sessions in the terminal and server-side cloud coding agents running on GitHub infrastructure.

I addressed two critical requirements for this harness:

Establishing a strict Privacy boundary: Server-side cloud agents were explicitly forbidden from operating on private security trackers and embargoed vulnerability discussions. This ensures private exploit code is never sent to external cloud model servers.
Enforcing human confirmation: In the terminal, all shell commands require interactive operator approval before execution. On GitHub servers, the agent is restricted strictly to creating draft pull requests; it has zero permission to push to protected branches or merge changes autonomously.

2. Aider Terminal Pair Programming (PR #1286)

Aider is a fast, terminal-based pair programming harness that interacts directly with local git working trees.

To make Aider work reliably with dozens of complex workflows, I solved two engineering problems:

Preventing context window flooding: Magpie contains over seventy specialized skills. Loading all skill files at startup exhausts model context windows and degrades reasoning quality. I implemented a progressive loading pattern where Aider ingests only the specific skill markdown file needed for the immediate task.
Enforcing human-reviewed commits: By default, Aider can automatically commit file modifications to git. In security engineering, unreviewed autonomous commits are unacceptable. I configured strict rules disabling automatic and dirty commits, guaranteeing that maintainers inspect every diff before a commit is created.

3. Block Goose and Model Context Protocol (PR #1217)

Goose, developed by Block, is an extensible open source CLI agent harness that automates complex engineering tasks through recipes and extensions.

For the Goose harness, I focused on deep protocol integration:

Native Model Context Protocol (MCP) integration: I configured native stdio MCP bridges connecting Goose directly to custom Magpie servers for mail triage, project metadata, and security tooling.
Declarative recipe workflows: I created reusable YAML recipe templates that allow maintainers to launch multi-step security triage routines with a single terminal command.
PreToolUse guardrails: I leveraged Goose extension hooks to intercept tool execution, giving maintainers the ability to inspect and approve operations before the agent modifies files.

4. Air-Gapped Local LLMs via Ollama, llama.cpp, and vLLM (PR #1206)

For security teams handling top-secret vulnerability disclosures, commercial cloud APIs are out of the question.

In PR #1206, I documented and calibrated the harness for 100% offline, sovereign operations:

Universal local serving: Documented complete integration with Ollama, llama.cpp server, and vLLM exposing local OpenAI-compatible endpoints.
Calibrating empirical model capability floors: Through rigorous evaluation testing, I established clear guidance on model sizes:
- 70B+ parameter reasoning models are required for multi-step security triage, where the agent must evaluate exploit mechanics and follow multi-step governance rules.
- 8B to 14B parameter models are calibrated for bounded, single-file audits, checklist verifications, and formatting tasks.
Air-gapped workstations: Enabled maintainers to run fully capable AI assistants on isolated machines with zero internet connectivity and zero cloud telemetry.

5. Cursor Composer and Agent CLI (PR #1204)

Cursor is one of the most widely used AI development harnesses, featuring both an in-editor Composer and a standalone terminal agent.

To integrate Cursor into our universal model:

Canonical skill discovery: Mapped canonical skill directory paths so Cursor automatically discovers and recommends workflows without requiring duplicated configuration files.
Modal approval alignment: Mapped Cursor's per-action approval modals to our proposal-then-confirm standard, ensuring the agent never applies destructive changes without consent.
Sandboxed CLI execution: Wrapped CLI invocations in clean-environment isolation containers to prevent unintended access to local developer tokens.

[4] Production Impact and Measurable Outcomes

Landing these five agent harnesses delivered immediate, concrete benefits to the Apache Magpie framework and the wider open source community:

100% Harness Neutrality Score: All five harnesses passed our automated vendor neutrality test suites with a perfect 10/10 score across capability contracts and 24/24 substrate tools.
Multi-Harness Support: Maintainers across diverse Apache projects can now execute identical security workflows from Cursor, Aider, Copilot, Goose, or fully offline local LLMs.
Zero Leaked Credentials: By wrapping runtimes with ambient credential stripping, local API keys and cloud tokens remain completely isolated from model sessions.
Proven Air-Gapped Feasibility: Established empirical benchmarks proving that open-weight 70B models can handle complex, multi-step vulnerability triage without sending a single byte to cloud servers.
Complete Maintainer Governance: Strict human-in-the-loop policies guarantee that every git mutation, draft advisory, and outbound communication remains under human authority.

[5] Key Takeaways for Engineering Teams

Building adapters across five distinct agent harnesses highlighted three vital lessons for the future of software engineering:

Decouple workflows from harnesses: AI developer tools change constantly. If your automation relies on proprietary features from a single harness, your codebase will age poorly. Writing workflows in clean, tool-agnostic markdown protects your engineering investment.
Safety is more important than autonomy: Fully autonomous agents that write code, commit to git, and deploy without confirmation are a liability in production. Real-world enterprise systems require proposal-then-confirm patterns where the agent drafts and the human decides.
Sovereign AI is ready for production: You no longer need to compromise between powerful AI capabilities and strict confidentiality. With properly calibrated open-weight models, teams can run enterprise-grade security workflows on entirely private infrastructure.

Open source software succeeds when developers have freedom of choice. By building universal agent harnesses for Apache Magpie, we showed that AI automation can be flexible, secure, and truly vendor-neutral.

Huge thanks to Jarek Potiuk for the rigorous architectural reviews, thoughtful feedback, and continuous push for production-grade engineering quality across these PRs. He is the final boss of code reviews in Apache Magpie :)

Thanks for reading, Connect with me :)

LinkedIn and GitHub

How I Built Universal AI Agent Harnesses for Apache Magpie