Factory vs Omnara
Omnara wins · 21–24 (17 drawn)
Agenticness — how well agents can access and operate the productAgenticness
How well agents can access and operate the product
Agent access
ai-native userPoint an agent at llms.txt or agent-oriented docs
weight 2 · round drawnFactory hosts an actual llms.txt at docs.factory.ai/llms.txt (HTTP 200) confirmed by direct probe, and its docs describe an agent-native platform with structured agent-oriented documentation (droid-cli, agent-readiness, missions) that an agent could be pointed at. missing for 10: independent/hands-on confirmation that an agent successfully consumes llms.txt in practice, and no explicit vendor statement encouraging users to point agents at llms.txt.
- [probe] “PROBE llms.txt: HTTP 200 at https://docs.factory.ai/llms.txt # Factory Documentation > Documentation for Factory, the agent-native software…”
- [claimed-docs] “Use the Droid CLI to run Factory from your terminal, editor, tests, and Git workflow.”
- [claimed-docs] “Run the /readiness-report slash command in a Factory App web or desktop session, or in the Droid CLI, to evaluate a repository's readiness l…”
Probe confirms llms.txt is live at docs.omnara.com/llms.txt (HTTP 200) with structured links to introduction, quickstart, etc., directly enabling agent-oriented doc consumption; openapi.json is also served for machine consumption. missing for 10: no independent/community confirmation that agents actually consume llms.txt effectively, and no explicit vendor documentation describing llms.txt as an intentional agent-facing feature.
- [probe] “PROBE llms.txt: HTTP 200 at https://docs.omnara.com/llms.txt # Omnara - [Introduction](https://docs.omnara.com/introduction.md): The API fo…”
- [probe] “PROBE openapi: HTTP 200 at https://docs.omnara.com/openapi.json — contains "openapi" key”
- [probe] “official CLI documented at https://docs.omnara.com/quickstart”
ai-native userRun the product headlessly / in CI for automation
weight 2 · round to FactoryFactory documents droid exec as a one-shot CLI command explicitly designed for CI/CD pipelines, shell scripts, and batch processing, with tiered autonomy controls for unattended operation. This directly satisfies headless/CI automation. Missing for 10: independent/hands-on third-party verification of CI usage and more detail on exit codes/output formats for pipeline integration.
- [claimed-docs] “droid exec runs as a one-shot command that completes a task and exits, making it ideal for CI/CD pipelines, shell scripts, and batch process…”
- [claimed-docs] “Droid exec uses tiered autonomy to control what operations can run without manual confirmation.”
- [claimed-docs] “Use the Droid CLI to run Factory from your terminal, editor, tests, and Git workflow.”
- [probe] “official CLI documented at https://docs.factory.ai/droid-cli/quickstart”
Omnara explicitly supports launching agents programmatically via CLI, REST API, or TypeScript SDK (omnara-docs-2), and API keys can be assigned operator roles for automated agent control (omnara-gh-1), which supports headless/CI-style usage. However, there is no explicit CI/CD pipeline example, GitHub Actions integration, or dedicated 'headless mode' documentation demonstrating a full automation workflow. missing for 10: concrete CI/CD integration example, explicit headless-mode docs, independent confirmation of automated pipeline usage.
- [claimed-docs] “Launch your first agent with Omnara via the dashboard, or programmatically with the CLI, the REST API, or the TypeScript SDK.”
- [github] “Assign organization and project roles to users and API keys. Separate who can manage access, configure agents, operate them, or only view th…”
- [github] “Self-hosted deployments can query agent history directly in Postgres for analytics, evals, prompt analysis, and training datasets.”
- [probe] “PROBE openapi: HTTP 200 at https://docs.omnara.com/openapi.json — contains "openapi" key”
- [probe] “official CLI documented at https://docs.omnara.com/quickstart”
ai-native userPlug MCP servers into this product so it can use their tools
weight 3 · round to OmnaraDocs explicitly state Droid CLI can connect MCP tools alongside Jira/Notion/Slack/Linear/PagerDuty integrations, confirming MCP client support. However, there is no detail on setup/configuration process, supported transport types, or independent/hands-on confirmation of MCP tool usage in practice. missing for 10: configuration details for adding MCP servers, examples of MCP tools being invoked, independent verification of functionality.
- [claimed-docs] “Connect Jira, Notion, Slack, Linear, PagerDuty, and MCP tools to keep development synchronized with team systems”
- [claimed-docs] “Use the Droid CLI to run Factory from your terminal, editor, tests, and Git workflow.”
Docs explicitly state users can 'Connect MCP servers to give them access to your services and data, or add custom tools' as a first-party feature. Missing for 10: independent hands-on confirmation of MCP server integration working, and details on setup/configuration flow or limitations.
- [claimed-docs] “Connect MCP servers to give them access to your services and data, or add custom tools for actions in your own application.”
ai-native userConnect an agent via an official MCP server
weight 3 · round to FactoryFactory is primarily a coding agent (Droid), and factory-docs-4 shows it can connect OUT to MCP tools (client-side), which wouldn't count under the na rule — but factory-probe-3 explicitly indicates an official MCP server is documented at docs.factory.ai/harness/mcp, flipping this to the applicable server-side capability. However, the evidence pack only confirms the page's existence via a probe hit with no detail on what the server exposes or how other agents connect to it. Missing for 10: detailed documentation of the MCP server's tool surface, setup instructions for external agents, and independent confirmation of a working connection.
- [probe] “official MCP server documented at https://docs.factory.ai/harness/mcp”
- [claimed-docs] “Connect Jira, Notion, Slack, Linear, PagerDuty, and MCP tools to keep development synchronized with team systems”
Omnaranone0/10Omnara is a platform for launching and managing agents (not itself a coding agent), so the axis of exposing an official MCP server applies. Evidence only shows Omnara connecting outward to MCP servers to give its agents tool access (client-side), with no documentation of Omnara itself running as an MCP server that external agents can connect to.
- [claimed-docs] “Connect MCP servers to give them access to your services and data, or add custom tools for actions in your own application.”
ai-native userUse an official CLI
weight 2 · round to FactoryFactory ships an official Droid CLI with documented quickstart, terminal/editor/Git integration, bash mode, one-shot exec for CI/CD, and slash-command features—clearly an official CLI for AI-native workflows. Missing for 10: independent/hands-on third-party corroboration beyond vendor docs.
- [claimed-docs] “Use the Droid CLI to run Factory from your terminal, editor, tests, and Git workflow.”
- [claimed-docs] “Press ! to toggle bash mode and run shell commands directly without AI interpretation.”
- [claimed-docs] “droid exec runs as a one-shot command that completes a task and exits, making it ideal for CI/CD pipelines, shell scripts, and batch process…”
- [probe] “official CLI documented at https://docs.factory.ai/droid-cli/quickstart”
Docs explicitly mention launching agents 'programmatically with the CLI' alongside REST API and SDK, and a probe confirms an official CLI documented on the quickstart page. Missing for 10: detailed CLI command reference/usage docs and independent hands-on confirmation of CLI functionality.
- [claimed-docs] “Launch your first agent with Omnara via the dashboard, or programmatically with the CLI, the REST API, or the TypeScript SDK.”
- [probe] “official CLI documented at https://docs.omnara.com/quickstart”
ai-native userDrive the product through a documented public API
weight 3 · round to OmnaraFactory documents a public API for creating and driving Droid sessions (factory-docs-13) plus CLI/exec automation paths (droid exec, CI/CD) that give programmatic control, but there's no discoverable OpenAPI/swagger spec (probe found only 404s), suggesting the API reference is thinner or less formally specified than a full REST API story would require. missing for 10: a published OpenAPI/swagger schema, broader API endpoint coverage beyond sessions, independent third-party confirmation of API usage.
- [claimed-docs] “Create and drive Droid sessions: manage their lifecycle, settings, and messages.”
- [claimed-docs] “droid exec runs as a one-shot command that completes a task and exits, making it ideal for CI/CD pipelines, shell scripts, and batch process…”
- [claimed-docs] “Droid exec uses tiered autonomy to control what operations can run without manual confirmation.”
- [probe] “PROBE openapi: all candidate paths 404 (https://docs.factory.ai/openapi.json, https://docs.factory.ai/swagger.json, https://docs.factory.ai/…”
Omnara documents a public REST API with an OpenAPI spec (probed live, HTTP 200), plus a CLI and TypeScript SDK for programmatic launching and control of agents, and describes itself as 'The API for Production-Grade Agents.' Missing for 10: independent third-party developer accounts of building against the API beyond vendor docs/probes.
- [claimed-docs] “Launch your first agent with Omnara via the dashboard, or programmatically with the CLI, the REST API, or the TypeScript SDK.”
- [probe] “PROBE llms.txt: HTTP 200 at https://docs.omnara.com/llms.txt # Omnara - [Introduction](https://docs.omnara.com/introduction.md): The API fo…”
- [probe] “PROBE openapi: HTTP 200 at https://docs.omnara.com/openapi.json — contains "openapi" key”
- [probe] “official CLI documented at https://docs.omnara.com/quickstart”
ai-native userIssue scoped/least-privilege API credentials for an agent
weight 2 · round to OmnaraFactorynone0/10No evidence of scoped or least-privilege API credential/token issuance for agents; docs mention API sessions and integrations (Jira, Slack, MCP) but nothing about credential scoping, permission tiers for API keys, or least-privilege access control. Tiered autonomy (factory-docs-7) governs action confirmation, not credential scoping.
Omnara supports role-based access via 'organization and project roles to users and API keys' distinguishing who can manage, configure, operate, or view agents, which is a form of scoped credential control (omnara-gh-1). However, there is no documentation of fine-grained scoping (e.g., per-resource or per-action API key scopes), token expiration, or least-privilege credential issuance workflows specifically for agents. missing for 10: explicit API key scoping/permission granularity docs, credential expiration/rotation support, independent verification of least-privilege enforcement.
- [github] “Assign organization and project roles to users and API keys. Separate who can manage access, configure agents, operate them, or only view th…”
ai-native userBuild against official SDKs
weight 2 · round to OmnaraFactory documents an API reference for creating/driving Droid sessions (factory-docs-13) and MCP/CLI integration points, suggesting some programmatic access, but there is no evidence of an official language SDK (Python/JS/etc.) and the OpenAPI spec probe returned 404s, indicating thin or unpublished API tooling. missing for 10: no named official SDK packages, no OpenAPI/schema availability, no code samples showing SDK usage.
- [claimed-docs] “Create and drive Droid sessions: manage their lifecycle, settings, and messages.”
- [probe] “PROBE openapi: all candidate paths 404 (https://docs.factory.ai/openapi.json, https://docs.factory.ai/swagger.json, https://docs.factory.ai/…”
- [probe] “official MCP server documented at https://docs.factory.ai/harness/mcp”
Docs and probes confirm an official REST API (OpenAPI spec live), a TypeScript SDK, and a CLI for launching/programming agents, which supports SDK-based building. However, evidence is limited to a single TypeScript SDK with no mention of other language SDKs (e.g., Python), no independent developer corroboration of SDK usage, and community discussion is mostly about UI/pricing rather than SDK development experience. Missing for 10: multi-language SDK coverage, independent hands-on SDK usage reports, deeper SDK reference docs beyond quickstart.
- [claimed-docs] “Launch your first agent with Omnara via the dashboard, or programmatically with the CLI, the REST API, or the TypeScript SDK.”
- [probe] “PROBE openapi: HTTP 200 at https://docs.omnara.com/openapi.json — contains "openapi" key”
- [probe] “official CLI documented at https://docs.omnara.com/quickstart”
ai-native userSubscribe to events via webhooks
weight 2 · round drawnFactorynone0/10No evidence of webhook subscription support; docs mention API sessions, MCP tools, and integrations (Jira, Slack, etc.) but nothing about webhooks for event notifications. missing for 10: any webhook documentation, event types, subscription endpoints, or third-party confirmation of webhook support.
Agentic features
ai-native userSet up automations that run autonomously in the background
weight 2 · round drawndroid exec supports one-shot autonomous runs with tiered autonomy suited for CI/CD, scripts, and batch automation, and the API supports creating/driving Droid sessions programmatically, which enables background automation setups. However, there's no evidence of a scheduling/trigger system (e.g., cron-like or event-driven automations) or a dedicated 'automations' dashboard for persistent background jobs. Missing for 10: native scheduling/triggers for autonomous background runs, independent hands-on confirmation of unattended long-running automations, and a dedicated automations management UI.
- [claimed-docs] “droid exec runs as a one-shot command that completes a task and exits, making it ideal for CI/CD pipelines, shell scripts, and batch process…”
- [claimed-docs] “Droid exec uses tiered autonomy to control what operations can run without manual confirmation.”
- [claimed-docs] “Create and drive Droid sessions: manage their lifecycle, settings, and messages.”
- [claimed-docs] “Use Factory Missions to plan and execute large, multi-feature projects with structured orchestration.”
Docs show agents can be launched programmatically (CLI/API/SDK), run on a connected machine, and persist across crashes/restarts/disconnects, which supports background operation, and users can queue tasks and set approval rules for autonomy. However, the docs emphasize human oversight (pause for approval, live corrections) rather than fully unattended scheduled automation, and no independent evidence confirms agents running autonomously without supervision. Missing for 10: evidence of scheduling/triggers for automation, independent confirmation of successful unattended background runs, and clarity on how 'autonomous' vs 'supervised' operation is enforced.
- [claimed-docs] “Launch your first agent with Omnara via the dashboard, or programmatically with the CLI, the REST API, or the TypeScript SDK.”
- [claimed-docs] “Agents keep their conversation history across crashes, restarts, and machine disconnects, so work can resume without starting a new conversa…”
- [claimed-docs] “Follow an agent’s progress live, send a correction while it’s working, or queue up its next task.”
- [claimed-docs] “You can decide which actions need approval, and agents can pause to ask questions when they need your input.”
- [claimed-docs] “or a machine you connect, such as a laptop or server”
ai-native userDelegate tasks to a built-in AI assistant inside the product
weight 3 · round drawnFactory's core product is built around delegating tasks to Droid agents via App, CLI, or exec, with clear docs on task delegation, custom droids, and session management. missing for 10: independent/hands-on corroboration beyond vendor docs, and quantified reliability of delegated task completion.
- [claimed-docs] “Delegate a task, review the diff, and merge from the App or your terminal.”
- [claimed-docs] “Use the Droid CLI to run Factory from your terminal, editor, tests, and Git workflow.”
- [claimed-docs] “delegate scoped tasks to Custom Droids via /droids, and package reusable procedures as Skills with /skills or /create-skill”
- [claimed-docs] “droid exec runs as a one-shot command that completes a task and exits, making it ideal for CI/CD pipelines, shell scripts, and batch process…”
- [claimed-docs] “Create and drive Droid sessions: manage their lifecycle, settings, and messages.”
Omnara's core product function is launching and delegating tasks to configurable AI agents (via dashboard, CLI, API, or SDK), which run using pluggable models (OpenAI, Anthropic, etc.), retain conversation history, and let users follow progress, send corrections, queue tasks, and approve actions — a textbook 'delegate to built-in AI assistant' workflow. Hands-on community reactions corroborate real usage and enthusiasm for this delegation workflow (e.g. 'using Claude Code from your phone is kind of great', 'this is genius, how I've wanted AI agents to work'). Missing for 10: no rigorous independent benchmark of assistant quality/reliability, and some community friction (UI copy bugs, GitHub OAuth errors) suggests rough edges in the delegation experience.
- [claimed-docs] “Launch your first agent with Omnara via the dashboard, or programmatically with the CLI, the REST API, or the TypeScript SDK.”
- [claimed-docs] “Use models from OpenAI, Anthropic, OpenRouter, or Amazon Bedrock, or connect a compatible model you host yourself.”
- [claimed-docs] “Follow an agent’s progress live, send a correction while it’s working, or queue up its next task.”
- [claimed-docs] “You can decide which actions need approval, and agents can pause to ask questions when they need your input.”
- [community] “For the skeptics: using Claude Code from your phone is kind of great. Think this sort of solution is excellent once you've figured out a goo…”
- [community] “Ok now this is genius, and how I've wanted AI agents to work for a while now. Gonna try this out!”
- [community] “There's a lot of negative feedback in this thread, so let me say I'm really excited to try this!... Omnara looks much more configurable and …”
ai-native userOperate the product with natural-language commands
weight 2 · round to FactoryFactory's Droid CLI and App let users delegate tasks, run commands, and manage workflows via natural-language prompts, with bash-mode fallback and slash commands (/droids, /skills, /readiness-report) layered on top of NL interaction, showing deep agentic natural-language operation across CLI, exec, and Missions. missing for 10: independent/hands-on user reports confirming NL command quality and reliability outside vendor docs.
- [claimed-docs] “Use the Droid CLI to run Factory from your terminal, editor, tests, and Git workflow.”
- [claimed-docs] “Press ! to toggle bash mode and run shell commands directly without AI interpretation.”
- [claimed-docs] “delegate scoped tasks to Custom Droids via /droids, and package reusable procedures as Skills with /skills or /create-skill”
- [claimed-docs] “droid exec runs as a one-shot command that completes a task and exits, making it ideal for CI/CD pipelines, shell scripts, and batch process…”
- [claimed-docs] “Use Factory Missions to plan and execute large, multi-feature projects with structured orchestration.”
- [probe] “official CLI documented at https://docs.factory.ai/droid-cli/quickstart”
Omnara's core interaction model is natural-language messaging with agents — users can 'send a correction while it's working, or queue up its next task' and agents 'pause to ask questions' for user input in natural language, which is the primary way users operate the product day-to-day. Missing for 10: independent/hands-on confirmation of the NL interface specifically (community threads focus on UI/pricing/tunneling, not command quality), and no detail on how broad or robust the NL command surface is beyond corrections/queuing.
- [claimed-docs] “Follow an agent’s progress live, send a correction while it’s working, or queue up its next task.”
- [claimed-docs] “You can decide which actions need approval, and agents can pause to ask questions when they need your input.”
- [claimed-docs] “You can also attach images and documents to a conversation.”
- [claimed-docs] “Launch your first agent with Omnara via the dashboard, or programmatically with the CLI, the REST API, or the TypeScript SDK.”
Api quality
ai-native userExplore an interactive API reference with runnable examples
weight 2 · round to OmnaraFactorynone0/10Factory does have an API reference section (docs.factory.ai/api-reference/sessions) but there's no evidence of an interactive, runnable-example explorer — the probe for standard OpenAPI/Swagger interactive docs (openapi.json, swagger.json, etc.) returned 404 at all candidate paths, suggesting no live 'try it' interface exists.
- [claimed-docs] “Create and drive Droid sessions: manage their lifecycle, settings, and messages.”
- [probe] “PROBE openapi: all candidate paths 404 (https://docs.factory.ai/openapi.json, https://docs.factory.ai/swagger.json, https://docs.factory.ai/…”
An OpenAPI spec is confirmed live (openapi.json) and docs.omnara.com hosts structured API/quickstart pages, suggesting a machine-readable API reference exists, but no evidence confirms an interactive 'try it now' console or runnable code examples within the docs. Missing for 10: explicit documentation or screenshots of an interactive API playground, confirmation of runnable/executable examples, and independent user reports of using such a feature.
- [probe] “PROBE openapi: HTTP 200 at https://docs.omnara.com/openapi.json — contains "openapi" key”
- [probe] “PROBE llms.txt: HTTP 200 at https://docs.omnara.com/llms.txt # Omnara - [Introduction](https://docs.omnara.com/introduction.md): The API fo…”
- [claimed-docs] “Launch your first agent with Omnara via the dashboard, or programmatically with the CLI, the REST API, or the TypeScript SDK.”
ai-native userDownload a machine-readable API spec (OpenAPI or equivalent)
weight 2 · round to OmnaraFactorynone0/10Factory has an API reference (sessions endpoints) but probes for standard OpenAPI/swagger spec locations all returned 404, indicating no downloadable machine-readable spec is published; missing for 10: any accessible OpenAPI/swagger JSON file or equivalent machine-readable spec.
- [probe] “PROBE openapi: all candidate paths 404 (https://docs.factory.ai/openapi.json, https://docs.factory.ai/swagger.json, https://docs.factory.ai/…”
- [claimed-docs] “Create and drive Droid sessions: manage their lifecycle, settings, and messages.”
A probe confirms a live, machine-readable OpenAPI spec at docs.omnara.com/openapi.json returning a valid 'openapi' key, alongside a documented REST API and SDKs referenced in quickstart docs. Missing for 10: no independent/community corroboration of developers actually consuming the spec.
- [probe] “PROBE openapi: HTTP 200 at https://docs.omnara.com/openapi.json — contains "openapi" key”
- [claimed-docs] “Launch your first agent with Omnara via the dashboard, or programmatically with the CLI, the REST API, or the TypeScript SDK.”
- [probe] “PROBE llms.txt: HTTP 200 at https://docs.omnara.com/llms.txt # Omnara - [Introduction](https://docs.omnara.com/introduction.md): The API fo…”
ai-native userRely on versioned APIs with a documented deprecation policy
weight 2 · round drawnFactorynone0/10There is an API reference (sessions) but no evidence of API versioning scheme or a documented deprecation policy; the OpenAPI spec probe even returned 404s across candidate paths, suggesting no formal versioned spec is exposed.
- [claimed-docs] “Create and drive Droid sessions: manage their lifecycle, settings, and messages.”
- [probe] “PROBE openapi: all candidate paths 404 (https://docs.factory.ai/openapi.json, https://docs.factory.ai/swagger.json, https://docs.factory.ai/…”
Omnaranone0/10No evidence of API versioning scheme or documented deprecation policy; the OpenAPI spec exists (omnara-probe-2) but nothing addresses version numbers, backward compatibility guarantees, or deprecation timelines. Missing for 10: versioned API endpoints/headers, published deprecation policy, changelog entries about breaking changes.
- [probe] “PROBE openapi: HTTP 200 at https://docs.omnara.com/openapi.json — contains "openapi" key”
Automation depth — how much of the product can run unattendedAutomation depth
How much of the product can run unattended
ai-native userPerform bulk operations across many items at once
weight 2 · round to Factorydroid exec is explicitly documented as a one-shot command 'ideal for CI/CD pipelines, shell scripts, and batch processing,' and the API lets users programmatically create/drive many Droid sessions, both enabling bulk automation across items. However, there's no explicit example, docs, or tooling demonstrating a native 'run across many files/repos/tickets at once' bulk operation feature — it relies on the user scripting droid exec in loops rather than a built-in bulk-operation UI/command. Missing for 10: a dedicated bulk-operation command or documented multi-item batch workflow example, and independent/hands-on evidence of it working at scale.
- [claimed-docs] “droid exec runs as a one-shot command that completes a task and exits, making it ideal for CI/CD pipelines, shell scripts, and batch process…”
- [claimed-docs] “Create and drive Droid sessions: manage their lifecycle, settings, and messages.”
- [claimed-docs] “Use Factory Missions to plan and execute large, multi-feature projects with structured orchestration.”
ai-native userDefine rules that trigger actions automatically on events
weight 3 · round to FactoryFactory's droid exec is explicitly designed to run as a one-shot command in CI/CD pipelines, shell scripts, and batch jobs, which implies it can be wired to external events (e.g., git hooks, CI triggers) with tiered autonomy controlling what runs unattended — but this is an execution mode, not a native rule/trigger definition system where a user declares 'on event X, do Y'. Missing for 10: explicit rule/trigger authoring UI or config, built-in event listeners (e.g., webhook triggers, issue-created triggers), and any documented automation-rules engine beyond CI invocation.
- [claimed-docs] “droid exec runs as a one-shot command that completes a task and exits, making it ideal for CI/CD pipelines, shell scripts, and batch process…”
- [claimed-docs] “Droid exec uses tiered autonomy to control what operations can run without manual confirmation.”
- [claimed-docs] “Open Software Factory to see your delivery lifecycle as an automation coverage map.”
Omnaranone0/10Omnara's evidence covers agent launching, conversation persistence, MCP/tool connections, and human-in-the-loop approval gates, but nothing describes a rule engine or event-trigger system where users define conditions that automatically fire actions. The closest item (approval configuration, omnara-docs-8) is about gating agent actions for human review, not automating actions on events.
ai-native userSchedule recurring jobs or workflows
weight 2 · round drawnFactorynone0/10Evidence shows droid exec for CI/CD one-shot automation and API session management, but no documentation of scheduling or recurring/cron-style job execution exists anywhere in the pack. missing for 10: any mention of scheduling, cron, recurring triggers, or timed/repeated workflow execution.
ai-native userVersion, review, and roll back my automations
weight 1 · round to FactoryFactory supports reviewing diffs and merging via git workflow (factory-docs-1, factory-docs-2), and packages automations as reusable Custom Droids/Skills (factory-docs-5), which implies some git-based versioning, but there is no explicit documentation of a version-history feature for Droids/Skills nor an explicit rollback mechanism for automations themselves. missing for 10: explicit versioning UI/history for Skills/Droids, dedicated rollback command or feature distinct from generic git revert.
- [claimed-docs] “Delegate a task, review the diff, and merge from the App or your terminal.”
- [claimed-docs] “Use the Droid CLI to run Factory from your terminal, editor, tests, and Git workflow.”
- [claimed-docs] “delegate scoped tasks to Custom Droids via /droids, and package reusable procedures as Skills with /skills or /create-skill”
Omnaranone0/10Evidence shows agent configs as YAML, conversation history persistence, and approval gating, but nothing about versioning automations, reviewing changes over time, or rolling back to prior configurations/states. Missing for 10: version history for agent configs, diff/review UI for changes, rollback/restore mechanism.
- [claimed-docs] “An agent starts as a small YAML config”
- [claimed-docs] “Agents keep their conversation history across crashes, restarts, and machine disconnects, so work can resume without starting a new conversa…”
- [claimed-docs] “You can decide which actions need approval, and agents can pause to ask questions when they need your input.”
Autonomous implementation — end-to-end implementation by the agent — multi-file changes, task completionAutonomous implementation
End-to-end implementation by the agent — multi-file changes, task completion
End to end feature delivery
developerHave an agent autonomously diagnose and fix a reported bug
weight 3 · round to FactoryFactory's Droid CLI/exec and delegated task workflow support autonomous code changes (diff review and merge) and integrations like Jira/Linear/PagerDuty for bug tracking, plus tiered autonomy for unattended operation, which together plausibly support autonomous bug diagnosis and fixing. However, no evidence explicitly describes an end-to-end 'diagnose a reported bug from ticket to verified fix' workflow or hands-on validation of bug-fixing accuracy. Missing for 10: explicit bug-diagnosis workflow documentation, independent/hands-on evidence of successful autonomous bug fixes, and details on root-cause diagnosis capability.
- [claimed-docs] “Delegate a task, review the diff, and merge from the App or your terminal.”
- [claimed-docs] “droid exec runs as a one-shot command that completes a task and exits, making it ideal for CI/CD pipelines, shell scripts, and batch process…”
- [claimed-docs] “Droid exec uses tiered autonomy to control what operations can run without manual confirmation.”
- [claimed-docs] “Connect Jira, Notion, Slack, Linear, PagerDuty, and MCP tools to keep development synchronized with team systems”
- [claimed-docs] “Automatically fix failing criteria with the /readiness-fix slash command in a Factory App web or desktop session, or in the Droid CLI”
Omnaranone0/10Omnara's evidence describes it as an orchestration/remote-monitoring layer for launching, tracking, and approving agent sessions (YAML config, model connections, MCP tools, live following/correction) rather than an agent that itself performs autonomous bug diagnosis and code fixes; community comments frame it as a wrapper around external coding agents like Claude Code rather than an implementer of fixes. No evidence shows Omnara autonomously diagnosing or resolving a reported bug end-to-end.
- [claimed-docs] “An agent starts as a small YAML config”
- [claimed-docs] “Launch your first agent with Omnara via the dashboard, or programmatically with the CLI, the REST API, or the TypeScript SDK.”
- [claimed-docs] “Connect MCP servers to give them access to your services and data, or add custom tools for actions in your own application.”
- [community] “For the skeptics: using Claude Code from your phone is kind of great. Think this sort of solution is excellent once you've figured out a goo…”
- [community] “My problem isn't running the agents, I have an SSH terminal that supports tabs on my phone. My problem is QAing and reviewing the code all t…”
product-managerGo from a mockup or design to a working implementation without an engineering handoff
weight 2 · round to FactoryFactory's agent-readiness docs explicitly describe the exact scenario: "A designer shares a mockup, and the system implements it without handoffs" (factory-docs-8), backed by delegation/review/merge workflow (factory-docs-1) and readiness tooling (factory-docs-9, factory-docs-10) that lets non-engineers trigger and validate implementation. Missing for 10: independent or hands-on corroboration of a PM-specific end-to-end mockup-to-merge case study, and detail on how a non-technical PM reviews/approves the diff without engineering involvement.
- [claimed-docs] “A designer shares a mockup, and the system implements it without handoffs.”
- [claimed-docs] “Delegate a task, review the diff, and merge from the App or your terminal.”
- [claimed-docs] “Run the /readiness-report slash command in a Factory App web or desktop session, or in the Droid CLI, to evaluate a repository's readiness l…”
- [claimed-docs] “Automatically fix failing criteria with the /readiness-fix slash command in a Factory App web or desktop session, or in the Droid CLI”
Omnaranone0/10Omnara is an orchestration/monitoring layer for launching and supervising coding agents (via YAML config, CLI, API, dashboard) rather than a design-to-code tool; there is no evidence of interpreting mockups/designs into implementations, and setup (YAML config, CLI, API keys) implies engineering involvement rather than a no-handoff PM workflow. The only tangential evidence is that images/documents can be attached to a conversation, which does not demonstrate a mockup-to-implementation pipeline.
- [claimed-docs] “You can also attach images and documents to a conversation.”
- [claimed-docs] “An agent starts as a small YAML config”
- [claimed-docs] “Launch your first agent with Omnara via the dashboard, or programmatically with the CLI, the REST API, or the TypeScript SDK.”
developerHave an agent implement a requested feature end-to-end, including writing tests
weight 3 · round to FactoryFactory's docs show agents can be delegated end-to-end feature tasks (delegate, diff review, merge), run in terminal/CI with test execution and git workflow, and orchestrate multi-feature projects via Missions, but no explicit first-party evidence confirms the agent autonomously writes tests as part of implementation. missing for 10: explicit documentation of test-writing behavior, independent/hands-on verification of end-to-end feature delivery.
- [claimed-docs] “Delegate a task, review the diff, and merge from the App or your terminal.”
- [claimed-docs] “Use the Droid CLI to run Factory from your terminal, editor, tests, and Git workflow.”
- [claimed-docs] “droid exec runs as a one-shot command that completes a task and exits, making it ideal for CI/CD pipelines, shell scripts, and batch process…”
- [claimed-docs] “Use Factory Missions to plan and execute large, multi-feature projects with structured orchestration.”
Omnara lets you launch and remotely manage coding agents (e.g., Claude Code, Codex) that can presumably write code, and community comments imply these agents 'write code' end-to-end, but there is no documentation or evidence specifically describing an agent completing a full feature implementation including test-writing within Omnara itself — it's positioned as an orchestration/monitoring layer, not the coding agent doing the work. missing for 10: explicit documentation or case study of an agent autonomously implementing a feature end-to-end, evidence of test generation/execution, and confirmation that Omnara (vs. the underlying model) performs the actual coding.
- [claimed-docs] “Launch your first agent with Omnara via the dashboard, or programmatically with the CLI, the REST API, or the TypeScript SDK.”
- [claimed-docs] “Follow an agent’s progress live, send a correction while it’s working, or queue up its next task.”
- [community] “My problem isn't running the agents, I have an SSH terminal that supports tabs on my phone. My problem is QAing and reviewing the code all t…”
Environment setup
developerHave an agent automatically clone the repo, install dependencies, and configure its own working environment
weight 2 · round to FactoryFactory's Droid CLI/exec can run shell commands autonomously (bash mode, tiered autonomy for unconfirmed operations) and operates within a repo's terminal/Git workflow, which implies it could run clone/install commands, but no evidence explicitly describes an agent autonomously cloning a repo or bootstrapping its own dev environment from scratch. missing for 10: explicit documentation of automated repo cloning, dependency installation, or environment provisioning as a first-class capability, and any hands-on example showing this workflow.
- [claimed-docs] “Use the Droid CLI to run Factory from your terminal, editor, tests, and Git workflow.”
- [claimed-docs] “Press ! to toggle bash mode and run shell commands directly without AI interpretation.”
- [claimed-docs] “droid exec runs as a one-shot command that completes a task and exits, making it ideal for CI/CD pipelines, shell scripts, and batch process…”
- [claimed-docs] “Droid exec uses tiered autonomy to control what operations can run without manual confirmation.”
Omnaranone0/10Omnara's evidence covers launching, monitoring, and remote-controlling agents (via dashboard/CLI/API/SDK) and connecting to a machine you own, but there's no documentation of the agent autonomously cloning a repo, installing dependencies, or self-configuring its own working environment — that setup appears to remain the developer's responsibility. Missing for 10: any mention of automated repo cloning, dependency installation, or environment bootstrap by the agent itself.
- [claimed-docs] “Launch your first agent with Omnara via the dashboard, or programmatically with the CLI, the REST API, or the TypeScript SDK.”
- [claimed-docs] “or a machine you connect, such as a laptop or server”
- [claimed-docs] “An agent starts as a small YAML config”
Interactive takeover
developerTake over an in-progress agent task in my editor, terminal, or browser to finish or redirect the work
weight 2 · round to OmnaraFactory explicitly supports multi-surface access (App/web, terminal via Droid CLI, browser) and delegating tasks, reviewing diffs, and merging from any of these surfaces, which implies continuity across surfaces. However, there's no explicit documentation of a 'takeover mid-task' handoff flow (e.g., pausing an in-progress session in one surface and resuming/redirecting it live in another) — the evidence shows task delegation and review/merge but not explicit interactive takeover semantics. Missing for 10: explicit documentation of resuming/redirecting an in-progress session across surfaces, and independent/hands-on confirmation of this handoff working smoothly.
- [claimed-docs] “Delegate a task, review the diff, and merge from the App or your terminal.”
- [claimed-docs] “Use the Droid CLI to run Factory from your terminal, editor, tests, and Git workflow.”
- [claimed-docs] “Create and drive Droid sessions: manage their lifecycle, settings, and messages.”
Docs directly describe following an agent live, sending a correction mid-task, and queuing its next task, plus pausing for approval/questions — matching the story's takeover/redirect ask across web/dashboard access; agents also persist across restarts/disconnects so work can be resumed from any connected machine. Community threads corroborate real-world use (e.g., interacting with agents from phone/terminal), though QA/copy friction is noted. Missing for 10: no explicit first-party confirmation of parity across editor, terminal, and browser specifically, and no independent hands-on account of a full mid-task takeover/redirect workflow.
- [claimed-docs] “Follow an agent’s progress live, send a correction while it’s working, or queue up its next task.”
- [claimed-docs] “You can decide which actions need approval, and agents can pause to ask questions when they need your input.”
- [claimed-docs] “Agents keep their conversation history across crashes, restarts, and machine disconnects, so work can resume without starting a new conversa…”
- [claimed-docs] “or a machine you connect, such as a laptop or server”
- [community] “For the skeptics: using Claude Code from your phone is kind of great. Think this sort of solution is excellent once you've figured out a goo…”
- [community] “My problem isn't running the agents, I have an SSH terminal that supports tabs on my phone. My problem is QAing and reviewing the code all t…”
developerSend follow-up instructions to an active agent session to steer its work without restarting
weight 2 · round to OmnaraFactory's API reference explicitly supports creating and driving Droid sessions including sending messages within an active session (factory-docs-13), and the CLI is interactive by nature, implying follow-up prompts are possible without restarting. However, there is no explicit documentation describing mid-task interruption/steering while the agent is actively executing a long-running task (e.g., Missions or droid exec), so the steering-while-running behavior is not directly evidenced. Missing for 10: explicit docs on interrupting/redirecting an in-progress autonomous run, and independent/hands-on confirmation that follow-up messages actually steer ongoing work rather than queue for the next turn.
- [claimed-docs] “Create and drive Droid sessions: manage their lifecycle, settings, and messages.”
- [claimed-docs] “Use the Droid CLI to run Factory from your terminal, editor, tests, and Git workflow.”
- [claimed-docs] “Use Factory Missions to plan and execute large, multi-feature projects with structured orchestration.”
Docs explicitly describe following an agent's progress live, sending a correction while it's working, or queuing its next task, which directly matches steering an active session without restarting; approvals/pause-for-input further support mid-session interaction. Missing for 10: independent hands-on confirmation that mid-session corrections actually steer behavior reliably (community threads focus on UI/pricing, not on validating this specific steering feature).
- [claimed-docs] “Follow an agent’s progress live, send a correction while it’s working, or queue up its next task.”
- [claimed-docs] “You can decide which actions need approval, and agents can pause to ask questions when they need your input.”
- [claimed-docs] “Agents keep their conversation history across crashes, restarts, and machine disconnects, so work can resume without starting a new conversa…”
Sandbox execution
developerHave an agent safely execute code and install dependencies inside an isolated sandbox
weight 3 · round drawnFactorynone0/10Evidence describes tiered autonomy, bash mode, and CI/CD execution (droid exec) but never mentions an isolated sandbox environment for code execution or dependency installation; no container/VM isolation is documented.
Omnaranone0/10Omnara is a control/orchestration layer for launching, monitoring, and messaging agents (via CLI/API/dashboard, connecting to your own machine or model), but nothing in the evidence describes a sandboxed or isolated execution environment for code/dependency installation—agents appear to run on a user-connected machine or laptop/server without mention of isolation, containerization, or sandboxing guarantees. missing for 10: any mention of sandbox/isolation architecture, dependency installation safety, or containerized execution environment.
- [claimed-docs] “Launch your first agent with Omnara via the dashboard, or programmatically with the CLI, the REST API, or the TypeScript SDK.”
- [claimed-docs] “or a machine you connect, such as a laptop or server”
Human oversight — keeping a human in the loop — approvals, checkpoints, interruptsHuman oversight
Keeping a human in the loop — approvals, checkpoints, interrupts
Approval controls
developerConfigure an agent to auto-approve all its actions instead of confirming each one
weight 2 · round drawnFactory's docs confirm 'tiered autonomy' in droid exec that controls what operations run without manual confirmation, implying a configurable auto-approve mode for CI/batch use, but there's no explicit documentation of a full 'auto-approve all actions' toggle or its exact configuration options/flags. missing for 10: explicit config syntax/flag for full auto-approval, independent confirmation of behavior, coverage of auto-approve in interactive (non-exec) sessions.
- [claimed-docs] “droid exec runs as a one-shot command that completes a task and exits, making it ideal for CI/CD pipelines, shell scripts, and batch process…”
- [claimed-docs] “Droid exec uses tiered autonomy to control what operations can run without manual confirmation.”
Docs state you can 'decide which actions need approval, and agents can pause to ask questions when they need your input' (omnara-docs-8), implying configurable approval granularity that could be set to auto-approve everything, but there's no explicit documentation of a dedicated 'approve all' toggle or default-auto-approve mode. missing for 10: explicit doc/example showing an 'auto-approve all actions' setting, independent/hands-on confirmation of this behavior.
- [claimed-docs] “You can decide which actions need approval, and agents can pause to ask questions when they need your input.”
product-managerApprove key agent decisions from my phone while agents continue working
weight 1 · round to OmnaraDocs mention delegating tasks, reviewing diffs, and merging 'from the App or your terminal' and tiered autonomy that gates operations needing confirmation, implying some human-in-the-loop review outside the terminal, but there is no explicit evidence of a mobile/phone app or of approving in-flight agent decisions remotely while agents keep working. Missing for 10: explicit mobile app/phone interface, evidence of asynchronous approval while agent continues running, and independent confirmation of this workflow.
- [claimed-docs] “Delegate a task, review the diff, and merge from the App or your terminal.”
- [claimed-docs] “Droid exec uses tiered autonomy to control what operations can run without manual confirmation.”
Docs explicitly describe mobile-friendly approval workflows: agents pause to ask questions and require approval on configurable actions, users can follow progress live, send corrections, and queue tasks while agents keep working, and community feedback confirms real hands-on use of approving/interacting with agents from a phone. missing for 10: no explicit phone-specific UI documentation or independent review confirming approval-from-phone works reliably at scale, and some community concerns about UI usability (copy/paste) slightly temper confidence.
- [claimed-docs] “Follow an agent’s progress live, send a correction while it’s working, or queue up its next task.”
- [claimed-docs] “You can decide which actions need approval, and agents can pause to ask questions when they need your input.”
- [community] “For the skeptics: using Claude Code from your phone is kind of great. Think this sort of solution is excellent once you've figured out a goo…”
engineering-leadSet tiered autonomy levels controlling what an agent can do without manual confirmation
weight 3 · round drawnFactory explicitly documents 'tiered autonomy' in droid exec to control which operations run without manual confirmation, directly matching the story, but this is scoped to the CI/CD-oriented droid exec mode rather than a broader, configurable set of autonomy tiers across all agent surfaces. missing for 10: detail on specific tier levels/permissions, configuration UI or granular controls, and evidence this applies uniformly across App/CLI sessions, not just droid exec.
- [claimed-docs] “droid exec runs as a one-shot command that completes a task and exits, making it ideal for CI/CD pipelines, shell scripts, and batch process…”
- [claimed-docs] “Droid exec uses tiered autonomy to control what operations can run without manual confirmation.”
Docs state you can decide which actions require approval and agents pause to ask for input when needed, which maps to configurable autonomy/oversight, and org/project roles further separate what different users can configure vs operate (omnara-docs-8, omnara-gh-1). However there's no explicit documentation of named 'tiers' or granular policy configuration, and no community/hands-on evidence validating this in practice. Missing for 10: explicit tiered-autonomy policy documentation (e.g., levels/thresholds), examples of configuring per-action approval rules, and independent corroboration that this works as described.
- [claimed-docs] “You can decide which actions need approval, and agents can pause to ask questions when they need your input.”
- [github] “Assign organization and project roles to users and API keys. Separate who can manage access, configure agents, operate them, or only view th…”
Model control
ai-native userHave each task prompt automatically routed to the most suitable underlying model
weight 2 · round drawnFactorynone0/10No evidence describes automatic routing of prompts to the most suitable underlying model; docs mention model-agnostic droid workflows, custom droids, and orchestration but nothing about auto-selecting models per task.
Omnaranone0/10Omnara lets users choose from multiple model providers (OpenAI, Anthropic, OpenRouter, Bedrock, self-hosted) but there is no evidence of automatic routing of prompts to the most suitable model — model selection appears manual/configured per agent, not dynamically routed.
- [claimed-docs] “Use models from OpenAI, Anthropic, OpenRouter, or Amazon Bedrock, or connect a compatible model you host yourself.”
- [claimed-docs] “Bring your own model keys for free”
engineering-leadSwitch away from automatic model selection to a specific model of my choice
weight 1 · round to OmnaraFactorynone0/10No evidence in the pack mentions model selection, automatic model routing, or the ability to choose a specific model over an automatic default; the docs cover CLI usage, integrations, missions, and readiness reports but nothing about model choice controls.
Docs confirm users can choose models from OpenAI, Anthropic, OpenRouter, Bedrock, or self-hosted ones and 'bring your own model keys', implying explicit model selection rather than forced auto-selection, but there is no direct documentation of an automatic model-selection mode being toggled off or a UI/config setting explicitly named 'model selection'. missing for 10: explicit description of a default/automatic model-selection mode and a documented switch/override mechanism, independent confirmation of this workflow.
- [claimed-docs] “Use models from OpenAI, Anthropic, OpenRouter, or Amazon Bedrock, or connect a compatible model you host yourself.”
- [claimed-docs] “Bring your own model keys for free”
Visibility monitoring
developerWatch what a running agent is doing in real time, including its current status
weight 3 · round to OmnaraFactory's session API lets you create and manage Droid sessions—including lifecycle, settings, and messages—implying some ability to track a running agent's state, and the App/CLI let you review diffs as work progresses. However, there is no explicit documentation of a live status dashboard, streaming logs, or real-time progress view of an in-flight agent. Missing for 10: dedicated real-time monitoring UI/stream, explicit 'live status' feature documentation, independent confirmation of live tracking.
- [claimed-docs] “Create and drive Droid sessions: manage their lifecycle, settings, and messages.”
- [claimed-docs] “Delegate a task, review the diff, and merge from the App or your terminal.”
- [claimed-docs] “Use the Droid CLI to run Factory from your terminal, editor, tests, and Git workflow.”
Docs explicitly state you can 'Follow an agent's progress live' and agents 'pause to ask questions when they need your input,' with approval controls for actions, matching real-time monitoring and status visibility. Community threads confirm the product's core use case is remote live monitoring of running agents via a UI/app. Missing for 10: independent hands-on confirmation of the live status UI's accuracy/latency and more detail on what 'status' fields are shown.
- [claimed-docs] “Follow an agent’s progress live, send a correction while it’s working, or queue up its next task.”
- [claimed-docs] “You can decide which actions need approval, and agents can pause to ask questions when they need your input.”
- [community] “For the skeptics: using Claude Code from your phone is kind of great. Think this sort of solution is excellent once you've figured out a goo…”
- [community] “Although I must say that Omnara's UI looks absolutely fantastic. Well done!”
developerGet notified when an agent completes a task or needs my input
weight 2 · round to OmnaraFactory integrates with Slack and PagerDuty and supports tiered autonomy that pauses for manual confirmation, implying some mechanism for alerting developers when input is needed, but there's no explicit documentation of a notification feature for task completion or input requests. missing for 10: explicit notification/alert documentation, evidence of completion pings, confirmation of Slack/PagerDuty being used specifically for task-status alerts.
- [claimed-docs] “Connect Jira, Notion, Slack, Linear, PagerDuty, and MCP tools to keep development synchronized with team systems”
- [claimed-docs] “Droid exec uses tiered autonomy to control what operations can run without manual confirmation.”
- [claimed-docs] “Delegate a task, review the diff, and merge from the App or your terminal.”
Docs describe live progress tracking, mid-task corrections, task queuing, and agents pausing to ask for input/approval — directly matching the notification/human-input story, and mobile/remote use is corroborated by community hands-on comments. Missing for 10: no explicit push/email notification mechanism documented, and no independent verification of notification reliability or latency.
- [claimed-docs] “Follow an agent’s progress live, send a correction while it’s working, or queue up its next task.”
- [claimed-docs] “You can decide which actions need approval, and agents can pause to ask questions when they need your input.”
- [community] “For the skeptics: using Claude Code from your phone is kind of great. Think this sort of solution is excellent once you've figured out a goo…”
- [community] “Ok now this is genius, and how I've wanted AI agents to work for a while now. Gonna try this out!”
Intent to spec — stories about intent to spec in this arenaIntent to spec
Stories about intent to spec in this arena
Natural language task intake
developerDescribe a feature or bug in plain language and have it automatically turned into a scoped implementation task
weight 3 · round to FactoryFactory supports delegating tasks described in plain language (via droid CLI, App, or Missions) which get turned into scoped, executable work with diffs to review and merge, and readiness tooling helps scope repo work automatically. However, there's no explicit documented workflow for turning a raw bug/feature description into a formal 'scoped implementation task' spec artifact (e.g., structured ticket/spec generation before execution) distinct from just running the agent. missing for 10: explicit intent-to-spec artifact generation/preview step, independent/hands-on evidence of accurate scoping from vague input, and detail on how ambiguous requests are clarified before execution.
- [claimed-docs] “Delegate a task, review the diff, and merge from the App or your terminal.”
- [claimed-docs] “delegate scoped tasks to Custom Droids via /droids, and package reusable procedures as Skills with /skills or /create-skill”
- [claimed-docs] “A designer shares a mockup, and the system implements it without handoffs.”
- [claimed-docs] “Run the /readiness-report slash command in a Factory App web or desktop session, or in the Droid CLI, to evaluate a repository's readiness l…”
- [claimed-docs] “Use Factory Missions to plan and execute large, multi-feature projects with structured orchestration.”
Omnaranone0/10Omnara's evidence describes launching, monitoring, and queuing tasks for coding agents (YAML configs, live progress, queueing next task, approvals) but nothing shows Omnara itself converting a plain-language feature/bug description into a scoped implementation task or spec — that logic would live in the underlying agent model, not in Omnara's own product surface.
- [claimed-docs] “Launch your first agent with Omnara via the dashboard, or programmatically with the CLI, the REST API, or the TypeScript SDK.”
- [claimed-docs] “Follow an agent’s progress live, send a correction while it’s working, or queue up its next task.”
- [claimed-docs] “An agent starts as a small YAML config”
developerAttach a marked-up screenshot or mockup to a task so the agent implements the correct visual change
weight 2 · round to OmnaraFactorynone0/10Evidence only mentions a designer sharing a mockup conceptually (factory-docs-8), with no documented capability to attach an image/screenshot with markup annotations to a task or session, nor any multimodal image-input feature described.
- [claimed-docs] “A designer shares a mockup, and the system implements it without handoffs.”
Omnara supports attaching images and documents to a conversation (omnara-docs-9), which could technically allow attaching a marked-up screenshot, but there is no evidence of markup/annotation tooling, task-specific attachment linking, or the agent explicitly interpreting visual diffs to implement UI changes. missing for 10: annotation/markup tooling, task-level (not just conversation-level) attachment workflow, evidence the agent actually parses/acts on visual markup, independent confirmation of this use case.
- [claimed-docs] “You can also attach images and documents to a conversation.”
Plan approval
developerReview and approve an agent's implementation plan before any code changes are made
weight 3 · round drawnFactory's tiered autonomy in droid exec implies some operations require manual confirmation before running, and Missions offer 'plan and execute' orchestration, suggesting a planning phase exists, but there is no explicit documentation of a discrete implementation-plan artifact that a developer reviews and approves before any code changes are made — the described workflow (docs-1) instead centers on reviewing the diff/output after changes. missing for 10: explicit plan-approval UI/step description, evidence of a pre-execution plan artifact, confirmation that no code is touched until plan is approved.
- [claimed-docs] “Droid exec uses tiered autonomy to control what operations can run without manual confirmation.”
- [claimed-docs] “Use Factory Missions to plan and execute large, multi-feature projects with structured orchestration.”
- [claimed-docs] “Delegate a task, review the diff, and merge from the App or your terminal.”
Omnara lets you configure which agent actions require approval and lets agents pause to ask for input, which could be used to gate on a plan before code changes, but there is no documented feature specifically for presenting/reviewing an implementation plan before code is written. Community feedback (omnara-comm-5) explicitly notes that reviewing/QAing agent work remains unsolved with tools like this, casting doubt on how robust the approval gate is for plan review. Missing for 10: dedicated plan-approval workflow/UI, evidence that approvals occur pre-implementation rather than mid-execution, and independent confirmation the feature works as intended.
- [claimed-docs] “You can decide which actions need approval, and agents can pause to ask questions when they need your input.”
- [community] “My problem isn't running the agents, I have an SSH terminal that supports tabs on my phone. My problem is QAing and reviewing the code all t…”
engineering-leadApprove a task's scope and contract before an agent is allowed to modify the repository
weight 2 · round drawnFactory documents tiered autonomy that limits what operations run without manual confirmation and diff review before merge, which implies some human-gate mechanics, but there's no explicit evidence of an engineering-lead approving a task's defined scope/contract *before* the agent is permitted to touch the repository — reviews are framed as post-hoc diff review rather than pre-execution scope sign-off. Missing for 10: explicit scope/contract definition step, an approval gate that blocks agent execution until lead sign-off, and evidence this is lead-specific rather than generic autonomy tiering.
- [claimed-docs] “Delegate a task, review the diff, and merge from the App or your terminal.”
- [claimed-docs] “Droid exec uses tiered autonomy to control what operations can run without manual confirmation.”
Omnara lets you configure which agent actions require approval and pause for input, which offers some pre-modification gating (omnara-docs-8), and role-based permissions can restrict who can operate agents (omnara-gh-1). However there is no documented workflow for an engineering-lead to review and approve a task's overall scope/contract before any repository changes occur — approvals are described as per-action, not a formal spec-review gate. Missing for 10: explicit scope/contract definition and lead sign-off step prior to agent execution, evidence of blocking all repo writes until approval, and independent confirmation this approval flow works as intended.
- [claimed-docs] “You can decide which actions need approval, and agents can pause to ask questions when they need your input.”
- [github] “Assign organization and project roles to users and API keys. Separate who can manage access, configure agents, operate them, or only view th…”
Ticket driven tasking
developerAssign a coding task to an agent directly from an existing issue or ticket
weight 3 · round to FactoryFactory documents integrations with Jira and Linear ('Connect Jira, Notion, Slack, Linear, PagerDuty, and MCP tools to keep development synchronized with team systems'), implying tickets can feed into agent workflows, but there is no explicit documentation of assigning/triggering a droid task directly from an issue or ticket UI. missing for 10: explicit workflow/screenshot showing task creation from a Jira/Linear ticket, confirmation that ticket assignment auto-triggers an agent session, independent corroboration.
- [claimed-docs] “Connect Jira, Notion, Slack, Linear, PagerDuty, and MCP tools to keep development synchronized with team systems”
Omnaranone0/10Evidence covers launching agents via dashboard/CLI/API/SDK, skills, MCP tools, and approvals, but nothing describes assigning a task directly from an existing GitHub issue, Jira ticket, or similar tracker item. A GitHub OAuth connection is mentioned only in the context of a login error, not issue-to-task assignment. Missing for 10: any documentation or demo of ticket/issue import, issue-linked task creation, or tracker integration triggering agent work.
- [claimed-docs] “Launch your first agent with Omnara via the dashboard, or programmatically with the CLI, the REST API, or the TypeScript SDK.”
- [community] “getting this error trying to connect github: github_unauthorized: GitHub OAuth error: The redirect_uri MUST match the registered callback UR…”
Openness — open source, data portability, and self-hosting storiesOpenness
Open source, data portability, and self-hosting stories
ai-native userDo everything through the API that I can do in the UI
weight 2 · round to OmnaraFactory documents a real API (session creation/lifecycle) alongside its CLI and UI, showing some functional overlap, but there's no OpenAPI spec (404s on all candidate URLs) and no evidence that UI-exclusive features like Missions, Software Factory's coverage map, or readiness-report/fix slash commands are exposed via the API. missing for 10: documented API parity for Missions, Software Factory, and readiness commands; a public OpenAPI/schema confirming full surface coverage.
- [claimed-docs] “Create and drive Droid sessions: manage their lifecycle, settings, and messages.”
- [claimed-docs] “Use Factory Missions to plan and execute large, multi-feature projects with structured orchestration.”
- [claimed-docs] “Open Software Factory to see your delivery lifecycle as an automation coverage map.”
- [claimed-docs] “Run the /readiness-report slash command in a Factory App web or desktop session, or in the Droid CLI, to evaluate a repository's readiness l…”
- [claimed-docs] “Automatically fix failing criteria with the /readiness-fix slash command in a Factory App web or desktop session, or in the Droid CLI”
- [probe] “PROBE openapi: all candidate paths 404 (https://docs.factory.ai/openapi.json, https://docs.factory.ai/swagger.json, https://docs.factory.ai/…”
Omnara explicitly markets itself as 'The API for Production-Grade Agents' with a published OpenAPI spec, and docs state you can launch, manage, and interact with agents via dashboard, CLI, REST API, or TypeScript SDK — indicating API-first parity with the UI. Role-based API keys (omnara-gh-1) further support programmatic control equivalent to UI actions like approvals and configuration. Missing for 10: an explicit statement or independent confirmation that every UI feature (e.g., approvals, skills, image/document attachments) has a documented 1:1 API equivalent, and hands-on verification beyond the OpenAPI probe.
- [claimed-docs] “Launch your first agent with Omnara via the dashboard, or programmatically with the CLI, the REST API, or the TypeScript SDK.”
- [probe] “PROBE llms.txt: HTTP 200 at https://docs.omnara.com/llms.txt # Omnara - [Introduction](https://docs.omnara.com/introduction.md): The API fo…”
- [probe] “PROBE openapi: HTTP 200 at https://docs.omnara.com/openapi.json — contains "openapi" key”
- [github] “Assign organization and project roles to users and API keys. Separate who can manage access, configure agents, operate them, or only view th…”
ai-native userExport all of my data in open formats and leave
weight 3 · round to OmnaraFactorynone0/10No evidence addresses data export or portability in open formats, nor any account-deletion/data-takeout mechanism; the docs focus on session management, CLI, and integrations, not exporting user data to leave the platform.
Omnara is open-source (Apache 2.0) and self-hostable, and self-hosted deployments can query agent history directly in Postgres for analytics/training datasets, which effectively enables data portability and exit without lock-in. However, there is no explicit documented 'export my data' feature, bulk export tool, or open-format export guarantee for the hosted/SaaS offering. Missing for 10: an explicit data-export mechanism/API endpoint for hosted users, documentation confirming full data portability in open formats, and independent confirmation of successful export/migration.
- [claimed-docs] “Omnara — The open-source alternative to Claude Managed Agents”
- [claimed-docs] “Free to self-host and build on. Open source under Apache 2.0.”
- [github] “Self-hosted deployments can query agent history directly in Postgres for analytics, evals, prompt analysis, and training datasets.”
ai-native userRead the product's source under an open license
weight 2 · round to OmnaraFactorynone0/10No evidence of an open-source license or publicly available source code for Factory/Droid; all evidence points to closed docs and a proprietary CLI/platform. missing for 10: any license file, GitHub repo, or open-source statement covering the product's source code.
Omnara explicitly markets itself as 'the open-source alternative to Claude Managed Agents' and states it is 'Free to self-host and build on. Open source under Apache 2.0,' with a public GitHub repository confirming the source is available. Missing for 10: independent verification of full license scope/completeness (e.g., some components possibly closed-source) and community confirmation of actual code openness beyond vendor claims.
- [claimed-docs] “Omnara — The open-source alternative to Claude Managed Agents”
- [claimed-docs] “Free to self-host and build on. Open source under Apache 2.0.”
- [github] “Assign organization and project roles to users and API keys. Separate who can manage access, configure agents, operate them, or only view th…”
- [github] “Self-hosted deployments can query agent history directly in Postgres for analytics, evals, prompt analysis, and training datasets.”
ai-native userSelf-host the core product
weight 3 · round to OmnaraFactorynone0/10Factory is presented as a cloud-hosted platform (Factory App, Droid CLI connecting to hosted services, API sessions) with no evidence of a self-hostable core server or on-prem deployment option anywhere in the docs or probes.
Omnara is explicitly positioned as open-source (Apache 2.0) and 'free to self-host and build on,' with a public GitHub repo and self-hosted deployment details (e.g., querying Postgres directly for analytics) confirming the capability exists in practice, not just marketing. Missing for 10: independent/hands-on report of someone actually self-hosting and confirming setup ease or parity with the hosted SaaS version.
- [claimed-docs] “Omnara — The open-source alternative to Claude Managed Agents”
- [claimed-docs] “Free to self-host and build on. Open source under Apache 2.0.”
- [github] “Assign organization and project roles to users and API keys. Separate who can manage access, configure agents, operate them, or only view th…”
- [github] “Self-hosted deployments can query agent history directly in Postgres for analytics, evals, prompt analysis, and training datasets.”
Pricing limits — free-tier ceilings, usage caps, and rate limits before you have to payPricing limits
Free-tier ceilings, usage caps, and rate limits before you have to pay
Enterprise licensing
engineering-leadLicense an enterprise deployment with SSO and commercial support for organization-wide rollout
weight 2 · round drawnFactorynone0/10No evidence pack items mention enterprise licensing, SSO, or commercial support offerings; documentation only covers product features like Droid CLI, MCP integration, and readiness reports. Missing for 10: any mention of SSO/SAML, enterprise tier, commercial support SLAs, or org-wide licensing terms.
Omnaranone0/10Evidence shows role-based access control and self-hosted deployments, but there is no mention of SSO, enterprise licensing, or commercial support offerings anywhere in the pack; pricing info only covers free self-hosting and a consumer subscription tier.
- [github] “Assign organization and project roles to users and API keys. Separate who can manage access, configure agents, operate them, or only view th…”
- [claimed-docs] “Free to self-host and build on. Open source under Apache 2.0.”
- [community] “Feels expensive for something that an engineer can hack in a couple of ours with tailscale and Claude Code. Has potential though. At $9 I'd …”
Model flexibility
engineering-leadBring my own LLM or API key so agents run on the model of my choice
weight 2 · round to OmnaraFactorynone0/10No evidence pack item mentions bringing your own LLM, custom API keys, or model selection/configuration options; all docs focus on CLI, integrations, and workflow features.
Docs explicitly state support for OpenAI, Anthropic, OpenRouter, Bedrock, or self-hosted compatible models, and pricing page confirms 'Bring your own model keys for free.' missing for 10: independent/hands-on confirmation of BYO-key setup working in practice, and details on per-model configuration granularity.
- [claimed-docs] “Use models from OpenAI, Anthropic, OpenRouter, or Amazon Bedrock, or connect a compatible model you host yourself.”
- [claimed-docs] “Bring your own model keys for free”
- [claimed-docs] “Free to self-host and build on. Open source under Apache 2.0.”
Usage quotas
engineering-leadSee and manage plan-based daily task and concurrency limits for agent workflows
weight 2 · round drawnFactorynone0/10No evidence of plan-based daily task limits, concurrency caps, or admin controls for managing such limits; docs cover CLI, sessions, missions, and integrations but nothing on usage/plan limit visibility or management.
Omnaranone0/10No evidence in the pack mentions plan-based daily task limits, concurrency limits, or any dashboard/UI for managing such quotas; pricing docs only mention self-hosting being free and 'bring your own model keys' with no concurrency/task-limit controls described.
- [claimed-docs] “Free to self-host and build on. Open source under Apache 2.0.”
- [claimed-docs] “Bring your own model keys for free”
Privacy posture — data-handling and privacy storiesPrivacy posture
Data-handling and privacy stories
ai-native userChoose where my data is stored (region/residency)
weight 2 · round drawnFactorynone0/10No evidence in the pack addresses data residency, regional storage options, or compliance/data-location controls; all evidence covers agent workflows, CLI, and integrations unrelated to data storage location.
Omnaranone0/10No evidence of any data residency/region selection feature; Omnara is self-hostable (Apache 2.0) which could give indirect control over storage location, but there's no documented region-choice option for the hosted service, and community comments even raise unresolved concerns about data going to a 3rd party and lack of E2E encryption. missing for 10: any documented region/residency selection mechanism, hosted-service data location controls, compliance certifications tied to region.
- [claimed-docs] “Free to self-host and build on. Open source under Apache 2.0.”
- [community] “How is this different from VibeTunnel which is not limited to just Claude and Codex... There's also Happy, Coder/Mux, and so many others tha…”
- [community] “I hope Omnara can be not too far from E2E encryption. The lack of E2E encryption was why I didn't chose Omnara.”
ai-native userPrevent my data from being used to train AI models
weight 3 · round drawnFactorynone0/10No evidence in the pack addresses data usage/training opt-out, privacy policy, or data retention controls; the docs focus entirely on product features like CLI, missions, and integrations.
ai-native userControl data retention and deletion
weight 2 · round to OmnaraFactorynone0/10No evidence pack items address data retention policies, deletion controls, or privacy settings for user data; all citations relate to product features like CLI, MCP integrations, and agent workflows rather than privacy posture.
Omnara offers a self-hosted, open-source deployment where operators can query and manage the Postgres agent-history database directly, implying some control over data retention/deletion, but there is no explicit documented deletion/retention policy, data-export tool, or 'right to be forgotten' feature for the hosted product. Community feedback also raises concerns about lack of end-to-end encryption and visibility of stored messages in the hosted service, suggesting limited user control over cloud-stored data. missing for 10: explicit data-retention policy, a documented deletion/export mechanism, confirmation that hosted-service data can be fully deleted on request.
- [claimed-docs] “Free to self-host and build on. Open source under Apache 2.0.”
- [github] “Self-hosted deployments can query agent history directly in Postgres for analytics, evals, prompt analysis, and training datasets.”
- [community] “If you can see the messages unfortunately thats a deal breaker for me. If its encrypted end-to-end than I'm in.”
- [community] “I hope Omnara can be not too far from E2E encryption. The lack of E2E encryption was why I didn't chose Omnara.”
ai-native userOpt out of telemetry and usage tracking
weight 2 · round drawnFactorynone0/10No evidence in the pack mentions telemetry, usage tracking, data collection, or any opt-out/privacy settings for Factory or the Droid CLI.
Repo integration — stories about repo integration in this arenaRepo integration
Stories about repo integration in this arena
Chat integration
developerTag an agent in a chat thread to discuss and delegate a bug or task
weight 2 · round to OmnaraFactory's docs mention connecting Slack as an integration to 'keep development synchronized with team systems,' implying some chat-based interaction, but there is no explicit evidence of an @-mention/tagging mechanism in a chat thread to discuss or delegate a specific bug/task to the agent. Missing for 10: explicit Slack @droid tagging workflow, thread-based task delegation UI, and confirmation that discussion happens inline in chat rather than just triggering external actions.
- [claimed-docs] “Connect Jira, Notion, Slack, Linear, PagerDuty, and MCP tools to keep development synchronized with team systems”
Omnara supports conversational interaction with an agent — following its progress live, sending corrections, queuing next tasks, and pausing for questions/approvals (omnara-docs-7, omnara-docs-8) — which covers discussing and delegating work, but there is no evidence of an actual '@tag' mechanism inside a shared team chat thread (e.g., Slack/Discord-style mention) for bringing an agent into a conversation. missing for 10: explicit tagging/mention syntax, multi-user shared thread support, and any integration showing agents being pulled into existing chat channels.
- [claimed-docs] “Follow an agent’s progress live, send a correction while it’s working, or queue up its next task.”
- [claimed-docs] “You can decide which actions need approval, and agents can pause to ask questions when they need your input.”
- [claimed-docs] “You can also attach images and documents to a conversation.”
Knowledge context
developerAdd a context file describing my codebase conventions so agents generate more relevant plans and code
weight 3 · round to OmnaraFactorynone0/10The evidence pack does not mention any context file mechanism (e.g., AGENTS.md, .factory config, or similar) for describing codebase conventions to guide agent behavior; it covers CLI usage, integrations, readiness reports, and missions but nothing about persistent repo-convention context files.
Omnara's 'Skills' feature lets users package instructions and supporting files for recurring agent work, which could function like a context file, but there is no explicit documentation describing a dedicated codebase-conventions context file that shapes agent plans/code generation. missing for 10: explicit codebase-conventions context file feature, evidence of it improving plan/code relevance, and any independent confirmation of this use case.
- [claimed-docs] “Skills let you package instructions and supporting files for work your agents do regularly.”
Project management integration
product-managerConnect issue trackers like Jira, Linear, ClickUp, or Monday.com so agents can manage tickets directly
weight 3 · round to FactoryDocs confirm Jira and Linear integration explicitly (factory-docs-4), but ClickUp and Monday.com are not mentioned anywhere in the evidence, and there's no detail on ticket management workflows (creating/updating tickets) beyond 'connecting' systems to sync development. Missing for 10: ClickUp integration, Monday.com integration, concrete ticket-management/CRUD workflows via these connectors, independent corroboration.
- [claimed-docs] “Connect Jira, Notion, Slack, Linear, PagerDuty, and MCP tools to keep development synchronized with team systems”
Version control integration
developerConnect a GitHub repository so an agent can access the code and open pull requests against it
weight 3 · round to FactoryDocs indicate Factory works within Git workflows (delegate tasks, review diffs, merge from App/terminal) and can run in CI/CD via droid exec, implying repo access and PR-opening capability, but there is no explicit documentation describing connecting/authorizing a GitHub repository or an explicit PR-creation feature. Missing for 10: explicit GitHub repo connection/auth flow documentation, explicit 'open pull request' feature description, and independent/hands-on confirmation of PR creation.
- [claimed-docs] “Delegate a task, review the diff, and merge from the App or your terminal.”
- [claimed-docs] “Use the Droid CLI to run Factory from your terminal, editor, tests, and Git workflow.”
- [claimed-docs] “droid exec runs as a one-shot command that completes a task and exits, making it ideal for CI/CD pipelines, shell scripts, and batch process…”
Omnaranone0/10No vendor documentation describes connecting a GitHub repository so an agent can access code and open pull requests; the only concrete evidence is a community report of a GitHub OAuth connection failure (redirect_uri mismatch), with no confirmation that repo access or PR creation actually works.
- [community] “getting this error trying to connect github: github_unauthorized: GitHub OAuth error: The redirect_uri MUST match the registered callback UR…”
developerGrant an agent access to my repositories with a one-click install, without complex setup
weight 2 · round to OmnaraFactorynone0/10Evidence shows Factory integrates with Git workflows, CLI, and external tools like Jira/Slack/MCP, but there is no mention of a one-click GitHub/repo install or simplified OAuth-based repo authorization flow. Setup appears to require CLI installation and configuration rather than a one-click grant.
Omnaradisputedcontradicted3/10There's no explicit vendor documentation describing a 'one-click' repo install flow, but community evidence shows a user attempting to connect GitHub and hitting a concrete OAuth failure (github_unauthorized: redirect_uri mismatch), indicating the integration is not smooth or reliable as implied. Missing for 10: first-party docs describing a one-click repo-connect flow, and confirmation the OAuth issue was resolved.
- [community] “getting this error trying to connect github: github_unauthorized: GitHub OAuth error: The redirect_uri MUST match the registered callback UR…”
- [github] “Assign organization and project roles to users and API keys. Separate who can manage access, configure agents, operate them, or only view th…”
Review quality gates — quality gates on changes — review flow, required checks, merge protectionReview quality gates
Quality gates on changes — review flow, required checks, merge protection
Ci remediation
developerTrigger an agent from CI/CD pipelines to fix a broken build or failing test
weight 2 · round to FactoryFactory explicitly documents `droid exec` as a one-shot CLI command designed for CI/CD pipelines, shell scripts, and batch processing, with tiered autonomy controls for unattended operation — directly enabling triggering an agent from CI to fix builds/tests. Missing for 10: a concrete worked example of a CI pipeline invoking droid exec on a failing test/build, and independent/hands-on corroboration beyond vendor docs.
- [claimed-docs] “droid exec runs as a one-shot command that completes a task and exits, making it ideal for CI/CD pipelines, shell scripts, and batch process…”
- [claimed-docs] “Droid exec uses tiered autonomy to control what operations can run without manual confirmation.”
- [probe] “official CLI documented at https://docs.factory.ai/droid-cli/quickstart”
Omnaranone0/10Omnara offers a REST API/CLI/SDK to launch agents programmatically (omnara-docs-2), which could theoretically be scripted into a CI pipeline, but there is no documentation, example, or community evidence showing an actual CI/CD integration (e.g., GitHub Actions trigger, webhook on failed build/test) for automatically fixing broken builds.
- [claimed-docs] “Launch your first agent with Omnara via the dashboard, or programmatically with the CLI, the REST API, or the TypeScript SDK.”
Diff review
developerConfigure an agent to automatically open a pull request when its task completes
weight 2 · round to FactoryDocs show git workflow integration (droid-cli git workflow, delegate task then review diff and merge, droid exec for CI/CD pipelines) which implies PR-related automation, but there is no explicit documentation of a feature to automatically open a pull request when a task completes. Missing for 10: explicit config/flag for auto-PR creation, first-party example of a droid opening a PR on completion, independent/hands-on confirmation.
- [claimed-docs] “Delegate a task, review the diff, and merge from the App or your terminal.”
- [claimed-docs] “droid exec runs as a one-shot command that completes a task and exits, making it ideal for CI/CD pipelines, shell scripts, and batch process…”
- [claimed-docs] “Use the Droid CLI to run Factory from your terminal, editor, tests, and Git workflow.”
Omnaranone0/10Omnara is a mission-control/dashboard layer for monitoring and interacting with coding agents (session tracking, MCP tools, approvals) but no evidence describes automated PR creation on task completion; community even notes GitHub OAuth integration errors and no PR-automation feature is mentioned.
- [community] “getting this error trying to connect github: github_unauthorized: GitHub OAuth error: The redirect_uri MUST match the registered callback UR…”
- [claimed-docs] “Follow an agent’s progress live, send a correction while it’s working, or queue up its next task.”
- [claimed-docs] “You can decide which actions need approval, and agents can pause to ask questions when they need your input.”
developerReview a diff of an agent's changes and approve it before it becomes a pull request
weight 3 · round to FactoryDocs explicitly describe the delegate-review-merge workflow ('Delegate a task, review the diff, and merge from the App or your terminal') and tiered autonomy controls that gate what runs without confirmation, supporting a review-before-PR gate. However, there is no detailed documentation of the diff review UI itself, approval mechanics, or independent/hands-on confirmation of this exact flow. Missing for 10: dedicated diff-review UI documentation, explicit approval-step mechanics, and independent verification of the review-before-PR gate.
- [claimed-docs] “Delegate a task, review the diff, and merge from the App or your terminal.”
- [claimed-docs] “Droid exec uses tiered autonomy to control what operations can run without manual confirmation.”
Omnaranone0/10Omnara's docs describe generic 'approve actions' and pause-for-input mechanisms, but there is no evidence of a diff-review UI or an approval gate specifically tied to turning agent changes into a pull request. A community comment on the same HN thread explicitly notes 'my problem is QAing and reviewing the code... none of these tools solves that,' reinforcing the absence of this capability.
- [claimed-docs] “You can decide which actions need approval, and agents can pause to ask questions when they need your input.”
- [community] “My problem isn't running the agents, I have an SSH terminal that supports tabs on my phone. My problem is QAing and reviewing the code all t…”
Scale parallelism — running many jobs at once — concurrency, fleets, queueingScale parallelism
Running many jobs at once — concurrency, fleets, queueing
Concurrent execution
engineering-leadRun many agent tasks concurrently to scale delivery throughput
weight 3 · round to FactoryFactory provides several building blocks for scaling agent work: droid exec is designed for one-shot, CI/CD, and batch processing use cases, the Sessions API lets you create and manage multiple Droid sessions programmatically, and Missions support orchestrating large multi-feature projects with structured coordination. Together these imply the ability to run many concurrent tasks, but no evidence explicitly states a documented concurrency limit, dashboard for tracking many simultaneous droids, or independent case study proving throughput scaling. Missing for 10: explicit concurrency/parallelism guarantees or limits, a multi-task monitoring UI description, and independent/hands-on validation of running many tasks simultaneously.
- [claimed-docs] “droid exec runs as a one-shot command that completes a task and exits, making it ideal for CI/CD pipelines, shell scripts, and batch process…”
- [claimed-docs] “Use Factory Missions to plan and execute large, multi-feature projects with structured orchestration.”
- [claimed-docs] “Create and drive Droid sessions: manage their lifecycle, settings, and messages.”
- [claimed-docs] “Open Software Factory to see your delivery lifecycle as an automation coverage map.”
Omnara lets you launch agents programmatically via CLI/REST/SDK, manage multiple agents with org/project roles, and queue tasks, which implies some capacity for parallel agent operation, but there is no explicit documentation or evidence of running many agents concurrently at scale, dashboards for fleet-wide monitoring, or throughput/performance claims. missing for 10: explicit concurrency/parallelism documentation, scaling benchmarks or case studies, dashboard support for managing many simultaneous agent sessions.
- [claimed-docs] “Launch your first agent with Omnara via the dashboard, or programmatically with the CLI, the REST API, or the TypeScript SDK.”
- [claimed-docs] “Follow an agent’s progress live, send a correction while it’s working, or queue up its next task.”
- [github] “Assign organization and project roles to users and API keys. Separate who can manage access, configure agents, operate them, or only view th…”
- [claimed-docs] “You can decide which actions need approval, and agents can pause to ask questions when they need your input.”
engineering-leadCreate agent sessions on behalf of other users in my organization
weight 2 · round to OmnaraFactorynone0/10The API reference (factory-docs-13) shows session creation/management exists, but nothing in the evidence indicates an org-admin or lead can create/manage sessions on behalf of other specific users within an organization. missing for 10: evidence of org-level user impersonation, admin controls for delegating sessions to teammates, or role-based session ownership across users.
- [claimed-docs] “Create and drive Droid sessions: manage their lifecycle, settings, and messages.”
Omnara supports organization/project roles that let admins manage access and configure or operate agents on behalf of others (omnara-gh-1), which implies some delegated control, but there is no explicit documentation of an engineering-lead directly creating or launching agent sessions attributed to another specific user. Missing for 10: explicit API/CLI support for creating a session 'as' another user, audit/ownership attribution evidence, and any hands-on confirmation of this workflow.
- [github] “Assign organization and project roles to users and API keys. Separate who can manage access, configure agents, operate them, or only view th…”
Deployment flexibility
developerUse a managed cloud offering to run agents without operating my own backend infrastructure
weight 2 · round to FactoryFactory's docs describe a hosted 'Factory App' (web/desktop) where users delegate tasks and review diffs without local setup, plus an API for creating and managing Droid sessions server-side, indicating a managed backend rather than self-hosted infra. Missing for 10: independent/hands-on confirmation of the cloud service's scalability and infrastructure details beyond first-party docs.
- [claimed-docs] “Delegate a task, review the diff, and merge from the App or your terminal.”
- [claimed-docs] “Run the /readiness-report slash command in a Factory App web or desktop session, or in the Droid CLI, to evaluate a repository's readiness l…”
- [claimed-docs] “Automatically fix failing criteria with the /readiness-fix slash command in a Factory App web or desktop session, or in the Droid CLI”
- [claimed-docs] “Create and drive Droid sessions: manage their lifecycle, settings, and messages.”
Omnara offers a dashboard/API/CLI to launch and manage agents (omnara-docs-2, omnara-probe-1/2/3) and a pricing tier separate from self-hosting (omnara-docs-11, omnara-comm-12/14 mention a paid ~$20/mo hosted tunnel/service), implying a managed cloud option exists. However, the product is primarily marketed as an open-source, self-hostable alternative to managed agent platforms, and there's no first-party documentation detailing SLAs, scaling, or infrastructure guarantees of the managed cloud service — community comments even question its value versus self-hosting with Tailscale. missing for 10: explicit vendor documentation of the managed cloud infrastructure's scaling/reliability guarantees, and independent confirmation that users successfully run agents purely via the hosted service without any self-hosted components.
- [claimed-docs] “Launch your first agent with Omnara via the dashboard, or programmatically with the CLI, the REST API, or the TypeScript SDK.”
- [claimed-docs] “Free to self-host and build on. Open source under Apache 2.0.”
- [community] “Not affiliated with that project, but have been using it for a few weeks and it blows every other 'GUI for the CLI agents' I've tried out of…”
- [community] “Feels expensive for something that an engineer can hack in a couple of ours with tailscale and Claude Code. Has potential though. At $9 I'd …”
- [probe] “PROBE llms.txt: HTTP 200 at https://docs.omnara.com/llms.txt # Omnara - [Introduction](https://docs.omnara.com/introduction.md): The API fo…”
engineering-leadSelf-host agent infrastructure locally, in containers, or on my own VMs
weight 2 · round to OmnaraFactorynone0/10No evidence describes self-hosting Factory's agent infrastructure locally, in containers, or on customer-owned VMs; all evidence points to Factory's own cloud-hosted App, CLI, and API rather than a deployable/self-hosted backend.
Omnara is explicitly open-source (Apache 2.0) and documented as 'free to self-host and build on,' with self-hosted deployments able to query Postgres directly for agent history, and agents can run on any connected machine (laptop or server) — consistent with local, container, or VM deployment. Missing for 10: explicit Docker/Kubernetes/VM deployment guides or configs, and independent hands-on confirmation of a successful self-hosted setup.
- [claimed-docs] “Free to self-host and build on. Open source under Apache 2.0.”
- [github] “Self-hosted deployments can query agent history directly in Postgres for analytics, evals, prompt analysis, and training datasets.”
- [claimed-docs] “or a machine you connect, such as a laptop or server”
- [claimed-docs] “Omnara — The open-source alternative to Claude Managed Agents”
Headless automation
developerRun an agent headlessly inside CI/CD pipelines and shell scripts
weight 2 · round to FactoryFactory explicitly documents droid exec as a one-shot headless command designed for CI/CD pipelines, shell scripts, and batch processing, with tiered autonomy controls for unattended operation. Missing for 10: independent/hands-on confirmation of real-world CI pipeline usage beyond first-party docs.
- [claimed-docs] “droid exec runs as a one-shot command that completes a task and exits, making it ideal for CI/CD pipelines, shell scripts, and batch process…”
- [claimed-docs] “Droid exec uses tiered autonomy to control what operations can run without manual confirmation.”
- [claimed-docs] “Use the Droid CLI to run Factory from your terminal, editor, tests, and Git workflow.”
Omnara documents a CLI, REST API and TypeScript SDK for launching agents programmatically (omnara-docs-2, omnara-probe-2/3), which could technically be invoked from a CI/CD pipeline or shell script, but there is no explicit CI/CD example, pipeline integration guide, or non-interactive/headless mode documentation. The product's core design emphasizes live human oversight (following progress, approvals, pausing for questions) rather than unattended automation. Missing for 10: explicit CI/CD pipeline docs/examples, confirmation of a fully non-interactive headless mode, and evidence of shell-script usage patterns.
- [claimed-docs] “Launch your first agent with Omnara via the dashboard, or programmatically with the CLI, the REST API, or the TypeScript SDK.”
- [probe] “PROBE openapi: HTTP 200 at https://docs.omnara.com/openapi.json — contains "openapi" key”
- [probe] “official CLI documented at https://docs.omnara.com/quickstart”
- [claimed-docs] “You can decide which actions need approval, and agents can pause to ask questions when they need your input.”
Not comparable on these axes
ai-native userGet AI-generated insights and suggestions from my data inside the product
weight 2 · not comparableFactoryn/aFactory is an agentic coding platform focused on delegating software development tasks (code diffs, PRs, readiness reports), not a data analytics or BI product that surfaces AI-generated insights/suggestions from a user's own data. This story is a category error for this product type.
Omnaranone0/10Omnara is a platform for launching, monitoring, and controlling AI coding agents (agent orchestration/remote control), but no evidence shows the product itself surfaces AI-generated insights or suggestions derived from a user's own data (e.g., analytics, dashboards summarizing usage, recommendations). The mention of querying agent history in Postgres for analytics is a raw data-access feature for self-hosters, not an AI-generated insight delivered inside the product.
- [github] “Self-hosted deployments can query agent history directly in Postgres for analytics, evals, prompt analysis, and training datasets.”
ai-native userTest against a sandbox environment without touching production data
weight 1 · not comparableFactorynone0/10Factory's evidence covers coding agent workflows (CLI, sessions, MCP, readiness reports) but nothing addresses a sandbox environment for testing separate from production data. No mention of sandbox mode, staging environment, or data isolation guarantees.
Omnaran/aOmnara is an agent orchestration/monitoring platform for launching and managing coding agents, not a data platform or API with production vs sandbox environments; the evidence pack shows no concept of sandbox/test environments distinct from production data. This axis is a category error for this product type.
ai-native userHave an agent automatically generate and run tests to validate its own code changes before proposing them
weight 2 · not comparableFactorynone0/10Docs mention integration with 'tests' as part of Git workflow and bash-mode shell execution, plus readiness-report/readiness-fix commands, but none of this describes an agent autonomously generating and running tests to validate its own code changes before proposing a diff. Missing for 10: explicit documentation of automated test generation, self-validation loop, or evidence droid runs tests as a pre-proposal gate.
- [claimed-docs] “Use the Droid CLI to run Factory from your terminal, editor, tests, and Git workflow.”
- [claimed-docs] “Press ! to toggle bash mode and run shell commands directly without AI interpretation.”
- [claimed-docs] “Run the /readiness-report slash command in a Factory App web or desktop session, or in the Droid CLI, to evaluate a repository's readiness l…”
- [claimed-docs] “Automatically fix failing criteria with the /readiness-fix slash command in a Factory App web or desktop session, or in the Droid CLI”
Omnaran/aOmnara is a remote agent-orchestration/monitoring platform (control, approvals, notifications, session relay) rather than a coding agent itself; it doesn't generate or execute code or tests—it relays the underlying agent's work. Self-generated test authoring/execution is an axis for the coding agent, not this orchestration layer, so it's a category mismatch here.
product-managerConvert user feedback submissions into structured tasks with proposed scope
weight 2 · not comparableFactorynone0/10Factory documents task delegation, Missions planning, and integrations with Jira/Linear/Notion, but nothing in the evidence describes ingesting user feedback submissions and outputting a structured task with proposed scope — this specific PM workflow is not documented.
Omnaran/aOmnara is an agent-orchestration/remote-control platform for coding agents, not a product-feedback or requirements-management tool; nothing in the evidence relates to converting user feedback into structured tasks with proposed scope. This is a wrong-axis story for this product category.
developerQuery generated documentation for any public or private repository
weight 1 · not comparableFactorynone0/10The evidence pack covers Factory's CLI, agent-readiness reports, missions, and integrations, but nothing describes generating or querying documentation for a repository's codebase (e.g., an auto-generated repo wiki/docs feature). This is a plausible capability for a repo-integrated dev tool, so the axis applies, but no supporting evidence exists.
engineering-leadHave failed CI workflows automatically diagnosed and fixed with a proposed pull request
weight 3 · not comparableFactory's droid exec is explicitly designed to run in CI/CD pipelines as a one-shot task with tiered autonomy, and Droid CLI integrates with Git workflows, suggesting the building blocks exist for automating CI fixes. However, there is no direct evidence of a dedicated feature that detects a failed CI workflow, diagnoses the failure, and automatically opens a proposed pull request end-to-end. missing for 10: explicit CI-failure detection/trigger integration, automatic diagnosis-to-PR workflow documentation, and any hands-on/independent proof of this specific use case.
- [claimed-docs] “droid exec runs as a one-shot command that completes a task and exits, making it ideal for CI/CD pipelines, shell scripts, and batch process…”
- [claimed-docs] “Droid exec uses tiered autonomy to control what operations can run without manual confirmation.”
- [claimed-docs] “Use the Droid CLI to run Factory from your terminal, editor, tests, and Git workflow.”
Omnaran/aOmnara is a remote agent-monitoring/mobile-control platform for launching and interacting with coding agents, not a CI/CD integration or code-review tool; there is no mention of CI workflow diagnosis or auto-generated PRs from failed builds. This is a wrong-axis story for this product category.
ai-native userHave incoming issues automatically triaged with severity suggested and routed to the right owner
weight 2 · not comparableFactorynone0/10Factory documents integrations with issue trackers like Jira, Linear, and PagerDuty (factory-docs-4) and generic custom droid/skill automation (factory-docs-5), but there is no evidence of an automated triage workflow that suggests severity or routes issues to an owner. missing for 10: any documented triage feature, severity classification logic, or owner-routing capability.
- [claimed-docs] “Connect Jira, Notion, Slack, Linear, PagerDuty, and MCP tools to keep development synchronized with team systems”
- [claimed-docs] “delegate scoped tasks to Custom Droids via /droids, and package reusable procedures as Skills with /skills or /create-skill”
Omnaran/aOmnara is an agent orchestration/mobile-control platform for launching and monitoring coding agents, not an issue-triage or incident-management tool; there is no concept of incoming issues, severity scoring, or ownership routing in its evidence pack. This axis is a category error for this product type.
engineering-leadHave every pull request automatically reviewed with AI-generated inline comments
weight 3 · not comparableFactorynone0/10Evidence shows Factory's droid CLI/exec can review diffs, be triggered in CI/CD, and connect to Git workflows, but nothing in the pack describes automatic PR review with AI-generated inline comments posted to pull requests. Missing for 10: any documentation of automated PR-triggered review, inline comment generation on PRs, or GitHub/GitLab PR integration specifics.
- [claimed-docs] “Delegate a task, review the diff, and merge from the App or your terminal.”
- [claimed-docs] “droid exec runs as a one-shot command that completes a task and exits, making it ideal for CI/CD pipelines, shell scripts, and batch process…”
- [claimed-docs] “Use the Droid CLI to run Factory from your terminal, editor, tests, and Git workflow.”
Omnaran/aOmnara is an agent-orchestration/mobile-control platform for launching and monitoring AI coding agents, not a PR/code-review tool; there is no evidence of GitHub PR integration, inline diff comments, or review-gate automation. In fact community feedback explicitly notes it does not address code review/QA needs, and a GitHub OAuth connection error is reported.
- [community] “My problem isn't running the agents, I have an SSH terminal that supports tabs on my phone. My problem is QAing and reviewing the code all t…”
- [community] “getting this error trying to connect github: github_unauthorized: GitHub OAuth error: The redirect_uri MUST match the registered callback UR…”
engineering-leadAutomatically fix failing agent-readiness criteria in my repository
weight 1 · not comparableFactory explicitly documents a /readiness-fix slash command that automatically fixes failing agent-readiness criteria, usable from the Factory App or Droid CLI, complementing the /readiness-report diagnostic command. This directly matches the story's ask, though only first-party docs support it with no independent/hands-on corroboration. Missing for 10: independent or hands-on verification that /readiness-fix reliably resolves criteria, and detail on which criteria types it can/cannot auto-fix.
- [claimed-docs] “Run the /readiness-report slash command in a Factory App web or desktop session, or in the Droid CLI, to evaluate a repository's readiness l…”
- [claimed-docs] “Automatically fix failing criteria with the /readiness-fix slash command in a Factory App web or desktop session, or in the Droid CLI”
Omnaran/aOmnara is an agent orchestration/remote-monitoring platform for launching and interacting with coding agents, not a repository review/quality-gate tool that assesses or auto-fixes 'agent-readiness criteria'. This story belongs to a different product category (review-quality-gate/linting tools), so it is a category error for Omnara.
engineering-leadRun a readiness report that evaluates how ready my repository is for autonomous agents
weight 2 · not comparableFactory explicitly documents a `/readiness-report` slash command (in App or Droid CLI) that evaluates a repository's agent-readiness level, plus a companion `/readiness-fix` command to auto-remediate failing criteria, directly matching the engineering-lead's story. Missing for 10: independent/hands-on validation of report contents and no sample report output shown.
- [claimed-docs] “Run the /readiness-report slash command in a Factory App web or desktop session, or in the Droid CLI, to evaluate a repository's readiness l…”
- [claimed-docs] “Automatically fix failing criteria with the /readiness-fix slash command in a Factory App web or desktop session, or in the Droid CLI”
- [claimed-docs] “A designer shares a mockup, and the system implements it without handoffs.”
engineering-leadHave security alerts automatically validated and remediated with an opened pull request
weight 2 · not comparableFactorynone0/10Evidence covers task delegation, CLI automation, readiness reports/fixes, and CI/CD execution, but nothing describes ingesting security alerts (e.g., from a scanner like Dependabot/Snyk) and automatically validating and remediating them via an opened PR. No security-alert-specific workflow or integration is documented.
Omnaran/aOmnara is a platform for remotely monitoring/controlling coding agents, not a security-scanning or SCA/SAST tool; there is no evidence of vulnerability detection, alert triage, or automated PR remediation workflows tied to security findings. This axis (security-alert validation and auto-remediation) belongs to a different product category (e.g., dependency/security scanners), so it does not apply here.