ChatGPT vs Gemini
free-tier · subscription-flat · subscription-per-seat · enterprise-custom
·free-tier · subscription-flat
ChatGPT wins · 29–9 (7 drawn)
Agenticness — how well agents can access and operate the productAgenticness
How well agents can access and operate the product
Agent access
ai-native userPoint an agent at llms.txt or agent-oriented docs
weight 2 · round to ChatGPTChatGPT's Browser and MCP features let a user direct the agent to fetch and use third-party documentation or websites (e.g. Context7 for developer docs, browsing arbitrary URLs), which would technically allow pointing it at an llms.txt or agent-oriented doc page. The evidence pack also shows ChatGPT's own docs site publishes an llms.txt (probe-1), indicating familiarity with the convention, but there is no explicit documented feature or example of a user instructing ChatGPT to consume an llms.txt file specifically. Missing for 10: explicit product support/example for llms.txt ingestion, confirmation that browsing normalizes/parses such files as agent-context rather than generic web content, independent hands-on verification.
- [probe] “PROBE llms.txt: HTTP 200 at https://learn.chatgpt.com/llms.txt # Codex > Build with the Codex CLI, IDE extension, and cloud automation to s…”
- [claimed-docs] “Browser lets ChatGPT open websites, gather current information, and take action while you stay in control.”
- [claimed-docs] “Use it to give ChatGPT or Codex access to third-party documentation, or to let it interact with developer tools like your browser or Figma.”
- [claimed-docs] “Model Context Protocol (MCP) connects models to tools and context. Use it to give ChatGPT or Codex access to third-party documentation”
- [claimed-docs] “to add Context7 (a free MCP server for developer documentation)”
Gemininone0/10No evidence that Gemini can be pointed at llms.txt or agent-oriented documentation formats; the evidence only covers uploading files, GitHub repos, and connecting to apps, none of which address consuming llms.txt or agent-facing docs conventions. Missing for 10: any mention of llms.txt support, agent-oriented doc parsing, or a documented workflow for feeding agent-discovery docs to Gemini.
ai-native userRun the product headlessly / in CI for automation
weight 2 · round to ChatGPTChatGPT's Codex offers a CLI and SDK explicitly documented for headless/CI use ('Use the Codex SDK to automate coding tasks, including jobs in CI'), plus CLI flags (--search, /fast) and a bulk-scan command (npx @openai/codex-security bulk-scan) that support non-interactive automation workflows. missing for 10: independent/hands-on confirmation of actual CI pipeline runs, and more detail on authentication/headless setup specifics for CI environments.
- [claimed-docs] “Use the Codex SDK to automate coding tasks, including jobs in CI.”
- [claimed-docs] “Inspect code, make changes, run commands, and automate repeatable work without leaving your terminal.”
- [claimed-docs] “Use its command-line interface (CLI) to scan repositories you own or have permission to assess, review findings over time”
- [claimed-docs] “Use `npx @openai/codex-security bulk-scan` to review repositories in one campaign.”
- [claimed-docs] “Use `/fast on`, `/fast off`, or `/fast status` in the CLI to change or inspect the current setting.”
- [claimed-docs] “In the CLI, pass `--search` to fetch live results for one run”
Gemininone0/10The evidence pack is dominated by consumer-facing Gemini Apps documentation (chat, image generation, Gemini Live, notebooks) and contains no first-party documentation of a headless mode, CI integration, or automation-friendly API/CLI usage. The only hint of a CLI tool appears in community complaints (gemini-comm-14, gemini-comm-15) about login failures for Workspace accounts trying to use 'Gemini CLI' for software development, which does not establish that headless/CI automation works.
- [community] “I love how fragmented Google's Gemini offerings are. I'm a Pro subscriber, but I now learn I should be a 'Gemini Code Assist Standard or Ent…”
- [community] “'Failed to login. Ensure your Google account is not a Workspace account.' ... now as a Workspace account I cant use Gemini CLI for my work, …”
ai-native userPlug MCP servers into this product so it can use their tools
weight 3 · round to ChatGPTChatGPT has first-party documentation for MCP support, explicitly describing configuring MCP servers (e.g., Context7) so ChatGPT/Codex can access third-party tools like documentation, browser, or Figma, with config stored in config.toml and scoping options for trusted projects, plus cross-client portability once configured. missing for 10: independent/hands-on community corroboration of MCP server usage specifically (community evidence covers plugins/browsing but not MCP directly).
- [claimed-docs] “let it interact with developer tools like your browser or Figma”
- [claimed-docs] “Once you configure your MCP servers, you can switch among those clients without redoing setup.”
- [claimed-docs] “Use it to give ChatGPT or Codex access to third-party documentation, or to let it interact with developer tools like your browser or Figma.”
- [claimed-docs] “to add Context7 (a free MCP server for developer documentation)”
- [claimed-docs] “you can also scope MCP servers to a project with `.codex/config.toml` (trusted projects only)”
- [claimed-docs] “Codex stores MCP configuration in `config.toml`”
- [claimed-docs] “Model Context Protocol (MCP) connects models to tools and context. Use it to give ChatGPT or Codex access to third-party documentation”
Gemininone0/10Evidence covers Gemini Apps' 'Connected Apps' feature (Gmail, Calendar, Maps, Flights) and coding-related uploads (GitHub repos, folders), but nothing describes plugging in Model Context Protocol (MCP) servers as a tool-connection mechanism. Missing for 10: any documentation or mention of MCP server support, MCP client configuration, or third-party tool registration via MCP standard.
- [claimed-docs] “Gemini Apps can connect to other apps to complete your request and give you more helpful responses. With your permission, Gemini can also he…”
- [claimed-docs] “Gemini Apps can connect to other apps to complete your request and give you more helpful responses.”
- [claimed-docs] “With your permission, Gemini can also help you take actions, like editing and managing your content in other apps.”
- [claimed-docs] “Boost productivity: Summarize emails from Gmail, create events in Google Calendar or third-party apps, or search for content across a variet…”
- [claimed-docs] “Get started with Connected Apps in Gemini **Important:** For now, Gemini can’t use Connected Apps in Gemini in Google Messages.”
ai-native userUse an official CLI
weight 2 · round to ChatGPTChatGPT ships an official Codex CLI with documented commands (--search, --cd, /memories, /fast, MCP config via config.toml, codex-security bulk-scan), and the llms.txt probe confirms 'Codex CLI' as a first-party interface for building and automating dev workflows, satisfying the AI-native official CLI story. Missing for 10: independent/hands-on community verification of the CLI itself, since community evidence only covers the general ChatGPT web product, not Codex CLI usage.
- [claimed-docs] “In Codex CLI, use /memories in an interactive session to control whether the current chat can use existing local memories or become an input…”
- [claimed-docs] “Run codex from the directory you want Codex to work in, or pass --cd (-C) to set it explicitly.”
- [claimed-docs] “In the CLI, pass `--search` to fetch live results for one run”
- [claimed-docs] “pass `--search` to fetch live results for one run”
- [claimed-docs] “Use the Codex SDK to automate coding tasks, including jobs in CI.”
- [claimed-docs] “Inspect code, make changes, run commands, and automate repeatable work without leaving your terminal.”
- [claimed-docs] “Codex reads AGENTS.md files before doing any work.”
- [claimed-docs] “Use rules to control which commands Codex can run outside the sandbox.”
- [claimed-docs] “Use `/fast on`, `/fast off`, or `/fast status` in the CLI to change or inspect the current setting.”
- [claimed-docs] “Use its command-line interface (CLI) to scan repositories you own or have permission to assess, review findings over time”
- [claimed-docs] “Use `npx @openai/codex-security bulk-scan` to review repositories in one campaign.”
- [probe] “PROBE llms.txt: HTTP 200 at https://learn.chatgpt.com/llms.txt # Codex > Build with the Codex CLI, IDE extension, and cloud automation to s…”
Community discussion confirms the existence of an official 'Gemini CLI' used for software development (comm-13, comm-14), but there is no first-party documentation of it in the evidence pack, and a hands-on report shows it fails to work for Workspace accounts ('Failed to login... now as a Workspace account I cant use Gemini CLI for my work') (comm-15). Missing for 10: official CLI documentation/spec, installation and usage guides, and evidence that login/account restrictions have been resolved.
- [community] “I have been using this for about a month and it's a beast, mostly thanks to 2.5pro being SOTA and also how it leverages that huge 1M context…”
- [community] “I love how fragmented Google's Gemini offerings are. I'm a Pro subscriber, but I now learn I should be a 'Gemini Code Assist Standard or Ent…”
- [community] “'Failed to login. Ensure your Google account is not a Workspace account.' ... now as a Workspace account I cant use Gemini CLI for my work, …”
ai-native userDrive the product through a documented public API
weight 3 · round to GeminiThere is documented evidence of programmatic/scriptable access via the Codex CLI and Codex SDK ('Use the Codex SDK to automate coding tasks, including jobs in CI', CLI flags like --search, --cd, /fast, config.toml), which lets an AI-native user drive parts of the product outside the chat UI. However, a probe for a standard documented public API (openapi.json/swagger) at the docs site returned 404 on all candidate paths, and none of the evidence describes a general, versioned public API for driving ChatGPT itself (as opposed to Codex-specific tooling or MCP client configuration). Missing for 10: an explicit REST/GraphQL API reference for ChatGPT product actions, OpenAPI/swagger spec, and independent corroboration that non-Codex ChatGPT features are API-drivable.
- [claimed-docs] “Use the Codex SDK to automate coding tasks, including jobs in CI.”
- [claimed-docs] “Inspect code, make changes, run commands, and automate repeatable work without leaving your terminal.”
- [claimed-docs] “Use its command-line interface (CLI) to scan repositories you own or have permission to assess, review findings over time”
- [claimed-docs] “Use `npx @openai/codex-security bulk-scan` to review repositories in one campaign.”
- [probe] “PROBE openapi: all candidate paths 404 (https://learn.chatgpt.com/openapi.json, https://learn.chatgpt.com/swagger.json, https://learn.chatgp…”
Community evidence confirms a public Gemini API exists and is actively used by developers (pricing tiers, model versions, Google AI Studio parameters like temperature and top_k), but the evidence pack contains no first-party API reference documentation, and one community report notes undocumented/hidden parameter behavior (fixed top_k, capped temperature) that limits full programmatic control. missing for 10: first-party API reference/docs citation, SDK/auth documentation, confirmation that documented parameters match actual behavior.
- [community] “API pricing is up to $2/M for input and $12/M for output. For comparison: Gemini 2.5 Pro was $1.25/M for input and $10/M for output, Gemini …”
- [community] “I'm sure this is a very impressive model, but gemini-3-pro-preview is failing spectacularly at my fairly basic python benchmark. In fact, ge…”
- [community] “My favorite benchmark is to analyze a very long audio file recording of a management meeting and produce very good notes along with a transc…”
- [community] “Temperature continues to be gated to maximum of 0.2, and there's still the hidden top_k of 64 that you can't turn off. I love the google AI …”
- [community] “It still failed my image identification test (a photoshopped picture of a dog with 5 legs)... Gemini 3 however, while still failing, at leas…”
ai-native userBuild against official SDKs
weight 2 · round to ChatGPTThe pack contains a single line noting 'Use the Codex SDK to automate coding tasks, including jobs in CI' (chatgpt-docs-91), indicating an official SDK exists for Codex/automation, but there is no linked documentation, API reference, code samples, or independent corroboration of this SDK's use. The openapi probe found no public API spec, reinforcing that developer-facing SDK documentation is thin in this evidence set. missing for 10: dedicated SDK documentation/reference pages, code examples, language support details, independent developer reports of building with the SDK.
- [claimed-docs] “Use the Codex SDK to automate coding tasks, including jobs in CI.”
- [probe] “PROBE openapi: all candidate paths 404 (https://learn.chatgpt.com/openapi.json, https://learn.chatgpt.com/swagger.json, https://learn.chatgp…”
Gemininone0/10The evidence pack only covers Gemini's consumer-facing app features (uploads, Gemini Live, Gems, connected apps) and does not document any official SDKs for building applications (e.g., Python/Node/Go SDKs for the Gemini API). Community mentions reference API pricing and a 'Gemini CLI' having login issues, but no concrete SDK documentation or hands-on developer usage evidence is present.
- [community] “API pricing is up to $2/M for input and $12/M for output. For comparison: Gemini 2.5 Pro was $1.25/M for input and $10/M for output, Gemini …”
- [community] “'Failed to login. Ensure your Google account is not a Workspace account.' ... now as a Workspace account I cant use Gemini CLI for my work, …”
ai-native userSubscribe to events via webhooks
weight 2 · round to ChatGPTChatGPT's Automations/Scheduled Tasks support event-driven triggers ('act on pull request feedback without polling on a fixed cadence') tied to Gmail, Slack, and GitHub events, which functions like a webhook-subscription mechanism, but this is not exposed as a general-purpose webhook subscription API — it's limited to a few pre-integrated services with no documented endpoint/config for arbitrary webhook URLs. Missing for 10: a generic webhook subscription API/config for third-party or custom events, documentation of payload/security handling, and any independent confirmation of reliability.
- [claimed-docs] “Scheduled tasks can now start when a supported event occurs in Gmail, Slack, or GitHub.”
- [claimed-docs] “Use an event trigger to triage new email, summarize channel activity, or act on pull request feedback without polling on a fixed cadence.”
- [claimed-docs] “scheduled tasks can run when a supported Gmail, Slack, or GitHub event occurs”
Agentic features
ai-native userGet AI-generated insights and suggestions from my data inside the product
weight 2 · round drawnChatGPT can analyze uploaded/connected data (files, projects, local folders, connected apps like Google Drive/SharePoint/Salesforce) and generate insights, summaries, and suggestions (e.g. 'Search the web, browse websites, compare sources, read files, analyze data, and summarize findings' and 'Create a deck, analyze files, draft a report, build a project plan'), with projects giving persistent context for tailored suggestions. Missing for 10: independent hands-on evidence specifically validating data-insight quality/accuracy (community evidence largely covers unrelated features like web search accuracy issues, not data-analysis insight quality).
- [claimed-docs] “Search the web, browse websites, compare sources, read files, analyze data, and summarize findings.”
- [claimed-docs] “Create a deck, analyze files, draft a report, build a project plan”
- [claimed-docs] “Use a project to organize related chats and give ChatGPT the context it needs.”
- [claimed-docs] “A local project gives chats access to one or more folders on your computer, such as a collection of source files or a codebase.”
- [claimed-docs] “Plugins can connect ChatGPT to the tools and information you use for work, such as Google Drive, SharePoint, Salesforce, or Gong.”
- [claimed-docs] “Turn research and analysis into documents, presentations, spreadsheets, and other finished work.”
Gemini's docs show robust support for generating insights from user data: uploading documents/spreadsheets/photos/videos for 'answers, summaries, and insights' (gemini-docs-25), Notebooks for organizing and querying sources (gemini-docs-11, gemini-docs-24, gemini-docs-29), Deep Research for in-depth analysis (gemini-docs-7), and Connected Apps that summarize Gmail and productivity data (gemini-docs-30). Independent hands-on reports corroborate real usage: navigating large codebases effortlessly (gemini-comm-13), replacing an OCR vendor for document ingestion with strong accuracy (gemini-comm-17), and NotebookLM being 'really powerful for comprehending large corpuses' (gemini-comm-19), though one report notes weaker accuracy on visual bounding-box tasks (gemini-comm-18). missing for 10: broader independent benchmarking specifically on insight-generation quality across data types, and resolution of the noted accuracy gaps in some visual tasks.
- [claimed-docs] “You can upload documents, spreadsheets, notebooks, photos, videos, and more to Gemini Apps to get answers, summaries, and insights about you…”
- [claimed-docs] “Notebooks provide a continuous chat experience that remembers your sources, instructions, and ongoing discussions.”
- [claimed-docs] “you can use notebooks to organize your projects in a dedicated, focused space.”
- [claimed-docs] “You can conduct in-depth and real-time research on almost any subject with Deep Research in Gemini Apps.”
- [claimed-docs] “You can change or add other sources, like your personal Gmail or Drive, for your research. You can also upload files and add NotebookLM note…”
- [claimed-docs] “Boost productivity: Summarize emails from Gmail, create events in Google Calendar or third-party apps, or search for content across a variet…”
- [community] “I have been using this for about a month and it's a beast, mostly thanks to 2.5pro being SOTA and also how it leverages that huge 1M context…”
- [community] “I work in fintech and we replaced an OCR vendor with Gemini at work for ingesting some PDFs... Processing time went from something like 12 m…”
- [community] “I've been using NotebookLM powered by Gemini 2.0 for three projects and it is really powerful for comprehending large corpuses you can't pos…”
- [community] “Unfortunately Gemini really seems to struggle on this, and no matter how we tried prompting it, it would generate wildly inaccurate bounding…”
ai-native userSet up automations that run autonomously in the background
weight 2 · round to ChatGPTChatGPT's Scheduled Tasks/Automations feature explicitly lets users schedule recurring or event-triggered tasks (Gmail, Slack, GitHub events) to run autonomously in the background, with a dashboard to review active, paused, and completed runs. This is well-documented first-party functionality directly matching the story. Missing for 10: independent hands-on verification of background automation reliability and no community corroboration specifically about the automations/scheduled-tasks feature.
- [claimed-docs] “Schedule recurring tasks to run in the background.”
- [claimed-docs] “Scheduled tasks can now start when a supported event occurs in Gmail, Slack, or GitHub.”
- [claimed-docs] “Schedule recurring tasks to run in the background... Review active, paused, and completed tasks and recent runs in Scheduled.”
- [claimed-docs] “scheduled tasks can run when a supported Gmail, Slack, or GitHub event occurs”
- [claimed-docs] “Use an event trigger to triage new email, summarize channel activity, or act on pull request feedback without polling on a fixed cadence.”
Google's docs explicitly describe scheduling recurring actions so Gemini becomes part of daily/weekly/monthly routines (gemini-docs-8), and the Daily Brief feature autonomously delivers a personalized digest without user prompting (gemini-docs-32/41), supported by action-taking integrations with other apps (gemini-docs-10/17/23/30). However, evidence is limited to first-party docs with no independent/hands-on confirmation of reliability, no detail on trigger conditions, failure handling, or scope of what 'recurring actions' can actually do autonomously. Missing for 10: independent verification that scheduled automations run reliably unattended, documentation on trigger/condition logic, and evidence of broader autonomous multi-step task execution beyond simple digests.
- [claimed-docs] “You can make Gemini a part of your daily, weekly, and monthly routines by scheduling recurring actions.”
- [claimed-docs] “Start your day with a personalized digest from Gemini. Your Daily Brief understands your rhythms and personal preferences to deliver your to…”
- [claimed-docs] “Start your day with a personalized digest from Gemini. Your Daily Brief understands your rhythms and personal preferences to deliver your to…”
- [claimed-docs] “Gemini Apps can connect to other apps to complete your request and give you more helpful responses. With your permission, Gemini can also he…”
- [claimed-docs] “Gemini Apps can connect to other apps to complete your request and give you more helpful responses.”
- [claimed-docs] “With your permission, Gemini can also help you take actions, like editing and managing your content in other apps.”
- [claimed-docs] “Boost productivity: Summarize emails from Gmail, create events in Google Calendar or third-party apps, or search for content across a variet…”
ai-native userDelegate tasks to a built-in AI assistant inside the product
weight 3 · round to ChatGPTChatGPT ships extensive built-in agentic delegation: Work/Codex modes carry tasks through to reviewable results, scheduled/background tasks and event-triggered automations, computer use and browser control, subagent delegation, and voice-initiated task delegation — all first-party features, not third-party add-ons. Missing for 10: independent hands-on verification of these newer agentic features (Work, subagents, Computer Use) beyond vendor docs.
- [claimed-docs] “Schedule recurring tasks to run in the background.”
- [claimed-docs] “Scheduled tasks can now start when a supported event occurs in Gmail, Slack, or GitHub.”
- [claimed-docs] “Turn on Work in the switcher when you want ChatGPT to carry a larger task through to a reviewable result.”
- [claimed-docs] “Run projects in parallel, work with files, use your computer, and keep long-running work moving from one desktop workspace.”
- [claimed-docs] “Unlike Work on the Web, local Work can operate on resources that remain on your computer without requiring you to upload files to a cloud co…”
- [claimed-docs] “In ChatGPT Work, ask ChatGPT to delegate independent work to subagents.”
- [claimed-docs] “ChatGPT Voice can start separate tasks for longer work, check existing tasks, and send follow-up instructions.”
- [claimed-docs] “Browser lets ChatGPT open websites, gather current information, and take action while you stay in control.”
- [claimed-docs] “With Computer Use, ChatGPT can see and operate graphical user interfaces on macOS or Windows.”
- [claimed-docs] “enter `/goal` to start Goal mode”
Gemini's docs show extensive task delegation: connected apps to take actions in Gmail/Calendar/Maps/Flights (gemini-docs-10, gemini-docs-23, gemini-docs-30), scheduled recurring actions (gemini-docs-8), custom Gems for specific goals (gemini-docs-9), Deep Research for autonomous multi-step research (gemini-docs-7), and Daily Brief proactive digests (gemini-docs-32, gemini-docs-41). Community evidence corroborates real-world delegated use cases like large codebase analysis and audio transcription (gemini-comm-13, gemini-comm-10). missing for 10: independent hands-on verification specifically of agentic task delegation/action-taking (vs. just Q&A), and evidence of reliability/success rate when Gemini executes multi-step delegated tasks.
- [claimed-docs] “You can conduct in-depth and real-time research on almost any subject with Deep Research in Gemini Apps.”
- [claimed-docs] “You can make Gemini a part of your daily, weekly, and monthly routines by scheduling recurring actions.”
- [claimed-docs] “To create a custom Gem to meet your goals, provide info about your request and requirements.”
- [claimed-docs] “Gemini Apps can connect to other apps to complete your request and give you more helpful responses. With your permission, Gemini can also he…”
- [claimed-docs] “With your permission, Gemini can also help you take actions, like editing and managing your content in other apps.”
- [claimed-docs] “Boost productivity: Summarize emails from Gmail, create events in Google Calendar or third-party apps, or search for content across a variet…”
- [claimed-docs] “Start your day with a personalized digest from Gemini. Your Daily Brief understands your rhythms and personal preferences to deliver your to…”
- [community] “I have been using this for about a month and it's a beast, mostly thanks to 2.5pro being SOTA and also how it leverages that huge 1M context…”
- [community] “My favorite benchmark is to analyze a very long audio file recording of a management meeting and produce very good notes along with a transc…”
ai-native userOperate the product with natural-language commands
weight 2 · round to ChatGPTChatGPT is fundamentally a natural-language interface: users type or speak requests and it carries out tasks (browsing, coding, file edits, image generation, scheduling, computer use, voice) via plain conversational commands, per extensive first-party docs and corroborating community reports of natural-language driven capability (e.g. running Docker/filesystem commands via prompts). Missing for 10: independent hands-on verification specifically of newer agentic features (Work, Goal mode, subagents) beyond vendor docs.
- [claimed-docs] “Browser lets ChatGPT open websites, gather current information, and take action while you stay in control.”
- [claimed-docs] “With Computer Use, ChatGPT can see and operate graphical user interfaces on macOS or Windows.”
- [claimed-docs] “Schedule recurring tasks to run in the background.”
- [claimed-docs] “ChatGPT Voice lets you talk through ideas and coordinate tasks in Chat, Work, and Codex in the ChatGPT desktop app. Start work, check progre…”
- [claimed-docs] “In the ChatGPT desktop app, enter `/goal` to start Goal mode.”
- [claimed-docs] “In ChatGPT Work, ask ChatGPT to delegate independent work to subagents.”
- [community] “I am beyond astounded. I was able to run a Docker image, utilize the fs inside of the container, and exit the container. Docker system comma…”
- [community] “This is huge, essentially adding what people have been building with LangChain Tools into the core product. The browser and file-upload/inte…”
Gemini is fundamentally operated via natural-language chat and voice (Gemini Live's free-flowing conversation, 'Hey Google' voice commands, text prompts for image generation, scheduling recurring actions, custom Gems, and connected-app actions) rather than rigid commands or GUIs, per extensive first-party docs. Missing for 10: independent/hands-on verification that complex natural-language action requests (e.g., connected-app task execution) reliably work as described, and community evidence is mixed on model quality/accuracy for specific tasks rather than confirming the NL interface itself.
- [claimed-docs] “With Gemini Live, you can have a natural, free-flowing conversation with Gemini. You can even interrupt it like when you want to add more de…”
- [claimed-docs] “With Gemini Live, you can have a natural, free-flowing conversation with Gemini.”
- [claimed-docs] “With Gemini Live, you can: * **Talk back and forth:** Talk to Gemini without typing, and Gemini will respond back verbally.”
- [claimed-docs] “you can have a natural, free-flowing conversation with Gemini. You can even interrupt it like when you want to add more details or change th…”
- [claimed-docs] “Gemini Apps can connect to other apps to complete your request and give you more helpful responses. With your permission, Gemini can also he…”
- [claimed-docs] “Gemini Apps can connect to other apps to complete your request and give you more helpful responses.”
- [claimed-docs] “With your permission, Gemini can also help you take actions, like editing and managing your content in other apps.”
- [claimed-docs] “You can make Gemini a part of your daily, weekly, and monthly routines by scheduling recurring actions.”
- [claimed-docs] “To create a custom Gem to meet your goals, provide info about your request and requirements.”
- [claimed-docs] “Boost productivity: Summarize emails from Gmail, create events in Google Calendar or third-party apps, or search for content across a variet…”
- [claimed-docs] “On Android, say “Hey Google” to ask Gemini for help with what’s on your screen”
Api quality
ai-native userRely on versioned APIs with a documented deprecation policy
weight 2 · round drawnChatGPTnone0/10No evidence of a versioned API or a documented deprecation policy is present; the OpenAPI probe explicitly returned 404s and none of the docs mention API versioning or deprecation practices.
- [probe] “PROBE openapi: all candidate paths 404 (https://learn.chatgpt.com/openapi.json, https://learn.chatgpt.com/swagger.json, https://learn.chatgp…”
Gemininone0/10The evidence pack is entirely consumer-focused Gemini Apps documentation (image generation, Gemini Live, connected apps, notebooks) and general community sentiment about model quality/pricing; there is no mention of API versioning, version lifecycle, or a documented deprecation policy anywhere in the pack.
Agents tasks — stories about agents tasks in this arenaAgents tasks
Stories about agents tasks in this arena
Agent mode
power-userDelegate a multi-step task that the assistant works on autonomously in the background and returns for my review
weight 3 · round to ChatGPTChatGPT's Tasks/automations feature explicitly supports scheduling and running multi-step work in the background (including event-triggered runs from Gmail/Slack/GitHub), with a review interface for active/completed runs, and 'Work' mode is described as carrying larger tasks through to a reviewable result; Codex further supports background coding tasks and subagent delegation. Missing for 10: independent hands-on verification of background task reliability/quality and no third-party review of the review/approval workflow.
- [claimed-docs] “Schedule recurring tasks to run in the background.”
- [claimed-docs] “Scheduled tasks can now start when a supported event occurs in Gmail, Slack, or GitHub.”
- [claimed-docs] “Schedule recurring tasks to run in the background... Review active, paused, and completed tasks and recent runs in Scheduled.”
- [claimed-docs] “Turn on Work in the switcher when you want ChatGPT to carry a larger task through to a reviewable result.”
- [claimed-docs] “scheduled tasks can run when a supported Gmail, Slack, or GitHub event occurs”
- [claimed-docs] “In ChatGPT Work, ask ChatGPT to delegate independent work to subagents.”
- [claimed-docs] “Start a Codex task to run the tests and investigate anything that doesn't pass.”
Deep Research lets users delegate an open-ended research task that Gemini works on autonomously and returns a report for review, and scheduled/recurring actions show background task automation; Connected Apps/Gems add some autonomous action-taking on the user's behalf. However there's no clear evidence of a general-purpose multi-step 'agent mode' that plans and executes across arbitrary apps end-to-end with an explicit review/approval step, nor independent hands-on confirmation of this workflow. Missing for 10: evidence of a broad autonomous agent (beyond Deep Research/scheduling) handling arbitrary multi-step tasks across apps, and independent/community validation of such background task delegation working reliably.
- [claimed-docs] “You can conduct in-depth and real-time research on almost any subject with Deep Research in Gemini Apps.”
- [claimed-docs] “You can make Gemini a part of your daily, weekly, and monthly routines by scheduling recurring actions.”
- [claimed-docs] “To create a custom Gem to meet your goals, provide info about your request and requirements.”
- [claimed-docs] “Gemini Apps can connect to other apps to complete your request and give you more helpful responses. With your permission, Gemini can also he…”
- [claimed-docs] “With your permission, Gemini can also help you take actions, like editing and managing your content in other apps.”
- [claimed-docs] “Boost productivity: Summarize emails from Gmail, create events in Google Calendar or third-party apps, or search for content across a variet…”
power-userHave the assistant operate a web browser on my behalf to research and complete tasks on websites
weight 3 · round to ChatGPTChatGPT's Browser feature explicitly lets it open websites, gather information, and take action on the user's behalf while staying in control, and can complete multi-step tasks like comparing options or completing actions on a website; this is reinforced by Computer Use (operating GUIs) and Work/Codex browser tab integration for signed-in site tasks. missing for 10: independent hands-on verification of complex multi-step website task completion, and detail on reliability/success rate limits.
- [claimed-docs] “Browser lets ChatGPT open websites, gather current information, and take action while you stay in control.”
- [claimed-docs] “Use it to compare options, complete a multi-step task on a website, or review a page you're building.”
- [claimed-docs] “With Computer Use, ChatGPT can see and operate graphical user interfaces on macOS or Windows.”
- [claimed-docs] “ChatGPT can see and operate graphical user interfaces on macOS or Windows”
- [claimed-docs] “Bring an open tab into a ChatGPT Work or Codex chat and work with the website where you're already signed in.”
- [claimed-docs] “Follow the sign-in request and enter your details in the sign-in flow, not in the chat. This doesn't connect your local browser profile.”
- [claimed-docs] “With Site tools (WebMCP), ChatGPT Work and Codex can use actions offered by a website in the desktop app's built-in browser.”
Deep Research lets Gemini conduct 'in-depth and real-time research' across web sources, and Connected Apps let it take actions in other apps (Gmail, Calendar, Maps, Flights) with permission — showing some research/task-automation capability. However, there is no evidence of Gemini actually driving a web browser (navigating pages, clicking, filling forms) to complete tasks on arbitrary websites, only sanctioned first-party app integrations. Missing for 10: explicit browser-control/agentic navigation evidence, hands-on demonstration of completing a multi-step website task, independent corroboration of this capability.
- [claimed-docs] “You can conduct in-depth and real-time research on almost any subject with Deep Research in Gemini Apps.”
- [claimed-docs] “You can change or add other sources, like your personal Gmail or Drive, for your research. You can also upload files and add NotebookLM note…”
- [claimed-docs] “Gemini Apps can connect to other apps to complete your request and give you more helpful responses. With your permission, Gemini can also he…”
- [claimed-docs] “With your permission, Gemini can also help you take actions, like editing and managing your content in other apps.”
- [claimed-docs] “Boost productivity: Summarize emails from Gmail, create events in Google Calendar or third-party apps, or search for content across a variet…”
power-userLet the assistant see and operate applications on my computer to complete work
weight 2 · round to ChatGPTChatGPT's Computer Use feature explicitly lets it see and operate GUIs on macOS/Windows, including testing desktop/mobile app flows, backed by Browser and Work agent capabilities that carry out multi-step tasks on the user's behalf. Documentation is extensive and detailed across multiple first-party sources describing screen operation, browser control, and file/app interaction. Missing for 10: independent hands-on verification of Computer Use reliability in practice.
- [claimed-docs] “With Computer Use, ChatGPT can see and operate graphical user interfaces on macOS or Windows.”
- [claimed-docs] “ChatGPT can see and operate graphical user interfaces on macOS or Windows”
- [claimed-docs] “ChatGPT can see and operate graphical user interfaces on macOS or Windows.”
- [claimed-docs] “Testing a macOS app, Windows app, iOS simulator flow, or another desktop app that ChatGPT is building.”
- [claimed-docs] “Browser lets ChatGPT open websites, gather current information, and take action while you stay in control.”
- [claimed-docs] “Use it to compare options, complete a multi-step task on a website, or review a page you're building.”
- [claimed-docs] “Run projects in parallel, work with files, use your computer, and keep long-running work moving from one desktop workspace.”
- [claimed-docs] “Unlike Work on the Web, local Work can operate on resources that remain on your computer without requiring you to upload files to a cloud co…”
- [claimed-docs] “Your ChatGPT data controls apply to content processed through ChatGPT, including screenshots taken by Computer Use.”
Gemini's Connected Apps feature lets it take actions in other apps (Gmail, Calendar, Maps, Flights) with permission, and on Android it can see what's on-screen to help with context, but this is limited to a curated set of app integrations/APIs rather than general computer-use/desktop automation across arbitrary applications. Missing for 10: broader OS-level screen perception, ability to click/type across arbitrary desktop apps, and independent hands-on evidence of an agentic computer-use mode.
- [claimed-docs] “Gemini Apps can connect to other apps to complete your request and give you more helpful responses. With your permission, Gemini can also he…”
- [claimed-docs] “Gemini Apps can connect to other apps to complete your request and give you more helpful responses.”
- [claimed-docs] “With your permission, Gemini can also help you take actions, like editing and managing your content in other apps.”
- [claimed-docs] “Boost productivity: Summarize emails from Gmail, create events in Google Calendar or third-party apps, or search for content across a variet…”
- [claimed-docs] “On Android, say “Hey Google” to ask Gemini for help with what’s on your screen”
- [claimed-docs] “Get started with Connected Apps in Gemini **Important:** For now, Gemini can’t use Connected Apps in Gemini in Google Messages.”
Tasks
power-userSchedule recurring or one-off tasks that run automatically and come back to me with results
weight 2 · round to ChatGPTChatGPT's Automations/Scheduled Tasks feature explicitly supports scheduling recurring or one-off tasks that run in the background and report back, plus event-triggered tasks from Gmail/Slack/GitHub, with a UI to review active, paused, and completed runs. missing for 10: independent/hands-on community corroboration of scheduled task reliability and no detail on notification/result-delivery mechanics beyond docs.
- [claimed-docs] “Schedule recurring tasks to run in the background.”
- [claimed-docs] “Scheduled tasks can now start when a supported event occurs in Gmail, Slack, or GitHub.”
- [claimed-docs] “Schedule recurring tasks to run in the background... Review active, paused, and completed tasks and recent runs in Scheduled.”
- [claimed-docs] “scheduled tasks can run when a supported Gmail, Slack, or GitHub event occurs”
- [claimed-docs] “Use an event trigger to triage new email, summarize channel activity, or act on pull request feedback without polling on a fixed cadence.”
Gemini docs confirm scheduling recurring actions (gemini-docs-8) and Daily Brief personalized digest (gemini-docs-32/41), showing one-off/recurring automated tasks that return results. However, evidence lacks detail on task complexity, multi-step agent tasks, notification delivery mechanisms, or independent/community corroboration of this specific feature working reliably. missing for 10: detailed documentation on scheduling flexibility, agentic multi-step task execution, notification/result delivery mechanisms, and independent hands-on validation of the scheduled-tasks feature.
- [claimed-docs] “You can make Gemini a part of your daily, weekly, and monthly routines by scheduling recurring actions.”
- [claimed-docs] “Start your day with a personalized digest from Gemini. Your Daily Brief understands your rhythms and personal preferences to deliver your to…”
- [claimed-docs] “Start your day with a personalized digest from Gemini. Your Daily Brief understands your rhythms and personal preferences to deliver your to…”
Apps devices — stories about apps devices in this arenaApps devices
Stories about apps devices in this arena
Apps
power-userUse an official desktop app with OS-level shortcuts and access to what is on my screen
weight 2 · round to ChatGPTThe evidence confirms an official ChatGPT desktop app (macOS/Windows) that can 'use your computer' and run 'Computer Use' to see and operate GUIs, including screenshot-based screen access, plus desktop-only features like floating Pets controls and multi-browser support. However, there is no documentation of OS-level global keyboard shortcuts (e.g., a system-wide hotkey to invoke the app) or of general 'what's on my screen' querying outside the specific Computer Use/testing use case. Missing for 10: explicit OS-level shortcut/hotkey documentation, general screen-content awareness beyond Computer Use testing scenarios, independent hands-on confirmation of these desktop-specific features.
- [claimed-docs] “Run projects in parallel, work with files, use your computer, and keep long-running work moving from one desktop workspace.”
- [claimed-docs] “Work in Edge, Brave, Opera, or Vivaldi as well as Chrome from the ChatGPT desktop app.”
- [claimed-docs] “With Computer Use, ChatGPT can see and operate graphical user interfaces on macOS or Windows.”
- [claimed-docs] “ChatGPT can see and operate graphical user interfaces on macOS or Windows”
- [claimed-docs] “ChatGPT can see and operate graphical user interfaces on macOS or Windows.”
- [claimed-docs] “Your ChatGPT data controls apply to content processed through ChatGPT, including screenshots taken by Computer Use.”
- [claimed-docs] “Type a request or start a voice conversation from the floating Pets controls in the ChatGPT desktop app on macOS and Windows.”
- [claimed-docs] “Testing a macOS app, Windows app, iOS simulator flow, or another desktop app that ChatGPT is building.”
Docs confirm an official Mac desktop app with a global OS-level keyboard shortcut (Option+Space) to summon Gemini without opening a browser [gemini-docs-15, gemini-docs-36, gemini-docs-20], but the only 'see what's on your screen' capability documented is for the Android mobile app via 'Hey Google' [gemini-docs-6], not the desktop app. Missing for 10: evidence of desktop screen-reading/screenshare capability, and independent/hands-on confirmation of the desktop shortcut and screen access working together.
- [claimed-docs] “you can also use the keyboard shortcut Option + Space to bring up the Gemini chat window after the app is installed and running”
- [claimed-docs] “The Gemini app provides quick access to AI features, allowing you to get help with writing, brainstorming, coding, and more, without needing…”
- [claimed-docs] “What you need to use the Gemini app on Mac * A personal Google Account you manage on your own, or a work or school account that your adminis…”
- [claimed-docs] “On Android, say “Hey Google” to ask Gemini for help with what’s on your screen”
knowledge-workerUse full-featured official mobile apps for iOS and Android
weight 2 · round to ChatGPTThe iOS App Store listing (chatgpt-docs-20/21/41/52/58/59) documents rich mobile features—voice mode, photo upload, image generation, stickers—confirming a full-featured iOS app, and doc references (chatgpt-docs-67) explicitly mention continuing tasks 'in ChatGPT Work on the web, iOS, or Android,' indicating Android parity. Missing for 10: no dedicated Android app store evidence pack, no independent hands-on reviews of the mobile apps' feature completeness or performance.
- [claimed-docs] “Photo upload—Snap or upload a picture to transcribe a handwritten recipe or get info about a landmark.”
- [claimed-docs] “Advanced Voice Mode–Tap the soundwave icon to have a real-time convo on the go.”
- [claimed-docs] “Advanced Voice Mode–Tap the soundwave icon to have a real-time convo on the go. Settle a dinner table debate, or practice a new language.”
- [claimed-docs] “Snap or upload a picture to transcribe a handwritten recipe or get info about a landmark.”
- [claimed-docs] “Image generation–Generate original images from a description, or transform existing ones with a few simple words.”
- [claimed-docs] “Choose an image and style, remix with emojis, and make personalized stickers that are ready to share in your group chat.”
- [claimed-docs] “continue a task that needs a website account in ChatGPT Work on the web, iOS, or Android”
Docs confirm a full-featured Gemini mobile app with rich functionality (Gemini Live voice conversations, screen-aware Android assistant via 'Hey Google', Maps/Flights integration, daily briefs, scheduling, image generation, coding help) showing strong Android support, and generic mobile-app language implies broader availability, but the evidence never explicitly names or details an iOS app or iOS-specific parity. missing for 10: explicit iOS app documentation/feature parity, App Store presence, and any cross-platform comparison confirming equal functionality on iOS vs Android.
- [claimed-docs] “On Android, say “Hey Google” to ask Gemini for help with what’s on your screen”
- [claimed-docs] “With Gemini Live, you can have a natural, free-flowing conversation with Gemini. You can even interrupt it like when you want to add more de…”
- [claimed-docs] “Make plans with Google Maps and Google Flights”
- [claimed-docs] “The Gemini app provides quick access to AI features, allowing you to get help with writing, brainstorming, coding, and more, without needing…”
- [claimed-docs] “get help with writing, brainstorming, coding, and more, without needing to open a web browser.”
- [claimed-docs] “Start your day with a personalized digest from Gemini. Your Daily Brief understands your rhythms and personal preferences to deliver your to…”
- [claimed-docs] “If you have access to Gemini Apps with a work or school account, some Gemini mobile app features aren’t available.”
- [claimed-docs] “Start your day with a personalized digest from Gemini. Your Daily Brief understands your rhythms and personal preferences to deliver your to…”
Custom bots
power-userBuild and share custom assistants with their own instructions and knowledge
weight 2 · round to GeminiDocs confirm a 'GPT builder' concept for creating custom GPTs with configurable actions/connected apps and a dedicated 'gpts-and-sharing' doc implying sharing, but no evidence details custom instructions or uploading a knowledge base, and no independent/hands-on corroboration of the sharing flow. Missing for 10: explicit mention of setting custom instructions, uploading knowledge files, and community/hands-on validation of building and sharing a custom GPT.
- [claimed-docs] “Allow GPT builders to use approved workspace apps or configure actions that interact with permitted third-party APIs.”
- [claimed-docs] “configure actions that interact with permitted third-party APIs”
- [claimed-docs] “GPT builders can use either connected apps or custom actions, but not both in the same GPT.”
Gemini's "Gems" feature (gemini-docs-9) lets power-users create a custom Gem with its own instructions and requirements, and Gems can be given knowledge via uploaded files/sources (gemini-docs-25), directly matching the core of this story. However, evidence does not confirm a straightforward sharing/publishing mechanism for Gems to other users, nor independent hands-on validation of the feature's robustness. Missing for 10: documentation of sharing/publishing custom Gems with other users, evidence of knowledge-base scope/limits, and independent community corroboration of Gems in practice.
- [claimed-docs] “To create a custom Gem to meet your goals, provide info about your request and requirements.”
- [claimed-docs] “You can upload documents, spreadsheets, notebooks, photos, videos, and more to Gemini Apps to get answers, summaries, and insights about you…”
Automation depth — how much of the product can run unattendedAutomation depth
How much of the product can run unattended
ai-native userPerform bulk operations across many items at once
weight 2 · round to ChatGPTChatGPT/Codex docs show concrete bulk-style automation: 'bulk-scan' scanning many repositories in one campaign (chatgpt-docs-98), a CLI for scanning multiple repos over time (chatgpt-docs-97), running multiple projects in parallel (chatgpt-docs-32), and delegating work to subagents (chatgpt-docs-103), plus background scheduled tasks with multiple runs (chatgpt-docs-37). This covers meaningful bulk/parallel automation but is concentrated in coding/security contexts rather than general-purpose bulk operations across arbitrary item sets, and there's no independent/hands-on corroboration of these bulk claims. Missing for 10: evidence of bulk operations on non-code items (e.g., bulk document/data processing), and independent verification that bulk-scan/subagent delegation works reliably at scale.
- [claimed-docs] “Use `npx @openai/codex-security bulk-scan` to review repositories in one campaign.”
- [claimed-docs] “Use its command-line interface (CLI) to scan repositories you own or have permission to assess, review findings over time”
- [claimed-docs] “Run projects in parallel, work with files, use your computer, and keep long-running work moving from one desktop workspace.”
- [claimed-docs] “In ChatGPT Work, ask ChatGPT to delegate independent work to subagents.”
- [claimed-docs] “Schedule recurring tasks to run in the background... Review active, paused, and completed tasks and recent runs in Scheduled.”
Gemini supports uploading whole code folders/repos and large codebases (a form of bulk ingestion) and can take actions across connected apps or run scheduled recurring actions, but there is no explicit documentation of a bulk/batch operation feature (e.g., applying an action to many items at once like bulk-editing files, mass-tagging, or batch task execution) as an AI-native automation capability. missing for 10: explicit bulk-action/batch-processing feature, evidence of applying one instruction across many discrete items simultaneously, independent corroboration of this specific capability.
- [claimed-docs] “To add a code folder: click More Uploads Import Code Upload folder.”
- [claimed-docs] “To add a code folder: click More Uploads ... Import Code ... Upload folder.”
- [claimed-docs] “You can make Gemini a part of your daily, weekly, and monthly routines by scheduling recurring actions.”
- [claimed-docs] “Gemini Apps can connect to other apps to complete your request and give you more helpful responses. With your permission, Gemini can also he…”
- [community] “I have been using this for about a month and it's a beast, mostly thanks to 2.5pro being SOTA and also how it leverages that huge 1M context…”
ai-native userDefine rules that trigger actions automatically on events
weight 3 · round to ChatGPTChatGPT's automations/scheduled tasks feature explicitly supports event-driven triggers (Gmail, Slack, GitHub) instead of just fixed schedules, letting users define rules that fire actions on external events, and lets them review active/paused/completed runs. Missing for 10: independent or hands-on verification that these event triggers work reliably in practice, and detail on how flexible/general the rule definitions can be beyond the three named integrations.
- [claimed-docs] “Schedule recurring tasks to run in the background.”
- [claimed-docs] “Scheduled tasks can now start when a supported event occurs in Gmail, Slack, or GitHub.”
- [claimed-docs] “Use an event trigger to triage new email, summarize channel activity, or act on pull request feedback without polling on a fixed cadence.”
- [claimed-docs] “Schedule recurring tasks to run in the background... Review active, paused, and completed tasks and recent runs in Scheduled.”
- [claimed-docs] “scheduled tasks can run when a supported Gmail, Slack, or GitHub event occurs”
Gemini supports 'scheduling recurring actions' as part of daily/weekly/monthly routines (gemini-docs-8), which is a form of automation, but this is time-based scheduling rather than user-defined conditional rules that trigger on arbitrary events. Connected Apps let Gemini take actions in other apps (gemini-docs-10, gemini-docs-23, gemini-docs-30) but only in response to a direct user request, not an automatic event trigger. Missing for 10: evidence of event-based (not just time-based) trigger conditions, ability to define custom 'if X happens then do Y' rules, and any confirmation of conditional logic beyond fixed recurring schedules.
- [claimed-docs] “You can make Gemini a part of your daily, weekly, and monthly routines by scheduling recurring actions.”
- [claimed-docs] “Gemini Apps can connect to other apps to complete your request and give you more helpful responses. With your permission, Gemini can also he…”
- [claimed-docs] “With your permission, Gemini can also help you take actions, like editing and managing your content in other apps.”
- [claimed-docs] “Boost productivity: Summarize emails from Gmail, create events in Google Calendar or third-party apps, or search for content across a variet…”
ai-native userSchedule recurring jobs or workflows
weight 2 · round to ChatGPTChatGPT's Scheduled Tasks/Automations feature explicitly supports recurring background jobs ('Schedule recurring tasks to run in the background,' with review of active/paused/completed runs) and even event-based triggers (Gmail, Slack, GitHub) as an alternative to fixed cadence. This directly matches the story of scheduling recurring jobs/workflows with first-party docs. Missing for 10: independent/hands-on community corroboration of the scheduling feature specifically (evidence pack community items don't address automations/scheduling) and more detail on reliability/limits in practice.
- [claimed-docs] “Schedule recurring tasks to run in the background.”
- [claimed-docs] “Schedule recurring tasks to run in the background... Review active, paused, and completed tasks and recent runs in Scheduled.”
- [claimed-docs] “Scheduled tasks can now start when a supported event occurs in Gmail, Slack, or GitHub.”
- [claimed-docs] “scheduled tasks can run when a supported Gmail, Slack, or GitHub event occurs”
- [claimed-docs] “Use an event trigger to triage new email, summarize channel activity, or act on pull request feedback without polling on a fixed cadence.”
Gemini docs explicitly confirm scheduling recurring actions ('daily, weekly, and monthly routines') via a dedicated feature (gemini-docs-8), and Daily Brief shows an example of a recurring automated digest (gemini-docs-32/41). However, evidence lacks detail on workflow complexity, chaining multiple steps/tools, error handling, or independent/community validation of this scheduling feature. missing for 10: detail on multi-step workflow scheduling, chaining across apps, reliability/error handling, and independent hands-on confirmation.
- [claimed-docs] “You can make Gemini a part of your daily, weekly, and monthly routines by scheduling recurring actions.”
- [claimed-docs] “Start your day with a personalized digest from Gemini. Your Daily Brief understands your rhythms and personal preferences to deliver your to…”
- [claimed-docs] “Start your day with a personalized digest from Gemini. Your Daily Brief understands your rhythms and personal preferences to deliver your to…”
ai-native userVersion, review, and roll back my automations
weight 1 · round to ChatGPTChatGPT supports scheduled/recurring automations with a review of active/paused/completed task runs (chatgpt-docs-11, chatgpt-docs-37, chatgpt-docs-47), which gives some review and status visibility, but there is no evidence of version history, diffing between automation versions, or a rollback mechanism to revert an automation to a prior state. missing for 10: explicit versioning of automations, diff/comparison between versions, and a documented rollback/revert feature.
- [claimed-docs] “Schedule recurring tasks to run in the background.”
- [claimed-docs] “Schedule recurring tasks to run in the background... Review active, paused, and completed tasks and recent runs in Scheduled.”
- [claimed-docs] “scheduled tasks can run when a supported Gmail, Slack, or GitHub event occurs”
Gemininone0/10No evidence of version control, review workflows, or rollback capabilities for automations (e.g., scheduled actions, Gems, or connected app actions). The evidence only covers scheduling recurring actions and connecting apps, with no mention of versioning, diffing, review approval, or rollback of automations.
Connectors apps — stories about connectors apps in this arenaConnectors apps
Stories about connectors apps in this arena
Connectors
power-userBrowse a directory of third-party apps and connectors and add them to the assistant
weight 2 · round to ChatGPTChatGPT documents a Plugins/connectors ecosystem — installing named plugins (Slack, Codex Security), connecting to third-party tools like Google Drive, SharePoint, Salesforce, and Gong, and GPT builders choosing 'connected apps' or custom actions from an approved list — which together describe browsing and adding third-party integrations to the assistant. Missing for 10: independent/hands-on confirmation of the actual browse-and-add directory UI/UX and its breadth beyond the named examples.
- [claimed-docs] “Install the Slack plugin to summarize channels or draft replies.”
- [claimed-docs] “Install the Codex Security plugin to scan authorized code and confirm plausible vulnerability findings.”
- [claimed-docs] “Plugins can connect ChatGPT to the tools and information you use for work, such as Google Drive, SharePoint, Salesforce, or Gong.”
- [claimed-docs] “Allow GPT builders to use approved workspace apps or configure actions that interact with permitted third-party APIs.”
- [claimed-docs] “GPT builders can use either connected apps or custom actions, but not both in the same GPT.”
- [claimed-docs] “configure actions that interact with permitted third-party APIs”
Gemini docs describe a 'Connected Apps' feature that lets users link Gmail, Calendar, Maps, Flights and 'a variety of productivity apps' so Gemini can search/create content and take actions, which matches the connectors-directory concept ([gemini-docs-10],[gemini-docs-17],[gemini-docs-23],[gemini-docs-30],[gemini-docs-33],[gemini-docs-19]). However, the evidence never shows a browsable app-store-like directory UI, doesn't clarify how many third-party (non-Google) apps are included, and there's no independent/hands-on confirmation of the browse-and-add experience. Missing for 10: explicit directory/marketplace UI screenshots or docs, breadth of third-party (non-Google) app listings, independent user corroboration of adding connectors.
- [claimed-docs] “Gemini Apps can connect to other apps to complete your request and give you more helpful responses. With your permission, Gemini can also he…”
- [claimed-docs] “Gemini Apps can connect to other apps to complete your request and give you more helpful responses.”
- [claimed-docs] “With your permission, Gemini can also help you take actions, like editing and managing your content in other apps.”
- [claimed-docs] “Boost productivity: Summarize emails from Gmail, create events in Google Calendar or third-party apps, or search for content across a variet…”
- [claimed-docs] “Get started with Connected Apps in Gemini **Important:** For now, Gemini can’t use Connected Apps in Gemini in Google Messages.”
- [claimed-docs] “Make plans with Google Maps and Google Flights”
knowledge-workerConnect my cloud drive, email, and calendar so the assistant can search and use them in answers
weight 3 · round to GeminiDocs show ChatGPT can connect to Google Drive/SharePoint via plugins and can trigger on Gmail events, giving cloud-drive and email connectivity, and MCP/plugins provide a general connector framework for third-party tools. However, there is no explicit mention of calendar integration, and all evidence is vendor documentation with no independent/hands-on confirmation that these connectors actually surface content in answers. Missing for 10: explicit calendar connector support, independent corroboration of connector functionality, and clearer 'search across all three data sources in one answer' evidence.
- [claimed-docs] “Plugins can connect ChatGPT to the tools and information you use for work, such as Google Drive, SharePoint, Salesforce, or Gong.”
- [claimed-docs] “Install the Slack plugin to summarize channels or draft replies.”
- [claimed-docs] “Scheduled tasks can now start when a supported event occurs in Gmail, Slack, or GitHub.”
- [claimed-docs] “scheduled tasks can run when a supported Gmail, Slack, or GitHub event occurs”
- [claimed-docs] “Model Context Protocol (MCP) connects models to tools and context. Use it to give ChatGPT or Codex access to third-party documentation”
- [claimed-docs] “Use it to give ChatGPT or Codex access to third-party documentation, or to let it interact with developer tools like your browser or Figma.”
Google's docs describe first-party 'Connected Apps' letting Gemini summarize Gmail, create Calendar events, search across productivity apps, and use personal Gmail/Drive as sources for Deep Research, directly matching the story's cloud drive/email/calendar connector requirement. Missing for 10: independent/hands-on verification of search quality and confirmation that calendar reading (not just event creation) works reliably.
- [claimed-docs] “Gemini Apps can connect to other apps to complete your request and give you more helpful responses. With your permission, Gemini can also he…”
- [claimed-docs] “Gemini Apps can connect to other apps to complete your request and give you more helpful responses.”
- [claimed-docs] “With your permission, Gemini can also help you take actions, like editing and managing your content in other apps.”
- [claimed-docs] “You can change or add other sources, like your personal Gmail or Drive, for your research. You can also upload files and add NotebookLM note…”
- [claimed-docs] “Boost productivity: Summarize emails from Gmail, create events in Google Calendar or third-party apps, or search for content across a variet…”
- [claimed-docs] “Get started with Connected Apps in Gemini **Important:** For now, Gemini can’t use Connected Apps in Gemini in Google Messages.”
Files analysis — stories about files analysis in this arenaFiles analysis
Stories about files analysis in this arena
Analysis
power-userHave the assistant write and run code on my data to produce charts, computed answers, and downloadable files
weight 3 · round to ChatGPTChatGPT's Code Interpreter/Data Analysis capability (documented via file drafting, spreadsheet/PDF handling, interactive visualizations, and downloadable finished files) lets it write and execute code against uploaded data to produce charts and computed answers, then export results as documents/spreadsheets/PDFs. Docs explicitly describe analyzing files, building interactive visualizations, and downloading completed files, and community evidence corroborates real code/file execution capability (e.g., running code/containers, file interpretation plugins). Missing for 10: no dedicated first-party doc page specifically titled 'Code Interpreter/Data Analysis' in this pack, and no hands-on example showing a specific chart-from-CSV walkthrough.
- [claimed-docs] “Draft and refine documents, presentations, spreadsheets, and PDF files. Review the result, ask for specific changes, and download the comple…”
- [claimed-docs] “make interactive [visualizations], and build or share websites and apps with [Sites]”
- [claimed-docs] “Create a deck, analyze files, draft a report, build a project plan”
- [claimed-docs] “Search the web, browse websites, compare sources, read files, analyze data, and summarize findings.”
- [claimed-docs] “Draft and refine [documents, presentations, spreadsheets, and PDF files]. Review the result, ask for specific changes, and download the comp…”
- [community] “This is huge, essentially adding what people have been building with LangChain Tools into the core product. The browser and file-upload/inte…”
Gemininone0/10The evidence pack shows Gemini can import/upload code folders and GitHub repos and get 'insights' from uploaded documents/spreadsheets, but there is no evidence of an actual code-execution/sandbox feature that writes and runs code to produce charts, computed answers, or downloadable output files. No docs or community reports describe this specific data-analysis-with-code-execution workflow.
- [claimed-docs] “To add a GitHub repository: More Uploads Import Code and enter the GitHub repository or branch URL.”
- [claimed-docs] “To add a code folder: click More Uploads Import Code Upload folder.”
- [claimed-docs] “You can upload documents, spreadsheets, notebooks, photos, videos, and more to Gemini Apps to get answers, summaries, and insights about you…”
Artifacts
knowledge-workerHave the assistant create and iteratively edit documents, presentations, and other files I can export
weight 2 · round to ChatGPTDocs strongly support creating and iteratively editing documents, presentations, spreadsheets, and PDFs, with review/annotation and export/download workflows (chatgpt-docs-8, chatgpt-docs-30, chatgpt-docs-60, chatgpt-docs-66, chatgpt-docs-68, chatgpt-docs-70), plus a dedicated artifacts/canvas viewer supporting annotations and comments for iterative refinement (chatgpt-docs-7, chatgpt-docs-18, chatgpt-docs-24, chatgpt-docs-101). Missing for 10: independent hands-on corroboration specifically of document/presentation export workflows (community evidence pack focuses on other capabilities like coding/search, not file export).
- [claimed-docs] “Draft and refine documents, presentations, spreadsheets, and PDF files.”
- [claimed-docs] “Draft and refine documents, presentations, spreadsheets, and PDF files. Review the result, ask for specific changes, and download the comple…”
- [claimed-docs] “Draft and refine [documents, presentations, spreadsheets, and PDF files]. Review the result, ask for specific changes, and download the comp…”
- [claimed-docs] “Create a deck, analyze files, draft a report, build a project plan”
- [claimed-docs] “Turn research and analysis into documents, presentations, spreadsheets, and other finished work.”
- [claimed-docs] “Use ChatGPT Work to turn notes, docs, research, or meeting materials into a structured deck.”
- [claimed-docs] “Annotations let you point to a specific part of a file and tell ChatGPT what to change.”
- [claimed-docs] “a document editor can provide tools to find a section or add a comment”
- [claimed-docs] “Use annotations to point at a specific part of a supported preview and request a focused revision.”
Evidence documents Gemini's ability to generate and iteratively edit images (Nano Banana) with export/download at set resolutions, and general writing assistance, but there is no evidence of a dedicated document or presentation creation/editing workflow (e.g., a canvas-style doc/slide editor) with export to file formats. Missing for 10: explicit documentation of creating/iterating on text documents or slide decks, and exporting them to standard file formats (docx/pptx/pdf).
- [claimed-docs] “You can create captivating images in seconds with Nano Banana 2. From work, play, or anything in between, Gemini Apps can help you generate …”
- [claimed-docs] “Local edits: Making quick and specific changes to parts of an image.”
- [claimed-docs] “Character consistency: Maintain the look of a person or character across various generated images, including the ability to combine multiple…”
- [claimed-docs] “Download images at at 2K resolution with a Google AI plan or at 1K without an AI plan.”
- [claimed-docs] “Blend text, images, and video to bring your ideas to life.”
- [claimed-docs] “The Gemini app provides quick access to AI features, allowing you to get help with writing, brainstorming, coding, and more, without needing…”
Files
knowledge-workerUpload documents, spreadsheets, and PDFs and get accurate analysis of their contents
weight 3 · round to GeminiDocs confirm document/spreadsheet/PDF upload and analysis (draft/refine documents, presentations, spreadsheets, PDFs; 'analyze files'; 'read files, analyze data, and summarize findings'), which directly matches the story. However, evidence is entirely first-party documentation with no independent hands-on validation of accuracy specifically for uploaded file analysis (community evidence covers unrelated web-search accuracy issues, not file analysis). Missing for 10: independent/hands-on verification of analysis accuracy on real uploaded documents/spreadsheets/PDFs, and detail on limits (file size, complex spreadsheet formulas, OCR quality).
- [claimed-docs] “Draft and refine documents, presentations, spreadsheets, and PDF files.”
- [claimed-docs] “Draft and refine documents, presentations, spreadsheets, and PDF files. Review the result, ask for specific changes, and download the comple…”
- [claimed-docs] “Draft and refine [documents, presentations, spreadsheets, and PDF files]. Review the result, ask for specific changes, and download the comp…”
- [claimed-docs] “Create a deck, analyze files, draft a report, build a project plan”
- [claimed-docs] “Search the web, browse websites, compare sources, read files, analyze data, and summarize findings.”
First-party docs explicitly state users can upload documents, spreadsheets, and more to get answers, summaries, and insights (gemini-docs-25), and independent hands-on evidence corroborates accurate document/PDF analysis (a fintech user reporting 96% OCR accuracy on PDFs, comm-17) plus praise for NotebookLM's ability to comprehend large source corpora (comm-19). Missing for 10: dedicated evidence on spreadsheet-specific accuracy and broader independent benchmarking across document types beyond the single fintech PDF case.
- [claimed-docs] “You can upload documents, spreadsheets, notebooks, photos, videos, and more to Gemini Apps to get answers, summaries, and insights about you…”
- [community] “I work in fintech and we replaced an OCR vendor with Gemini at work for ingesting some PDFs... Processing time went from something like 12 m…”
- [community] “I've been using NotebookLM powered by Gemini 2.0 for three projects and it is really powerful for comprehending large corpuses you can't pos…”
Memory context — stories about memory context in this arenaMemory context
Stories about memory context in this arena
Memory
power-userHave the assistant remember relevant context from previous chats and apply it in new conversations
weight 3 · round drawnChatGPT's Memories feature explicitly carries useful context from earlier chats into future conversations, with dedicated docs on memory controls (personalize.md, customization/memories.md) and even CLI-level controls (/memories) for whether a chat can use or contribute to memories. Missing for 10: independent/hands-on community corroboration of cross-chat memory recall working reliably, and detail on limits/scope of what memories retain.
- [claimed-docs] “Memories let ChatGPT and Codex carry useful context from earlier work into future work.”
- [claimed-docs] “Memories let ChatGPT carry useful context from earlier chats into future work.”
- [claimed-docs] “In Codex CLI, use /memories in an interactive session to control whether the current chat can use existing local memories or become an input…”
- [claimed-docs] “use `/memories` to choose whether a chat can use local memories or contribute to future memories”
Gemini docs explicitly describe a personalization feature where 'the memory of your past Gemini chats' informs new conversations, plus user controls to review/delete that activity, confirming both the capability and its persistence across sessions. Missing for 10: independent hands-on validation of cross-chat memory recall accuracy and more detail on scope/duration of retained context.
- [claimed-docs] “You can get a personalized experience in Gemini Apps based on: The memory of your past Gemini chats”
- [claimed-docs] “You can review your prompts, delete your Gemini Apps activity, and turn off Keep Activity in My Activity at any time.”
power-userSet persistent custom instructions and preferences that shape every response
weight 1 · round to ChatGPTChatGPT supports persistent custom instructions/personalization and Memories that carry context across chats, plus Codex's global AGENTS.md for persistent personal instructions in coding workflows, and fine-grained control via /memories. Missing for 10: independent hands-on verification that custom instructions reliably shape *every* response over long-term use, and no detail on limits/scope conflicts between memories and per-chat overrides.
- [claimed-docs] “Memories let ChatGPT and Codex carry useful context from earlier work into future work.”
- [claimed-docs] “Memories let ChatGPT carry useful context from earlier chats into future work.”
- [claimed-docs] “In Codex, these personal instructions are stored in your global `AGENTS.md` file.”
- [claimed-docs] “use `/memories` to choose whether a chat can use local memories or contribute to future memories”
- [claimed-docs] “In Codex CLI, use /memories in an interactive session to control whether the current chat can use existing local memories or become an input…”
Gemini offers a 'memory of past chats' personalization feature (gemini-docs-18) and custom Gems that let users pre-set instructions/goals (gemini-docs-9), which together approximate persistent customization, but there's no documented single 'custom instructions' setting that explicitly shapes every response across all chats. Missing for 10: explicit persistent instruction/preference settings applied globally to every response, independent verification of memory consistency, and detail on how memory interacts with different chat sessions.
- [claimed-docs] “You can get a personalized experience in Gemini Apps based on: The memory of your past Gemini chats”
- [claimed-docs] “To create a custom Gem to meet your goals, provide info about your request and requirements.”
- [claimed-docs] “Start your day with a personalized digest from Gemini. Your Daily Brief understands your rhythms and personal preferences to deliver your to…”
Projects
knowledge-workerOrganize related chats and files into a project or space that shares context and instructions
weight 2 · round to ChatGPTChatGPT's Projects feature explicitly lets users organize related chats, share context/instructions, and even attach local folders/files for shared context, directly matching the story. Memories also reinforce carrying context across chats. Missing for 10: independent/hands-on corroboration of the Projects feature working as described, and detail on instruction-sharing UI beyond docs claims.
- [claimed-docs] “Use a project to organize related chats and give ChatGPT the context it needs.”
- [claimed-docs] “A local project gives chats access to one or more folders on your computer, such as a collection of source files or a codebase.”
- [claimed-docs] “Memories let ChatGPT and Codex carry useful context from earlier work into future work.”
- [claimed-docs] “Memories let ChatGPT carry useful context from earlier chats into future work.”
Gemini's 'Notebooks' feature explicitly provides a continuous, focused space that remembers sources, instructions, and ongoing chat history, and lets users upload documents, code, and other files into that same organized context (gemini-docs-11, gemini-docs-24, gemini-docs-25, gemini-docs-29). Custom Gems (gemini-docs-9) further let users set persistent instructions for recurring tasks, and independent commentary confirms NotebookLM/Notebooks are genuinely useful for organizing large source sets into a coherent project workspace (gemini-comm-19), though that same source flags UI/display shortcomings. Missing for 10: no first-party doc detailing multi-chat organization within one persistent 'project' the way competitors define it, and no hands-on evidence of shared-instruction consistency across many chats in the same notebook.
- [claimed-docs] “Notebooks provide a continuous chat experience that remembers your sources, instructions, and ongoing discussions.”
- [claimed-docs] “you can use notebooks to organize your projects in a dedicated, focused space.”
- [claimed-docs] “You can upload documents, spreadsheets, notebooks, photos, videos, and more to Gemini Apps to get answers, summaries, and insights about you…”
- [claimed-docs] “You can change or add other sources, like your personal Gmail or Drive, for your research. You can also upload files and add NotebookLM note…”
- [claimed-docs] “To create a custom Gem to meet your goals, provide info about your request and requirements.”
- [community] “I've been using NotebookLM powered by Gemini 2.0 for three projects and it is really powerful for comprehending large corpuses you can't pos…”
Multimodal — stories about multimodal in this arenaMultimodal
Stories about multimodal in this arena
Images
knowledge-workerGenerate and edit images from natural-language prompts
weight 2 · round to ChatGPTChatGPT's docs explicitly cover image generation and editing from natural-language prompts, including editing via Comment/annotations, reference images, and Canvas view for reviewing multiple images, plus mobile app support for generating/transforming images and creating stickers. This directly matches the knowledge-worker story of generating and editing images conversationally. Missing for 10: no independent/hands-on community corroboration specifically for image generation quality or reliability.
- [claimed-docs] “Ask ChatGPT to generate or edit images.”
- [claimed-docs] “use Comment to add precise feedback to one or more images”
- [claimed-docs] “Generate or edit images, make interactive visualizations, and build or share websites and apps with Sites.”
- [claimed-docs] “Ask ChatGPT to generate or edit images. Use image generation for UI assets, banners, backgrounds, illustrations”
- [claimed-docs] “Switch between Focused view to inspect one image and Canvas view to see the images generated in the same chat.”
- [claimed-docs] “In Canvas view, use Comment to add precise feedback to one or more images.”
- [claimed-docs] “Add a reference image when you want ChatGPT to transform an existing asset or use it as visual guidance.”
- [claimed-docs] “Image generation–Generate original images from a description, or transform existing ones with a few simple words.”
- [claimed-docs] “Choose an image and style, remix with emojis, and make personalized stickers that are ready to share in your group chat.”
Gemini docs explicitly describe generating and editing images via natural-language prompts (Nano Banana 2), including local edits, character consistency, blending images, and resolution export options. missing for 10: independent hands-on corroboration of image editing quality and no community evidence testing this specific feature.
- [claimed-docs] “You can create captivating images in seconds with Nano Banana 2. From work, play, or anything in between, Gemini Apps can help you generate …”
- [claimed-docs] “Local edits: Making quick and specific changes to parts of an image.”
- [claimed-docs] “Character consistency: Maintain the look of a person or character across various generated images, including the ability to combine multiple…”
- [claimed-docs] “You can create captivating images in seconds with Nano Banana 2.”
- [claimed-docs] “Download images at at 2K resolution with a Google AI plan or at 1K without an AI plan.”
- [claimed-docs] “Blend text, images, and video to bring your ideas to life.”
knowledge-workerShare screenshots and photos and have the assistant accurately interpret what is in them
weight 2 · round to ChatGPTFirst-party docs confirm ChatGPT accepts photo/screenshot uploads and interprets their content (e.g., transcribing a handwritten recipe or identifying a landmark from a photo), which directly matches the knowledge-worker use case of sharing images for analysis. Missing for 10: independent/hands-on evidence corroborating accuracy of image interpretation, and explicit documentation of screenshot-specific analysis (e.g., UI screenshots) rather than just general photo uploads.
- [claimed-docs] “Photo upload—Snap or upload a picture to transcribe a handwritten recipe or get info about a landmark.”
- [claimed-docs] “Snap or upload a picture to transcribe a handwritten recipe or get info about a landmark.”
- [claimed-docs] “Image generation–Generate original images from a description, or transform existing ones with a few simple words.”
Geminidisputedcontradicted4/10Docs confirm users can upload photos/screenshots and get insights (gemini-docs-25) and get screen-context help via 'Hey Google' (gemini-docs-6), but hands-on reports show real accuracy failures: Gemini generated 'wildly inaccurate bounding boxes' on images (gemini-comm-18) and failed to correctly interpret a manipulated photo, missing key details even in the latest version (gemini-comm-12). A positive OCR use case exists (gemini-comm-17) but that's for PDFs, not general screenshot/photo scene interpretation, so the core 'accurately interpret' claim is contradicted by concrete hands-on failures. Missing for 10: independent benchmarks specifically on screenshot/UI interpretation accuracy, and resolution of the documented bounding-box/detail-recognition failures.
- [claimed-docs] “You can upload documents, spreadsheets, notebooks, photos, videos, and more to Gemini Apps to get answers, summaries, and insights about you…”
- [claimed-docs] “On Android, say “Hey Google” to ask Gemini for help with what’s on your screen”
- [community] “Unfortunately Gemini really seems to struggle on this, and no matter how we tried prompting it, it would generate wildly inaccurate bounding…”
- [community] “It still failed my image identification test (a photoshopped picture of a dog with 5 legs)... Gemini 3 however, while still failing, at leas…”
- [community] “I work in fintech and we replaced an OCR vendor with Gemini at work for ingesting some PDFs... Processing time went from something like 12 m…”
Voice
knowledge-workerHave a natural, real-time voice conversation with the assistant
weight 2 · round drawnChatGPT ships Advanced Voice Mode for real-time spoken conversation on mobile ('Tap the soundwave icon to have a real-time convo on the go') and ChatGPT Voice on desktop that lets users talk through ideas, start work, check progress, or change direction without switching to typing. This directly matches the knowledge-worker story of natural, real-time voice conversation. Missing for 10: independent hands-on corroboration of voice quality/latency and any community verification beyond vendor docs.
- [claimed-docs] “Advanced Voice Mode–Tap the soundwave icon to have a real-time convo on the go.”
- [claimed-docs] “Advanced Voice Mode–Tap the soundwave icon to have a real-time convo on the go. Settle a dinner table debate, or practice a new language.”
- [claimed-docs] “ChatGPT Voice lets you talk through ideas and coordinate tasks in Chat, Work, and Codex in the ChatGPT desktop app. Start work, check progre…”
- [claimed-docs] “Start work, check progress, or change direction without switching back to typing.”
- [claimed-docs] “ChatGPT Voice lets you talk through ideas and coordinate tasks in Chat, Work, and Codex in the ChatGPT desktop app.”
- [claimed-docs] “ChatGPT Voice can start separate tasks for longer work, check existing tasks, and send follow-up instructions.”
Gemini Live is explicitly documented to support natural, free-flowing, interruptible voice conversations where Gemini talks back verbally in real time, directly matching the story. Missing for 10: independent/hands-on community corroboration specifically validating the voice conversation experience (community evidence covers other Gemini features/models but not Live voice interactions).
- [claimed-docs] “With Gemini Live, you can have a natural, free-flowing conversation with Gemini. You can even interrupt it like when you want to add more de…”
- [claimed-docs] “With Gemini Live, you can have a natural, free-flowing conversation with Gemini.”
- [claimed-docs] “With Gemini Live, you can: * **Talk back and forth:** Talk to Gemini without typing, and Gemini will respond back verbally.”
- [claimed-docs] “you can have a natural, free-flowing conversation with Gemini. You can even interrupt it like when you want to add more details or change th…”
Openness — open source, data portability, and self-hosting storiesOpenness
Open source, data portability, and self-hosting stories
ai-native userDo everything through the API that I can do in the UI
weight 2 · round to ChatGPTThere is evidence of programmatic access via the Codex CLI/SDK for coding automation and CI jobs, and a security-scan CLI, but nothing shows that the broad set of UI-only capabilities (Canvas/image editing, Computer Use, Voice, Work desktop automations, Sites, MCP-in-chat, memories, scheduled tasks) are exposed through a general ChatGPT API with parity to the UI. A probe for an OpenAPI/swagger spec on the docs site also returned 404s, reinforcing the lack of a documented full API surface. missing for 10: documented API endpoints mirroring Canvas/image tools, Computer Use, Voice, Work automations, Sites, and MCP-in-chat; independent confirmation of API-UI feature parity.
- [claimed-docs] “Use the Codex SDK to automate coding tasks, including jobs in CI.”
- [claimed-docs] “Inspect code, make changes, run commands, and automate repeatable work without leaving your terminal.”
- [claimed-docs] “Use its command-line interface (CLI) to scan repositories you own or have permission to assess, review findings over time”
- [probe] “PROBE openapi: all candidate paths 404 (https://learn.chatgpt.com/openapi.json, https://learn.chatgpt.com/swagger.json, https://learn.chatgp…”
Gemininone0/10The evidence pack details many Gemini Apps UI-only features (Gems, Deep Research, Connected Apps, Daily Brief, Live conversations, Notebooks, scheduling) with no corresponding API documentation showing these are exposed programmatically. Community evidence (e.g., gemini-comm-11) even notes the API/AI Studio path is more restricted (capped temperature, hidden top_k) than the full product, reinforcing that API and UI capability sets are not equivalent.
- [claimed-docs] “You can conduct in-depth and real-time research on almost any subject with Deep Research in Gemini Apps.”
- [claimed-docs] “To create a custom Gem to meet your goals, provide info about your request and requirements.”
- [claimed-docs] “Gemini Apps can connect to other apps to complete your request and give you more helpful responses. With your permission, Gemini can also he…”
- [claimed-docs] “With Gemini Live, you can have a natural, free-flowing conversation with Gemini. You can even interrupt it like when you want to add more de…”
- [claimed-docs] “Start your day with a personalized digest from Gemini. Your Daily Brief understands your rhythms and personal preferences to deliver your to…”
- [community] “Temperature continues to be gated to maximum of 0.2, and there's still the hidden top_k of 64 that you can't turn off. I love the google AI …”
ai-native userExport all of my data in open formats and leave
weight 3 · round drawnChatGPTnone0/10No evidence in the pack of a data export feature producing open/portable formats, nor any mention of account data export or deletion workflow. Evidence only covers general product features (Codex, Work, MCP, image gen, etc.) with nothing about exporting personal chat/data history or leaving the platform with your data intact.
Gemininone0/10Evidence covers activity deletion/review (gemini-docs-31) but no mention of exporting data in open/portable formats or data-export tooling for leaving the platform. Missing for 10: any documentation of a data export/download feature (e.g., Google Takeout integration), open-format export options, or independent confirmation of data portability.
- [claimed-docs] “You can review your prompts, delete your Gemini Apps activity, and turn off Keep Activity in My Activity at any time.”
Privacy posture — data-handling and privacy storiesPrivacy posture
Data-handling and privacy stories
ai-native userChoose where my data is stored (region/residency)
weight 2 · round drawnChatGPTnone0/10No evidence in the pack addresses data residency or regional storage choice for ChatGPT; the evidence only touches general data controls (e.g., Computer Use screenshots) without any region-selection capability.
Gemininone0/10The evidence pack covers Gemini's features and privacy/data-use settings (e.g., activity retention, human review opt-outs) but contains no mention of data residency or regional storage controls for AI-native/enterprise users. missing for 10: any documentation of region selection, data residency guarantees, or enterprise data-location controls.
ai-native userPrevent my data from being used to train AI models
weight 3 · round to GeminiChatGPTnone0/10The evidence pack contains no documentation or mention of data-training opt-out controls, privacy settings, or 'improve the model for everyone' toggles; only tangential data-controls mention (chatgpt-docs-89) refers to Computer Use screenshots, not training opt-out generally. Missing for 10: any reference to training-data opt-out settings, business/API data-usage policies, or privacy dashboard controls.
- [claimed-docs] “Your ChatGPT data controls apply to content processed through ChatGPT, including screenshots taken by Computer Use.”
Google's support docs state that chats and uploaded files in Gemini Apps are not reviewed by humans or used to improve generative AI models, and users can manage/delete their activity via 'My Activity' and toggle 'Keep Activity' off (gemini-docs-38, gemini-docs-31). However, a separate doc notes that audio/video from Gemini Live can be used to train models unless the user opts out, showing the training-data control is feature-specific rather than a single unified setting (gemini-docs-37). Missing for 10: a clear, centralized opt-out control covering all Gemini surfaces (not just chat text), independent/hands-on verification that opted-out data is truly excluded from training, and clarity for enterprise/Workspace accounts.
- [claimed-docs] “Your chats and uploaded files in Gemini Apps won’t be reviewed by human reviewers or otherwise used to improve generative AI models.”
- [claimed-docs] “You can let Google use your audio and Gemini Live videos and screenshares to improve and develop its services, including training generative…”
- [claimed-docs] “You can review your prompts, delete your Gemini Apps activity, and turn off Keep Activity in My Activity at any time.”
ai-native userControl data retention and deletion
weight 2 · round to GeminiThe only relevant evidence is a passing reference that 'ChatGPT data controls apply to content processed through ChatGPT, including screenshots taken by Computer Use,' implying some data-control/retention settings exist, but no documentation details how to view, export, or delete data, set retention periods, or manage memory deletion. Missing for 10: explicit data retention/export/delete documentation, memory deletion controls, and independent confirmation that these controls work as described.
- [claimed-docs] “Your ChatGPT data controls apply to content processed through ChatGPT, including screenshots taken by Computer Use.”
Google's support docs explicitly describe controls to review prompts, delete Gemini Apps activity, and turn off 'Keep Activity' (gemini-docs-31), plus explicit statements that chats/files are not used for model training by default (gemini-docs-38) while audio/video in Gemini Live can be opted in/out of training use (gemini-docs-37) — directly matching retention/deletion control for an AI-native user. Missing for 10: independent/hands-on verification that deletion actually removes data, and detail on data export or full account-level deletion workflows.
- [claimed-docs] “You can review your prompts, delete your Gemini Apps activity, and turn off Keep Activity in My Activity at any time.”
- [claimed-docs] “You can let Google use your audio and Gemini Live videos and screenshares to improve and develop its services, including training generative…”
- [claimed-docs] “Your chats and uploaded files in Gemini Apps won’t be reviewed by human reviewers or otherwise used to improve generative AI models.”
ai-native userOpt out of telemetry and usage tracking
weight 2 · round to GeminiChatGPTnone0/10None of the evidence addresses telemetry/usage-tracking opt-out controls for ChatGPT; the docs cover data controls tangentially (e.g., data usage for Computer Use screenshots) but nothing about disabling telemetry or usage analytics.
Docs show Gemini provides some privacy controls: reviewing/deleting prompts and turning off 'Keep Activity' (gemini-docs-31), and settings around whether audio/video/chats are used to improve models (gemini-docs-37, gemini-docs-38). This addresses usage-history and training-data opt-out but not a full 'telemetry/usage tracking' opt-out (e.g., diagnostic/product analytics separate from chat history) and there's no independent verification of these controls' effectiveness. Missing for 10: explicit documentation of a general telemetry/analytics opt-out toggle beyond activity/training-data settings, and independent/hands-on confirmation that opting out actually stops tracking.
- [claimed-docs] “You can review your prompts, delete your Gemini Apps activity, and turn off Keep Activity in My Activity at any time.”
- [claimed-docs] “You can let Google use your audio and Gemini Live videos and screenshares to improve and develop its services, including training generative…”
- [claimed-docs] “Your chats and uploaded files in Gemini Apps won’t be reviewed by human reviewers or otherwise used to improve generative AI models.”
Research answers — stories about research answers in this arenaResearch answers
Stories about research answers in this arena
Research
knowledge-workerLaunch a deep research run that autonomously searches many sources and returns a cited report
weight 3 · round to GeminiDocs show ChatGPT can search the web, browse multiple sources, compare them, and produce cited results in-chat (docs-86, docs-90), and can turn research into finished documents/reports (docs-66, docs-68, docs-60). However, there is no explicit mention of a dedicated 'Deep Research' autonomous multi-source research mode/run with a structured long-form cited report as its own distinct feature — the evidence only shows general web-search-with-citations and document drafting capabilities. missing for 10: an explicitly named deep-research mode, evidence of autonomous long-running multi-source research runs, and a structured final cited-report output format.
- [claimed-docs] “Search results and citations appear in the chat when ChatGPT uses web search.”
- [claimed-docs] “Search the web, browse websites, compare sources, read files, analyze data, and summarize findings.”
- [claimed-docs] “Create a deck, analyze files, draft a report, build a project plan”
- [claimed-docs] “Turn research and analysis into documents, presentations, spreadsheets, and other finished work.”
- [claimed-docs] “Draft and refine [documents, presentations, spreadsheets, and PDF files]. Review the result, ask for specific changes, and download the comp…”
- [claimed-docs] “In the CLI, pass `--search` to fetch live results for one run”
Gemini Apps has a documented Deep Research feature that conducts in-depth, real-time research across sources and allows adding personal sources (Gmail, Drive, uploads, NotebookLM notebooks) for the research, matching the autonomous multi-source research story. missing for 10: explicit documentation of citation formatting in the final report and independent hands-on evidence of report quality/citation accuracy from third parties.
- [claimed-docs] “You can conduct in-depth and real-time research on almost any subject with Deep Research in Gemini Apps.”
- [claimed-docs] “You can change or add other sources, like your personal Gmail or Drive, for your research. You can also upload files and add NotebookLM note…”
knowledge-workerGet answers grounded in current web results with citations back to the sources
weight 2 · round to ChatGPTChatGPT's web search docs explicitly state that search results and citations appear in chat when web search is used, and describe searching, browsing, comparing sources, and summarizing findings — directly matching the story. Community feedback shows mixed satisfaction with search accuracy (e.g., comparisons to Perplexity, a weather inaccuracy) but doesn't concretely show citations failing to appear, so this doesn't rise to a dispute. Missing for 10: independent verification of citation accuracy/consistency across many queries and no first-party detail on citation formatting/source diversity.
- [claimed-docs] “Search results and citations appear in the chat when ChatGPT uses web search.”
- [claimed-docs] “Search the web, browse websites, compare sources, read files, analyze data, and summarize findings.”
- [claimed-docs] “In the CLI, pass `--search` to fetch live results for one run”
- [community] “AKA Bing Search in ChatGPT. So it is not it's own search engine and is still using Bing for its results just like the rest of them.”
- [community] “I gave it a quick spin and my initial impression is much worse than perplexity.”
- [community] “I asked it the current weather in my area and the temperature was off by 23 degrees F.”
Gemini's Deep Research feature explicitly supports 'in-depth and real-time research on almost any subject,' implying grounded, current-web-based answers, and users can add further sources for research context. However, the evidence pack never explicitly documents inline citation links back to specific web sources, nor independent confirmation that citations are reliably shown. Missing for 10: explicit documentation of citation/source-linking UI, independent verification that citations are accurate and consistently displayed.
- [claimed-docs] “You can conduct in-depth and real-time research on almost any subject with Deep Research in Gemini Apps.”
- [claimed-docs] “You can change or add other sources, like your personal Gmail or Drive, for your research. You can also upload files and add NotebookLM note…”
Trust controls — stories about trust controls in this arenaTrust controls
Stories about trust controls in this arena
Admin
team-adminManage members, permissions, and data policies for my organization's workspace
weight 2 · round to ChatGPTOnly a single explicit doc line notes that a workspace administrator can control access to ChatGPT Work, plugins, browser use, and network access, plus general mentions of GPT builders using 'approved workspace apps' and data controls applying to processed content. There is no evidence of a full admin console covering member invitation/removal, granular role-based permissions, or explicit data retention/training-opt-out policy controls. missing for 10: admin console/member management UI, granular role/permission settings, explicit data-retention and training-opt-out policy controls, independent corroboration of these admin features.
- [claimed-docs] “Your workspace administrator can control access to ChatGPT Work, plugins, browser use, and network access.”
- [claimed-docs] “Allow GPT builders to use approved workspace apps or configure actions that interact with permitted third-party APIs.”
- [claimed-docs] “configure actions that interact with permitted third-party APIs”
- [claimed-docs] “Your ChatGPT data controls apply to content processed through ChatGPT, including screenshots taken by Computer Use.”
Gemininone0/10Evidence only shows end-user features (image generation, Live conversations, notebooks, connected apps) and passing mentions that a 'work or school account' administrator can 'enable Gemini,' but there is no evidence of an admin console for managing members, setting permissions, or configuring org-wide data policies.
Data controls
knowledge-workerExport my complete chat history and account data
weight 1 · round drawnChatGPTnone0/10The evidence pack contains no mention of data export, account data download, or chat history export features anywhere in the docs, community, or probe results; there is only a passing reference to 'data controls' applying to Computer Use content, which does not address exporting complete chat history or account data. Missing for 10: any documentation of an export data feature, its scope (chats, files, settings), format, or process, and any independent corroboration it works.
- [claimed-docs] “Your ChatGPT data controls apply to content processed through ChatGPT, including screenshots taken by Computer Use.”
Gemininone0/10The evidence only covers reviewing and deleting Gemini activity via My Activity (gemini-docs-31) and privacy settings for Live audio (gemini-docs-37, gemini-docs-38), but there is no mention of exporting chat history or account data (e.g., via Google Takeout or a dedicated export tool). missing for 10: any documentation of an export/download feature for chat history or account data, independent confirmation of export completeness.
- [claimed-docs] “You can review your prompts, delete your Gemini Apps activity, and turn off Keep Activity in My Activity at any time.”
knowledge-workerControl whether my conversations are used to train models
weight 3 · round to GeminiChatGPTnone0/10The evidence pack covers ChatGPT's agentic/feature capabilities (Codex, Work, MCP, Computer Use, etc.) but contains no documentation or mention of data controls, training opt-out settings, or 'Improve the model for everyone' toggles that let a user control whether their conversations are used for model training.
Google's own support docs state that by default chats/uploaded files aren't used to improve generative AI models unless a user opts in via 'Keep Activity', and that users can review, delete activity, or turn off Keep Activity at any time; separately, users must explicitly consent to let Google use Gemini Live audio/video for model training. This gives knowledge-workers clear, documented control over whether conversations feed model training. Missing for 10: independent/hands-on verification that these settings work as described and clarity on enterprise/Workspace account defaults.
- [claimed-docs] “You can review your prompts, delete your Gemini Apps activity, and turn off Keep Activity in My Activity at any time.”
- [claimed-docs] “You can let Google use your audio and Gemini Live videos and screenshares to improve and develop its services, including training generative…”
- [claimed-docs] “Your chats and uploaded files in Gemini Apps won’t be reviewed by human reviewers or otherwise used to improve generative AI models.”
Not comparable on these axes
ai-native userConnect an agent via an official MCP server
weight 3 · not comparableChatGPT/Codex has official first-party documentation for MCP support: users can add official MCP servers (e.g., Context7 for docs, Figma, browser tools) to extend the agent, configuration is stored in config.toml, servers can be scoped per-project, and setup carries across clients without redoing it. This directly evidences 'connecting an agent via an official MCP server.' Missing for 10: independent/hands-on community corroboration of MCP server usage and broader detail on the range of officially supported/verified servers beyond the Context7 example.
- [claimed-docs] “let it interact with developer tools like your browser or Figma”
- [claimed-docs] “Once you configure your MCP servers, you can switch among those clients without redoing setup.”
- [claimed-docs] “Use it to give ChatGPT or Codex access to third-party documentation, or to let it interact with developer tools like your browser or Figma.”
- [claimed-docs] “to add Context7 (a free MCP server for developer documentation)”
- [claimed-docs] “you can also scope MCP servers to a project with `.codex/config.toml` (trusted projects only)”
- [claimed-docs] “Codex stores MCP configuration in `config.toml`”
Geminin/aGemini is itself an AI assistant/agent product; the evidence shows only that it can connect outward to other apps (Connected Apps, code import) as a client, not that it exposes an official MCP server for other agents to connect to it. Per the agent-role exception, this axis does not apply unless Gemini documents running as an MCP server, which no evidence here shows.
- [claimed-docs] “Gemini Apps can connect to other apps to complete your request and give you more helpful responses. With your permission, Gemini can also he…”
- [claimed-docs] “Gemini Apps can connect to other apps to complete your request and give you more helpful responses.”
- [claimed-docs] “With your permission, Gemini can also help you take actions, like editing and managing your content in other apps.”
- [claimed-docs] “Get started with Connected Apps in Gemini **Important:** For now, Gemini can’t use Connected Apps in Gemini in Google Messages.”
ai-native userIssue scoped/least-privilege API credentials for an agent
weight 2 · not comparableChatGPT/Codex docs show some least-privilege mechanisms for agent tool access — MCP servers can be scoped to a trusted project via `.codex/config.toml` (chatgpt-docs-55), sandbox command rules can restrict what Codex can run outside the sandbox (chatgpt-docs-95), and workspace admins can control access to browser use, plugins, and network access (chatgpt-docs-88), plus GPT builders can configure scoped third-party API actions (chatgpt-docs-34, chatgpt-docs-44). However there is no evidence of a user-facing mechanism to actually issue/generate a scoped or least-privilege API credential/token specifically for an agent to use. missing for 10: explicit scoped API key/token issuance workflow, granularity/expiry controls on such credentials, and any documentation or independent confirmation that a user can mint a restricted-permission credential for agent use.
- [claimed-docs] “you can also scope MCP servers to a project with `.codex/config.toml` (trusted projects only)”
- [claimed-docs] “Use rules to control which commands Codex can run outside the sandbox.”
- [claimed-docs] “Your workspace administrator can control access to ChatGPT Work, plugins, browser use, and network access.”
- [claimed-docs] “Allow GPT builders to use approved workspace apps or configure actions that interact with permitted third-party APIs.”
- [claimed-docs] “configure actions that interact with permitted third-party APIs”
Geminin/aGemini is an AI product/app, not a platform issuing API credentials with scoped permissions for agents; the evidence pack covers consumer features (imports, images, connected apps) with no mention of credential scoping or API key/permission management for agents, so this axis is a category error for this product's evidenced role.
ai-native userExplore an interactive API reference with runnable examples
weight 2 · not comparableChatGPTnone0/10The evidence pack contains no documentation of an interactive API reference or runnable code examples; a direct probe for OpenAPI/swagger specs on the docs site returned 404 across all candidate paths, and no other citation mentions an API reference sandbox or runnable snippets.
- [probe] “PROBE openapi: all candidate paths 404 (https://learn.chatgpt.com/openapi.json, https://learn.chatgpt.com/swagger.json, https://learn.chatgp…”
Geminin/aThis story asks for an interactive API reference with runnable examples, which is a developer-portal/API-docs concern, not a fair axis for Gemini Apps (the consumer product surfaced in this evidence pack). The evidence pack covers Gemini's app features (Gems, Connected Apps, Deep Research, Live) but contains no mention of an interactive API reference, making this a wrong-axis question for the product as evidenced.
ai-native userDownload a machine-readable API spec (OpenAPI or equivalent)
weight 2 · not comparableChatGPTnone0/10The probe explicitly checked for an OpenAPI/machine-readable spec at all standard locations and found only 404s, with no other evidence pack item showing a downloadable API spec for ChatGPT itself.
- [probe] “PROBE openapi: all candidate paths 404 (https://learn.chatgpt.com/openapi.json, https://learn.chatgpt.com/swagger.json, https://learn.chatgp…”
Geminin/aGemini here refers to the consumer AI assistant app/product; the story asks for a downloadable machine-readable API spec (OpenAPI), which is a developer-platform artifact, not a consumer app feature. This axis applies to API/platform products, not to the Gemini Apps experience described in the evidence.
ai-native userTest against a sandbox environment without touching production data
weight 1 · not comparableCodex documentation indicates commands run inside a sandbox by default, with rules governing what can run 'outside the sandbox' (chatgpt-docs-95), and Codex Security's scanning is scoped to repos you own/have permission to assess (chatgpt-docs-97/98), implying isolated execution rather than direct production access. However, there is no explicit documentation describing a dedicated 'test/staging' environment distinct from production data, no detail on how production systems/data are excluded, and no independent/hands-on confirmation of this isolation. Missing for 10: explicit sandbox-vs-production data separation docs, details on network/data isolation guarantees, and independent verification of the sandbox boundary holding in practice.
- [claimed-docs] “Use rules to control which commands Codex can run outside the sandbox.”
- [claimed-docs] “Use its command-line interface (CLI) to scan repositories you own or have permission to assess, review findings over time”
- [claimed-docs] “Use `npx @openai/codex-security bulk-scan` to review repositories in one campaign.”
ai-native userRead the product's source under an open license
weight 2 · not comparableChatGPTn/aChatGPT is a closed-source, proprietary SaaS product; open-sourcing its source code is not a plausible axis for this kind of product (unlike an open-source framework or tool), so this is a category mismatch rather than a missing capability.
ai-native userSelf-host the core product
weight 3 · not comparableChatGPTn/aChatGPT is a closed, hosted proprietary product with no evidence of a self-hostable core model or server; self-hosting is not a fair axis for this SaaS product category (it does not ship open weights or an installable core).