[codex-app-server-test-client] Plugin Install/Uninstall Analytics Smoke Test (#27100)

## This PR

The original [combined remote plugin analytics PR
#26281](https://github.com/openai/codex/pull/26281) mixed reusable
analytics test infrastructure, two manual smoke workflows, a metadata
refactor, and the final identity behavior. This PR adds the
account-mutating validation workflow separately so its cleanup and
recovery guarantees can be reviewed without the final analytics behavior
change.

- Add a manually invoked remote plugin install/uninstall smoke workflow.
- Require explicit account-mutation confirmation and an initially
uninstalled plugin.
- Validate the current `codex_plugin_installed` contract, where
`plugin_id` is the backend ID.
- Restore and verify the original uninstalled state, with a dedicated
recovery command.

This baseline intentionally does not require `codex_plugin_uninstalled`,
because production does not emit that event yet. The final PR will
update this smoke to require local `plugin_id`, `remote_plugin_id`, and
uninstall emission. Review this PR as the net diff against #27099.

## Testing

- `just test -p codex-app-server-test-client` (3 focused
capture/validation tests passed)
- The live workflow was previously exercised on the green combined
reference branch, and the original uninstalled account state was
restored.
- CI is green across the required platform matrix.

## Split Overview

```text
main
├── #27093  Debug analytics capture
│   └── #27099  Non-mutating plugin smoke
│       └── #27100  Remote install/uninstall smoke  ← you are here
└── #27102  Plugin telemetry metadata refactor

After #27093, #27099, #27100, and #27102 merge:
└── Final PR: add remote_plugin_id to plugin analytics
```

Review order and dependencies:

1. [#27093 Add debug-only analytics event
capture](https://github.com/openai/codex/pull/27093) (based on `main`)
2. [#27099 Add a plugin analytics smoke
workflow](https://github.com/openai/codex/pull/27099) (stacked on
#27093)
3. [#27100 Add a remote plugin analytics mutation smoke
workflow](https://github.com/openai/codex/pull/27100) **(this PR,
stacked on #27099)**
4. [#27102 Centralize plugin telemetry metadata
construction](https://github.com/openai/codex/pull/27102) (independent,
based on `main`)
5. Final remote-ID behavior PR (created after PRs 1-4 merge)

The original [#26281](https://github.com/openai/codex/pull/26281)
remains open as the green aggregate reference until the final PR is
published.
This commit is contained in:
jameswt-oai authored and GitHub committed 2026-06-16 12:28:45 -07:00
1 parent 7baf7e467e
commit 8a40200880
7 files changed
+798 -3

No files matched your search

@@ -88,6 +88,8 @@ use url::Url;
use uuid::Uuid;
mod loopback_responses_server;
mod plugin_analytics_capture;
mod plugin_analytics_mutation_smoke;
mod plugin_analytics_smoke;
const NOTIFICATIONS_TO_OPT_OUT: &[&str] = &[
@@ -285,6 +287,29 @@ enum CliCommand {
#[arg(long)]
capture_file: Option<PathBuf>,
},
/// Install and uninstall one remote plugin while validating analytics capture.
#[command(name = "plugin-analytics-mutation-smoke")]
PluginAnalyticsMutationSmoke {
/// Backend remote plugin id. The plugin must be initially uninstalled.
#[arg(long)]
remote_plugin_id: String,
/// Acknowledge that this command mutates the active account's plugin state.
#[arg(long)]
confirm_account_mutation: bool,
/// JSONL output path. Defaults to a PID-specific file under the system temp directory.
#[arg(long)]
capture_file: Option<PathBuf>,
},
/// Best-effort recovery command that uninstalls one remote plugin.
#[command(name = "plugin-remote-uninstall")]
PluginRemoteUninstall {
/// Backend remote plugin id to uninstall.
#[arg(long)]
remote_plugin_id: String,
/// Acknowledge that this command mutates the active account's plugin state.
#[arg(long)]
confirm_account_mutation: bool,
},
}
pub async fn run() -> Result<()> {
@@ -447,6 +472,47 @@ pub async fn run() -> Result<()> {
let codex_bin = codex_bin.context("plugin-analytics-smoke requires --codex-bin")?;
plugin_analytics_smoke::run(&codex_bin, &config_overrides, &plugin_id, capture_file)
}
CliCommand::PluginAnalyticsMutationSmoke {
remote_plugin_id,
confirm_account_mutation,
capture_file,
} => {
ensure_dynamic_tools_unused(&dynamic_tools, "plugin-analytics-mutation-smoke")?;
if url.is_some() {
bail!(
"plugin-analytics-mutation-smoke requires --codex-bin and does not support --url"
);
}
let codex_bin =
codex_bin.context("plugin-analytics-mutation-smoke requires --codex-bin")?;
plugin_analytics_mutation_smoke::run(
&codex_bin,
&config_overrides,
&remote_plugin_id,
plugin_analytics_mutation_smoke::AccountMutationConfirmation::from_flag(
confirm_account_mutation,
),
capture_file,
)
}
CliCommand::PluginRemoteUninstall {
remote_plugin_id,
confirm_account_mutation,
} => {
ensure_dynamic_tools_unused(&dynamic_tools, "plugin-remote-uninstall")?;
if url.is_some() {
bail!("plugin-remote-uninstall requires --codex-bin and does not support --url");
}
let codex_bin = codex_bin.context("plugin-remote-uninstall requires --codex-bin")?;
plugin_analytics_mutation_smoke::run_cleanup(
&codex_bin,
&config_overrides,
&remote_plugin_id,
plugin_analytics_mutation_smoke::AccountMutationConfirmation::from_flag(
confirm_account_mutation,
),
)
}
}
}