[codex-app-server-test-client & codex-app-server] Plugin Usage Analytics Smoke Test (#27099)

## This PR

The original [combined remote plugin analytics PR
#26281](https://github.com/openai/codex/pull/26281) mixed reusable
analytics test infrastructure, two manual smoke workflows, a metadata
refactor, and the final identity behavior. This PR establishes a
non-mutating end-to-end plugin smoke workflow before any analytics
identity semantics change.

- Add `plugin-analytics-smoke` to the existing app-server test client.
- Exercise plugin disable, enable, and use through production app-server
RPC paths.
- Isolate config writes in a temporary file and use a loopback Responses
API server.
- Capture analytics without sending them to the production analytics
backend.
- Validate the current local `plugin_id`, names, capability metadata,
thread, turn, and model fields.

This is intentionally a baseline smoke workflow. It does not assert
`remote_plugin_id`; the final PR will update it when that field exists.
Review this PR as the net diff against #27093.

## Testing

- The test-client target compiles successfully.
- The combined reference branch exercised the manual smoke against the
live remote plugin service.
- CI is green across the required platform matrix.

## Split Overview

```text
main
├── #27093  Debug analytics capture
│   └── #27099  Non-mutating plugin smoke           ← you are here
│       └── #27100  Remote install/uninstall smoke
└── #27102  Plugin telemetry metadata refactor

After #27093, #27099, #27100, and #27102 merge:
└── Final PR: add remote_plugin_id to plugin analytics
```

Review order and dependencies:

1. [#27093 Add debug-only analytics event
capture](https://github.com/openai/codex/pull/27093) (based on `main`)
2. [#27099 Add a plugin analytics smoke
workflow](https://github.com/openai/codex/pull/27099) **(this PR,
stacked on #27093)**
3. [#27100 Add a remote plugin analytics mutation smoke
workflow](https://github.com/openai/codex/pull/27100) (stacked on this
PR)
4. [#27102 Centralize plugin telemetry metadata
construction](https://github.com/openai/codex/pull/27102) (independent,
based on `main`)
5. Final remote-ID behavior PR (created after PRs 1-4 merge)

The original [#26281](https://github.com/openai/codex/pull/26281)
remains open as the green aggregate reference until the final PR is
published.
This commit is contained in:
jameswt-oai authored and GitHub committed 2026-06-16 10:11:41 -07:00
1 parent a544f5a612
commit a376781a3c
5 files changed
+779

No files matched your search

@@ -87,6 +87,9 @@ use tungstenite::stream::MaybeTlsStream;
use url::Url;
use uuid::Uuid;
mod loopback_responses_server;
mod plugin_analytics_smoke;
const NOTIFICATIONS_TO_OPT_OUT: &[&str] = &[
// v2 item deltas.
"command/exec/outputDelta",
@@ -272,6 +275,16 @@ enum CliCommand {
#[arg(long, default_value_t = 15)]
hold_seconds: u64,
},
/// Exercise remote plugin analytics through production app-server RPC paths.
#[command(name = "plugin-analytics-smoke")]
PluginAnalyticsSmoke {
/// Installed local plugin id, such as `linear@openai-curated-remote`.
#[arg(long)]
plugin_id: String,
/// JSONL output path. Defaults to a PID-specific file under the system temp directory.
#[arg(long)]
capture_file: Option<PathBuf>,
},
}
pub async fn run() -> Result<()> {
@@ -423,6 +436,17 @@ pub async fn run() -> Result<()> {
hold_seconds,
)
}
CliCommand::PluginAnalyticsSmoke {
plugin_id,
capture_file,
} => {
ensure_dynamic_tools_unused(&dynamic_tools, "plugin-analytics-smoke")?;
if url.is_some() {
bail!("plugin-analytics-smoke requires --codex-bin and does not support --url");
}
let codex_bin = codex_bin.context("plugin-analytics-smoke requires --codex-bin")?;
plugin_analytics_smoke::run(&codex_bin, &config_overrides, &plugin_id, capture_file)
}
}
}
@@ -1440,6 +1464,14 @@ impl CodexClient {
}
fn spawn_stdio(codex_bin: &Path, config_overrides: &[String]) -> Result<Self> {
Self::spawn_stdio_with_env(codex_bin, config_overrides, &[])
}
fn spawn_stdio_with_env(
codex_bin: &Path,
config_overrides: &[String],
environment: &[(OsString, OsString)],
) -> Result<Self> {
let codex_bin_display = codex_bin.display();
let mut cmd = Command::new(codex_bin);
if let Some(codex_bin_parent) = codex_bin.parent() {
@@ -1453,6 +1485,9 @@ impl CodexClient {
for override_kv in config_overrides {
cmd.arg("--config").arg(override_kv);
}
for (name, value) in environment {
cmd.env(name, value);
}
let mut codex_app_server = cmd
.arg("app-server")
.stdin(Stdio::piped())