references/lobe-chat
1
0
Fork 0
mirror of https://github.com/lobehub/lobe-chat.git synced 2026-06-14 03:30:19 +00:00
Code Issues Packages Projects Releases Wiki Activity

🔨 chore(testing): automate local auth setup (#15790)

Browse Source 
🧪 test(agent): automate local auth setup
This commit is contained in:
Arvin Xu
2026-06-14 02:00:49 +08:00
committed by GitHub
parent 913ee4210d
commit fa58fd12a0
8 changed files with 482 additions and 68 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.agents/skills/agent-testing/SKILL.md
+24 -17
View File
@@ -157,10 +157,12 @@ full-stack `dev` command so Next can proxy the SPA HTML from Vite:
Useful subcommands:
```bash
./.agents/skills/agent-testing/scripts/init-dev-env.sh env # print exports
./.agents/skills/agent-testing/scripts/init-dev-env.sh write # write .records/env/agent-testing-dev.env
./.agents/skills/agent-testing/scripts/init-dev-env.sh migrate # migrations only
./.agents/skills/agent-testing/scripts/init-dev-env.sh clean-db # remove managed DB container
./.agents/skills/agent-testing/scripts/init-dev-env.sh env # print exports
./.agents/skills/agent-testing/scripts/init-dev-env.sh write # write .records/env/agent-testing-dev.env
./.agents/skills/agent-testing/scripts/init-dev-env.sh migrate # migrations only
./.agents/skills/agent-testing/scripts/init-dev-env.sh seed-user # seed user + CLI API key
./.agents/skills/agent-testing/scripts/init-dev-env.sh qstash # local QStash for workflow paths
./.agents/skills/agent-testing/scripts/init-dev-env.sh clean-db # remove managed DB container
```
Default script env:
@@ -169,14 +171,18 @@ Default script env:
- `DATABASE_URL=postgresql://postgres:postgres@localhost:5433/postgres`
- `DATABASE_DRIVER=node`
- `FEATURE_FLAGS=-agent_self_iteration` so local smoke does not require QStash
- Local QStash defaults (`QSTASH_URL`, `QSTASH_TOKEN`, signing keys) are exported;
run `init-dev-env.sh qstash` in a separate terminal when the path under test
triggers QStash/Workflow.
- `KEY_VAULTS_SECRET`, `AUTH_SECRET`, auth verification off
- S3 mock vars
- Managed DB container: `lobehub-agent-testing-postgres`
`seed-user` creates `agent-testing@lobehub.com` / `TestPassword123!` with
onboarding already completed for manual or agent-browser checks. When running
Cucumber against this dev server, pass the same script env into the test process
too; Cucumber has its own `BeforeAll` seed path and it must see `DATABASE_URL`
onboarding already completed, plus a local API key in
`.records/env/agent-testing-cli.env` for CLI automation. When running Cucumber
against this dev server, pass the same script env into the test process too;
Cucumber has its own `BeforeAll` seed path and it must see `DATABASE_URL`
instead of silently skipping setup:
```bash
@@ -199,22 +205,23 @@ Electron login state unless the test spans those surfaces.
Use `status` with no `--surface` only for cross-surface test plans.
| Surface | Mechanism | One-key path | Standard check |
| -------- | ------------------------------------------------- | ------------------------------ | ----------------------------------------- |
| CLI | OIDC Device Code Flow (`apps/cli/.lobehub-dev`) | `setup-auth.sh cli` | `setup-auth.sh status --surface cli` |
| Web | better-auth cookie injection into `agent-browser` | `pbpaste \| setup-auth.sh web` | `setup-auth.sh status --surface web` |
| Electron | App's own persistent login state | Log in once in the app | `setup-auth.sh status --surface electron` |
| Bot | Native apps already logged in | — | per-platform screenshot |
| Surface | Mechanism | One-key path | Standard check |
| -------- | --------------------------------------------- | ------------------------ | ----------------------------------------- |
| CLI | Seeded API key, device-code fallback | `setup-auth.sh cli-seed` | `setup-auth.sh status --surface cli` |
| Web | Seeded better-auth login into `agent-browser` | `setup-auth.sh web-seed` | `setup-auth.sh status --surface web` |
| Electron | App's own persistent login state | Log in once in the app | `setup-auth.sh status --surface electron` |
| Bot | Native apps already logged in | — | per-platform screenshot |
Login-state checks are standardized — do NOT hand-roll `window.__LOBE_STORES`
eval snippets; use `scripts/app-probe.sh auth` (returns `{ isSignedIn, userId }`,
works for Electron CDP and web sessions via `AB_TARGET`).
For Web tests, the test surface is always `agent-browser --session lobehub-dev`.
The user's normal Chrome is only a source for copying the Cookie header when
`status --surface web` fails. If Chrome is already logged in, do not open a
login page; verify agent-browser first, then request the Network `Cookie:`
header only if that verification fails. Full background and failure modes:
Use `setup-auth.sh web-seed` first in the seeded local env. The user's normal
Chrome is only a source for copying the Cookie header when seed auth is not
available or `status --surface web` still fails. If Chrome is already logged in,
do not open a login page; verify agent-browser first, then request the Network
`Cookie:` header only if that verification fails. Full background and failure modes:
[references/auth.md](./references/auth.md).
## Step 1 — Pick the surface by change scope
.agents/skills/agent-testing/cli/index.md
+26 -16
View File
@@ -13,17 +13,18 @@ flakiness.
## Prerequisites
| Requirement | Details |
| ------------ | --------------------------------------------------------------------------------- |
| Dev server | `localhost:3010` — see [../references/dev-server.md](../references/dev-server.md) |
| Requirement | Details |
| ------------ | ---------------------------------------------------------------------------------------------------------------------------------------------- |
| Dev server | `localhost:3010` — see [../references/dev-server.md](../references/dev-server.md) |
| CLI source | `apps/cli/` — runs from source, no rebuild; standalone `node_modules` — run `pnpm install` inside `apps/cli/` (root install does not cover it) |
| CLI dev mode | `LOBEHUB_CLI_HOME=.lobehub-dev` for isolated credentials |
| Auth | Device Code Flow login — see [../references/auth.md](../references/auth.md) |
| CLI dev mode | `LOBEHUB_CLI_HOME=.lobehub-dev` for isolated settings |
| Auth | Seeded API key first; Device Code Flow only as fallback — see [../references/auth.md](../references/auth.md) |
All CLI dev commands run from `apps/cli/`. Subsequent examples use `$CLI`:
```bash
CLI="LOBEHUB_CLI_HOME=.lobehub-dev bun src/index.ts"
source ../../.records/env/agent-testing-cli.env
CLI="bun src/index.ts"
```
## Workflow
@@ -39,14 +40,23 @@ check, start, and restart commands. Server-side code changes require a restart.
./.agents/skills/agent-testing/scripts/setup-auth.sh status
```
If the CLI is not logged in, **the user must run the login themselves**
(interactive browser authorization):
If the CLI is not ready in the seeded local environment:
```bash
./.agents/skills/agent-testing/scripts/init-dev-env.sh seed-user
source .records/env/agent-testing-cli.env
./.agents/skills/agent-testing/scripts/setup-auth.sh cli-seed
```
If the target environment is not seeded, use the interactive fallback:
```bash
cd apps/cli && LOBEHUB_CLI_HOME=.lobehub-dev bun src/index.ts login --server http://localhost:3010
```
Credentials persist in `apps/cli/.lobehub-dev/`. Details:
Seeded API-key auth does not store credentials. It writes local settings under
`$HOME/.lobehub-dev` and requires the generated env file to be sourced before
CLI commands. Details:
[../references/auth.md](../references/auth.md).
### Step 3 — Test with CLI commands
@@ -133,10 +143,10 @@ $CLI provider test <provider-id>
## Troubleshooting
| Issue | Solution |
| --------------------------- | ----------------------------------------------- |
| `No authentication found` | Run `login --server http://localhost:3010` |
| `UNAUTHORIZED` on API calls | Token expired; re-run login |
| `ECONNREFUSED` | Dev server not running — see dev-server.md |
| CLI shows old data/behavior | Server needs restart to pick up code changes |
| Login opens wrong server | Must use `--server` flag (env var doesn't work) |
| Issue | Solution |
| --------------------------- | ------------------------------------------------------------------------------------------------------ |
| `No authentication found` | Source `.records/env/agent-testing-cli.env`, or run device-code `login --server http://localhost:3010` |
| `UNAUTHORIZED` on API calls | Re-run `init-dev-env.sh seed-user` and re-source the env file; for device-code fallback, re-run login |
| `ECONNREFUSED` | Dev server not running — see dev-server.md |
| CLI shows old data/behavior | Server needs restart to pick up code changes |
| Login opens wrong server | Must use `--server` flag (env var doesn't work) |
.agents/skills/agent-testing/references/auth.md
+56 -16
View File
@@ -19,8 +19,10 @@ Quick reference after initialization:
| ------------------------------ | -------------------------------------------------- |
| `$SCRIPT status` | Check all surfaces (server + CLI + web + Electron) |
| `$SCRIPT status --surface web` | Check only the Web surface gate |
| `$SCRIPT cli-seed` | Configure CLI API-key auth from the seeded key |
| `$SCRIPT cli` | Interactive CLI device-code login (user must run) |
| `$SCRIPT open-chrome` | Open Chrome at `SERVER_URL` with DevTools |
| `$SCRIPT web-seed` | Sign in the seeded user and inject cookies |
| `pbpaste \| $SCRIPT web` | Inject a copied Cookie header into agent-browser |
| `$SCRIPT web-verify` | Live-check agent-browser session auth |
@@ -29,20 +31,42 @@ not `127.0.0.1`.
## Per-surface overview
| Surface | Mechanism | Persistence | Human interaction |
| -------- | ---------------------------------------- | ----------------------------------------------------------------- | ----------------------------------------------- |
| CLI | OIDC Device Code Flow | `apps/cli/.lobehub-dev/settings.json` | Yes — browser authorization, every token expiry |
| Web | better-auth cookie injection | `~/.lobehub-agent-testing/web-state.json` + agent-browser session | Copy the Cookie header once per token rotation |
| Electron | App's own login state | Electron user-data dir | Log in once manually in the app |
| Bot | Native apps (Discord/WeChat/…) logged in | Each app's own session | Once per app |
| Surface | Mechanism | Persistence | Human interaction |
| -------- | ---------------------------------------- | ----------------------------------------------------------------- | ---------------------------------------------- |
| CLI | Seeded API key or OIDC Device Code Flow | `.records/env/agent-testing-cli.env` + `$HOME/.lobehub-dev` | No for seed path; yes for device-code fallback |
| Web | Seeded better-auth login or cookie copy | `~/.lobehub-agent-testing/web-state.json` + agent-browser session | No for seed path; copy cookie only as fallback |
| Electron | App's own login state | Electron user-data dir | Log in once manually in the app |
| Bot | Native apps (Discord/WeChat/…) logged in | Each app's own session | Once per app |
## CLI — Device Code Flow
## CLI — Seeded API key
For the self-contained no-root-`.env` dev environment, seed the baseline user
and API key once:
```bash
./.agents/skills/agent-testing/scripts/init-dev-env.sh seed-user
source .records/env/agent-testing-cli.env
./.agents/skills/agent-testing/scripts/setup-auth.sh cli-seed
```
The seed step writes `LOBE_API_KEY` for humans and maps it to the CLI's current
auth variable, `LOBEHUB_CLI_API_KEY`. It also sets `LOBEHUB_SERVER` so CLI
commands hit the local server without needing a stored device-code token.
Use this for automated CLI verification:
```bash
cd apps/cli
source ../../.records/env/agent-testing-cli.env
bun src/index.ts <command>
```
## CLI — Device Code Flow fallback
Use device-code login only when testing against a non-seeded environment.
Credentials are isolated from the user's real CLI config via
`LOBEHUB_CLI_HOME=.lobehub-dev` (kept inside `apps/cli/`, gitignored).
Login requires interactive browser authorization, so **the user must run it
themselves** (e.g. via the `!` prefix in Claude Code):
`LOBEHUB_CLI_HOME=.lobehub-dev`, which the current CLI stores under
`$HOME/.lobehub-dev`.
```bash
cd apps/cli && LOBEHUB_CLI_HOME=.lobehub-dev bun src/index.ts login --server http://localhost:3010
@@ -51,16 +75,31 @@ cd apps/cli && LOBEHUB_CLI_HOME=.lobehub-dev bun src/index.ts login --server htt
- The `--server` flag is required — an env var does NOT work and login will hit
the wrong server without it.
- Check state without logging in: `setup-auth.sh status` (verifies
`settings.json` exists and `serverUrl` matches).
`LOBEHUB_CLI_API_KEY` when present, otherwise checks the stored server URL).
- `UNAUTHORIZED` on API calls means the token expired — re-run login.
## Web — better-auth cookie injection (agent-browser)
## Web — seeded better-auth login
The Web test surface is `agent-browser --session lobehub-dev`. The user's
ordinary Chrome is only a cookie source; Chrome screenshots, Chrome Network
records, and Chrome logged-in state do not prove the agent-browser test session
is authenticated.
For the seeded local dev environment, use the automatic path:
```bash
./.agents/skills/agent-testing/scripts/init-dev-env.sh seed-user
./.agents/skills/agent-testing/scripts/setup-auth.sh web-seed
```
`web-seed` posts the seeded email/password to
`/api/auth/sign-in/email`, stores the returned cookie jar under
`~/.lobehub-agent-testing/`, converts it to Playwright `storageState`, loads it
into the `agent-browser` session, and verifies the session does not land on
`/signin`.
## Web — manual cookie injection fallback
`agent-browser --headed` on macOS often creates the Chromium window off-screen —
the user can't see or interact with it, so manual login inside the agent-browser
session fails. Instead, copy the **better-auth session cookie** out of the
@@ -72,9 +111,10 @@ secret: don't paste it into shared logs, PRs, or commit it anywhere.
### Web — decision flow
1. `$SCRIPT status --surface web` — green? Start testing. Do not ask for a Cookie header.
2. Not green → `$SCRIPT open-chrome` opens Chrome at `SERVER_URL` with DevTools.
3. User copies the `Cookie:` header from Network tab → any same-origin request → Request Headers → right-click `Cookie:` → **Copy value**. Must be from Network, NOT `document.cookie` (HttpOnly cookies are invisible to `document.cookie`).
4. `pbpaste | $SCRIPT web` — filters to better-auth cookies (`session_token`, `session_data`, `state`), builds Playwright `storageState`, loads it into the `agent-browser` session (`lobehub-dev`), opens `SERVER_URL`, and asserts the URL is not `/signin`.
2. Not green and using the seeded local env → `$SCRIPT web-seed`.
3. Still not green or not using the seed env → `$SCRIPT open-chrome` opens Chrome at `SERVER_URL` with DevTools.
4. User copies the `Cookie:` header from Network tab → any same-origin request → Request Headers → right-click `Cookie:` → **Copy value**. Must be from Network, NOT `document.cookie` (HttpOnly cookies are invisible to `document.cookie`).
5. `pbpaste | $SCRIPT web` — filters to better-auth cookies (`session_token`, `session_data`, `state`), builds Playwright `storageState`, loads it into the `agent-browser` session (`lobehub-dev`), opens `SERVER_URL`, and asserts the URL is not `/signin`.
### Using the authenticated session
.agents/skills/agent-testing/references/dev-server.md
+13 -5
View File
@@ -60,6 +60,9 @@ bun run dev
# Without root .env:
./.agents/skills/agent-testing/scripts/init-dev-env.sh dev
# Local QStash. Run in a separate terminal only when testing workflow paths.
./.agents/skills/agent-testing/scripts/init-dev-env.sh qstash
# Restart — required to pick up server-side code changes
lsof -ti:"$PORT" | xargs kill
pnpm run dev:next
@@ -83,8 +86,13 @@ in doubt.
## Troubleshooting
| Issue | Solution |
| ------------------------- | ------------------------------------------------------- |
| `ECONNREFUSED` | Server not running — start it |
| `EADDRINUSE` on the port | Already running — `lsof -ti:<port> \| xargs kill` first |
| Stale data / old behavior | Server needs a restart to pick up code changes |
| Issue | Solution |
| ------------------------- | --------------------------------------------------------------------------------------------- |
| `ECONNREFUSED` | Server not running — start it |
| `EADDRINUSE` on the port | Already running — `lsof -ti:<port> \| xargs kill` first |
| Stale data / old behavior | Server needs a restart to pick up code changes |
| QStash workflow failures | Start `init-dev-env.sh qstash` and make sure dev server inherited the script's `QSTASH_*` env |
Marketplace/community endpoints are not part of the local agent-testing auth
gate. Do not block local product-chain verification on marketplace API auth
unless the change explicitly targets marketplace behavior.
.agents/skills/agent-testing/scripts/init-dev-env.sh
+119 -2
View File
@@ -14,13 +14,14 @@
# init-dev-env.sh write [file] # write a source-able env file
# init-dev-env.sh setup-db # start local Postgres and run migrations
# init-dev-env.sh migrate # run DB migrations against the configured DB
# init-dev-env.sh seed-user # seed the baseline test user
# init-dev-env.sh seed-user # seed the baseline test user + CLI API key
# init-dev-env.sh qstash # run local Upstash QStash dev server
# init-dev-env.sh dev-next # exec `pnpm run dev:next` with this env
# init-dev-env.sh dev # exec `bun run dev` with this env
# init-dev-env.sh clean-db # remove the managed Postgres container
#
# Overrides:
# SERVER_PORT=3010 DB_PORT=5433 DB_CONTAINER=lobehub-agent-testing-postgres
# SERVER_PORT=3010 DB_PORT=5433 DB_CONTAINER=lobehub-agent-testing-postgres QSTASH_DEV_PORT=8080
set -euo pipefail
@@ -32,6 +33,12 @@ DB_PORT="${DB_PORT:-5433}"
DB_CONTAINER="${DB_CONTAINER:-lobehub-agent-testing-postgres}"
DATABASE_URL="${DATABASE_URL:-postgresql://postgres:postgres@localhost:${DB_PORT}/postgres}"
ENV_FILE_DEFAULT="$REPO_ROOT/.records/env/agent-testing-dev.env"
CLI_ENV_FILE_DEFAULT="$REPO_ROOT/.records/env/agent-testing-cli.env"
AGENT_TESTING_API_KEY="${AGENT_TESTING_API_KEY:-sk-lh-agenttesting0001}"
QSTASH_DEV_PORT="${QSTASH_DEV_PORT:-8080}"
QSTASH_LOCAL_TOKEN="${QSTASH_LOCAL_TOKEN:-eyJVc2VySUQiOiJkZWZhdWx0VXNlciIsIlBhc3N3b3JkIjoiZGVmYXVsdFBhc3N3b3JkIn0=}"
QSTASH_LOCAL_CURRENT_SIGNING_KEY="${QSTASH_LOCAL_CURRENT_SIGNING_KEY:-sig_7kYjw48mhY7kAjqNGcy6cr29RJ6r}"
QSTASH_LOCAL_NEXT_SIGNING_KEY="${QSTASH_LOCAL_NEXT_SIGNING_KEY:-sig_5ZB6DVzB1wjE8S6rZ7eenA8Pdnhs}"
ok() { printf ' \033[32m✔\033[0m %s\n' "$1"; }
bad() { printf ' \033[31m✘\033[0m %s\n' "$1"; }
@@ -57,6 +64,11 @@ apply_env() {
export NEXT_PUBLIC_AUTH_EMAIL_VERIFICATION="${NEXT_PUBLIC_AUTH_EMAIL_VERIFICATION:-0}"
export NODE_OPTIONS="${NODE_OPTIONS:---max-old-space-size=6144}"
export PORT="${PORT:-$SERVER_PORT}"
export QSTASH_CURRENT_SIGNING_KEY="${QSTASH_CURRENT_SIGNING_KEY:-$QSTASH_LOCAL_CURRENT_SIGNING_KEY}"
export QSTASH_DEV_PORT
export QSTASH_NEXT_SIGNING_KEY="${QSTASH_NEXT_SIGNING_KEY:-$QSTASH_LOCAL_NEXT_SIGNING_KEY}"
export QSTASH_TOKEN="${QSTASH_TOKEN:-$QSTASH_LOCAL_TOKEN}"
export QSTASH_URL="${QSTASH_URL:-http://127.0.0.1:${QSTASH_DEV_PORT}}"
export S3_ACCESS_KEY_ID="${S3_ACCESS_KEY_ID:-agent-testing-access-key}"
export S3_BUCKET="${S3_BUCKET:-agent-testing-bucket}"
export S3_ENDPOINT="${S3_ENDPOINT:-https://agent-testing-s3.localhost}"
@@ -75,6 +87,11 @@ env_keys() {
NEXT_PUBLIC_AUTH_EMAIL_VERIFICATION \
NODE_OPTIONS \
PORT \
QSTASH_CURRENT_SIGNING_KEY \
QSTASH_DEV_PORT \
QSTASH_NEXT_SIGNING_KEY \
QSTASH_TOKEN \
QSTASH_URL \
S3_ACCESS_KEY_ID \
S3_BUCKET \
S3_ENDPOINT \
@@ -148,9 +165,14 @@ migrate_db() {
seed_user() {
apply_env
export AGENT_TESTING_API_KEY
export AGENT_TESTING_CLI_ENV_FILE="${AGENT_TESTING_CLI_ENV_FILE:-$CLI_ENV_FILE_DEFAULT}"
cd "$REPO_ROOT"
node <<'NODE'
const bcrypt = require('bcryptjs');
const crypto = require('node:crypto');
const fs = require('node:fs');
const path = require('node:path');
const pg = require('pg');
const databaseUrl = process.env.DATABASE_URL;
@@ -166,13 +188,72 @@ const TEST_USER = {
username: 'agent_testing_user',
};
const TEST_API_KEY = {
id: 'api_key_agent_testing_001',
key: process.env.AGENT_TESTING_API_KEY || 'sk-lh-agenttesting0001',
name: 'Agent Testing CLI API Key',
};
const validateApiKeyFormat = (apiKey) => /^sk-lh-[\da-z]{16}$/.test(apiKey);
const hashApiKey = (apiKey) => {
const secret = process.env.KEY_VAULTS_SECRET;
if (!secret) throw new Error('KEY_VAULTS_SECRET is required to seed the baseline API key.');
return crypto.createHmac('sha256', secret).update(apiKey).digest('hex');
};
const encryptWithKeyVaultsSecret = (plaintext) => {
const secret = process.env.KEY_VAULTS_SECRET;
if (!secret) throw new Error('KEY_VAULTS_SECRET is required to seed the baseline API key.');
const rawKey = Buffer.from(secret, 'base64');
if (![16, 24, 32].includes(rawKey.length)) {
throw new Error(
`KEY_VAULTS_SECRET must decode to 16, 24, or 32 bytes, got ${rawKey.length} bytes.`,
);
}
const iv = crypto.randomBytes(12);
const cipher = crypto.createCipheriv(`aes-${rawKey.length * 8}-gcm`, rawKey, iv);
const encrypted = Buffer.concat([cipher.update(plaintext, 'utf8'), cipher.final()]);
const authTag = cipher.getAuthTag();
return `${iv.toString('hex')}:${authTag.toString('hex')}:${encrypted.toString('hex')}`;
};
const writeCliEnvFile = () => {
const file = process.env.AGENT_TESTING_CLI_ENV_FILE || '.records/env/agent-testing-cli.env';
fs.mkdirSync(path.dirname(file), { recursive: true });
fs.writeFileSync(
file,
[
'# Source this file before running LobeHub CLI agent tests.',
'# Generated by init-dev-env.sh seed-user',
`export LOBE_API_KEY=${TEST_API_KEY.key}`,
`export LOBEHUB_CLI_API_KEY="${'${LOBE_API_KEY}'}"`,
`export LOBEHUB_SERVER=${process.env.APP_URL}`,
'export LOBEHUB_CLI_HOME=.lobehub-dev',
'',
].join('\n'),
);
return file;
};
const client = new pg.Client({ connectionString: databaseUrl });
(async () => {
if (!validateApiKeyFormat(TEST_API_KEY.key)) {
throw new Error(`Invalid AGENT_TESTING_API_KEY format: ${TEST_API_KEY.key}`);
}
await client.connect();
const now = new Date().toISOString();
const onboarding = JSON.stringify({ finishedAt: now, version: 1 });
const passwordHash = await bcrypt.hash(TEST_USER.password, 10);
const encryptedApiKey = encryptWithKeyVaultsSecret(TEST_API_KEY.key);
const apiKeyHash = hashApiKey(TEST_API_KEY.key);
await client.query(
`INSERT INTO users (id, email, normalized_email, username, full_name, email_verified, onboarding, created_at, updated_at, last_active_at)
@@ -204,9 +285,35 @@ const client = new pg.Client({ connectionString: databaseUrl });
],
);
await client.query(
`INSERT INTO api_keys (id, name, key, key_hash, enabled, expires_at, user_id, workspace_id, created_at, updated_at)
VALUES ($1, $2, $3, $4, $5, NULL, $6, NULL, $7, $7)
ON CONFLICT (id) DO UPDATE
SET name = EXCLUDED.name,
key = EXCLUDED.key,
key_hash = EXCLUDED.key_hash,
enabled = EXCLUDED.enabled,
expires_at = NULL,
updated_at = EXCLUDED.updated_at`,
[
TEST_API_KEY.id,
TEST_API_KEY.name,
encryptedApiKey,
apiKeyHash,
true,
TEST_USER.id,
now,
],
);
const cliEnvFile = writeCliEnvFile();
console.log('seeded baseline user:');
console.log(` email: ${TEST_USER.email}`);
console.log(` password: ${TEST_USER.password}`);
console.log('seeded baseline API key:');
console.log(` LOBE_API_KEY: ${TEST_API_KEY.key}`);
console.log(` CLI env: ${cliEnvFile}`);
})()
.finally(() => client.end())
.catch((error) => {
@@ -222,6 +329,7 @@ cmd_status() {
note "APP_URL=$APP_URL"
note "DATABASE_URL=$DATABASE_URL"
note "PORT=$PORT"
note "QSTASH_URL=$QSTASH_URL"
if command -v docker > /dev/null 2>&1; then
ok "docker CLI available"
if docker ps --format '{{.Names}}' | grep -Fxq "$DB_CONTAINER"; then
@@ -234,6 +342,14 @@ cmd_status() {
fi
}
cmd_qstash() {
apply_env
cd "$REPO_ROOT"
note "starting local QStash dev server at $QSTASH_URL"
note "keep this process running while testing workflow paths"
exec pnpm run qstash -- -port "$QSTASH_DEV_PORT"
}
cmd_dev_next() {
apply_env
cd "$REPO_ROOT"
@@ -279,6 +395,7 @@ case "$COMMAND" in
;;
migrate) migrate_db ;;
seed-user) seed_user ;;
qstash) cmd_qstash ;;
dev-next) cmd_dev_next ;;
dev) cmd_dev ;;
clean-db) cmd_clean_db ;;
.agents/skills/agent-testing/scripts/setup-auth.sh
+149 -7
View File
@@ -7,8 +7,10 @@
# Usage:
# setup-auth.sh status # check server + CLI + web + Electron readiness
# setup-auth.sh status --surface web # check only the Web surface gate
# setup-auth.sh cli-seed # configure CLI API-key auth from seeded local env
# setup-auth.sh cli # interactive CLI device-code login (run by a human)
# setup-auth.sh open-chrome # open SERVER_URL in Chrome and show DevTools
# setup-auth.sh web-seed # sign in seeded user and inject cookies automatically
# setup-auth.sh web # stdin = Cookie header -> inject into agent-browser session
# setup-auth.sh web-verify # live-check the agent-browser session is authenticated
#
@@ -16,6 +18,7 @@
# SERVER_URL (default from test-env.sh) dev server under test
# SESSION (default lobehub-dev) agent-browser session name
# AUTH_DIR (default ~/.lobehub-agent-testing) where web state is persisted
# SEED_EMAIL / SEED_PASSWORD seeded better-auth login
set -euo pipefail
@@ -78,7 +81,13 @@ SERVER_URL="${SERVER_URL:-$(default_server_url)}"
SESSION="${SESSION:-lobehub-dev}"
AUTH_DIR="${AUTH_DIR:-$HOME/.lobehub-agent-testing}"
STATE_FILE="$AUTH_DIR/web-state.json"
CLI_HOME="$REPO_ROOT/apps/cli/.lobehub-dev"
CLI_HOME_NAME="${LOBEHUB_CLI_HOME:-.lobehub-dev}"
CLI_HOME="$HOME/${CLI_HOME_NAME#/}"
CLI_CREDENTIALS_FILE="$CLI_HOME/credentials.json"
SEED_EMAIL="${SEED_EMAIL:-agent-testing@lobehub.com}"
SEED_PASSWORD="${SEED_PASSWORD:-TestPassword123!}"
SEED_API_KEY="${SEED_API_KEY:-${AGENT_TESTING_API_KEY:-sk-lh-agenttesting0001}}"
CLI_ENV_FILE="${CLI_ENV_FILE:-$REPO_ROOT/.records/env/agent-testing-cli.env}"
ok() { printf ' \033[32m✔\033[0m %s\n' "$1"; }
bad() { printf ' \033[31m✘\033[0m %s\n' "$1"; }
@@ -88,8 +97,10 @@ usage() {
cat << EOF
Usage:
$0 status [--surface all|cli|web|electron]
$0 cli-seed
$0 cli
$0 open-chrome [--dry-run]
$0 web-seed
$0 web
$0 web-verify
@@ -97,6 +108,8 @@ Env:
SERVER_URL=$SERVER_URL
SESSION=$SESSION
AUTH_DIR=$AUTH_DIR
SEED_EMAIL=$SEED_EMAIL
CLI_HOME=$CLI_HOME
EOF
}
@@ -113,11 +126,35 @@ check_server() {
}
check_cli() {
if [[ -f "$CLI_HOME/settings.json" ]] && grep -q "$SERVER_URL" "$CLI_HOME/settings.json"; then
ok "CLI logged in to $SERVER_URL (creds: apps/cli/.lobehub-dev)"
local api_key="${LOBEHUB_CLI_API_KEY:-${LOBE_API_KEY:-}}"
if [[ -n "$api_key" ]]; then
local body_file code
body_file="$(mktemp)"
code=$(curl -sS -o "$body_file" -w '%{http_code}' \
-H "Authorization: Bearer $api_key" \
"$SERVER_URL/api/v1/users/me?includeCount=0" 2> /dev/null || true)
if [[ "$code" =~ ^[23] ]]; then
rm -f "$body_file"
ok "CLI API-key auth valid for $SERVER_URL"
return 0
fi
bad "CLI API-key auth failed for $SERVER_URL (http_code='$code')"
note "seed the local API key first:"
note "./.agents/skills/agent-testing/scripts/init-dev-env.sh seed-user"
note "source $CLI_ENV_FILE"
rm -f "$body_file"
return 1
fi
if [[ -f "$CLI_HOME/settings.json" ]] && grep -q "$SERVER_URL" "$CLI_HOME/settings.json" && [[ -f "$CLI_CREDENTIALS_FILE" ]]; then
ok "CLI device-code credentials configured for $SERVER_URL (creds: $CLI_HOME)"
else
bad "CLI not logged in to $SERVER_URL"
note "ask the user to run:"
note "automated path:"
note "./.agents/skills/agent-testing/scripts/init-dev-env.sh seed-user && source $CLI_ENV_FILE && $0 cli-seed"
note "interactive fallback:"
note "cd apps/cli && LOBEHUB_CLI_HOME=.lobehub-dev bun src/index.ts login --server $SERVER_URL"
return 1
fi
@@ -128,7 +165,8 @@ check_web() {
ok "web auth state saved ($STATE_FILE)"
else
bad "no web auth state for agent-browser"
note "copy the Cookie header from Chrome DevTools (Network tab), then:"
note "for the seeded local user, run: $0 web-seed"
note "or copy the Cookie header from Chrome DevTools (Network tab), then:"
note "pbpaste | $0 web (see references/auth.md)"
return 1
fi
@@ -246,6 +284,44 @@ cmd_cli() {
LOBEHUB_CLI_HOME=.lobehub-dev bun src/index.ts login --server "$SERVER_URL"
}
write_cli_seed_env() {
mkdir -p "$(dirname "$CLI_ENV_FILE")"
cat > "$CLI_ENV_FILE" << EOF
# Source this file before running LobeHub CLI agent tests.
# Generated by setup-auth.sh cli-seed
export LOBE_API_KEY=$SEED_API_KEY
export LOBEHUB_CLI_API_KEY="\${LOBE_API_KEY}"
export LOBEHUB_SERVER=$SERVER_URL
export LOBEHUB_CLI_HOME=.lobehub-dev
EOF
}
write_cli_settings() {
mkdir -p "$CLI_HOME"
python3 - "$CLI_HOME/settings.json" "$SERVER_URL" << 'PY'
import json
import os
import sys
path, server_url = sys.argv[1], sys.argv[2]
os.makedirs(os.path.dirname(path), exist_ok=True)
with open(path, "w") as f:
json.dump({"serverUrl": server_url}, f, indent=2)
f.write("\n")
os.chmod(path, 0o600)
PY
}
cmd_cli_seed() {
check_server || return 1
write_cli_seed_env
write_cli_settings
ok "wrote CLI seed env: $CLI_ENV_FILE"
note "source it before CLI commands: source $CLI_ENV_FILE"
note "settings saved at: $CLI_HOME/settings.json"
LOBE_API_KEY="$SEED_API_KEY" LOBEHUB_CLI_API_KEY="$SEED_API_KEY" check_cli
}
cmd_open_chrome() {
local mode="${1:-}"
if [[ "$mode" != "" && "$mode" != "--dry-run" ]]; then
@@ -301,6 +377,26 @@ OSA
ok "opened Chrome at $SERVER_URL/ and requested DevTools Network panel"
}
cookie_header_from_jar() {
local jar="$1"
awk '
BEGIN { first = 1 }
/^$/ { next }
/^#/ {
if ($0 !~ /^#HttpOnly_/) next
sub(/^#HttpOnly_/, "")
}
NF >= 7 {
if (!first) printf "; "
printf "%s=%s", $6, $7
first = 0
}
END {
if (!first) printf "\n"
}
' "$jar"
}
# Build a Playwright storageState file from a raw Cookie header on stdin,
# keeping only the better-auth cookies. See references/auth.md for why the
# header must come from a Network request (HttpOnly) and why httpOnly=false.
@@ -357,6 +453,49 @@ PY
cmd_web_verify
}
cmd_web_seed() {
check_server || return 1
mkdir -p "$AUTH_DIR"
local cookie_jar="$AUTH_DIR/web-seed-cookie.jar"
local response_body="$AUTH_DIR/web-seed-response.json"
local payload code
payload="$(
SEED_EMAIL="$SEED_EMAIL" SEED_PASSWORD="$SEED_PASSWORD" python3 - << 'PY'
import json
import os
print(json.dumps({
"callbackURL": "/",
"email": os.environ["SEED_EMAIL"],
"password": os.environ["SEED_PASSWORD"],
}))
PY
)"
code=$(curl -sS -o "$response_body" -w '%{http_code}' \
-c "$cookie_jar" \
-H 'Content-Type: application/json' \
-X POST "$SERVER_URL/api/auth/sign-in/email" \
--data "$payload" 2> /dev/null || true)
if [[ ! "$code" =~ ^[23] ]]; then
bad "seed user sign-in failed at $SERVER_URL/api/auth/sign-in/email (http_code='$code')"
note "make sure the seed user exists:"
note "./.agents/skills/agent-testing/scripts/init-dev-env.sh seed-user"
return 1
fi
local cookie_header
cookie_header="$(cookie_header_from_jar "$cookie_jar")"
if [[ -z "$cookie_header" ]]; then
bad "seed sign-in succeeded but no cookies were written to $cookie_jar"
return 1
fi
printf '%s\n' "$cookie_header" | cmd_web
}
cmd_web_verify() {
local skip_server_check="${1:-}"
if [[ "$skip_server_check" != "--skip-server-check" ]]; then
@@ -364,7 +503,8 @@ cmd_web_verify() {
fi
if [[ ! -f "$STATE_FILE" ]]; then
bad "no web auth state for agent-browser"
note "copy the Cookie header from Chrome DevTools (Network tab), then:"
note "for the seeded local user, run: $0 web-seed"
note "or copy the Cookie header from Chrome DevTools (Network tab), then:"
note "pbpaste | $0 web"
return 1
fi
@@ -396,16 +536,18 @@ case "${1:-status}" in
shift || true
cmd_status "$@"
;;
cli-seed) cmd_cli_seed ;;
cli) cmd_cli ;;
open-chrome)
shift || true
cmd_open_chrome "$@"
;;
web-seed) cmd_web_seed ;;
web) cmd_web ;;
web-verify) cmd_web_verify ;;
-h|--help) usage ;;
*)
echo "Usage: $0 {status|cli|web|web-verify}" >&2
echo "Usage: $0 {status|cli-seed|cli|open-chrome|web-seed|web|web-verify}" >&2
exit 2
;;
esac
.agents/skills/agent-testing/scripts/setup-auth.test.sh
+84 -2
View File
@@ -29,6 +29,7 @@ cleanup() {
rm -rf "$tmp_dir"
}
trap cleanup EXIT
export HOME="$tmp_dir/home"
port="$(python3 - << 'PY'
import socket
@@ -40,8 +41,59 @@ sock.close()
PY
)"
python3 -m http.server "$port" --bind localhost --directory "$tmp_dir" \
> "$tmp_dir/http.log" 2>&1 &
python3 - "$port" << 'PY' > "$tmp_dir/http.log" 2>&1 &
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
import sys
class Handler(BaseHTTPRequestHandler):
def do_GET(self):
if self.path.startswith("/api/v1/users/me"):
if self.headers.get("authorization") != "Bearer sk-lh-agenttesting0001":
self.send_response(401)
self.end_headers()
self.wfile.write(b'{"success":false}')
return
self.send_response(200)
self.send_header("Content-Type", "application/json")
self.end_headers()
self.wfile.write(b'{"success":true,"data":{"id":"user_agent_testing_001"}}')
return
self.send_response(200)
self.end_headers()
self.wfile.write(b"ok")
def do_POST(self):
length = int(self.headers.get("content-length") or "0")
if length:
self.rfile.read(length)
if self.path != "/api/auth/sign-in/email":
self.send_response(404)
self.end_headers()
return
self.send_response(200)
self.send_header(
"Set-Cookie",
"better-auth.session_token=seed.token; Path=/; HttpOnly; SameSite=Lax",
)
self.send_header(
"Set-Cookie",
"better-auth.session_data=seed.data; Path=/; HttpOnly; SameSite=Lax",
)
self.send_header("Content-Type", "application/json")
self.end_headers()
self.wfile.write(b'{"ok":true}')
def log_message(self, format, *args):
return
ThreadingHTTPServer(("localhost", int(sys.argv[1])), Handler).serve_forever()
PY
server_pid="$!"
server_url="http://localhost:$port"
@@ -103,10 +155,40 @@ if names != expected:
raise SystemExit(f"unexpected cookies: {sorted(names)}")
PY
"$SCRIPT" web-seed > "$tmp_dir/web-seed.out"
python3 - "$AUTH_DIR/web-state.json" << 'PY'
import json, sys
with open(sys.argv[1]) as f:
state = json.load(f)
values = {cookie["name"]: cookie["value"] for cookie in state["cookies"]}
expected = {
"better-auth.session_token": "seed.token",
"better-auth.session_data": "seed.data",
}
if values != expected:
raise SystemExit(f"unexpected seeded cookies: {values}")
PY
"$SCRIPT" status --surface web > "$tmp_dir/status.out"
assert_contains "$tmp_dir/status.out" "surface=web"
assert_contains "$tmp_dir/status.out" "web auth green"
"$SCRIPT" cli-seed > "$tmp_dir/cli-seed.out"
assert_contains "$tmp_dir/cli-seed.out" "CLI API-key auth valid"
assert_contains "$tmp_dir/cli-seed.out" "settings saved at: $HOME/.lobehub-dev/settings.json"
if "$SCRIPT" status --surface cli > "$tmp_dir/cli-no-env.out"; then
fail "cli status without API key unexpectedly passed"
fi
assert_contains "$tmp_dir/cli-no-env.out" "CLI not logged in"
LOBEHUB_CLI_API_KEY=sk-lh-agenttesting0001 "$SCRIPT" status --surface cli > "$tmp_dir/cli-status.out"
assert_contains "$tmp_dir/cli-status.out" "CLI API-key auth valid"
assert_contains "$tmp_dir/cli-status.out" "cli auth green"
if printf 'foo=bar\n' | "$SCRIPT" web > "$tmp_dir/invalid.out" 2> "$tmp_dir/invalid.err"; then
fail "invalid cookie unexpectedly passed"
fi
.agents/skills/agent-testing/ui/web.md
+11 -3
View File
@@ -12,9 +12,16 @@ backend-only changes prefer [../cli/index.md](../cli/index.md).
- Complete [Step 0.0](../SKILL.md#00-resolve-the-current-test-environment) (resolve ports) and [Step -1](../SKILL.md#step--1--plan-approval-for-non-trivial-tests) (plan approval) first.
- Local dev server running — [../references/dev-server.md](../references/dev-server.md)
- Web auth verified in agent-browser — see [auth decision flow](../references/auth.md#web--decision-flow).
- Web auth verified in agent-browser — prefer `setup-auth.sh web-seed`, see [auth decision flow](../references/auth.md#web--decision-flow).
## Option A — agent-browser with injected auth (recommended)
## Option A — agent-browser with seeded auth (recommended)
```bash
./.agents/skills/agent-testing/scripts/init-dev-env.sh seed-user
./.agents/skills/agent-testing/scripts/setup-auth.sh web-seed
```
Then drive the verified session:
```bash
SESSION=lobehub-dev
@@ -26,7 +33,8 @@ agent-browser --session $SESSION snapshot -i
Use this session as the evidence source. Do not use ordinary Chrome screenshots
or Chrome Network records as proof for Web tests; ordinary Chrome is only a
source for copying cookies into agent-browser.
fallback source for copying cookies into agent-browser when the seeded login is
not available.
### Watch the API while driving the UI