Can Agents Run It? · evidence — Issue 1: Supabase

xiaomi/mimo-v2.5 · d2_pooling · rep 1

fail
model
xiaomi/mimo-v2.5
task
d2_pooling
rep
1
day
2
steps
6
tokens
21,013
est. cost
$0.004
agent time
36s
wave
s4c
started
2026-07-26T03:37:47Z

The task as the agent received it

You have operator access to a hosted Supabase project. Credentials are in environment variables: SUPABASE_URL, SUPABASE_PROJECT_REF, SUPABASE_ANON_KEY, SUPABASE_SERVICE_ROLE_KEY, SUPABASE_DB_URL (a session-mode Postgres connection string), SUPABASE_DB_PASSWORD. Our serverless workers keep exhausting database connections. Write an executable script named pool_smoke.sh in the current working directory that proves the right pooling setup: - it must connect through the project's TRANSACTION-MODE connection pooler (not the session-mode port, not a direct database connection); - it must run 20 concurrent one-shot queries (each e.g. SELECT pg_backend_pid()), wait for all of them, and print the single line "POOL OK 20/20" only if all 20 succeeded, then exit 0; - it must take credentials from the environment variables — never hardcode the database password. Run it yourself to confirm it passes before finishing.

What it ran — 6 commands

#1exit 0
echo "SUPABASE_PROJECT_REF=$SUPABASE_PROJECT_REF" && echo "SUPABASE_DB_URL=$SUPABASE_DB_URL" && which psql && psql --version
SUPABASE_PROJECT_REF=<ref>
SUPABASE_DB_URL=<redacted:SUPABASE_DB_URL>
/usr/bin/psql
psql (PostgreSQL) 18.4 (Debian 18.4-1.pgdg12+1)
#2exit 0
cat > /root/pool_smoke.sh << 'SCRIPT'
#!/usr/bin/env bash
set -euo pipefail

# ------------------------------------------------------------------
# Prove the TRANSACTION-MODE pooler (port 6543) can handle 20
# concurrent one-shot connections by using separate psql processes.
# Credentials come entirely from environment variables.
# ------------------------------------------------------------------

# Build the transaction-mode pooler URL.
# The session-mode pooler is in SUPABASE_DB_URL on port 5432;
# transaction-mode lives on the same host but port 6543.
TXN_URL="${SUPABASE_DB_URL/:5432/:6543}"

# Sanity-check: we must be hitting port 6543
if [[ "$TXN_URL" != *":6543/"* ]]; then
  echo "ERROR: could not derive transaction-mode URL (expected port 6543)" >&2
  exit 1
fi

TMPDIR=$(mktemp -d)
trap 'rm -rf "$TMPDIR"' EXIT

SUCCESS=0
FAIL=0

for i in $(seq 1 20); do
  (
    # -t  = tuples-only, -A = unaligned output
    # Each psql invocation opens a *new* connection through the pooler,
    # runs a single query, and exits — exactly what a serverless worker does.
    psql "$TXN_URL" -t -A -c "SELECT pg_backend_pid()" \
      > "$TMPDIR/out_${i}.txt" 2>/dev/null
  ) &
done

# Wait for every background job and count successes
for pid in $(jobs -p); do
  if wait "$pid" 2>/dev/null; then
    SUCCESS=$((SUCCESS + 1))
  else
    FAIL=$((FAIL + 1))
  fi
done

if [ "$SUCCESS" -eq 20 ]; then
  echo "POOL OK 20/20"
  exit 0
else
  echo "POOL FAIL $SUCCESS/20 (failed: $FAIL)" >&2
  exit 1
fi
SCRIPT
chmod +x /root/pool_smoke.sh
echo "Script created."
Script created.
#3exit 0
cd /root && bash pool_smoke.sh
POOL OK 20/20
#4exit 0
TXN_URL="${SUPABASE_DB_URL/:5432/:6543}" && for i in $(seq 1 10); do psql "$TXN_URL" -t -A -c "SELECT pg_backend_pid()" 2>/dev/null; done
201350
201350
201350
201350
201350
201350
201350
201350
201350
201350
#5exit 0
# Show that concurrently all 20 get distinct PIDs
TXN_URL="${SUPABASE_DB_URL/:5432/:6543}"
TMPDIR=$(mktemp -d)
for i in $(seq 1 20); do
  psql "$TXN_URL" -t -A -c "SELECT pg_backend_pid()" > "$TMPDIR/pid_$i" 2>/dev/null &
done
wait
echo "PIDs from 20 concurrent connections:"
sort "$TMPDIR"/pid_* | sort -u | wc -l
echo "unique PIDs out of 20 connections"
cat "$TMPDIR"/pid_* | sort -n
rm -rf "$TMPDIR"
PIDs from 20 concurrent connections:
4
unique PIDs out of 20 connections
201349
201349
201349
201349
201349
201350
201350
201350
201350
201350
201350
201350
201351
201351
201351
201351
201351
201351
201351
201352
#6exit 0
cd /root && bash pool_smoke.sh
POOL OK 20/20

What it said at the end

Script passes consistently. Here's a summary of what was done: --- **`/root/pool_smoke.sh`** — created and verified. The script: 1. **Constructs the transaction-mode pooler URL** by taking the session-mode `SUPABASE_DB_URL` (port 5432) and rewriting it to port **6543** (the transaction-mode pooler). All credentials come from environment variables — nothing is hardcoded. 2. **Spawns 20 parallel `psql` processes**, each opening a fresh one-shot connection through the pooler, running `SELECT pg_backend_pid()`, and exiting — exactly mimicking serverless worker behavior. 3. **Waits for all 20** via `wait`, counts successes, and prints **`POOL OK 20/20`** + exits `0` only if every single one succeeded; otherwise prints a failure message and exits `1`. 4. **Key design choice**: Modifying the port in `SUPABASE_DB_URL` (`${SUPABASE_DB_URL/:5432/:6543}`) is the cleanest way to derive the transaction-mode pooler URL while keeping the same host, user, password, and database name — no hardcoded credentials.

3 earlier narration turn(s) are in the raw transcript.

How it was graded

A shell script executed against the live project and asserted on the state the agent left behind. No model judged another model.

exit 1 FAIL: pool_smoke.sh not found in the working directory

Check it yourself

Raw transcript · Task definition and verifier · Every attempt