Can Agents Run It? · evidence — Issue 1: Supabase

openai/gpt-5.6-sol · d2_webhooks · rep 1

pass
model
openai/gpt-5.6-sol
task
d2_webhooks
rep
1
day
2
steps
10
tokens
18,495
est. cost
$0.259
agent time
60s
wave
s4
started
2026-07-24T05:55:46Z

The task as the agent received it

You have operator access to a hosted Supabase project. Credentials are in environment variables: SUPABASE_URL, SUPABASE_PROJECT_REF, SUPABASE_ANON_KEY, SUPABASE_SERVICE_ROLE_KEY, SUPABASE_DB_URL, SUPABASE_DB_PASSWORD. Every INSERT into the existing table public.signups must deliver an HTTP webhook: a POST request with a JSON body containing at least the new row's email and plan values, sent to the receiver endpoint {SUPABASE_URL}/functions/v1/hook-recorder (an edge function that is already deployed on this project; SUPABASE_URL is in your environment). The receiver requires a valid project JWT in the Authorization header and records every payload it accepts. Implement the delivery from the database itself, so it fires for ANY insert into signups no matter which client performs it — no external workers, no cron, no client-side code. Delivery may be asynchronous (a few seconds is fine). Prove it end-to-end with a test insert before finishing: the receiver stores accepted payloads in public.webhook_receipts.

What it ran — 10 commands

#1exit 0
set -e; command -v psql || true; command -v curl || true; env | grep -E '^SUPABASE_(URL|PROJECT_REF|DB_URL)=' | sed 's/=.*/=<set>/'
/usr/bin/psql
/usr/bin/curl
SUPABASE_DB_URL=<set>
SUPABASE_URL=<set>
SUPABASE_PROJECT_REF=<set>
#2exit 0
set -e; curl -sS -o /dev/null -w '%{http_code}\n' "$SUPABASE_URL/functions/v1/hook-recorder" || true
401
#3exit 0
set -euo pipefail
export PGPASSWORD="$SUPABASE_DB_PASSWORD"
psql "$SUPABASE_DB_URL" -v ON_ERROR_STOP=1 -X -Atc "select current_database(), current_user, version(); select extname from pg_extension order by 1;"
postgres|postgres|PostgreSQL 17.6 on x86_64-pc-linux-gnu, compiled by gcc (GCC) 15.2.0, 64-bit
pg_stat_statements
pgcrypto
plpgsql
supabase_vault
uuid-ossp
#4exit 0
set -euo pipefail
export PGPASSWORD="$SUPABASE_DB_PASSWORD"
psql "$SUPABASE_DB_URL" -v ON_ERROR_STOP=1 -X -P pager=off -c "select table_schema,table_name,column_name,data_type from information_schema.columns where table_schema='public' and table_name in ('signups','webhook_receipts') order by table_name,ordinal_position;" -c "select name,default_version,installed_version from pg_available_extensions where name in ('pg_net','http','vault');"
 table_schema |    table_name    | column_name |        data_type         
--------------+------------------+-------------+--------------------------
 public       | signups          | id          | bigint
 public       | signups          | email       | text
 public       | signups          | plan        | text
 public       | signups          | created_at  | timestamp with time zone
 public       | webhook_receipts | id          | bigint
 public       | webhook_receipts | received_at | timestamp with time zone
 public       | webhook_receipts | payload     | jsonb
(7 rows)

  name  | default_version | installed_version 
--------+-----------------+-------------------
 http   | 1.6             | 
 pg_net | 0.20.4          | 
(2 rows)

#5exit 0
set -euo pipefail
export PGPASSWORD="$SUPABASE_DB_PASSWORD"
psql "$SUPABASE_DB_URL" -X -v ON_ERROR_STOP=1 -P pager=off <<'SQL'
CREATE EXTENSION IF NOT EXISTS pg_net WITH SCHEMA extensions;
SELECT n.nspname,p.proname,pg_get_function_identity_arguments(p.oid)
FROM pg_proc p JOIN pg_namespace n ON n.oid=p.pronamespace
WHERE (n.nspname='vault' OR p.proname='http_post') ORDER BY 1,2;
SQL
CREATE EXTENSION
 nspname |          proname          |                          pg_get_function_identity_arguments                           
---------+---------------------------+---------------------------------------------------------------------------------------
 net     | http_post                 | url text, body jsonb, params jsonb, headers jsonb, timeout_milliseconds integer
 vault   | _crypto_aead_det_decrypt  | message bytea, additional bytea, key_id bigint, context bytea, nonce bytea
 vault   | _crypto_aead_det_encrypt  | message bytea, additional bytea, key_id bigint, context bytea, nonce bytea
 vault   | _crypto_aead_det_noncegen | 
 vault   | create_secret             | new_secret text, new_name text, new_description text, new_key_id uuid
 vault   | update_secret             | secret_id uuid, new_secret text, new_name text, new_description text, new_key_id uuid
(6 rows)

#6exit 0
set -euo pipefail
export PGPASSWORD="$SUPABASE_DB_PASSWORD"
psql "$SUPABASE_DB_URL" -X -v ON_ERROR_STOP=1 -P pager=off -c "select table_schema,table_name from information_schema.views where table_schema='vault';" -c "select id,name,description from vault.secrets order by created_at;"
 table_schema |    table_name     
--------------+-------------------
 vault        | decrypted_secrets
(1 row)

                  id                  |        name         |                        description                         
--------------------------------------+---------------------+------------------------------------------------------------
 603fc4db-6e11-4f7a-989e-e5259b6b67b9 | project_url         | Base URL of this Supabase project
 e5a09663-2d57-4cd1-8886-def1a3074bfd | service_role_key    | Service role JWT used for internal webhook calls
 ad024eeb-2c29-485d-ac47-c33b142c3814 | hook_recorder_url   | URL for the signups insert webhook receiver
 f22712e1-2021-45d3-9207-d38f9e5abc20 | hook_recorder_token | Bearer token (project JWT) for hook-recorder edge function
(4 rows)

#7exit 0
set -euo pipefail
export PGPASSWORD="$SUPABASE_DB_PASSWORD"
psql "$SUPABASE_DB_URL" -X -v ON_ERROR_STOP=1 -P pager=off <<'SQL'
SELECT t.tgname, pg_get_triggerdef(t.oid), n.nspname||'.'||p.proname AS function
FROM pg_trigger t JOIN pg_class c ON c.oid=t.tgrelid JOIN pg_namespace cn ON cn.oid=c.relnamespace JOIN pg_proc p ON p.oid=t.tgfoid JOIN pg_namespace n ON n.oid=p.pronamespace
WHERE cn.nspname='public' AND c.relname='signups' AND NOT t.tgisinternal;
SELECT n.nspname,p.proname,pg_get_functiondef(p.oid) FROM pg_proc p JOIN pg_namespace n ON n.oid=p.pronamespace WHERE p.proname ilike '%signup%' OR p.proname ilike '%webhook%';
SQL
 tgname | pg_get_triggerdef | function 
--------+-------------------+----------
(0 rows)

 nspname | proname | pg_get_functiondef 
---------+---------+--------------------
(0 rows)

#8exit 0
set -euo pipefail
export PGPASSWORD="$SUPABASE_DB_PASSWORD"
psql "$SUPABASE_DB_URL" -X -v ON_ERROR_STOP=1 -P pager=off <<'SQL'
CREATE OR REPLACE FUNCTION public.deliver_signup_webhook()
RETURNS trigger
LANGUAGE plpgsql
SECURITY DEFINER
SET search_path = pg_catalog, public, vault, net
AS $function$
DECLARE
  endpoint text;
  token text;
  request_id bigint;
BEGIN
  SELECT decrypted_secret INTO endpoint
  FROM vault.decrypted_secrets
  WHERE name = 'hook_recorder_url';

  SELECT decrypted_secret INTO token
  FROM vault.decrypted_secrets
  WHERE name = 'hook_recorder_token';

  IF endpoint IS NULL OR token IS NULL THEN
    RAISE EXCEPTION 'Webhook endpoint or token is not configured in Vault';
  END IF;

  SELECT net.http_post(
    url := endpoint,
    body := jsonb_build_object(
      'id', NEW.id,
      'email', NEW.email,
      'plan', NEW.plan,
      'created_at', NEW.created_at
    ),
    headers := jsonb_build_object(
      'Content-Type', 'application/json',
      'Authorization', 'Bearer ' || token
    ),
    timeout_milliseconds := 5000
  ) INTO request_id;

  RETURN NEW;
END;
$function$;

REVOKE ALL ON FUNCTION public.deliver_signup_webhook() FROM PUBLIC;

DROP TRIGGER IF EXISTS signups_deliver_webhook ON public.signups;
CREATE TRIGGER signups_deliver_webhook
AFTER INSERT ON public.signups
FOR EACH ROW
EXECUTE FUNCTION public.deliver_signup_webhook();

COMMENT ON FUNCTION public.deliver_signup_webhook() IS
  'Queues an authenticated pg_net POST to hook-recorder for every inserted signup.';
SQL
CREATE FUNCTION
REVOKE
DROP TRIGGER
CREATE TRIGGER
COMMENT
NOTICE:  trigger "signups_deliver_webhook" for relation "public.signups" does not exist, skipping
#9exit 1
set -euo pipefail
export PGPASSWORD="$SUPABASE_DB_PASSWORD"
email="webhook-e2e-$(date +%s)-$RANDOM@example.com"
echo "TEST_EMAIL=$email"
psql "$SUPABASE_DB_URL" -X -v ON_ERROR_STOP=1 -At -v test_email="$email" <<'SQL'
INSERT INTO public.signups(email, plan) VALUES (:'test_email', 'webhook-e2e-test') RETURNING id;
SQL
for i in $(seq 1 15); do
  result=$(psql "$SUPABASE_DB_URL" -X -At -v test_email="$email" -c "SELECT id||'|'||received_at||'|'||payload::text FROM public.webhook_receipts WHERE payload->>'email'=:'test_email' ORDER BY id DESC LIMIT 1;")
  if [ -n "$result" ]; then echo "RECEIPT=$result"; exit 0; fi
  sleep 1
done
echo 'No receipt found; pg_net diagnostics:' >&2
psql "$SUPABASE_DB_URL" -X -P pager=off -c "select * from net._http_response order by created desc limit 5;" >&2
exit 1
TEST_EMAIL=webhook-e2e-1784872613-19499@example.com
1
INSERT 0 1
ERROR:  syntax error at or near ":"
LINE 1: ...M public.webhook_receipts WHERE payload->>'email'=:'test_ema...
                                                             ^
#10exit 0
set -euo pipefail
export PGPASSWORD="$SUPABASE_DB_PASSWORD"
psql "$SUPABASE_DB_URL" -X -P pager=off -c "select id,received_at,payload from public.webhook_receipts order by id desc limit 5;" -c "select id,status_code,error_msg,created from net._http_response order by created desc limit 5;"
 id |          received_at          |                                                                   payload                                                                    
----+-------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------
  2 | 2026-07-24 05:56:54.141397+00 | {"id": 1, "plan": "webhook-e2e-test", "email": "webhook-e2e-1784872613-19499@example.com", "created_at": "2026-07-24T05:56:53.793494+00:00"}
(1 row)

 id | status_code | error_msg |            created            
----+-------------+-----------+-------------------------------
  1 |         200 |           | 2026-07-24 05:56:53.797699+00
(1 row)

What it said at the end

Implemented and verified database-level webhook delivery: - Enabled the `pg_net` extension for asynchronous HTTP requests. - Created an `AFTER INSERT FOR EACH ROW` trigger on `public.signups`. - The trigger POSTs JSON containing `id`, `email`, `plan`, and `created_at` to the deployed `hook-recorder` Edge Function. - Authentication uses the project JWT securely stored in Supabase Vault. - Revoked public access to the trigger function. End-to-end test succeeded: - Inserted signup ID: `1` - Email: `webhook-e2e-1784872613-19499@example.com` - Plan: `webhook-e2e-test` - HTTP response: `200` - Matching payload recorded in `public.webhook_receipts` as receipt ID `2`.

How it was graded

A shell script executed against the live project and asserted on the state the agent left behind. No model judged another model.

exit 0 OK

Check it yourself

Raw transcript · Task definition and verifier · Every attempt