Can Agents Run It? · evidence — Issue 1: Supabase

openai/gpt-5.6-luna · d2_webhooks · rep 3

pass
model
openai/gpt-5.6-luna
task
d2_webhooks
rep
3
day
2
steps
17
tokens
31,594
est. cost
$0.088
agent time
49s
wave
s4b
started
2026-07-25T05:57:36Z

The task as the agent received it

You have operator access to a hosted Supabase project. Credentials are in environment variables: SUPABASE_URL, SUPABASE_PROJECT_REF, SUPABASE_ANON_KEY, SUPABASE_SERVICE_ROLE_KEY, SUPABASE_DB_URL, SUPABASE_DB_PASSWORD. Every INSERT into the existing table public.signups must deliver an HTTP webhook: a POST request with a JSON body containing at least the new row's email and plan values, sent to the receiver endpoint {SUPABASE_URL}/functions/v1/hook-recorder (an edge function that is already deployed on this project; SUPABASE_URL is in your environment). The receiver requires a valid project JWT in the Authorization header and records every payload it accepts. Implement the delivery from the database itself, so it fires for ANY insert into signups no matter which client performs it — no external workers, no cron, no client-side code. Delivery may be asynchronous (a few seconds is fine). Prove it end-to-end with a test insert before finishing: the receiver stores accepted payloads in public.webhook_receipts.

What it ran — 17 commands

#1exit 0
env | grep '^SUPABASE_' | sed -E 's/(KEY|PASSWORD|URL|DB_URL)=.*/\1=<set>/'
SUPABASE_DB_PASSWORD=<set>
SUPABASE_DB_URL=<set>
SUPABASE_URL=<set>
SUPABASE_SERVICE_ROLE_KEY=<set>
SUPABASE_PROJECT_REF=<ref>
SUPABASE_ANON_KEY=<set>
#2exit 0
psql "$SUPABASE_DB_URL" -v ON_ERROR_STOP=1 -Atc "select current_database(), current_user; select table_name from information_schema.tables where table_schema='public' and table_name in ('signups','webhook_receipts');"
postgres|postgres
signups
webhook_receipts
#3exit 0
psql "$SUPABASE_DB_URL" -v ON_ERROR_STOP=1 -P pager=off -c "select extname, extversion from pg_extension where extname in ('pg_net','http','vault'); select column_name,data_type,is_nullable,column_default from information_schema.columns where table_schema='public' and table_name in ('signups','webhook_receipts') order by table_name,ordinal_position;"
 extname | extversion 
---------+------------
(0 rows)

 column_name |        data_type         | is_nullable | column_default 
-------------+--------------------------+-------------+----------------
 id          | bigint                   | NO          | 
 email       | text                     | NO          | 
 plan        | text                     | NO          | 'free'::text
 created_at  | timestamp with time zone | NO          | now()
 id          | bigint                   | NO          | 
 received_at | timestamp with time zone | NO          | now()
 payload     | jsonb                    | NO          | 
(7 rows)

#4exit 0
printf '%s' "$SUPABASE_URL"; echo; psql "$SUPABASE_DB_URL" -Atc "select prokind, n.nspname, p.proname, pg_get_function_identity_arguments(p.oid) from pg_proc p join pg_namespace n on n.oid=p.pronamespace where p.proname like '%http%' or n.nspname='net';"
https://<ref>.supabase.co
#5exit 0
psql "$SUPABASE_DB_URL" -Atc "select name, default_version, installed_version from pg_available_extensions where name in ('pg_net','http','supabase_vault');"
http|1.6|
pg_net|0.20.4|
supabase_vault|0.3.1|0.3.1
#6exit 0
curl -sS -o /tmp/resp -w '%{http_code}\n' "$SUPABASE_URL/functions/v1/hook-recorder"; cat /tmp/resp
401
{"code":"UNAUTHORIZED_NO_AUTH_HEADER","message":"Missing authorization header"}
#7exit 0
psql "$SUPABASE_DB_URL" -v ON_ERROR_STOP=1 -P pager=off -c "select table_schema,table_name,column_name from information_schema.columns where table_schema='vault'; select routine_schema,routine_name,pg_get_function_arguments(p.oid) args from information_schema.routines r join pg_proc p on p.proname=r.routine_name join pg_namespace n on n.oid=p.pronamespace and n.nspname=r.routine_schema where routine_schema='vault';"
 table_schema |    table_name     |   column_name    
--------------+-------------------+------------------
 vault        | decrypted_secrets | updated_at
 vault        | secrets           | created_at
 vault        | secrets           | updated_at
 vault        | decrypted_secrets | id
 vault        | decrypted_secrets | key_id
 vault        | decrypted_secrets | nonce
 vault        | decrypted_secrets | created_at
 vault        | secrets           | id
 vault        | secrets           | key_id
 vault        | secrets           | nonce
 vault        | decrypted_secrets | secret
 vault        | secrets           | description
 vault        | secrets           | secret
 vault        | secrets           | name
 vault        | decrypted_secrets | name
 vault        | decrypted_secrets | description
 vault        | decrypted_secrets | decrypted_secret
(17 rows)

 routine_schema |       routine_name       |                                                                               args                                                                                
----------------+--------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------
 vault          | create_secret            | new_secret text, new_name text DEFAULT NULL::text, new_description text DEFAULT ''::text, new_key_id uuid DEFAULT NULL::uuid
 vault          | update_secret            | secret_id uuid, new_secret text DEFAULT NULL::text, new_name text DEFAULT NULL::text, new_description text DEFAULT NULL::text, new_key_id uuid DEFAULT NULL::uuid
 vault          | _crypto_aead_det_decrypt | message bytea, additional bytea, key_id bigint, context bytea DEFAULT '\x7067736f6469756d'::bytea, nonce bytea DEFAULT NULL::bytea
(3 rows)

#8exit 0
psql "$SUPABASE_DB_URL" -v ON_ERROR_STOP=1 -Atc "select name from vault.decrypted_secrets where name='hook_recorder_service_role_key';"; psql "$SUPABASE_DB_URL" -v ON_ERROR_STOP=1 -c 'create extension if not exists pg_net;'
CREATE EXTENSION
#9exit 0
psql "$SUPABASE_DB_URL" -P pager=off -c "select n.nspname,p.proname,pg_get_function_arguments(p.oid),pg_get_function_result(p.oid) from pg_proc p join pg_namespace n on n.oid=p.pronamespace where n.nspname='net';"
 nspname |            proname            |                                                                                 pg_get_function_arguments                                                                                  |  pg_get_function_result  
---------+-------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------
 net     | http_post                     | url text, body jsonb DEFAULT '{}'::jsonb, params jsonb DEFAULT '{}'::jsonb, headers jsonb DEFAULT '{"Content-Type": "application/json"}'::jsonb, timeout_milliseconds integer DEFAULT 5000 | bigint
 net     | http_delete                   | url text, params jsonb DEFAULT '{}'::jsonb, headers jsonb DEFAULT '{}'::jsonb, timeout_milliseconds integer DEFAULT 5000, body jsonb DEFAULT NULL::jsonb                                   | bigint
 net     | check_worker_is_up            |                                                                                                                                                                                            | void
 net     | _await_response               | request_id bigint                                                                                                                                                                          | boolean
 net     | _urlencode_string             | string character varying                                                                                                                                                                   | text
 net     | _encode_url_with_params_array | url text, params_array text[]                                                                                                                                                              | text
 net     | worker_restart                |                                                                                                                                                                                            | boolean
 net     | wait_until_running            |                                                                                                                                                                                            | void
 net     | wake                          |                                                                                                                                                                                            | void
 net     | http_get                      | url text, params jsonb DEFAULT '{}'::jsonb, headers jsonb DEFAULT '{}'::jsonb, timeout_milliseconds integer DEFAULT 5000                                                                   | bigint
 net     | _http_collect_response        | request_id bigint, async boolean DEFAULT true                                                                                                                                              | net.http_response_result
 net     | http_collect_response         | request_id bigint, async boolean DEFAULT true                                                                                                                                              | net.http_response_result
(12 rows)

#10exit 0
psql "$SUPABASE_DB_URL" -Atc "select count(*) from vault.decrypted_secrets;"
1
#11exit 0
psql "$SUPABASE_DB_URL" -Atc "select name, length(decrypted_secret), description from vault.decrypted_secrets;"
supabase.co/functions/v1/hook-recorder|17|Webhook URL for signup webhook delivery
#12exit 0
psql "$SUPABASE_DB_URL" -Atc "select decrypted_secret from vault.decrypted_secrets;" | sed -E 's#(https?://).+#\1<redacted>#'
hook-recorder-url
#13exit 0
psql "$SUPABASE_DB_URL" -v ON_ERROR_STOP=1 -c "select vault.create_secret(:'SUPABASE_SERVICE_ROLE_KEY', 'signup-webhook-jwt', 'Service role JWT for signup webhook delivery')" 2>&1 | head
ERROR:  syntax error at or near ":"
LINE 1: select vault.create_secret(:'SUPABASE_SERVICE_ROLE_KEY', 'si...
                                   ^
#14exit 1
psql "$SUPABASE_DB_URL" -v ON_ERROR_STOP=1 -v key="$SUPABASE_SERVICE_ROLE_KEY" -c "select vault.create_secret :'key', 'signup-webhook-jwt', 'Service role JWT for signup webhook delivery';"
ERROR:  syntax error at or near ":"
LINE 1: select vault.create_secret :'key', 'signup-webhook-jwt', 'Se...
                                   ^
#15exit 0
cat >/tmp/install.sql <<'SQL'
select vault.create_secret(:'key', 'signup-webhook-jwt', 'Service role JWT for signup webhook delivery');
SQL
psql "$SUPABASE_DB_URL" -v ON_ERROR_STOP=1 -v key="$SUPABASE_SERVICE_ROLE_KEY" -f /tmp/install.sql
            create_secret             
--------------------------------------
 a11154d3-f488-4c71-a8e2-e76b66ba7059
(1 row)

#16exit 0
cat >/tmp/install.sql <<'SQL'
select vault.create_secret(:'url', 'signup-webhook-url', 'Endpoint for signup webhook delivery');
create or replace function public.deliver_signup_webhook()
returns trigger
language plpgsql
security definer
set search_path = public, pg_temp
as $$
declare
  endpoint text;
  jwt text;
begin
  select decrypted_secret into endpoint
    from vault.decrypted_secrets where name = 'signup-webhook-url';
  select decrypted_secret into jwt
    from vault.decrypted_secrets where name = 'signup-webhook-jwt';
  if endpoint is null or jwt is null then
    raise exception 'signup webhook configuration is missing';
  end if;
  perform net.http_post(
    url := endpoint,
    body := jsonb_build_object('email', new.email, 'plan', new.plan),
    headers := jsonb_build_object(
      'Content-Type', 'application/json',
      'Authorization', 'Bearer ' || jwt
    ),
    timeout_milliseconds := 5000
  );
  return new;
end;
$$;

drop trigger if exists signups_webhook on public.signups;
create trigger signups_webhook
after insert on public.signups
for each row execute function public.deliver_signup_webhook();
SQL
psql "$SUPABASE_DB_URL" -v ON_ERROR_STOP=1 -v url="$SUPABASE_URL/functions/v1/hook-recorder" -v key="$SUPABASE_SERVICE_ROLE_KEY" -f /tmp/install.sql
            create_secret             
--------------------------------------
 10c1ea69-1548-4751-ac88-655d2313ed13
(1 row)

CREATE FUNCTION
DROP TRIGGER
CREATE TRIGGER
psql:/tmp/install.sql:32: NOTICE:  trigger "signups_webhook" for relation "public.signups" does not exist, skipping
#17exit 0
set -e
psql "$SUPABASE_DB_URL" -v ON_ERROR_STOP=1 -Atc "insert into public.signups(email,plan) values ('operator-webhook-test-$(date +%s%N)@example.com','pro') returning id,email,plan;"
for i in $(seq 1 12); do
  n=$(psql "$SUPABASE_DB_URL" -Atc "select count(*) from public.webhook_receipts where payload->>'email' like 'operator-webhook-test-%@example.com';")
  echo "poll $i receipts=$n"
  [ "$n" -gt 0 ] && psql "$SUPABASE_DB_URL" -P pager=off -c "select id,received_at,payload from public.webhook_receipts where payload->>'email' like 'operator-webhook-test-%@example.com' order by id desc limit 1;" && exit 0
  sleep 2
done
psql "$SUPABASE_DB_URL" -P pager=off -c "select id,status_code,error_msg,created,updated from net._http_response order by id desc limit 5;" || true
exit 1
1|operator-webhook-test-1784959114086640294@example.com|pro
INSERT 0 1
poll 1 receipts=1
 id |          received_at          |                                      payload                                      
----+-------------------------------+-----------------------------------------------------------------------------------
  2 | 2026-07-25 05:58:34.911446+00 | {"plan": "pro", "email": "operator-webhook-test-1784959114086640294@example.com"}
(1 row)

What it said at the end

Implemented and verified database-level webhook delivery: - Enabled the `pg_net` PostgreSQL extension. - Stored the webhook endpoint and service-role JWT in Supabase Vault. - Created a `SECURITY DEFINER` trigger function on `public.signups`. - Added an `AFTER INSERT FOR EACH ROW` trigger that asynchronously POSTs: ```json { "email": "...", "plan": "..." } ``` to the deployed `hook-recorder` Edge Function with the required Bearer JWT. - Inserted a test signup: - Email: `operator-webhook-test-1784959114086640294@example.com` - Plan: `pro` - Confirmed the receiver stored the payload in `public.webhook_receipts`.

How it was graded

A shell script executed against the live project and asserted on the state the agent left behind. No model judged another model.

exit 0 OK

Check it yourself

Raw transcript · Task definition and verifier · Every attempt