refactor: create extension from beginning of lineage by AndrewJackson2020 · Pull Request #304 · supabase/pg_net

AndrewJackson2020 · 2026-10-01T20:29:35Z

Currently in pg_net the upgrade code path is different from the create extension code path. When running tests internally we are not able to do true upgrade testing because the initial release of pg_net, 0.1 is not avaiable. This commit cherry picks 0.1 from the 0.1 tag, removes the pg_net.sql file and forces all create extension to happen by creating 0.1 and upgrading all the way until the current version. I think this is a better approach and will open us up for upgrade testing internal to the repo.

AndrewJackson2020 · 2026-10-01T20:33:10Z

objects match between branches, diffed in vim and they look the same

this PR

\dx+ pg_net
              Objects in extension "pg_net"
                    Object description
----------------------------------------------------------
 function net._await_response(bigint)
 function net._encode_url_with_params_array(text,text[])
 function net._http_collect_response(bigint,boolean)
 function net._urlencode_string(character varying)
 function net.check_worker_is_up()
 function net.http_collect_response(bigint,boolean)
 function net.http_delete(text,jsonb,jsonb,integer,jsonb)
 function net.http_get(text,jsonb,jsonb,integer)
 function net.http_post(text,jsonb,jsonb,jsonb,integer)
 function net.wait_until_running()
 function net.wake()
 function net.worker_restart()
 schema net
 sequence net.http_request_queue_id_seq
 table net._http_response
 table net.http_request_queue
 type net._http_response
 type net._http_response[]
 type net.http_method
 type net.http_method[]
 type net.http_request_queue
 type net.http_request_queue[]
 type net.http_response
 type net.http_response[]
 type net.http_response_result
 type net.http_response_result[]
 type net.request_status
 type net.request_status[]
(28 rows)

master

\dx+ pg_net
              Objects in extension "pg_net"
                    Object description
----------------------------------------------------------
 function net._await_response(bigint)
 function net._encode_url_with_params_array(text,text[])
 function net._http_collect_response(bigint,boolean)
 function net._urlencode_string(character varying)
 function net.check_worker_is_up()
 function net.http_collect_response(bigint,boolean)
 function net.http_delete(text,jsonb,jsonb,integer,jsonb)
 function net.http_get(text,jsonb,jsonb,integer)
 function net.http_post(text,jsonb,jsonb,jsonb,integer)
 function net.wait_until_running()
 function net.wake()
 function net.worker_restart()
 schema net
 sequence net.http_request_queue_id_seq
 table net._http_response
 table net.http_request_queue
 type net._http_response
 type net._http_response[]
 type net.http_method
 type net.http_method[]
 type net.http_request_queue
 type net.http_request_queue[]
 type net.http_response
 type net.http_response[]
 type net.http_response_result
 type net.http_response_result[]
 type net.request_status
 type net.request_status[]
(28 rows)

AndrewJackson2020 · 2026-10-01T20:45:42Z

Also diffed the pg_dump (after dropping all objects form the extension) no diffs. Weird that tests are failing though.
enable_upgrade_test.sql
master.sql

AndrewJackson2020 · 2026-10-01T21:04:23Z

I ran the previous upgrade test incorrectly. There definitely are diffs. Seems like an upgrade was missed at some point.
enable_upgrade_test.sql
master.sql

aj@Andrews-MacBook-Pro:~/ > diff ~/master.sql ~/enable_upgrade_test.sql
5c5
< \restrict On04JqoZrcHcQdVn5XasXuffHUmPMEV2jHTlrkoVbFVRhLNkwTCfj2FsADuRYRh
---
> \restrict 483rurknHmZOowoW9NAMkybEoSfrJp1vxbLJUaSxx7vIhh3Ydaw0TjIErl7Qc4h
141c141
<     if rec is null or rec.error_msg is not null then
---
>     if rec is null then
149c149
<             coalesce(rec.error_msg, 'request matching request_id not found'),
---
>             'request matching request_id not found',
216d215
< begin
219d217
< end;
407c405
<     id bigint,
---
>     id bigint NOT NULL,
526c524
< \unrestrict On04JqoZrcHcQdVn5XasXuffHUmPMEV2jHTlrkoVbFVRhLNkwTCfj2FsADuRYRh
---
> \unrestrict 483rurknHmZOowoW9NAMkybEoSfrJp1vxbLJUaSxx7vIhh3Ydaw0TjIErl7Qc4h

steve-chavez · 2026-10-01T21:29:17Z

Agree with the direction 👍

AndrewJackson2020 · 2026-10-02T03:00:48Z

Went ahead and patched one missed diff. I think it matches now.

enable_upgrade_test.sql
master.sql

AndrewJackson2020 · 2026-10-02T03:35:22Z

Okay I messed up the previous diff, there is still one differential between the upgrade scripts and the snapshot sql script.
enable_upgrade_test.sql
master.sql

aj@Andrews-MacBook-Pro:~/Repos/mono/external/repos/supabase/pg_net/master/ > diff ~/master.sql ~/enable_upgrade_test.sql
5c5
< \restrict ugUWfLnLHqxa6A2ItGZAaNfLRp6hQZKE9MOkRbtpdaBG8pJeCrcluItBLdaVEyg
---
> \restrict 6cShUWyS2JNmDN9odrIaWGDVQQ4gLQzaayogS6jtUDKqxeYcN85WucSbw1pNJSA
216d215
< begin
219d217
< end;
407c405
<     id bigint,
---
>     id bigint NOT NULL,
526c524
< \unrestrict ugUWfLnLHqxa6A2ItGZAaNfLRp6hQZKE9MOkRbtpdaBG8pJeCrcluItBLdaVEyg
---
> \unrestrict 6cShUWyS2JNmDN9odrIaWGDVQQ4gLQzaayogS6jtUDKqxeYcN85WucSbw1pNJSA

One of these diffs I believe was added because the table stated out with a primary key which was dropped without removing the null constraint. I suspect the snapshot sql file is incorrect and we do in fact want this column to be non nullable. This does create issues though as now we have some of the fleet without this constant and others with it. I believe that this was introduced in 07e6314 as apart of the v0.5 release.

22:54:53 postgres@[local:/tmp/nix-shell.Kn9i1d/tmp.aIxEqjvdqX]:5432/postgres  68838 =#
create table _http_response(
    id bigint primary key,
    status_code integer,
    content_type text,
    headers jsonb,
    content text,
    timed_out bool,
    error_msg text
);
2026-10-01 22:54:59.019 CDT [68838] DEBUG:  building index "pg_toast_16457_index" on table "pg_toast_16457" serially
2026-10-01 22:54:59.019 CDT [68838] DEBUG:  index "pg_toast_16457_index" can safely use deduplication
2026-10-01 22:54:59.019 CDT [68838] DEBUG:  CREATE TABLE / PRIMARY KEY will create implicit index "_http_response_pkey" for table "_http_response"
2026-10-01 22:54:59.019 CDT [68838] DEBUG:  building index "_http_response_pkey" on table "_http_response" serially
2026-10-01 22:54:59.019 CDT [68838] DEBUG:  index "_http_response_pkey" can safely use deduplication
CREATE TABLE
22:54:59 postgres@[local:/tmp/nix-shell.Kn9i1d/tmp.aIxEqjvdqX]:5432/postgres  68838 =#
\d _http_response
              Table "public._http_response"
    Column    |  Type   | Collation | Nullable | Default
--------------+---------+-----------+----------+---------
 id           | bigint  |           | not null |
 status_code  | integer |           |          |
 content_type | text    |           |          |
 headers      | jsonb   |           |          |
 content      | text    |           |          |
 timed_out    | boolean |           |          |
 error_msg    | text    |           |          |
Indexes:
    "_http_response_pkey" PRIMARY KEY, btree (id)

22:55:03 postgres@[local:/tmp/nix-shell.Kn9i1d/tmp.aIxEqjvdqX]:5432/postgres  68838 =#
alter table _http_response drop constraint if exists _http_response_pkey cascade;
ALTER TABLE
22:55:11 postgres@[local:/tmp/nix-shell.Kn9i1d/tmp.aIxEqjvdqX]:5432/postgres  68838 =#
\d _http_response
              Table "public._http_response"
    Column    |  Type   | Collation | Nullable | Default
--------------+---------+-----------+----------+---------
 id           | bigint  |           | not null |
 status_code  | integer |           |          |
 content_type | text    |           |          |
 headers      | jsonb   |           |          |
 content      | text    |           |          |
 timed_out    | boolean |           |          |
 error_msg    | text    |           |          |

AndrewJackson2020 · 2026-10-02T13:37:00Z

last diff is shown below. seems like it was introduced in 8c059af. pg_net.sql was updated but not the corresponding upgrade script for the v0.7.1 release.vk/

master

CREATE FUNCTION net.http_collect_response(request_id bigint, async boolean DEFAULT true) RETURNS net.http_response_result
    LANGUAGE plpgsql
    AS $$
begin
  raise notice 'The net.http_collect_response function is deprecated.';
  select net._http_collect_response(request_id, async);
end;
$$;

and in this commit

CREATE FUNCTION net.http_collect_response(request_id bigint, async boolean DEFAULT true) RETURNS net.http_response_result
    LANGUAGE plpgsql
    AS $$
  raise notice 'The net.http_collect_response function is deprecated.';
  select net._http_collect_response(request_id, async);
$$;

hanefi

Looks good overall. However I do not feel confident to approve it and would like another set of eyes to take a look.

steve-chavez · 2026-10-08T15:21:18Z

Nitpick: looks like this should be a refactor and not a fix.

AndrewJackson2020 · 2026-10-14T21:03:08Z

Nitpick: looks like this should be a refactor and not a fix.

Changed. Though I will say that this refactor exposed some inconsistencies.

AndrewJackson2020 · 2026-10-20T03:16:06Z

I think this PR should be good now.

I added a new file in the root called pg_net.sql that essentially is a point in time snapshot of the state of the extension built via pg_dump. Its state is enforced via a CI job.

Used the script in this PR to take a schema dump of the current state of the extension here and in master branch.

@utkarash2991 please take a look at this when you have a minute, would like to merge this week if possible (and then move onto other pg_net stuff that require schema changes like the row level security stuff)

aj@Andrews-MacBook-Pro:~/Repos/mono/external/repos/supabase/pg_net/master/ > diff ~/pg_net.branch.sql ~/pg_net.master.sql
214c214
< begin;
---
> begin

pg_net.branch.sql
pg_net.master.sql

AndrewJackson2020 · 2026-10-21T17:05:23Z

@utkarash2991 @imor @mkaruza

any objections to merging this by tomorrow? IMO this PR is basically blocking any other PR that requires schema change and would like to get this merged.

utkarash2991 · 2026-10-21T19:42:23Z

+    volatile
+    language plpgsql
+as $$
+begin;


@AndrewJackson2020 I think there is a problem here:

psql (17.6) Type "help" for help. postgres=# create or replace function t_collect() returns void language plpgsql as $$ begin; raise notice 'The net.http_collect_response function is deprecated.'; end; $$;postgres$# postgres$# postgres$# postgres$# ERROR: syntax error at or near ";" LINE 2: begin; ^ postgres=# set check_function_bodies = off; SET postgres=# create or replace function t_collect() returns void language plpgsql as $$ begin; raise notice 'The net.http_collect_response function is deprecated.'; end; $$;postgres$# postgres$# postgres$# postgres$# CREATE FUNCTION postgres=# select t_collect(); ERROR: syntax error at or near ";" LINE 2: begin; ^ QUERY: begin; raise notice 'The net.http_collect_response function is deprecated.'; end; CONTEXT: compilation of PL/pgSQL function "t_collect" near line 1

IIUC the scripts are passing due to this code logic in extension.c : https://github.com/postgres/postgres/blob/master/src/backend/commands/extension.c#L1337

So the function wrong syntax is not parsed during creation. Thats why it wasn't caught .

May be for sanity checks lets add the test calling those functions ?

If this function is never called in test then agree that should be added here, agreed.

Another idea: I have a PR that integrates plpgsql_check into the build system and test suite. Maybe we should consider merging that first? I just tested on that branch and it would definitely have caught this issue.

aj@Andrews-MacBook-Pro:~/Repos/mono/external/repos/supabase/pg_net/master/ > git diff diff --git a/sql/pg_net.sql b/sql/pg_net.sql index 385ee2f..740b267 100644 --- a/sql/pg_net.sql +++ b/sql/pg_net.sql @@ -345,7 +345,7 @@ create or replace function net.http_collect_response( returns net.http_response_result language plpgsql as $$ -begin +begin; raise notice 'The net.http_collect_response function is deprecated.'; return net._http_collect_response(request_id, async); end;

15:07:17 postgres@[local:/tmp/nix-shell.GBVekV/tmp.hRvcDL8lq9]:5432/postgres 80995 =# SELECT p.oid, p.proname, plpgsql_check_function(p.oid) FROM pg_catalog.pg_namespace n JOIN pg_catalog.pg_proc p ON pronamespace = n.oid JOIN pg_catalog.pg_language l ON p.prolang = l.oid WHERE l.lanname = 'plpgsql' AND p.prorettype <> 2279; oid | proname | plpgsql_check_function -------+-----------------------+-------------------------------------------------------------------------------- 16430 | http_collect_response | error:42601:syntax error at or near ";" 16430 | http_collect_response | Query: 16430 | http_collect_response | begin; 16430 | http_collect_response | -- ^ 16430 | http_collect_response | raise notice 'The net.http_collect_response function is deprecated.'; 16430 | http_collect_response | return net._http_collect_response(request_id, async); 16430 | http_collect_response | end; 16430 | http_collect_response | 16430 | http_collect_response | Context: compilation of PL/pgSQL function "http_collect_response" near line 1 (9 rows)

#318

I added the semicolon. Turns out that function never worked from day 1 because it needed to call the last statement with RETURN and not SELECT. Maybe we should just get rid of the thing entirely since it was marked for deprecation anyway?

Thinking about this more we should definitely not be removing this function in thisPR. @utkarash2991 does this PR look good after the above changes? If so would like to merge this week.

utkarash2991 · 2026-10-21T20:02:09Z

+END;
+$$;
+EOF) \
+  --command="\! pg_dump --schema=net --user=postgres --schema-only | grep -v -- -- | grep -v '^SET .*;' | grep -v 'SELECT pg_catalog.set_config' | pgpp > pg_net.sql"


I think the grep -v -- -- can create problem in some cases. I got that its been added to remove the pg_dump comment headers. But I believe it can lead to loss of contents in function body .
Please see below, for ex:

The comments in the function got removed in pg_net.sql generated file:

-- request_id reference request_id bigint, -- when `true`, return immediately. when `false` wait for the request to complete before returning async bool default true ) -- http response composite wrapped in a result type returns net.http_response_result language plpgsql as $$ declare rec net._http_response; req_exists boolean; begin if not async then perform net._await_response(request_id); end if; select * into rec from net._http_response where id = request_id; if rec is null or rec.error_msg is not null then -- The request is either still processing or the request_id provided does not exist -- TODO: request in progress is indistinguishable from request that doesn't exist -- No request matching request_id found return ( 'ERROR', coalesce(rec.error_msg, 'request matching request_id not found'), null )::net.http_response_result; end if; ....

Ideally the function body looses comments which can be harmless but what if the function body written like:

-- input CREATE FUNCTION f() AS $body$ -- an indented comment inside the body select 1 where x like '%--%'; $body$; -- after grep -v -- -- CREATE FUNCTION f() AS $body$ $body$;

I think we need to fix this

Yeah good catch. will fix.

This is fixed. I ended up just removing that statement altogether. pglast (the postgres SQL formatter that is the last stage of the pipe) by default removes comments anyway.

[nix-shell:~/Repos/mono/external/repos/supabase/pg_net/master]$ pgpp --help | grep commen -C, --preserve-comments preserve comments in the statement

utkarash2991 · 2026-10-28T19:09:09Z

@AndrewJackson2020 LGTM.

AndrewJackson2020 · 2026-10-29T15:55:51Z

@AndrewJackson2020 LGTM.

Thank you. Will double check and merge today.

AndrewJackson2020 · 2026-10-29T16:07:33Z

final checkout looks good

143c143
<   select net._http_collect_response(request_id, async);
---
>   return net._http_collect_response(request_id, async);

master.sql
enable_upgrade_test.sql

AndrewJackson2020 · 2026-11-11T22:02:24Z

One last comparison between the current master and this branch before merging. As expected the only diff is the non functioning SQL statement.

bash-5.3$ diff pg_net_master.sql pg_net_enable_upgrade_test.sql
'143c143
<   select net._http_collect_response(request_id, async);
---
>   return net._http_collect_response(request_id, async);

pg_net_enable_upgrade_test.sql
pg_net_master.sql

Currently in pg_net the upgrade code path is different from the create extension code path. When running tests internally we are not able to do true upgrade testing because the initial release of pg_net, 0.1 is not avaiable. This commit cherry picks 0.1 from the 0.1 tag, removes the pg_net.sql file and forces all create extension to happen by creating 0.1 and upgrading all the way until the current version. This commit also corrects a missing migration that was added pg_net.sql but not in the uprade script during 8b55e96. Regarding this missing migraiton, this may need an additional PR to correct somehow. It is not hard to imagine instances in the wild that do not have this modified function that should. I think this is a better approach and will open us up for upgrade testing internal to the repo. This commit

AndrewJackson2020 changed the title ~~fix: create extension from beginning of lineaage~~ fix: create extension from beginning of lineage Oct 2, 2026

AndrewJackson2020 force-pushed the enable_upgrade_test branch from 37fbfdc to 1c7b517 Compare October 2, 2026 02:56

AndrewJackson2020 marked this pull request as ready for review October 2, 2026 03:08

AndrewJackson2020 requested a review from a team as a code owner October 2, 2026 03:08

AndrewJackson2020 requested a review from steve-chavez October 2, 2026 03:09

AndrewJackson2020 force-pushed the enable_upgrade_test branch from 1c7b517 to 67ef171 Compare October 2, 2026 03:32

AndrewJackson2020 force-pushed the enable_upgrade_test branch 2 times, most recently from a6421a0 to 5eaf0f6 Compare October 2, 2026 20:42

AndrewJackson2020 mentioned this pull request Oct 5, 2026

Schema-qualify hypopg_reset() and hypopg_get_indexdef() calls supabase/index_advisor#12

Closed

hanefi reviewed Oct 6, 2026

View reviewed changes

Comment thread .gitignore Resolved

Comment thread Makefile Resolved

AndrewJackson2020 mentioned this pull request Oct 14, 2026

chore: add plpgsql_check to testing #318

Merged

AndrewJackson2020 force-pushed the enable_upgrade_test branch 2 times, most recently from 6aebc3d to 00ff5c2 Compare October 14, 2026 21:02

AndrewJackson2020 changed the title ~~fix: create extension from beginning of lineage~~ refactor: create extension from beginning of lineage Oct 14, 2026

steve-chavez approved these changes Oct 14, 2026

View reviewed changes

AndrewJackson2020 mentioned this pull request Oct 19, 2026

fix: add row level security over shared tables #303

Open

AndrewJackson2020 force-pushed the enable_upgrade_test branch 4 times, most recently from 0250e90 to 95412db Compare October 20, 2026 03:14

AndrewJackson2020 force-pushed the enable_upgrade_test branch from 95412db to 055d3f6 Compare October 20, 2026 21:14

AndrewJackson2020 mentioned this pull request Oct 21, 2026

refactor: add schema snapshot supabase/vault#66

Merged

utkarash2991 reviewed Oct 21, 2026

View reviewed changes

AndrewJackson2020 force-pushed the enable_upgrade_test branch 2 times, most recently from ac61c5c to 5db137b Compare October 21, 2026 20:57

utkarash2991 approved these changes Oct 28, 2026

View reviewed changes

AndrewJackson2020 force-pushed the enable_upgrade_test branch 6 times, most recently from 7671b82 to 788d5bc Compare October 29, 2026 18:23

AndrewJackson2020 force-pushed the enable_upgrade_test branch 2 times, most recently from 6be8cb4 to c5836a5 Compare November 11, 2026 21:57

AndrewJackson2020 added 2 commits November 11, 2026 17:00

fix: explicit void in function event_monitor_close

325c7de

AndrewJackson2020 force-pushed the enable_upgrade_test branch from c5836a5 to 325c7de Compare November 11, 2026 23:00

Conversation

AndrewJackson2020 commented Oct 1, 2026

Uh oh!

AndrewJackson2020 commented Oct 1, 2026

Uh oh!

AndrewJackson2020 commented Oct 1, 2026

Uh oh!

AndrewJackson2020 commented Oct 1, 2026

Uh oh!

steve-chavez commented Oct 1, 2026

Uh oh!

AndrewJackson2020 commented Oct 2, 2026

Uh oh!

AndrewJackson2020 commented Oct 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

AndrewJackson2020 commented Oct 2, 2026

Uh oh!

hanefi left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

steve-chavez commented Oct 8, 2026

Uh oh!

AndrewJackson2020 commented Oct 14, 2026

Uh oh!

AndrewJackson2020 commented Oct 20, 2026

Uh oh!

AndrewJackson2020 commented Oct 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

utkarash2991 Oct 21, 2026

Choose a reason for hiding this comment

Uh oh!

utkarash2991 Oct 21, 2026

Choose a reason for hiding this comment

Uh oh!

AndrewJackson2020 Oct 21, 2026

Choose a reason for hiding this comment

Uh oh!

AndrewJackson2020 Oct 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

AndrewJackson2020 Oct 28, 2026

Choose a reason for hiding this comment

Uh oh!

utkarash2991 Oct 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

AndrewJackson2020 Oct 21, 2026

Choose a reason for hiding this comment

Uh oh!

AndrewJackson2020 Oct 21, 2026

Choose a reason for hiding this comment

Uh oh!

utkarash2991 commented Oct 28, 2026

Uh oh!

AndrewJackson2020 commented Oct 29, 2026

Uh oh!

AndrewJackson2020 commented Oct 29, 2026

Uh oh!

AndrewJackson2020 commented Nov 11, 2026

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

AndrewJackson2020 commented Oct 2, 2026 •

edited

Loading

AndrewJackson2020 commented Oct 21, 2026 •

edited

Loading

AndrewJackson2020 Oct 21, 2026 •

edited

Loading

utkarash2991 Oct 21, 2026 •

edited

Loading