diff --git a/apps/docs/content/guides/deployment/branching.mdx b/apps/docs/content/guides/deployment/branching.mdx index 23342dd4c89c8..d4f3ed3ca116a 100644 --- a/apps/docs/content/guides/deployment/branching.mdx +++ b/apps/docs/content/guides/deployment/branching.mdx @@ -15,7 +15,8 @@ Supabase branches create separate environments that spin off from your main proj - **Preview Branches**: Preview branches are ephemeral and best suited for focused testing. They are automatically deleted when a PR is merged or closed. - **Persistent Branches**: Persistent branches are long-lived and recommended for environments like staging, QA, or development. They aren't automatically paused or deleted due to inactivity or when a PR is merged or closed. - **Managing Branches**: You can create, review, and merge branches either automatically via our [GitHub integration](/docs/guides/deployment/branching/github-integration) or directly [through the dashboard](/docs/guides/deployment/branching/dashboard) (currently in beta). All branches show up in the branches page in the dashboard, regardless of how they were created. -- **Data-less**: New branches do not start with any data from your main project. This is meant to better protect your sensitive production data. To start your branches with data, you can use a [seed file](/docs/guides/deployment/branching/github-integration#seeding) if using the GitHub integration. +- **Cloned from your main project**: Each new branch is created as a clone of your main project. It starts with that project's Edge Functions deployed and the configuration set. +- **Data-less by default**: By default, new branches do not start with any data or storage objects from your main project. This is meant to better protect your sensitive production data. To start your branches with data, you can use a [seed file](/docs/guides/deployment/branching/github-integration#seeding) if using the GitHub integration, or the [Include data](/docs/guides/deployment/branching/dashboard#include-production-data) option if you create the branch from the dashboard. ## Deploying to production diff --git a/apps/docs/content/guides/deployment/branching/dashboard.mdx b/apps/docs/content/guides/deployment/branching/dashboard.mdx index ad1ae7021c858..e10efb1c2cede 100644 --- a/apps/docs/content/guides/deployment/branching/dashboard.mdx +++ b/apps/docs/content/guides/deployment/branching/dashboard.mdx @@ -41,6 +41,24 @@ Once you've enabled the feature, you can create a new branch: 1. Click the arrows next to the branch name in the top menu bar. (The top menu bar has the format `YOUR_ORGANIZATION / YOUR_PROJECT / CURRENT_BRANCH_NAME`.) 2. Click `Create branch`. +The new branch is a clone of your base project. It starts with the project's schema, Edge Functions, and the configuration set. + +### Include production data + +By default, a branch starts without any of your production data or storage objects. If your project has the Point-in-Time Recovery add-on, you can turn on **Include data** when you create the branch to copy your production data into it. + + + +A branch created with **Include data** holds a copy of your production data, treat it with the same care as production. + + + + + +A branch uses a larger disk and matches the compute size of your project, which increases its cost. + + + ## Making changes to a branch Use the branch selector in the top bar to change to your branch. Any changes you make (including SQL run in the SQL editor, table editor changes, and configuration changes) are now made against the currently selected branch. diff --git a/apps/docs/content/guides/deployment/branching/github-integration.mdx b/apps/docs/content/guides/deployment/branching/github-integration.mdx index c04cb14f4fe61..cb81e3bff7239 100644 --- a/apps/docs/content/guides/deployment/branching/github-integration.mdx +++ b/apps/docs/content/guides/deployment/branching/github-integration.mdx @@ -83,19 +83,25 @@ Enable the **Automatic branching** option in your GitHub Integration configurati When a new branch is created in GitHub, a corresponding branch is created in Supabase. (You can enable the **Supabase changes only** option to only create Supabase branches when Supabase files change.) +Every Supabase branch, preview or persistent, is created as a clone of your base project. The new branch starts with the Edge Functions and configuration of that project. Its database schema is not cloned. Instead, it is built from the migrations you commit to your repository. + ### Configuration You can test configuration changes on your Preview Branch by configuring the `config.toml` file in your Supabase directory. See the [Configuration docs](/docs/guides/deployment/branching/configuration) for more information. +Your branch starts with the configuration of your base project. The settings in your `config.toml` file are applied on top of that clone. + A comment is added to your PR with the deployment status of your preview branch. ### Migrations -The migrations in the `migrations` subdirectory of your Supabase directory are automatically run. +The migrations in the `migrations` subdirectory of your Supabase directory are automatically run when the branch is created. Each later commit runs only the migrations that haven't been applied yet. + +If you want to rerun existing migrations, reset the branch from the Supabase dashboard to start from scratch. Note that existing data on your branch will also be dropped by a reset. ### Seeding -No production data is copied to your Preview branch. This is meant to protect your sensitive production data. +Cloning your base project copies its Edge Functions and configuration, but not its data or storage objects. This is meant to protect your sensitive production data. Your branch starts with the tables your migrations create, and the only rows in them are the ones your seed files add. You can seed your Preview Branch with sample data using the `seed.sql` file in your Supabase directory. See the [Seeding docs](/docs/guides/local-development/seeding-your-database) for more information. diff --git a/apps/docs/content/guides/deployment/branching/troubleshooting.mdx b/apps/docs/content/guides/deployment/branching/troubleshooting.mdx index 6e77b2c17683d..1fd9194c40c4f 100644 --- a/apps/docs/content/guides/deployment/branching/troubleshooting.mdx +++ b/apps/docs/content/guides/deployment/branching/troubleshooting.mdx @@ -27,7 +27,9 @@ You might want to roll back changes you've made in an earlier migration change. To fix this, push the latest changes, then delete the preview branch in Supabase and reopen it. -The new preview branch is reseeded from the `./supabase/seed.sql` file by default. Any additional data changes made on the old preview branch are lost. This is equivalent to running `supabase db reset` locally. All migrations are rerun in sequential order. +The new preview branch is a fresh clone of your base project and is reseeded from the `./supabase/seed.sql` file by default. Any additional data changes made on the old preview branch are lost. + +To rerun migrations that your base project has already applied, reset the branch from the Supabase dashboard instead. A reset reruns all migrations in sequential order and drops existing data on the branch. ### Deployment failures diff --git a/apps/docs/content/guides/deployment/branching/working-with-branches.mdx b/apps/docs/content/guides/deployment/branching/working-with-branches.mdx index ecdceb086583f..c007f17cf5fe7 100644 --- a/apps/docs/content/guides/deployment/branching/working-with-branches.mdx +++ b/apps/docs/content/guides/deployment/branching/working-with-branches.mdx @@ -181,7 +181,7 @@ After completing the steps above, you should receive a Slack message whenever an Migrations are run in sequential order. Each migration builds upon the previous one. -The preview branch has a record of which migrations have been applied, and only applies new migrations for each commit. This can create an issue when rolling back migrations. +The preview branch inherits the migration history of your base project, so it only applies migrations that haven't been run yet. This can create an issue when rolling back migrations. ### Using ORM or custom seed scripts @@ -237,7 +237,9 @@ You might want to roll back changes you've made in an earlier migration change. To fix this, push the latest changes, then delete the preview branch in Supabase and reopen it. -The new preview branch is reseeded from the `./supabase/seed.sql` file by default. Any additional data changes made on the old preview branch are lost. This is equivalent to running `supabase db reset` locally. All migrations are rerun in sequential order. +The new preview branch is a fresh clone of your base project and is reseeded from the `./supabase/seed.sql` file by default. Any additional data changes made on the old preview branch are lost. + +To rerun migrations that your base project has already applied, reset the branch from the Supabase dashboard instead. A reset reruns all migrations in sequential order and drops existing data on the branch. ### Seeding behavior diff --git a/apps/studio/TANSTACK_MIGRATION.md b/apps/studio/TANSTACK_MIGRATION.md index fb8ba8cc114de..13644b1c7eacd 100644 --- a/apps/studio/TANSTACK_MIGRATION.md +++ b/apps/studio/TANSTACK_MIGRATION.md @@ -231,6 +231,7 @@ These are the layout-only TanStack files. Most hold a single product layout comp ### Project shell — `/workers/*` - [x] A `routes/project/$ref/workers/index.tsx` ← `pages/project/[ref]/workers/index.tsx` +- [x] A `routes/project/$ref/workers/$name.tsx` ← `pages/project/[ref]/workers/[name].tsx` ### Project shell — `/functions/*` diff --git a/apps/studio/components/interfaces/ConnectSheet/Connect.constants.ts b/apps/studio/components/interfaces/ConnectSheet/Connect.constants.ts index 86071ccaef9b8..a588f3320dd39 100644 --- a/apps/studio/components/interfaces/ConnectSheet/Connect.constants.ts +++ b/apps/studio/components/interfaces/ConnectSheet/Connect.constants.ts @@ -386,6 +386,8 @@ export const PGBOUNCER_ENABLED_BUT_NO_IPV4_ADDON_TEXT = 'Purchase IPv4 add-on or use Shared Pooler if on a IPv4 network' export const IPV4_ADDON_TEXT = 'Connections are IPv4 proxied with IPv4 add-on' +export const CONNECTION_SOURCE_LOAD_BALANCER = 'load-balancer' + export type ConnectionStringMethod = 'direct' | 'transaction' | 'session' export const connectionStringMethodOptions: Record< diff --git a/apps/studio/components/interfaces/ConnectSheet/ConnectConfigSection.tsx b/apps/studio/components/interfaces/ConnectSheet/ConnectConfigSection.tsx index bb21ae25718f1..9f96b39af66cf 100644 --- a/apps/studio/components/interfaces/ConnectSheet/ConnectConfigSection.tsx +++ b/apps/studio/components/interfaces/ConnectSheet/ConnectConfigSection.tsx @@ -126,12 +126,14 @@ export function ConnectConfigSection({ layout="horizontal" label={field.label} description={field.description} + name={`connect-${field.id}`} > - {field.value ?? 'error'} + + {INVALIDATED_SLOT_BEHAVIOR_LABELS[field.value ?? 'error']} + -

Error

+

Block startup

Blocks startup for manual recovery.

-

Recreate

+

Recreate slot

Replaces destination tables and runs a new, billable initial sync.

@@ -222,7 +229,7 @@ export const AdvancedSettings = ({ } layout="horizontal" - description="How old query results can be while BigQuery applies ongoing changes." + description="Set the maximum age of query results while BigQuery applies ongoing changes, or leave blank for the freshest results." > @@ -233,7 +240,6 @@ export const AdvancedSettings = ({ step={1} value={field.value ?? ''} onChange={handleNumberChange(field)} - placeholder="Default: None (Freshest results)" /> minutes diff --git a/apps/studio/components/interfaces/Database/Replication/DestinationPanel/DestinationForm/AnalyticsBucket/AnalyticsBucket.utils.ts b/apps/studio/components/interfaces/Database/Replication/DestinationPanel/DestinationForm/AnalyticsBucket/AnalyticsBucket.utils.ts index 3aeb5cd90606e..d443262582e26 100644 --- a/apps/studio/components/interfaces/Database/Replication/DestinationPanel/DestinationForm/AnalyticsBucket/AnalyticsBucket.utils.ts +++ b/apps/studio/components/interfaces/Database/Replication/DestinationPanel/DestinationForm/AnalyticsBucket/AnalyticsBucket.utils.ts @@ -21,9 +21,9 @@ type AnalyticsBucketValidationOptions = { // Fields that are always required regardless of the namespace / access key selections. const ANALYTICS_BUCKET_REQUIRED_FIELDS: AnalyticsBucketValidationIssue[] = [ - { path: 'warehouseName', message: 'Bucket is required' }, - { path: 's3Region', message: 'S3 region is required' }, - { path: 's3AccessKeyId', message: 'S3 access key ID is required' }, + { path: 'warehouseName', message: 'Bucket is required.' }, + { path: 's3Region', message: 'S3 region is required.' }, + { path: 's3AccessKeyId', message: 'S3 access key ID is required.' }, ] export const getAnalyticsBucketValidationIssues = ( @@ -43,8 +43,8 @@ export const getAnalyticsBucketValidationIssues = ( if (!hasValidNamespace) { issues.push( isCreatingNewNamespace - ? { path: 'newNamespaceName', message: 'Namespace name is required' } - : { path: 'namespace', message: 'Namespace is required' } + ? { path: 'newNamespaceName', message: 'Namespace name is required.' } + : { path: 'namespace', message: 'Namespace is required.' } ) } @@ -53,7 +53,7 @@ export const getAnalyticsBucketValidationIssues = ( data.s3SecretAccessKey?.trim().length && !data.s3AccessKeyId?.trim().length ) { - issues.push({ path: 's3AccessKeyId', message: 'S3 access key ID is required' }) + issues.push({ path: 's3AccessKeyId', message: 'S3 access key ID is required.' }) } const currentS3AccessKeyId = data.s3AccessKeyId?.trim() @@ -69,7 +69,7 @@ export const getAnalyticsBucketValidationIssues = ( (!options.secretsOptional || hasChangedStoredS3AccessKey) && !data.s3SecretAccessKey?.trim().length ) { - issues.push({ path: 's3SecretAccessKey', message: 'S3 secret access key is required' }) + issues.push({ path: 's3SecretAccessKey', message: 'S3 secret access key is required.' }) } return issues diff --git a/apps/studio/components/interfaces/Database/Replication/DestinationPanel/DestinationForm/BigQuery/BigQuery.schema.ts b/apps/studio/components/interfaces/Database/Replication/DestinationPanel/DestinationForm/BigQuery/BigQuery.schema.ts index abd86b79faf9d..b6fa2279c4915 100644 --- a/apps/studio/components/interfaces/Database/Replication/DestinationPanel/DestinationForm/BigQuery/BigQuery.schema.ts +++ b/apps/studio/components/interfaces/Database/Replication/DestinationPanel/DestinationForm/BigQuery/BigQuery.schema.ts @@ -7,11 +7,11 @@ export const BigQueryFormSchema = z.object({ connectionPoolSize: z .number() .int() - .min(1, 'Connection pool size must be greater than 0') + .min(1, 'Connection pool size must be greater than 0.') .optional(), maxStalenessMins: z .number() - .int('Maximum staleness must be a whole number of minutes') - .min(0, 'Maximum staleness must be 0 or greater') + .int('Maximum staleness must be a whole number of minutes.') + .min(0, 'Maximum staleness must be 0 or greater.') .optional(), }) diff --git a/apps/studio/components/interfaces/Database/Replication/DestinationPanel/DestinationForm/BigQuery/BigQuery.utils.ts b/apps/studio/components/interfaces/Database/Replication/DestinationPanel/DestinationForm/BigQuery/BigQuery.utils.ts index 5bc5181d1558a..423632f5c4fe8 100644 --- a/apps/studio/components/interfaces/Database/Replication/DestinationPanel/DestinationForm/BigQuery/BigQuery.utils.ts +++ b/apps/studio/components/interfaces/Database/Replication/DestinationPanel/DestinationForm/BigQuery/BigQuery.utils.ts @@ -7,14 +7,14 @@ export type BigQueryValidationIssue = { message: string } -export const BIGQUERY_SERVICE_ACCOUNT_JSON_MESSAGE = 'Service account key must be valid JSON' +export const BIGQUERY_SERVICE_ACCOUNT_JSON_MESSAGE = 'Service account key must be valid JSON.' const BIGQUERY_REQUIRED_FIELDS: { path: Exclude message: string }[] = [ - { path: 'projectId', message: 'Project ID is required' }, - { path: 'datasetId', message: 'Dataset ID is required' }, + { path: 'projectId', message: 'Project ID is required.' }, + { path: 'datasetId', message: 'Dataset ID is required.' }, ] const isValidJsonString = (value: string) => { @@ -39,7 +39,7 @@ export const getBigQueryValidationIssues = ( if (!serviceAccountKey) { if (!secretsOptional) { - issues.push({ path: 'serviceAccountKey', message: 'Service account key is required' }) + issues.push({ path: 'serviceAccountKey', message: 'Service account key is required.' }) } return issues } diff --git a/apps/studio/components/interfaces/Database/Replication/DestinationPanel/DestinationForm/BigQuery/Fields.test.tsx b/apps/studio/components/interfaces/Database/Replication/DestinationPanel/DestinationForm/BigQuery/Fields.test.tsx new file mode 100644 index 0000000000000..6322f20270dce --- /dev/null +++ b/apps/studio/components/interfaces/Database/Replication/DestinationPanel/DestinationForm/BigQuery/Fields.test.tsx @@ -0,0 +1,90 @@ +import { fireEvent, screen, waitFor } from '@testing-library/react' +import { useForm } from 'react-hook-form' +import { Form } from 'ui' +import { describe, expect, it, vi } from 'vitest' + +import type { DestinationPanelSchemaType } from '../DestinationForm.schema' +import { BigQueryFields } from './Fields' +import { customRender } from '@/tests/lib/custom-render' + +const TestForm = ({ serviceAccountKey = '' }: { serviceAccountKey?: string }) => { + const form = useForm({ + defaultValues: { projectId: '', datasetId: '', serviceAccountKey }, + }) + + return ( +
+ + + ) +} + +describe('BigQueryFields', () => { + it('imports a JSON file into an editable service account key field', async () => { + const { container } = customRender() + const contents = '{"type":"service_account"}' + const file = new File([contents], 'service-account.json', { type: 'application/json' }) + Object.defineProperty(file, 'text', { value: vi.fn().mockResolvedValue(contents) }) + + const fileInput = container.querySelector('input[type="file"]') + expect(fileInput).not.toBeNull() + fireEvent.change(fileInput!, { target: { files: [file] } }) + + const textarea = screen.getByPlaceholderText( + '{"type": "service_account", "project_id": "...", ...}' + ) + await waitFor(() => expect(textarea).toHaveValue(contents)) + + fireEvent.change(textarea, { target: { value: `${contents}\n` } }) + expect(textarea).toHaveValue(`${contents}\n`) + expect(textarea).toHaveClass('max-h-[calc(13lh+1rem)]') + }) + + it('imports a dropped JSON file into the service account key field', async () => { + customRender() + const contents = '{"type":"service_account"}' + const file = new File([contents], 'service-account.json', { type: 'application/json' }) + Object.defineProperty(file, 'text', { value: vi.fn().mockResolvedValue(contents) }) + + const textarea = screen.getByPlaceholderText( + '{"type": "service_account", "project_id": "...", ...}' + ) + fireEvent.drop(textarea.closest('div')!, { + dataTransfer: { files: [file] }, + }) + + await waitFor(() => expect(textarea).toHaveValue(contents)) + }) + + it('rejects an oversized file without replacing the existing key', async () => { + const { container } = customRender() + const file = new File(['x'.repeat(5001)], 'service-account.json', { + type: 'application/json', + }) + const readFile = vi.fn() + Object.defineProperty(file, 'text', { value: readFile }) + + const fileInput = container.querySelector('input[type="file"]') + fireEvent.change(fileInput!, { target: { files: [file] } }) + + expect( + await screen.findByText('Service account key must be 5,000 characters or fewer.') + ).toBeInTheDocument() + expect(readFile).not.toHaveBeenCalled() + expect(screen.getByDisplayValue('existing-key')).toBeInTheDocument() + }) + + it('preserves the existing key when the selected file cannot be read', async () => { + const { container } = customRender() + const file = new File(['{}'], 'service-account.json', { type: 'application/json' }) + Object.defineProperty(file, 'text', { + value: vi.fn().mockRejectedValue(new Error('File read failed')), + }) + + const fileInput = container.querySelector('input[type="file"]') + fireEvent.change(fileInput!, { target: { files: [file] } }) + + expect(await screen.findByText('Could not read the selected JSON file.')).toBeInTheDocument() + expect(screen.getByDisplayValue('existing-key')).toBeInTheDocument() + }) +}) diff --git a/apps/studio/components/interfaces/Database/Replication/DestinationPanel/DestinationForm/BigQuery/Fields.tsx b/apps/studio/components/interfaces/Database/Replication/DestinationPanel/DestinationForm/BigQuery/Fields.tsx index 9e7db5d75cfd9..bb4e4397d957f 100644 --- a/apps/studio/components/interfaces/Database/Replication/DestinationPanel/DestinationForm/BigQuery/Fields.tsx +++ b/apps/studio/components/interfaces/Database/Replication/DestinationPanel/DestinationForm/BigQuery/Fields.tsx @@ -1,10 +1,54 @@ +import { Upload } from 'lucide-react' +import { useRef, useState, type ChangeEvent, type DragEvent } from 'react' import type { UseFormReturn } from 'react-hook-form' -import { FormControl, FormField, Input, TextArea } from 'ui' +import { Button, cn, FormControl, FormField, Input, TextArea } from 'ui' import { FormItemLayout } from 'ui-patterns/form/FormItemLayout/FormItemLayout' import { STORED_SECRET_PLACEHOLDER } from '../DestinationForm.constants' import type { DestinationPanelSchemaType } from '../DestinationForm.schema' +const MAX_SERVICE_ACCOUNT_KEY_LENGTH = 5000 + +const readServiceAccountFile = async ( + file: File, + form: UseFormReturn, + isCurrentRequest: () => boolean +) => { + if (file.size > MAX_SERVICE_ACCOUNT_KEY_LENGTH) { + if (isCurrentRequest()) { + form.setError('serviceAccountKey', { + message: 'Service account key must be 5,000 characters or fewer.', + }) + } + return + } + + try { + const contents = await file.text() + if (!isCurrentRequest()) return + + if (contents.length > MAX_SERVICE_ACCOUNT_KEY_LENGTH) { + form.setError('serviceAccountKey', { + message: 'Service account key must be 5,000 characters or fewer.', + }) + return + } + + form.setValue('serviceAccountKey', contents, { + shouldDirty: true, + shouldTouch: true, + shouldValidate: true, + }) + form.clearErrors('serviceAccountKey') + } catch { + if (isCurrentRequest()) { + form.setError('serviceAccountKey', { + message: 'Could not read the selected JSON file.', + }) + } + } +} + export const BigQueryFields = ({ form, editMode, @@ -12,6 +56,41 @@ export const BigQueryFields = ({ form: UseFormReturn editMode: boolean }) => { + const serviceAccountFileInputRef = useRef(null) + const fileReadRequestIdRef = useRef(0) + const [isDraggingFile, setIsDraggingFile] = useState(false) + + const handleServiceAccountFile = async (file: File | undefined) => { + if (!file) return + const requestId = ++fileReadRequestIdRef.current + await readServiceAccountFile(file, form, () => requestId === fileReadRequestIdRef.current) + } + + const handleServiceAccountFileInputChange = async (event: ChangeEvent) => { + const file = event.target.files?.[0] + event.target.value = '' + await handleServiceAccountFile(file) + } + + const handleDragOver = (event: DragEvent) => { + event.preventDefault() + event.stopPropagation() + setIsDraggingFile(true) + } + + const handleDragLeave = (event: DragEvent) => { + event.preventDefault() + event.stopPropagation() + setIsDraggingFile(false) + } + + const handleDrop = async (event: DragEvent) => { + event.preventDefault() + event.stopPropagation() + setIsDraggingFile(false) + await handleServiceAccountFile(event.dataTransfer.files?.[0]) + } + return (

BigQuery settings

@@ -57,23 +136,58 @@ export const BigQueryFields = ({ label="Service account key" description={ editMode - ? 'Stored credentials are hidden. Enter new credentials to replace them.' - : 'Service account credentials JSON for authenticating with BigQuery' + ? 'Stored credentials are hidden. Paste or upload new credentials to replace them.' + : 'Paste or upload your service account credentials JSON file for authenticating with BigQuery.' } > - -