Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update directions for snapshot #761

Merged
merged 1 commit into from
Oct 16, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions packages/snaplet/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
.snaplet/snapshots/*
30 changes: 16 additions & 14 deletions packages/snaplet/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,21 @@ It seeds our database using two main methods:

[`seed.ts`](./seed.ts): This resets the database and seeds it with a set of default data. It is used to seed the database for local development esepcially for data that is not available in the production database.

## Capturing snapshots

`bunx @snaplet/snapshot snapshot capture`: This captures a snapshot of the database and saves it locally. It can then be shared with other developers by uploading the snapshot to snaplet's cloud storage.

**⚠️ Capturing snapshots** requires access to the production database.

```shell
# set to production database url
export SNAPLET_SOURCE_DATABASE_URL=postgresql://postgres:postgres@127.0.0.1:54322/postgres
bunx @snaplet/snapshot snapshot capture
```

## Restoring from snapshot

`bunx snaplet snapshot restore --no-reset`: This restores the database from a snapshot hosted in snaplet's cloud storage. It is used to restore the database for local development and restores production-like data. This is useful for testing and debugging.
`bunx @snaplet/snapshot snapshot restore --no-reset`: This restores the database from a snapshot hosted in snaplet's cloud storage. It is used to restore the database for local development and restores production-like data. This is useful for testing and debugging.

**⚠️ When restoring from snapshot** migrations are not run and can make your local database inconsistent with the production database or even fail to restore some data. To mitigate this, remove any migrations that are not in production yet. See below for how to remove migrations to overcome this.

Expand All @@ -26,22 +38,12 @@ git diff --name-only --diff-filter=A origin/main..HEAD -- supabase/migrations |
### Restore

```shell
# set target database url to local development database
export SNAPLET_TARGET_DATABASE_URL=$SUPABASE_DB_URL
# now run the snapshot restore command
bunx supabase db reset && \
bunx snaplet snapshot restore --no-reset --latest && \
bunx @snaplet/snapshot snapshot restore --no-reset --latest && \
git checkout ./supabase/migrations && \
bunx supabase db push --local --include-all
psql $SUPABASE_DB_URL -c "insert into send_accounts (user_id, address, chain_id, init_code) select u.id as user_id, c.address, '845337' as chain_id, CONCAT( '\\x00', upper( CONCAT( md5(random() :: text), md5(random() :: text), md5(random() :: text), md5(random() :: text) ) ) ) :: bytea as init_code from auth.users u join chain_addresses c on c.user_id = u.id where user_id not in ( select user_id from send_accounts );"
```

## Capturing snapshots

`bunx snaplet snapshot capture`: This captures a snapshot of the database and saves it locally. It can then be shared with other developers by uploading the snapshot to snaplet's cloud storage.

**⚠️ Capturing snapshots** requires access to the production database.

```shell
export SNAPLET_SOURCE_DATABASE_URL=postgresql://postgres:postgres@127.0.0.1:54322/postgres
bunx snaplet snapshot capture
bunx snaplet snapshot share <snapshot-id>
```
19 changes: 19 additions & 0 deletions packages/snaplet/snaplet.config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -51,8 +51,27 @@ export default defineConfig({
shovel: false,
// @ts-ignore
pgtap: false,
public: {
// activity: false,
},
},
// TODO: figure out how much data we need to snapshot
// subset: {
// targets: [
// // {
// // table: "public.activity",
// // orderBy: `"activity"."created_at" desc`,
// // percent: 10
// // },
// // {
// // table: "public.send_account_transfers",
// // orderBy: `"send_account_transfers"."block_num" desc`,
// // percent: 5
// // }
// ],
// },
transform: {
$mode: 'auto',
auth: {
users: ({ row }) => {
let phone: string
Expand Down
Loading