Skip to content

Commit

Permalink
feat: migrate script & function
Browse files Browse the repository at this point in the history
  • Loading branch information
oliver-oloughlin committed Feb 14, 2024
1 parent e512dfe commit b7f3048
Show file tree
Hide file tree
Showing 5 changed files with 756 additions and 1 deletion.
35 changes: 35 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,9 @@ _Supported Deno verisons:_ **^1.40.0**
- [Zod](#zod)
- [zodModel()](#zodmodel)
- [Kv-Schemas](#kv-schemas)
- [Migrate](#migrate)
- [Script](#script)
- [Function](#function)
- [Blob Storage](#blob-storage)
- [Development](#development)
- [License](#license)
Expand Down Expand Up @@ -1289,6 +1292,38 @@ const PostSchema = z.object({
})
```

## Migrate

A helper script and function for migrating entries from a source KV instance to
a target KV instance. Only migrates `kvdex` entries by default, but optionally
allows for migrating all entries.

### Script

Run the migrate script and provide --source and --target arguments. Optionally
pass --all to migrate all entries.

```console
deno run -A --unstable-kv jsr:@olli/kvdex/ext/migrate --source=./source.sqlite3 --target=./target.sqlite3
```

### Function

Use the migrate function and pass a source KV instance and a target KV instance.
Optionally pass `all: true` to migrate all entries.

```ts
import { migrate } from "jsr:@olli/kvdex/ext/migrate"

const source = await Deno.openKv("./source.sqlite3")
const target = await Deno.openKv("./target.sqlite3")

await migrate({
source,
target,
})
```

## Blob Storage

To store large blob sizes, and bypass the data limit of a single atomic
Expand Down
4 changes: 3 additions & 1 deletion deno.json
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,8 @@
"version": "0.34.1",
"exports": {
".": "./mod.ts",
"./ext/zod": "./ext/zod.ts"
"./ext/zod": "./ext/zod.ts",
"./ext/migrate": "./ext/migrate.ts"
},
"imports": {
"assert": "jsr:@std/[email protected]/assert",
Expand All @@ -13,6 +14,7 @@
"ulid": "jsr:@std/[email protected]",
"concat": "jsr:@std/[email protected]/concat",
"deep_merge": "jsr:@std/[email protected]/deep_merge",
"parse_args": "jsr:@std/[email protected]/parse_args",
"zlib": "node:zlib",
"zod": "npm:zod@3"
},
Expand Down
9 changes: 9 additions & 0 deletions deno.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

63 changes: 63 additions & 0 deletions ext/migrate.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
import { parseArgs } from "parse_args"
import { KVDEX_KEY_PREFIX } from "../src/constants.ts"

if (import.meta.main) {
const { source, target, all } = parseArgs(Deno.args, {
string: ["source", "target"],
boolean: ["all"],
})

if (!source) {
console.log(
"A source KV path to export from must be provided using the --source argument",
)
Deno.exit()
}

if (!target) {
console.log(
"A target KV path to export to must be provided using the --target argument",
)
Deno.exit()
}

using sourceKv = await Deno.openKv(source)
using targetKv = await Deno.openKv(target)

await migrate({
source: sourceKv,
target: targetKv,
all,
})
}

export type MigrateOptions = {
/** Source KV. */
source: Deno.Kv

/** Target KV. */
target: Deno.Kv

/**
* Flag indicating whether to migrate all entries or only kvdex specific entries.
*
* @default false
*/
all?: boolean
}

/**
* Migrate entries from a source KV instance to a target KV instance.
*
* @param options - Migrate options
*/
export async function migrate({
source,
target,
all,
}: MigrateOptions): Promise<void> {
const iter = source.list({ prefix: all ? [] : [KVDEX_KEY_PREFIX] })
for await (const { key, value } of iter) {
await target.set(key, value)
}
}
Loading

0 comments on commit b7f3048

Please sign in to comment.