Skip to content

Commit

Permalink
Merge pull request #21 from sebastianwessel/20-feat-typescript-suppor…
Browse files Browse the repository at this point in the history
…t-run-typescript-in-the-quickjs-sandbox

feat: Typescript support - Run Typescript in the QuickJS sandbox #20
  • Loading branch information
sebastianwessel authored Jul 11, 2024
2 parents eb2e632 + 4adc3ae commit dadbd83
Show file tree
Hide file tree
Showing 12 changed files with 289 additions and 15 deletions.
8 changes: 4 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,18 +1,18 @@
# QuickJS - Execute JavaScript in a WebAssembly QuickJS Sandbox
# QuickJS - Execute JavaScript and TypeScript in a WebAssembly QuickJS Sandbox

This TypeScript package allows you to safely execute JavaScript code within a WebAssembly sandbox using the QuickJS engine. Perfect for isolating and running untrusted code securely, it leverages the lightweight and fast QuickJS engine compiled to WebAssembly, providing a robust environment for code execution.
This TypeScript package allows you to safely execute **JavaScript AND TypeScript code** within a WebAssembly sandbox using the QuickJS engine. Perfect for isolating and running untrusted code securely, it leverages the lightweight and fast QuickJS engine compiled to WebAssembly, providing a robust environment for code execution.

## Features

- **Security**: Run untrusted JavaScript code in a safe, isolated environment.
- **Security**: Run untrusted JavaScript and TypeScript code in a safe, isolated environment.
- **Basic Node.js modules**: Provides basic standard Node.js module support for common use cases.
- **File System**: Can mount a virtual file system.
- **Custom Node Modules**: Custom node modules are mountable.
- **Fetch Client**: Can provide a fetch client to make http(s) calls.
- **Test-Runner**: Includes a test runner and chai based `expect`.
- **Performance**: Benefit from the lightweight and efficient QuickJS engine.
- **Versatility**: Easily integrate with existing TypeScript projects.
- **Simplicity**: User-friendly API for executing and managing JavaScript code in the sandbox.
- **Simplicity**: User-friendly API for executing and managing JavaScript and TypeScript code in the sandbox.

**[View the full documentation](https://sebastianwessel.github.io/quickjs/)**

Expand Down
6 changes: 3 additions & 3 deletions docs/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Package @sebastianwessel/quickjs

This TypeScript package allows you to safely execute JavaScript code within a WebAssembly sandbox using the QuickJS engine. Perfect for isolating and running untrusted code securely, it leverages the lightweight and fast QuickJS engine compiled to WebAssembly, providing a robust environment for code execution.
This TypeScript package allows you to safely execute **JavaScript and TypeScript code** within a WebAssembly sandbox using the QuickJS engine. Perfect for isolating and running untrusted code securely, it leverages the lightweight and fast QuickJS engine compiled to WebAssembly, providing a robust environment for code execution.

## Documentation

Expand All @@ -21,15 +21,15 @@ This TypeScript package allows you to safely execute JavaScript code within a We

## Features

- **Security**: Run untrusted JavaScript code in a safe, isolated environment.
- **Security**: Run untrusted JavaScript and TypeScript code in a safe, isolated environment.
- **Basic Node.js modules**: Provides basic standard Node.js module support for common use cases.
- **File System**: Can mount a virtual file system.
- **Custom Node Modules**: Custom node modules are mountable.
- **Fetch Client**: Can provide a fetch client to make http(s) calls.
- **Test-Runner**: Includes a test runner and chai based `expect`.
- **Performance**: Benefit from the lightweight and efficient QuickJS engine..
- **Versatility**: Easily integrate with existing TypeScript projects.
- **Simplicity**: User-friendly API for executing and managing JavaScript code in the sandbox.
- **Simplicity**: User-friendly API for executing and managing JavaScript and TypeScript code in the sandbox.

## Basic Usage

Expand Down
51 changes: 51 additions & 0 deletions docs/sandboxed-code.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,8 @@ The most common use case is, to execute a give JavaScript code in the QuickJS we
The `evalCode` function described below, is intend to be used, when a given JavaScript code should be executed and optional a result value is returned.
It is recommended, to always return a value, for better validation on the host side, that the code was executed as expected.

In the sandbox, the executed code "lives" in `/src/index.js`. If custom files are added via configuration, source files should be placed below `/src`. Nested directories are supported.

```typescript
import { quickJS } from '@sebastianwessel/quickjs'

Expand Down Expand Up @@ -84,3 +86,52 @@ console.log(result)
The `validateCode` functions returns a unified result object, and does not throw.

For further information, it is hight recommended to [read "Basic Understanding"](./basic.md).

## Execute TypeScript in the Sandbox

Executing TypeScript in the sandboxed runtime, is similar to executing JavaScript. An additional transpile step will be applied to the given code. Additionally each file below `/src` with file extension`.ts` in the custom file system will be transpiled.

The TypeScript code is only transpiled, but not type-checked!
If checking types is required, it should be done and handled, before using this library.

**Requirements:**

- optional dependency package `typescript` must be installed on the host system
- `createRuntime` option `transformTypescript` must be set to `true`

Example:

```typescript
import { quickJS } from '@sebastianwessel/quickjs'

const { createRuntime } = await quickJS()

// Create a runtime instance (sandbox)
const { evalCode } = await createRuntime({
transformTypescript: true,
mountFs: {
src: {
'test.ts': `export const testFn = (value: string): string => {
console.log(value)
return value
}`,
},
},
})


const result = await evalCode(`
import { testFn } from './test.js'
const t = (value:string):number=>value.length
console.log(t('abc'))
export default testFn('hello')
`)

console.log(result) // { ok: true, data: 'hello' }
// console log on host:
// 3
// hello
```
9 changes: 9 additions & 0 deletions example/server/worker.ts
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,15 @@ class MyThreadWorker extends ThreadWorker<InputData, ResponseData> {
allowFetch: true,
enableTestUtils: true,
env: {},
transformTypescript: true,
mountFs: {
src: {
'test.ts': `export const testFn = (value: string): string => {
console.log(value)
return value
}`,
},
},
})

const result = await evalCode(data.content)
Expand Down
2 changes: 1 addition & 1 deletion jsr.json
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
"$schema": "https://jsr.io/schema/config-file.v1.json",
"name": "@sebastianwessel/quickjs",
"version": "1.2.0",
"description": "A typescript package to execute javascript code in a webassembly quickjs sandbox",
"description": "A typescript package to execute JavaScript and TypeScript code in a webassembly quickjs sandbox",
"exports": "./dist/esm/index.js",
"publish": {
"include": ["dist/**/*.js", "dist/**/*.d.ts", "README.md", "package.json"],
Expand Down
10 changes: 9 additions & 1 deletion package.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "@sebastianwessel/quickjs",
"version": "1.2.0",
"description": "A typescript package to execute javascript code in a webassembly quickjs sandbox",
"description": "A typescript package to execute JavaScript and TypeScript code in a webassembly quickjs sandbox",
"type": "module",
"engines": {
"node": ">=18.0.0"
Expand Down Expand Up @@ -78,6 +78,14 @@
"quickjs-emscripten-core": "^0.29.2",
"rate-limiter-flexible": "^5.0.3"
},
"peerDependencies": {
"typescript": ">= 5.5.3"
},
"peerDependenciesMeta": {
"typescript": {
"optional": true
}
},
"exports": {
"./package.json": "./package.json",
".": {
Expand Down
6 changes: 4 additions & 2 deletions src/createVirtualFileSystem.ts
Original file line number Diff line number Diff line change
Expand Up @@ -18,13 +18,15 @@ import type { RuntimeOptions } from './types/RuntimeOptions.js'
* @returns filesystem fs and volume vol
*/
export const createVirtualFileSystem = (runtimeOptions: RuntimeOptions = {}) => {
const customFileSystem = runtimeOptions.mountFs

// setup node_modules folder
const virtualFS: Record<string, any> = {
'/': {
...runtimeOptions.mountFs,
...customFileSystem,
src: {
// biome-ignore lint/complexity/useLiteralKeys: <explanation>
...(runtimeOptions.mountFs?.['src'] ? (runtimeOptions.mountFs?.['src'] as NestedDirectoryJSON) : {}),
...(customFileSystem?.['src'] ? (customFileSystem?.['src'] as NestedDirectoryJSON) : {}),
},
node_modules: {
...runtimeOptions?.nodeModules,
Expand Down
157 changes: 157 additions & 0 deletions src/getTypescriptSupport.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,157 @@
import { join } from 'node:path'
import type { IFs, NestedDirectoryJSON } from 'memfs'
import type { default as TS } from 'typescript'

export type TranspileNestedJsonOptions = {
fileExtensions?: string[]
}

export type TranspileVirtualFsOptions = {
fileExtensions?: string[]
startPath?: string
}

/**
* Add support for handling typescript files and code.
* Requires the optional dependency 'typescript'.
*/
export const getTypescriptSupport = async (enabled = false, options?: TS.CompilerOptions) => {
if (!enabled) {
return {
transpileFile: (
value: string,
_compilerOptions?: TS.CompilerOptions,
_fileName?: string,
_diagnostics?: TS.Diagnostic[],
_moduleName?: string,
) => value,
transpileNestedDirectoryJSON: (
mountFsJson: NestedDirectoryJSON,
_option?: TranspileNestedJsonOptions,
): NestedDirectoryJSON => mountFsJson,
transpileVirtualFs: (fs: IFs, _options?: TranspileVirtualFsOptions): IFs => {
return fs
},
}
}

let ts: typeof TS
try {
ts = await import('typescript')
} catch (error) {
throw new Error('Package "typescript" is missing')
}

const compilerOptions: TS.CompilerOptions = {
module: 99, // ESNext
target: 99, // ES2023
//lib: ['ESNext'],
allowJs: true,
// moduleResolution: 100,
skipLibCheck: true,
esModuleInterop: true,
strict: false,
allowSyntheticDefaultImports: true,
...options,
}

/**
* Transpile a single File
*
* @param params source typescript code
* @returns javascript code
*/
const transpileFile: typeof ts.transpile = (
input: string,
cpOptions = compilerOptions,
fileName?: string,
diagnostics?: TS.Diagnostic[],
moduleName?: string,
) => ts.transpile(input, cpOptions, fileName, diagnostics, moduleName)

/**
* Iterates through the given JSON - NestedDirectoryJSON for defining the virtual file system.
* Replace every typescript file with the transpiled javascript version and renames the file to *.js
*
* @param mountFsJson
* @param fileExtensions
* @returns
*/
const transpileNestedDirectoryJSON = (
mountFsJson: NestedDirectoryJSON,
option?: TranspileNestedJsonOptions,
): NestedDirectoryJSON => {
const opt = {
fileExtensions: ['ts'],
...option,
}

const transformJson = (
obj: NestedDirectoryJSON,
transformer: (key: string, value: any) => [string, any],
): NestedDirectoryJSON => {
if (typeof obj === 'object' && obj !== null) {
const newObj: any = Array.isArray(obj) ? [] : {}
for (const key in obj) {
if (obj[key]) {
const [newKey, newValue] = transformer(key, obj[key])
newObj[newKey] = transformJson(newValue, transformer)
}
}
return newObj
}
return obj
}

const transpileTypescript = (key: string, value: string | Buffer): [string, any] => {
if (!opt.fileExtensions.includes(key)) {
return [key, value]
}
const newFileName = key.replace('.ts', '.js')
const tsSource = typeof value === 'string' ? value : value.toString()
const jsSource = transpileFile(tsSource, compilerOptions)
return [newFileName, jsSource]
}

return transformJson(mountFsJson, transpileTypescript)
}

const transpileVirtualFs = (fs: IFs, options?: TranspileVirtualFsOptions) => {
const opt = {
startPath: '/src',
...options,
}

const transformFileSystem = (startPath: string): void => {
if (!fs.existsSync(startPath)) {
throw new Error(`Directory not found: ${startPath}`)
}

const files = fs.readdirSync(startPath)
for (const file of files) {
const filePath = join(startPath, file as string)
const stat = fs.lstatSync(filePath)

if (stat.isDirectory()) {
// ignore node_modules
if ((file as string) !== 'node_modules') {
transformFileSystem(filePath)
}
} else if (stat.isFile()) {
const newFilePath = join(startPath, (file as string).replace('.ts', '.js'))
const content = fs.readFileSync(filePath, 'utf8')
const tsSource = typeof content === 'string' ? content : content.toString()
const jsSource = transpileFile(tsSource, compilerOptions, newFilePath)
fs.renameSync(filePath, newFilePath)
fs.writeFileSync(newFilePath, jsSource)
}
}
}

transformFileSystem(opt.startPath)

return fs
}

return { transpileFile, transpileNestedDirectoryJSON, transpileVirtualFs }
}
2 changes: 2 additions & 0 deletions src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -63,3 +63,5 @@ export * from './types/QuickJSEvalReturn.js'
export * from './adapter/fetch.js'
export * from './createVirtualFileSystem.js'
export * from './types/OkResponseCheck.js'
export * from './getTypescriptSupport.js'
export * from './modulePathNormalizer.js'
11 changes: 8 additions & 3 deletions src/quickJS.ts
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ import type { IFs } from 'memfs'
import { createTimeInterval } from './createTimeInterval.js'
import { createVirtualFileSystem } from './createVirtualFileSystem.js'
import { getModuleLoader } from './getModuleLoader.js'
import { getTypescriptSupport } from './getTypescriptSupport.js'
import { modulePathNormalizer } from './modulePathNormalizer.js'
import type { OkResponseCheck } from './types/OkResponseCheck.js'

Expand All @@ -31,6 +32,9 @@ export const quickJS = async (wasmVariantName = '@jitl/quickjs-ng-wasmfile-relea

const fs = existingFs ?? createVirtualFileSystem(runtimeOptions).fs

const { transpileVirtualFs, transpileFile } = await getTypescriptSupport(runtimeOptions.transformTypescript)
transpileVirtualFs(fs)

vm.runtime.setModuleLoader(getModuleLoader(fs, runtimeOptions), modulePathNormalizer)

const arena = new Arena(vm, { isMarshalable: true })
Expand All @@ -47,7 +51,7 @@ export const quickJS = async (wasmVariantName = '@jitl/quickjs-ng-wasmfile-relea
`)

const dispose = () => {
let err: unknown | undefined
let err: unknown
try {
arena.dispose()
} catch (error) {
Expand Down Expand Up @@ -105,7 +109,8 @@ export const quickJS = async (wasmVariantName = '@jitl/quickjs-ng-wasmfile-relea
}, 0)

try {
const evalResult = arena.evalCode(code, filename, {
const jsCode = transpileFile(code)
const evalResult = arena.evalCode(jsCode, filename, {
strict: true,
strip: true,
backtraceBarrier: true,
Expand Down Expand Up @@ -199,7 +204,7 @@ export const quickJS = async (wasmVariantName = '@jitl/quickjs-ng-wasmfile-relea
}
}

return { vm: arena, dispose, evalCode, validateCode }
return { vm: arena, dispose, evalCode, validateCode, mountedFs: fs }
}

return { createRuntime }
Expand Down
Loading

0 comments on commit dadbd83

Please sign in to comment.