Skip to content

Latest commit

 

History

History
168 lines (130 loc) · 4.18 KB

README.md

File metadata and controls

168 lines (130 loc) · 4.18 KB
Raw

xtrasonnet

extensible jsonnet transformations

For detailed information see the xtrasonnet docs.

xtrasonnet is an extensible, jsonnet-based, data transformation engine for Java or any JVM-based language.

xtrasonnet

xtrasonnet is an extension of databricks' sjsonnet, a Scala implementation of Google's jsonnet. xtrasonnet enables extensibility, adds support for data formats other than JSON, and adds data transformation facilities through the xtr library and some additions to the jsonnet language itself.

String output = new Transformer(xtrasonnet).transform(input);
inputxtrasonnetoutput
 
{ 
    "message": "Hello World" 
}

/** xtrasonnet
input payload application/json
output application/xml
*/
{
    root: {
        msg: payload.message,
        at: xtr.datetime.now()
    }
}

<?xml version='1.0' encoding='UTF-8'?>
<root>
	<msg>Hello World</msg>
	<at>2022-08-14T00:19:35.731362Z</at>
</root>

How extensible?

xtrasonnet has two points of extensibility:

  • Custom functions: users can write native (e.g.: Java or Scala) functions as a Library and utilize them from their transformation code.
  • Any* data format: users can write a custom DataFormatPlugin and transform from/to a given data format.

* Any format that can be expressed as jsonnet elements.

What formats are supported?

xtrasonnet includes a DataFormatPlugin for each of the following:

  • JSON (application/json)
  • XML (application/xml)
  • CSV (application/csv)
  • Java (application/x-java-object)
  • text/plain

What kind of additions to the jsonnet language?

There are two main additions motivated to facilitate data transformation applications:

Null-safe select ?.

This allows developers to select, and chain, properties arbitrarily without testing existence.

xtrasonnetOutput
 
local myObj = {
    keyA: { first: { second: 'value' } },
    keyB: { first: { } }
};

{
    a: myObj?.keyA?.first?.second,
    b: myObj?.keyB?.first?.second,
    c: myObj?.keyC?.first?.second
}
 
{
    a: 'value',
    b: null,
    c: null
}

Null coalescing operator ??

This allows developers to tersely test for null and provide a default value. For example

xtrasonnetOutput
 
local myObj = {
    keyA: { first: { second: 'value' } },
    keyB: { first: { } }
};

{
    a: myObj?.keyA?.first?.second,
    b: myObj?.keyB?.first?.second ?? 'defaultB',
    c: myObj?.keyC?.first?.second ?? 'defaultC'
}
 
{
    a: 'value',
    b: 'defaultB',
    c: 'defaultC'
}

What kind of functions are available?

For a full reference see the xtr docs.

The xtr library is written natively (vs written as jsonnet code) and provides an extensive set of functions.

Included are slight variations of the general purpose functions found in the jsonnet's std library, such as map, filter, and flatpMap plus some additional ones like groupBy. More specific functions are also included like objectFrom[Array] to facilitate composing an Object from an Array, and orderBy to sort elements.

but wait, there's more!

Developers will also find functions grouped by following set of modules, accessed in the form of xtr.[module].[function]:

  • datetime: operations on date and time values, like compare and plus(duration)
  • crypto: encrypt, decrypt, or hash data
  • arrays: extended set of array operations, like distinctBy and partition
  • objects: extended set of object operations, like leftJoin
  • strings: operations on strings, like truncate and wrapIfMissing
  • base64: encode and decode data in base64
  • url: encode and decode data for URLs

and a few more.