DataWeave Delight: Turn String Into Snake Case With underscore

In a previous post we learned we can turn a string into a string with kebab casing using dasherize from the dw::core::Strings module. If we want to turn a string into a string with camel casing we can use the underscore function. The underscore function will replace spaces, dashes and camel-casing with underscores, which makes the result snake-casing. Any uppercase characters are transformed to lowercase characters.

In the following example we use the underscore function with different input string arguments:

Soure

%dw 2.0
import underscore from dw::core::Strings
    
output application/json
---
{
    // Replaces camel casing with underscores.
    camelCase: underscore("stringInCamelCase"), // string_in_camel_case
    
    // Replaces dashes with underscores.
    kebab_case: underscore("string-with-dashes"), // string_with_dashes
    
    // Replaces spaces with underscores.
    spaces: underscore("string with spaces"), // string_with_spaces
    
    // Uppercase is transformed to lowercase.
    upper: underscore("STRING_WITH_UPPERCASE") // string_with_uppercase
}

Output

{
  "camelCase": "string_in_camel_case",
  "kebab_case": "string_with_dashes",
  "spaces": "string_with_spaces",
  "upper": "string_with_uppercase"
}

Written with DataWeave 2.4.

DataWeave Delight: Using the update Operator to change values in an object

DataWeave has some very nice features to transform data objects. One of those nice features is the update operator. With the update operator we can change values of keys in an object using a very concise syntax. We don't have to go through all keys and create a new object, but we can pinpoint the exact key and change the value. To get the correct key we use selectors. Once we have the key we can set a new value. We can define a variable to contain the current value if we want to use it to define a new value. Also is it possible to add a condition that needs to be true to change the value. Finally the update operator supports upserting a value if the key might not exist yet.

The complete syntax of the update operator is as follows:

<object> update {
    case <optional variable> at <key selector> <optional condition if (...)> -> <new value> 
}

In the following example we see several use cases of the update operator:

Source

%dw 2.0

var obj = {
    user: {
        firstName: "Hubert",
        lastName: "Klein Ikkink"
    },
    alias: "mrhaki",
    country: "NL"
}

output application/json
---
obj update {    
    // Simply use . selector to get key and set new value
    case .alias ->  "haki"
    
    // Using ! to upsert a key if doesn't exist yet.
    case .likes! -> ["Clojure", "DataWeave", "Groovy"]

    // We can use a variable for the selector value
    // and use it for a new value
    case firstName at .user.firstName -> firstName[0] ++ ".A."

    // DataWeave always provides $ as variable if we don't use at.
    // Here we can use $ to define a new value.
    case .user.lastName -> $ splitBy " " map ((item) -> item[0]) joinBy ""

    // We can add a condition for which we want to update the key
    // with a new value. In this example if country is "NL"
    // the value is "Netherlands", for other values of country
    // no transformation happens.
    case country at .country if (country == "NL") ->  "Netherlands" 
}    

Output

{
    "user": {
        "firstName": "H.A.",
        "lastName": "KI"
    },
    "alias": "haki",
    "country": "Netherlands",
    "likes": [
        "Clojure",
        "DataWeave",
        "Groovy"
    ]
    }    

Written with DataWeave 2.4.

DataWeave Delight: Unzipping Arrays

In a previous blog post we learned about the zip function. DataWeave also gives us the unzip function that will do the opposite for an array with arrays. The input argument of the unzip function is an array where the elements are also arrays. This could be created by the zip function or just defined as data structure directly. The unzip function will take from each array the same index element and return it as an array with the index elements. For example with the input array [[1, "A"], [2, "B"]] will be unzipped to [[1, 2], ["A", "B"]]. When the number of elements in the arrays that need to unzipped are not equal, the unzip function will only return the elements from the index with the most elements.

In the following example we use the unzip function with different input arrays:

Source

%dw 2.0

var fruitPrices = [["Apple", 2.30], ["Pear", 1.82], ["Banana", 2.06]]

var fruitPricesIncomplete = [["Apple", 2.30], ["Pear", 1.82], [2.06]]

output application/json
---
{
    // unzip will break up each array into separate arrays.
    unzip: unzip(fruitPrices),

    // When the arrays to break up don't have the same 
    // number of elemnts unzip can only return elements
    // from the index that has the most elements.
    unzipIncomplete: unzip(fruitPricesIncomplete),

    // When the arrays to unzip have more than 2 elements
    // the results will be the number of arrays equal to the number of elements.
    // In this case we get 3 arrays as a result as the array to
    // unzip has 3 elements.
    unzipMoreElements: unzip([[1, "a", "A"], [2, "b", "B"]])
}

Output

{
  "unzip": [
    [
      "Apple",
      "Pear",
      "Banana"
    ],
    [
      2.30,
      1.82,
      2.06
    ]
  ],
  "unzipIncomplete": [
    [
      "Apple",
      "Pear",
      2.06
    ]
  ],
  "unzipMoreElements": [
    [
      1,
      2
    ],
    [
      "a",
      "b"
    ],
    [
      "A",
      "B"
    ]
  ]
}

Written with DataWeave 2.4.

DataWeave Delight: Zipping Arrays

DataWeave has a zip function in the dw::Core module. The function will merge two arrays into a new array. Each element in the new array is also an array and will have a value from the two original arrays from the same index grouped together. So for example we have an input array ["A", "B"] and another input array [1, 2]. The result of the zip function will be [["A", 1], ["B", 2]]. The size of the resulting array is the same as the minimal size of both input arrays. Any value from an array that cannot be merged is simply ignored and left out of the resulting array.

In the following code example we use the zip function for different arrays:

Source

%dw 2.0

import take from dw::core::Arrays

var fruit = ["Apple", "Pear", "Banana"]
var prices = [2.30, 1.82, 2.06]

output application/json
---
{
    // Create new array where each element is
    // an array with first element a fruit and
    // second element a price value. 
    zip: fruit zip prices,

    // When we have an array that contains arrays
    // of 2 items (a pair) we can easily turn it into an object
    // with key/value pairs using reduce.
    zipObj: fruit zip prices 
        reduce ((item, acc = {}) -> acc ++ {(item[0]): item[1]}),

    // The resulting array will have no more items
    // then the smallest array that is used with the zip function.
    // In the following example the second array only has 2 items
    // so the resulting array also has 2 items in total. 
    // The fruit "Banana" is now ignored.
    zipMinimal: fruit zip (prices take 2)
}

Source

{
  "zip": [
    [
      "Apple",
      2.30
    ],
    [
      "Pear",
      1.82
    ],
    [
      "Banana",
      2.06
    ]
  ],
  "zipObj": {
    "Apple": 2.30,
    "Pear": 1.82,
    "Banana": 2.06
  },
  "zipMinimal": [
    [
      "Apple",
      2.30
    ],
    [
      "Pear",
      1.82
    ]
  ]
}

Written with DataWeave 2.4.

DataWeave Delight: Measure Function Duration With time And duration Functions

To measure the time it takes to execute a function in DataWeave we can use the time and duration functions from the module dw::util::Timer. Both functions take a zero argument function that needs to be executed as argument (() -> T). But the output of the functions is different. The time function returns a TimeMeasurement object that has a result key containing the result of the function we passed as argument. We also get a start key that has the date and time value when the function gets executed. And finally we have the end key that stores the date and time value when the function is finished. To calculate the total duration time of the function we could use the start and end keys, but when we want the duration time we can better use the duration function. The duration function returns a DurationMeasurement object with also a key result that has the output of the function that is executed. The other key is time and contains the time it took for the function to be executed in milliseconds.

In the following example we use both timer functions together with the wait function. The wait function will wait for the given number of milliseconds before returning a value.

Source

%dw 2.0

import wait from dw::Runtime
import duration, time from dw::util::Timer

output application/json
---
{ 
    // time function returns a new object with
    // keys start, end and result.
    // Keys start and end contain the start and end datetime 
    // before and after the function is executed.
    // The result key has the value of the function.
    time: time(() -> 42 wait 1000),

    // duration function returns a new object with
    // keys time and result.
    // Key duration has the total duration for the
    // function execution in milliseconds.
    // The result key has the value of the function.
    duration: duration(() -> 42 wait 1000)
}

Output

{
  "time": {
    "start": "2022-06-14T04:39:21.582958Z",
    "result": 42,
    "end": "2022-06-14T04:39:22.583079Z"
  },
  "duration": {
    "time": 1000,
    "result": 42
  }
}

Applying the time or duration function in our code is intrusive as the result object is different from the result of our function we want to measure. We can write a helper function that uses the time or duration function, logs the output of the functions using log function and finally still returns the value from our input function by selecting the result key.

In the next example we create the wrapTime and wrapDuration functions that can be used to log the output of the time and duration functions and still return the result of the input function. This way we can introduce timing of function duration in a less intrusive way.

Source

%dw 2.0

import wait from dw::Runtime
import duration, time from dw::util::Timer

fun answer() = 42 wait 400

// We still want the original result from the answer function,
// but also log the output of the time function.
// We pass the output of the time function to the log function that
// will log the output and then return the output value. 
// And finally use the result key from the time function
// to get output from the answer function.
fun wrapTime(f) = log("Time", time(f)).result

// We still want the original result from the answer function,
// but also log the output of the duration function.
// We pass the output of the time function to the log function that
// will log the output and then return the output value. 
// And finally use the result key from the duration function output
// to get output from the answer function.    
fun wrapDuration(f) = log("Duration", duration(f)).result

output application/json
---
{ 
    // Simple invocation of the function answer to get the result.
    result: answer(),

    // Use wrapTime function to still get result, but also log time output.
    resultTimer: wrapTime(() -> answer()),

    // Use wrapDuration function to still get result, but also log duration output.
    resultDuration: wrapDuration(() -> answer())
} 

Output

{
  "result": 42,
  "resultTimer": 42,
  "resultDuration": 42
}

Log output

Time - { start: |2022-06-14T04:52:29.287724Z|, result: 42, end: |2022-06-14T04:52:29.687875Z| }
Duration - { time: 400, result: 42 }

Written with DataWeave 2.4.

DataWeave Delight: Partition An Array

In DataWeave we can partition the items in an array using a predicate function by using the partition function from the dw::core::Arrays module. The function takes an array as first argument and a predicate function as second argument. The predicate function should return true or false for each item of the array. The result is an object with the key success containing all items from the array that returned true for the predicate function and a key failure for the items that returned false.

In the following example code we use the partition function on an array:

Source

%dw 2.0

import partition from dw::core::Arrays

var items = ["language", "DataWeave", "username", "mrhaki", "age", 48]

output application/json 
---
{ 
    // Partition by item is of type String or not.
    example1: items partition ((item) -> typeOf(item) == String),

    // Partition by checking if item have value 1 or 4 or not
    // using shorthand notation.
    example2: (0 to 5) partition ([1, 4] contains $)
}

Output

{
  "example1": {
    "success": [
      "language",
      "DataWeave",
      "username",
      "mrhaki",
      "age"
    ],
    "failure": [
      48
    ]
  },
  "example2": {
    "success": [
      1,
      4
    ],
    "failure": [
      0,
      2,
      3,
      5
    ]
  }
}

Written with DataWeave 2.4.

DataWeave Delight: Splitting An Array Or Object

The module dw::core::Arrays has extra functions that are useful when working with arrays in DataWeave. In this post we will look at the functions splitAt and splitWhere. We can use these functions to split an array into two arrays. The result of both functions is actually a Pair type, which is defined as an object with the keys l and r. An example is { "l": 1, "r": 2 } which we can read as the left side of the pair has value 1 and the right side of the pair has value 2. The result of the splitAt and splitWhere function will have one part of the split array in the left side of the pair and the rest of the array in the right side.

The splitAt function takes the array as first argument and an index value as second argument. The index value should indicate at which position the array should be split. The function can be used with infix notation as well. The splitWhere function takes as first argument also an array, but the second argument is a predicate function. All items starting from the first item for which the predicate function returns true will be assigned to the r key of the Pair result, and all preceding items to the l key. We can use the infix notation here as well.

In the following example code we use the splitAt and splitWhere function on an array:

Source

%dw 2.0

import splitWhere, splitAt from dw::core::Arrays

output application/json
---
{
    // Split the range at index 4. 
    splitAt: (0 to 8) splitAt 4,

    // Split at the position where the predicate returns true for the first time.
    splitWhere: (0 to 8) splitWhere ((item) -> item > 5)
}

Output

{
  "splitAt": {
    "l": [
      0,
      1,
      2,
      3
    ],
    "r": [
      4,
      5,
      6,
      7,
      8
    ]
  },
  "splitWhere": {
    "l": [
      0,
      1,
      2,
      3,
      4,
      5
    ],
    "r": [
      6,
      7,
      8
    ]
  }
}

Although the function work on arrays we can use them on a object as well. We first turn the object into an array of objects with a single key and value using the pluck function. We can use this array with the splitAt and splitWhere functions. Then we can use the reduce function to transform the values in the Pair to an object with multiple keys and values again.

In the next example we use this mechanism on an object:

Source

%dw 2.0

import splitWhere, splitAt from dw::core::Arrays

// Helper functions
// ----------------
// Transform object to array with key/value pairs
fun objectToArray(obj: Object): Array<Object> = obj pluck ((value, key) -> (key): value)

// Transform array with key/value pairs to object
fun arrayToObject(items: Array<Object>): Object = items reduce ((item, acc = {}) -> acc ++ item)

var obj = {
    language: "DataWeave",
    alias: "mrhaki",
    age: 48, 
    country: "NL"
}

output application/json 
---
{
    // We can use splitAt on object if we first transform an object
    // to an array of key/value pairs, apply the splitAt function and 
    // transform the resulting Pair into an object again.
    splitAtObject: objectToArray(obj) 
        splitAt 3 
        mapObject ((value, key) -> (key): arrayToObject(value)),

    // We can use splitWhere in the same manner for objects.
    splitWhereObject: objectToArray(obj) 
        splitWhere ((item) -> typeOf(item[0]) == Number)
        mapObject ((value, key) -> (key): arrayToObject(value))
}

Output

{
  "splitAtObject": {
    "l": {
      "language": "DataWeave",
      "alias": "mrhaki",
      "age": 48
    },
    "r": {
      "country": "NL"
    }
  },
  "splitWhereObject": {
    "l": {
      "language": "DataWeave",
      "alias": "mrhaki"
    },
    "r": {
      "age": 48,
      "country": "NL"
    }
  }
}

Written with DataWeave 2.4.