Search

Dark theme | Light theme

June 27, 2022

DataWeave Delight: Getting The Ordinal Value For A Number

The dw::core::Strings has a lot of functions to deal with strings. One of the functions is ordinalize that takes a number as argument and returns the ordinal value as string.

The following exmaple shows several numbers transformed to their ordinal values:

Source

%dw 2.0

import ordinalize from dw::core::Strings

output application/json
---
{
    "1": ordinalize(1),
    "2": ordinalize(2),
    "3": ordinalize(3),
    "10": ordinalize(10),
    "location": "The restaurant is at the " ++ ordinalize(4) ++ " floor."
}

Output

{
  "1": "1st",
  "2": "2nd",
  "3": "3rd",
  "10": "10th",
  "location": "The restaurant is at the 4th floor."
}

Written with DataWeave 2.4.

DataWeave Delight: Check Type Of Value

To check if a value is of a certain type in DataWeave we must use the is operator. We must specify the type after the is operator and the value before the is operator. For example to check if the value 42 is a Number we write 42 is Number. The result is a Boolean that is either true or false. In the previous example the result is true.

In the following examples we use is on several types:

Source

%dw 2.0

import every from dw::core::Arrays

// Closed object type with two properties alias and name
// that must have the String type.
type User = {|alias: String,name: String|}

// Helper function to check if argument value
// is an Array type and
// only contains elements of type String.
fun checkArrayString(value: Any): Boolean = 
    (value is Array) and
    ((value as Array) every ((element) -> element is String))

// Literal type.
type Language = "DataWeave" | "Java" | "Groovy" | "Clojure"

output application/json
---
{
    string: "DataWeave" is String,
    not_string: 42 is String,
    
    number: 42 is Number,

    // We can also use the Null type to check.
    null_value: null is Null,

    object: {alias: "mrhaki", name: "Hubert A. Klein Ikkink"} is Object,

    // We can check on our own object type.
    user: {alias: "mrhaki", name: "Hubert"} is User,

    // Only the alias property is valid.
    not_user1: {alias: "mrhaki", firstName: "Hubert", lastName: "Klein Ikkink"} is User,

    // User is a closed object so extra properties 
    // don't make it of type User anymore.
    not_user2: {alias: "mrhaki", name: "Hubert", location: "Tilburg"} is User,

    // Type of alias property is not correct.
    not_user3: {alias: 42, name: "Hubert"} is User,
    
    array: ["one", "two"] is Array,

    // To also check the types of the elements
    // we must use an extra check as defined
    // in the checkArrayString helper function.
    array_string: checkArrayString(["one", "two"]),
    not_array_string: checkArrayString([1, 2]),
    not_array: checkArrayString(42),

    // Literal types are supported.
    language: "DataWeave" is Language,
}

Output

{
  "string": true,
  "not_string": false,
  "number": true,
  "null_value": true,
  "object": true,
  "user": true,
  "not_user1": false,
  "not_user2": false,
  "not_user3": false,
  "array": true,
  "array_string": true,
  "not_array_string": false,
  "not_array": false,
  "language": true
}

Written with DataWeave 2.4.

June 26, 2022

DataWeave Delight: Calculating Remainder After Division

To calculate the modulo of two numbers in DataWeave we can use the mod function from the dw::Core module. We provide two arguments where the first argument is the number that needs to be divided by the number provided as second argument. The number that remains after a division of the input arguments is returned as a result.

In the following example we use different arguments for the mod function:

Source

%dw 2.0
output application/json
---
{
    result1: 6 mod 3,
    result2: 5 mod 3,
    result3: 5 mod -3,
    result4: -5 mod 3,
    result5: -5 mod -3,
    result6: 1.5 mod 1,
    result7: 5.095 mod 3
}

Output

{
  "result1": 0,
  "result2": 2,
  "result3": 2,
  "result4": -2,
  "result5": -2,
  "result6": 0.5,
  "result7": 2.095
}

Written with DataWave 2.4.

June 24, 2022

DataWeave Delight: Unzipping Arrays

In a previous blog post we learned about the zip function. DataWeave also gives us the unzip function that will do the opposite for an array with arrays. The input argument of the unzip function is an array where the elements are also arrays. This could be created by the zip function or just defined as data structure directly. The unzip function will take from each array the same index element and return it as an array with the index elements. For example with the input array [[1, "A"], [2, "B"]] will be unzipped to [[1, 2], ["A", "B"]]. When the number of elements in the arrays that need to unzipped are not equal, the unzip function will only return the elements from the index with the most elements.

In the following example we use the unzip function with different input arrays:

Source

%dw 2.0

var fruitPrices = [["Apple", 2.30], ["Pear", 1.82], ["Banana", 2.06]]

var fruitPricesIncomplete = [["Apple", 2.30], ["Pear", 1.82], [2.06]]

output application/json
---
{
    // unzip will break up each array into separate arrays.
    unzip: unzip(fruitPrices),

    // When the arrays to break up don't have the same 
    // number of elemnts unzip can only return elements
    // from the index that has the most elements.
    unzipIncomplete: unzip(fruitPricesIncomplete),

    // When the arrays to unzip have more than 2 elements
    // the results will be the number of arrays equal to the number of elements.
    // In this case we get 3 arrays as a result as the array to
    // unzip has 3 elements.
    unzipMoreElements: unzip([[1, "a", "A"], [2, "b", "B"]])
}

Output

{
  "unzip": [
    [
      "Apple",
      "Pear",
      "Banana"
    ],
    [
      2.30,
      1.82,
      2.06
    ]
  ],
  "unzipIncomplete": [
    [
      "Apple",
      "Pear",
      2.06
    ]
  ],
  "unzipMoreElements": [
    [
      1,
      2
    ],
    [
      "a",
      "b"
    ],
    [
      "A",
      "B"
    ]
  ]
}

Written with DataWeave 2.4.

June 21, 2022

DataWeave Delight: Zipping Arrays

DataWeave has a zip function in the dw::Core module. The function will merge two arrays into a new array. Each element in the new array is also an array and will have a value from the two original arrays from the same index grouped together. So for example we have an input array ["A", "B"] and another input array [1, 2]. The result of the zip function will be [["A", 1], ["B", 2]]. The size of the resulting array is the same as the minimal size of both input arrays. Any value from an array that cannot be merged is simply ignored and left out of the resulting array.

In the following code example we use the zip function for different arrays:

Source

%dw 2.0

import take from dw::core::Arrays

var fruit = ["Apple", "Pear", "Banana"]
var prices = [2.30, 1.82, 2.06]

output application/json
---
{
    // Create new array where each element is
    // an array with first element a fruit and
    // second element a price value. 
    zip: fruit zip prices,

    // When we have an array that contains arrays
    // of 2 items (a pair) we can easily turn it into an object
    // with key/value pairs using reduce.
    zipObj: fruit zip prices 
        reduce ((item, acc = {}) -> acc ++ {(item[0]): item[1]}),

    // The resulting array will have no more items
    // then the smallest array that is used with the zip function.
    // In the following example the second array only has 2 items
    // so the resulting array also has 2 items in total. 
    // The fruit "Banana" is now ignored.
    zipMinimal: fruit zip (prices take 2)
}

Source

{
  "zip": [
    [
      "Apple",
      2.30
    ],
    [
      "Pear",
      1.82
    ],
    [
      "Banana",
      2.06
    ]
  ],
  "zipObj": {
    "Apple": 2.30,
    "Pear": 1.82,
    "Banana": 2.06
  },
  "zipMinimal": [
    [
      "Apple",
      2.30
    ],
    [
      "Pear",
      1.82
    ]
  ]
}

Written with DataWeave 2.4.

June 14, 2022

DataWeave Delight: Measure Function Duration With time And duration Functions

To measure the time it takes to execute a function in DataWeave we can use the time and duration functions from the module dw::util::Timer. Both functions take a zero argument function that needs to be executed as argument (() -> T). But the output of the functions is different. The time function returns a TimeMeasurement object that has a result key containing the result of the function we passed as argument. We also get a start key that has the date and time value when the function gets executed. And finally we have the end key that stores the date and time value when the function is finished. To calculate the total duration time of the function we could use the start and end keys, but when we want the duration time we can better use the duration function. The duration function returns a DurationMeasurement object with also a key result that has the output of the function that is executed. The other key is time and contains the time it took for the function to be executed in milliseconds.

In the following example we use both timer functions together with the wait function. The wait function will wait for the given number of milliseconds before returning a value.

Source

%dw 2.0

import wait from dw::Runtime
import duration, time from dw::util::Timer

output application/json
---
{ 
    // time function returns a new object with
    // keys start, end and result.
    // Keys start and end contain the start and end datetime 
    // before and after the function is executed.
    // The result key has the value of the function.
    time: time(() -> 42 wait 1000),

    // duration function returns a new object with
    // keys time and result.
    // Key duration has the total duration for the
    // function execution in milliseconds.
    // The result key has the value of the function.
    duration: duration(() -> 42 wait 1000)
}

Output

{
  "time": {
    "start": "2022-06-14T04:39:21.582958Z",
    "result": 42,
    "end": "2022-06-14T04:39:22.583079Z"
  },
  "duration": {
    "time": 1000,
    "result": 42
  }
}

Applying the time or duration function in our code is intrusive as the result object is different from the result of our function we want to measure. We can write a helper function that uses the time or duration function, logs the output of the functions using log function and finally still returns the value from our input function by selecting the result key.

In the next example we create the wrapTime and wrapDuration functions that can be used to log the output of the time and duration functions and still return the result of the input function. This way we can introduce timing of function duration in a less intrusive way.

Source

%dw 2.0

import wait from dw::Runtime
import duration, time from dw::util::Timer

fun answer() = 42 wait 400

// We still want the original result from the answer function,
// but also log the output of the time function.
// We pass the output of the time function to the log function that
// will log the output and then return the output value. 
// And finally use the result key from the time function
// to get output from the answer function.
fun wrapTime(f) = log("Time", time(f)).result

// We still want the original result from the answer function,
// but also log the output of the duration function.
// We pass the output of the time function to the log function that
// will log the output and then return the output value. 
// And finally use the result key from the duration function output
// to get output from the answer function.    
fun wrapDuration(f) = log("Duration", duration(f)).result

output application/json
---
{ 
    // Simple invocation of the function answer to get the result.
    result: answer(),

    // Use wrapTime function to still get result, but also log time output.
    resultTimer: wrapTime(() -> answer()),

    // Use wrapDuration function to still get result, but also log duration output.
    resultDuration: wrapDuration(() -> answer())
} 

Output

{
  "result": 42,
  "resultTimer": 42,
  "resultDuration": 42
}

Log output

Time - { start: |2022-06-14T04:52:29.287724Z|, result: 42, end: |2022-06-14T04:52:29.687875Z| }
Duration - { time: 400, result: 42 }

Written with DataWeave 2.4.

March 16, 2022

DataWeave Delight: Partition An Array

In DataWeave we can partition the items in an array using a predicate function by using the partition function from the dw::core::Arrays module. The function takes an array as first argument and a predicate function as second argument. The predicate function should return true or false for each item of the array. The result is an object with the key success containing all items from the array that returned true for the predicate function and a key failure for the items that returned false.

In the following example code we use the partition function on an array:

Source

%dw 2.0

import partition from dw::core::Arrays

var items = ["language", "DataWeave", "username", "mrhaki", "age", 48]

output application/json 
---
{ 
    // Partition by item is of type String or not.
    example1: items partition ((item) -> typeOf(item) == String),

    // Partition by checking if item have value 1 or 4 or not
    // using shorthand notation.
    example2: (0 to 5) partition ([1, 4] contains $)
}

Output

{
  "example1": {
    "success": [
      "language",
      "DataWeave",
      "username",
      "mrhaki",
      "age"
    ],
    "failure": [
      48
    ]
  },
  "example2": {
    "success": [
      1,
      4
    ],
    "failure": [
      0,
      2,
      3,
      5
    ]
  }
}

Written with DataWeave 2.4.

March 15, 2022

DataWeave Delight: Splitting An Array Or Object

The module dw::core::Arrays has extra functions that are useful when working with arrays in DataWeave. In this post we will look at the functions splitAt and splitWhere. We can use these functions to split an array into two arrays. The result of both functions is actually a Pair type, which is defined as an object with the keys l and r. An example is { "l": 1, "r": 2 } which we can read as the left side of the pair has value 1 and the right side of the pair has value 2. The result of the splitAt and splitWhere function will have one part of the split array in the left side of the pair and the rest of the array in the right side.

The splitAt function takes the array as first argument and an index value as second argument. The index value should indicate at which position the array should be split. The function can be used with infix notation as well. The splitWhere function takes as first argument also an array, but the second argument is a predicate function. All items starting from the first item for which the predicate function returns true will be assigned to the r key of the Pair result, and all preceding items to the l key. We can use the infix notation here as well.

In the following example code we use the splitAt and splitWhere function on an array:

Source

%dw 2.0

import splitWhere, splitAt from dw::core::Arrays

output application/json
---
{
    // Split the range at index 4. 
    splitAt: (0 to 8) splitAt 4,

    // Split at the position where the predicate returns true for the first time.
    splitWhere: (0 to 8) splitWhere ((item) -> item > 5)
}

Output

{
  "splitAt": {
    "l": [
      0,
      1,
      2,
      3
    ],
    "r": [
      4,
      5,
      6,
      7,
      8
    ]
  },
  "splitWhere": {
    "l": [
      0,
      1,
      2,
      3,
      4,
      5
    ],
    "r": [
      6,
      7,
      8
    ]
  }
}

Although the function work on arrays we can use them on a object as well. We first turn the object into an array of objects with a single key and value using the pluck function. We can use this array with the splitAt and splitWhere functions. Then we can use the reduce function to transform the values in the Pair to an object with multiple keys and values again.

In the next example we use this mechanism on an object:

Source

%dw 2.0

import splitWhere, splitAt from dw::core::Arrays

// Helper functions
// ----------------
// Transform object to array with key/value pairs
fun objectToArray(obj: Object): Array<Object> = obj pluck ((value, key) -> (key): value)

// Transform array with key/value pairs to object
fun arrayToObject(items: Array<Object>): Object = items reduce ((item, acc = {}) -> acc ++ item)

var obj = {
    language: "DataWeave",
    alias: "mrhaki",
    age: 48, 
    country: "NL"
}

output application/json 
---
{
    // We can use splitAt on object if we first transform an object
    // to an array of key/value pairs, apply the splitAt function and 
    // transform the resulting Pair into an object again.
    splitAtObject: objectToArray(obj) 
        splitAt 3 
        mapObject ((value, key) -> (key): arrayToObject(value)),

    // We can use splitWhere in the same manner for objects.
    splitWhereObject: objectToArray(obj) 
        splitWhere ((item) -> typeOf(item[0]) == Number)
        mapObject ((value, key) -> (key): arrayToObject(value))
}

Output

{
  "splitAtObject": {
    "l": {
      "language": "DataWeave",
      "alias": "mrhaki",
      "age": 48
    },
    "r": {
      "country": "NL"
    }
  },
  "splitWhereObject": {
    "l": {
      "language": "DataWeave",
      "alias": "mrhaki"
    },
    "r": {
      "age": 48,
      "country": "NL"
    }
  }
}

Written with DataWeave 2.4.

March 14, 2022

DataWeave Delight: Importing Functions With An Alias

When we want to reference functions from other modules than the dw::Core module we must us the import statement. We can use the as keyword to map a function name that we want to import to another name that we want to use in our DataWeave code. We provide an alias for the function that we can use in our code. This could be useful if we think our code gets more readable in a certain context by using different function names, but still want to use the original function implementation. Next to providing an alias for function names, we can also provide an alias for a module name, when we use the import <ModuleName> as <NewModuleName> statement. In our DataWeave code we can then refer to the new alias to call functions from the module.

In the following example code we use import using as to define aliases for functions and a module:

Source

%dw 2.0

// Use an alias for functions.
import upper as upperCase from dw::Core
import firstWith as findFirst from dw::core::Arrays

// Assign an alias to a module, so we can reference
// functions from the module as Str::<function>
import dw::core::Strings as Str

output application/json
---
{
    // Use alias upperCase for upper function.
    upperCase: upperCase("DataWeave"),

    // Use alias findFirst for firstWith function.
    findFirst: [1, 2, 3, 4, 5] findFirst ((item, index) -> item > 3),

    // Use function from module with alias.
    str: Str::reverse("DataWeave")
} 

Output

{
  "upperCase": "DATAWEAVE",
  "findFirst": 4,
  "str": "evaeWataD"
}

Written with DataWeave 2.4.

March 1, 2022

DataWeave Delight: Taking or dropping items from an array

The module dw::core::Array has several functions to create a new array by taking or dropping a number of elements of the source array. With the take function has as first input argument the original array and the second is a number indicating the number of items to take. For example [1, 2] take 1 will return [1]. Notice that we can use the functions also in the infix notation. The drop function works similarly, but will skip the number of items we define as second argument to the function. So [1, 2] drop 1 will return [2].

In the following example we use the take and drop function with several numbers to use or skip items from the original array:

Source

%dw 2.0

import take from dw::core::Arrays

var list = ['Simple', 'list', 'with', 5, 'items']

output application/json
---
{
    // Return array with first element "Simple"
    "take 1 item": list take 1,

    // Return array with first 2 elements.
    "take 2 items": list take 2,

    // Empty list returned as we take 0 items.
    "take 0 items": list take 0,

    // Return array with all elements as 6 is larger 
    // than number of items in the list.
    "take 6 items": list take 6
}

Output

{
  "take 1 item": [
    "Simple"
  ],
  "take 2 items": [
    "Simple",
    "list"
  ],
  "take 0 items": [
    
  ],
  "take 6 items": [
    "Simple",
    "list",
    "with",
    5,
    "items"
  ]
}

Source

%dw 2.0

import drop from dw::core::Arrays

var list = ['Simple', 'list', 'with', 5, 'items']

output application/json
---
{
    // Return array where first element of original 
    // array is skipped.
    "drop 1 item": list drop 1,

    // Return array with elements from index 3.
    "drop 3 items": list drop 3,

    // Empty array returned as we drop all items.
    "drop 5 items": list drop 5,

    // Return empty array as 6 is larger than
    // number of items in the list.
    "drop 6 items": list drop 6,

    // Range as array of numbers.
    "drop from range": (0 to 10) drop 8
}

Output

{
  "drop 1 item": [
    "list",
    "with",
    5,
    "items"
  ],
  "drop 3 items": [
    5,
    "items"
  ],
  "drop 5 items": [
    
  ],
  "drop 6 items": [
    
  ],
  "drop from range": [
    8,
    9,
    10
  ]
}

We can also specify a condition using a predicate function to determine how many items to take or drop. In that case we use the takeWhile and dropWhile functions. Instead of specifying a number of items we use predicate function. The predicate function should return true or false. The function is applied to the items of the original array and as long as the return value is true it will be included in the resulting array with takeWhile or skipped with dropWhile.

In the next example we use both functions:

Source

%dw 2.0

import takeWhile, dropWhile from dw::core::Arrays

var list = (0 to 10)

var object = {
    name: "mrhaki", 
    livesIn: "Tilburg", 
    worksAt: 'JDriven'
}

// Helper function to get key from object
// with only 1 key/value pair.
fun getKey(obj: Object): Key = keysOf(obj)[0]

output application/json
---
{
    // Return array with first items smaller than 6.
    take: list takeWhile ((item) -> item < 6),

    // Once a false is returned for the condition
    // no more items are processed.
    // This time we use shorthand notation for the function argument.
    irregularTake: [1, 5, 7, 2, 4, 6] takeWhile $ < 6,

    // Return array where first items smaller than 6 are skipped.
    drop: list dropWhile ((item) -> item < 6),

    // With a bit of transformation we can also use the 
    // functions on objects. 
    // First we transform the object into an array, 
    // apply the function and transform to object again.
    objectTake: object
        pluck ((value, key) -> (key): value)
        takeWhile ((item) -> sizeOf(getKey(item)) == 4)
        reduce ((result, acc = {}) -> acc ++ result),

    objectDrop: object
        pluck ((value, key) -> (key): value)
        dropWhile ((item) -> getKey(item) ~= "name")
        reduce ((result, acc = {}) -> acc ++ result)        
}

Output

{
  "take": [
    0,
    1,
    2,
    3,
    4,
    5
  ],
  "irregularTake": [
    1,
    5
  ],
  "drop": [
    6,
    7,
    8,
    9,
    10
  ],
  "objectTake": {
    "name": "mrhaki"
  },
  "objectDrop": {
    "livesIn": "Tilburg",
    "worksAt": "JDriven"
  }
}

Written with DataWeave 2.4.