Search

Dark theme | Light theme

March 16, 2022

DataWeave Delight: Partition An Array

In DataWeave we can partition the items in an array using a predicate function by using the partition function from the dw::core::Arrays module. The function takes an array as first argument and a predicate function as second argument. The predicate function should return true or false for each item of the array. The result is an object with the key success containing all items from the array that returned true for the predicate function and a key failure for the items that returned false.

In the following example code we use the partition function on an array:

Source

%dw 2.0

import partition from dw::core::Arrays

var items = ["language", "DataWeave", "username", "mrhaki", "age", 48]

output application/json 
---
{ 
    // Partition by item is of type String or not.
    example1: items partition ((item) -> typeOf(item) == String),

    // Partition by checking if item have value 1 or 4 or not
    // using shorthand notation.
    example2: (0 to 5) partition ([1, 4] contains $)
}

Output

{
  "example1": {
    "success": [
      "language",
      "DataWeave",
      "username",
      "mrhaki",
      "age"
    ],
    "failure": [
      48
    ]
  },
  "example2": {
    "success": [
      1,
      4
    ],
    "failure": [
      0,
      2,
      3,
      5
    ]
  }
}

Written with DataWeave 2.4.

March 15, 2022

DataWeave Delight: Splitting An Array Or Object

The module dw::core::Arrays has extra functions that are useful when working with arrays in DataWeave. In this post we will look at the functions splitAt and splitWhere. We can use these functions to split an array into two arrays. The result of both functions is actually a Pair type, which is defined as an object with the keys l and r. An example is { "l": 1, "r": 2 } which we can read as the left side of the pair has value 1 and the right side of the pair has value 2. The result of the splitAt and splitWhere function will have one part of the split array in the left side of the pair and the rest of the array in the right side.

The splitAt function takes the array as first argument and an index value as second argument. The index value should indicate at which position the array should be split. The function can be used with infix notation as well. The splitWhere function takes as first argument also an array, but the second argument is a predicate function. All items starting from the first item for which the predicate function returns true will be assigned to the r key of the Pair result, and all preceding items to the l key. We can use the infix notation here as well.

In the following example code we use the splitAt and splitWhere function on an array:

Source

%dw 2.0

import splitWhere, splitAt from dw::core::Arrays

output application/json
---
{
    // Split the range at index 4. 
    splitAt: (0 to 8) splitAt 4,

    // Split at the position where the predicate returns true for the first time.
    splitWhere: (0 to 8) splitWhere ((item) -> item > 5)
}

Output

{
  "splitAt": {
    "l": [
      0,
      1,
      2,
      3
    ],
    "r": [
      4,
      5,
      6,
      7,
      8
    ]
  },
  "splitWhere": {
    "l": [
      0,
      1,
      2,
      3,
      4,
      5
    ],
    "r": [
      6,
      7,
      8
    ]
  }
}

Although the function work on arrays we can use them on a object as well. We first turn the object into an array of objects with a single key and value using the pluck function. We can use this array with the splitAt and splitWhere functions. Then we can use the reduce function to transform the values in the Pair to an object with multiple keys and values again.

In the next example we use this mechanism on an object:

Source

%dw 2.0

import splitWhere, splitAt from dw::core::Arrays

// Helper functions
// ----------------
// Transform object to array with key/value pairs
fun objectToArray(obj: Object): Array<Object> = obj pluck ((value, key) -> (key): value)

// Transform array with key/value pairs to object
fun arrayToObject(items: Array<Object>): Object = items reduce ((item, acc = {}) -> acc ++ item)

var obj = {
    language: "DataWeave",
    alias: "mrhaki",
    age: 48, 
    country: "NL"
}

output application/json 
---
{
    // We can use splitAt on object if we first transform an object
    // to an array of key/value pairs, apply the splitAt function and 
    // transform the resulting Pair into an object again.
    splitAtObject: objectToArray(obj) 
        splitAt 3 
        mapObject ((value, key) -> (key): arrayToObject(value)),

    // We can use splitWhere in the same manner for objects.
    splitWhereObject: objectToArray(obj) 
        splitWhere ((item) -> typeOf(item[0]) == Number)
        mapObject ((value, key) -> (key): arrayToObject(value))
}

Output

{
  "splitAtObject": {
    "l": {
      "language": "DataWeave",
      "alias": "mrhaki",
      "age": 48
    },
    "r": {
      "country": "NL"
    }
  },
  "splitWhereObject": {
    "l": {
      "language": "DataWeave",
      "alias": "mrhaki"
    },
    "r": {
      "age": 48,
      "country": "NL"
    }
  }
}

Written with DataWeave 2.4.

March 14, 2022

DataWeave Delight: Importing Functions With An Alias

When we want to reference functions from other modules than the dw::Core module we must us the import statement. We can use the as keyword to map a function name that we want to import to another name that we want to use in our DataWeave code. We provide an alias for the function that we can use in our code. This could be useful if we think our code gets more readable in a certain context by using different function names, but still want to use the original function implementation. Next to providing an alias for function names, we can also provide an alias for a module name, when we use the import <ModuleName> as <NewModuleName> statement. In our DataWeave code we can then refer to the new alias to call functions from the module.

In the following example code we use import using as to define aliases for functions and a module:

Source

%dw 2.0

// Use an alias for functions.
import upper as upperCase from dw::Core
import firstWith as findFirst from dw::core::Arrays

// Assign an alias to a module, so we can reference
// functions from the module as Str::<function>
import dw::core::Strings as Str

output application/json
---
{
    // Use alias upperCase for upper function.
    upperCase: upperCase("DataWeave"),

    // Use alias findFirst for firstWith function.
    findFirst: [1, 2, 3, 4, 5] findFirst ((item, index) -> item > 3),

    // Use function from module with alias.
    str: Str::reverse("DataWeave")
} 

Output

{
  "upperCase": "DATAWEAVE",
  "findFirst": 4,
  "str": "evaeWataD"
}

Written with DataWeave 2.4.

March 1, 2022

DataWeave Delight: Taking or dropping items from an array

The module dw::core::Array has several functions to create a new array by taking or dropping a number of elements of the source array. With the take function has as first input argument the original array and the second is a number indicating the number of items to take. For example [1, 2] take 1 will return [1]. Notice that we can use the functions also in the infix notation. The drop function works similarly, but will skip the number of items we define as second argument to the function. So [1, 2] drop 1 will return [2].

In the following example we use the take and drop function with several numbers to use or skip items from the original array:

Source

%dw 2.0

import take from dw::core::Arrays

var list = ['Simple', 'list', 'with', 5, 'items']

output application/json
---
{
    // Return array with first element "Simple"
    "take 1 item": list take 1,

    // Return array with first 2 elements.
    "take 2 items": list take 2,

    // Empty list returned as we take 0 items.
    "take 0 items": list take 0,

    // Return array with all elements as 6 is larger 
    // than number of items in the list.
    "take 6 items": list take 6
}

Output

{
  "take 1 item": [
    "Simple"
  ],
  "take 2 items": [
    "Simple",
    "list"
  ],
  "take 0 items": [
    
  ],
  "take 6 items": [
    "Simple",
    "list",
    "with",
    5,
    "items"
  ]
}

Source

%dw 2.0

import drop from dw::core::Arrays

var list = ['Simple', 'list', 'with', 5, 'items']

output application/json
---
{
    // Return array where first element of original 
    // array is skipped.
    "drop 1 item": list drop 1,

    // Return array with elements from index 3.
    "drop 3 items": list drop 3,

    // Empty array returned as we drop all items.
    "drop 5 items": list drop 5,

    // Return empty array as 6 is larger than
    // number of items in the list.
    "drop 6 items": list drop 6,

    // Range as array of numbers.
    "drop from range": (0 to 10) drop 8
}

Output

{
  "drop 1 item": [
    "list",
    "with",
    5,
    "items"
  ],
  "drop 3 items": [
    5,
    "items"
  ],
  "drop 5 items": [
    
  ],
  "drop 6 items": [
    
  ],
  "drop from range": [
    8,
    9,
    10
  ]
}

We can also specify a condition using a predicate function to determine how many items to take or drop. In that case we use the takeWhile and dropWhile functions. Instead of specifying a number of items we use predicate function. The predicate function should return true or false. The function is applied to the items of the original array and as long as the return value is true it will be included in the resulting array with takeWhile or skipped with dropWhile.

In the next example we use both functions:

Source

%dw 2.0

import takeWhile, dropWhile from dw::core::Arrays

var list = (0 to 10)

var object = {
    name: "mrhaki", 
    livesIn: "Tilburg", 
    worksAt: 'JDriven'
}

// Helper function to get key from object
// with only 1 key/value pair.
fun getKey(obj: Object): Key = keysOf(obj)[0]

output application/json
---
{
    // Return array with first items smaller than 6.
    take: list takeWhile ((item) -> item < 6),

    // Once a false is returned for the condition
    // no more items are processed.
    // This time we use shorthand notation for the function argument.
    irregularTake: [1, 5, 7, 2, 4, 6] takeWhile $ < 6,

    // Return array where first items smaller than 6 are skipped.
    drop: list dropWhile ((item) -> item < 6),

    // With a bit of transformation we can also use the 
    // functions on objects. 
    // First we transform the object into an array, 
    // apply the function and transform to object again.
    objectTake: object
        pluck ((value, key) -> (key): value)
        takeWhile ((item) -> sizeOf(getKey(item)) == 4)
        reduce ((result, acc = {}) -> acc ++ result),

    objectDrop: object
        pluck ((value, key) -> (key): value)
        dropWhile ((item) -> getKey(item) ~= "name")
        reduce ((result, acc = {}) -> acc ++ result)        
}

Output

{
  "take": [
    0,
    1,
    2,
    3,
    4,
    5
  ],
  "irregularTake": [
    1,
    5
  ],
  "drop": [
    6,
    7,
    8,
    9,
    10
  ],
  "objectTake": {
    "name": "mrhaki"
  },
  "objectDrop": {
    "livesIn": "Tilburg",
    "worksAt": "JDriven"
  }
}

Written with DataWeave 2.4.

February 23, 2022

DataWeave Delight: Reversing order of keys in objects

An object in DataWeave consists of key and value pairs. If we want to create a new object sorted by key or value we can use the orderBy function from the dw::Core module. The orderBy function takes as first argument the object. The second argument is a criteria function that can the value and key to determine how to order the object. But if we want to reverse the ordering of the key and value pairs we cannot simply use the orderBy function, because we don't have access to the index value of the key/value pair in the criteria function. We would love to have that index value, so we could negate the index value to reverse the ordering. What we need to do is first turn our object in an array of objects using the pluck function, where each object has our key and value. We can now use the orderBy function on that array of objects to reverse the order. Finally we need to reduce the array of objects into a new object using the reduce function, where the ordering of keys will be reversed.

In the following example we use :

Source

%dw 2.0

var object = {
    name: "Hubert Klein Ikkink",
    alias: "mrhaki",
    city: "Tilburg"
}

output application/json
---
{
    reverseKeys: object 
        // Turn into array: [{name: "Hubert Klein Ikkink"}, ...]
        pluck ((value, key, index) -> {(key): value})

        // Reverse the ordering based on negated index value
        orderBy ((item, index) -> -index) 

        // Transform array back to an object.
        reduce ((item, accumulator = {}) -> accumulator ++ item)
}

Output

{
  "reverseKeys": {
    "city": "Tilburg",
    "alias": "mrhaki",
    "name": "Hubert Klein Ikkink"
  }
}

Written with DataWeave 2.4.

DataWeave Delight: Reversing the items in an array

To reverse the order of the items in the array we have two possible solutions in DataWeave. The first solution uses the index operator to access items in an array ([...]). We can define a range as value for the index operator to get all elements that have a index value that is part of the range. For example we could use [1 to 3] to get the second up to the fourth items from the array. The value -1 can be used to indicate the last index in the array, so we could get a new array where the items are in reversed order by using the range [-1 to 0].

The second option is to use the orderBy function for an array from the dw::Core module. The function takes as first argument an array and the second argument is a criteria function that is applied for each element. The criteria function will get the array element value and index. The output of the function is used for the ordering of the resulting array. To reverse the order we can negate the index value of the criteria function and we get a new array in reverse order: ((item, index) -> -index).

In the following example we use both ways to reverse the order of an array:

Source

%dw 2.0

var list = [2, 1, 4, 5, 3]

output application/json
---
{
    // Using index operator.
    reverseIndices: list[-1 to 0],

    // Using orderBy with criteria function.
    reverseOrderBy: list orderBy ((item, index) -> -index),

    // Using orderBy with shorthand notation for criteria function.
    reverseOrderByShorthand: list orderBy -$$
}

Output

{
  "reverseOrderBy": [
    3,
    5,
    4,
    1,
    2
  ],
  "reverseIndices": [
    3,
    5,
    4,
    1,
    2
  ],
  "reverseOrderByShorthand": [
    3,
    5,
    4,
    1,
    2
  ]
}

Written with DataWeave 2.4.

February 17, 2022

DataWeave Delight: Removing duplicates from an array

If we want to remove duplicate items from an array in DataWeave we can use the distinctBy function from the dw::Core module. The first argument is an array and the second argument is a criteria function. The criteria function gets an item of the array and the index value of that item in the array. The return value of the function defines the uniqueness of that item. This can simply be the item value itself or something more complex like a calculation or selector.

In the following example we use several criteria functions to remove duplicate items from an array:

Source

%dw 2.0

var items = [
    { id: "DW", name: "DataWeave" },
    { id: "CLJ", name: "Clojure" },
    { id: "DW", name: "DataWeave" }
]
var s = "aabccdeff"

output application/json 
---
{ 
    // An array with numbers where we want to remove duplicates.
    numbers1: [1, 5, 5, 6, 2, 3, 3, 1] distinctBy ((item) -> item),

    // We can also use a bit more complex function to define uniqueness.
    numbers2: [1, -2, 2, 3, -1] distinctBy (item) -> abs(item),

    // We can use shorthand notation $ and
    // distinctBy also works on objects in the array.
    items1: items distinctBy $,

    // Instead of doing a distinctBy on the whole object
    // we can also use matching function and use properties
    // from the object to define unique objects.
    items2: items distinctBy ((item, index) -> item.id),

    // With the help of toArray and joinBy we can remove duplicates
    // from a string value.
    string: dw::util::Coercions::toArray(s) distinctBy $ joinBy ""
}

Output

{
  "numbers1": [
    1,
    5,
    6,
    2,
    3
  ],
  "numbers2": [
    1,
    -2,
    3
  ],
  "items1": [
    {
      "id": "DW",
      "name": "DataWeave"
    },
    {
      "id": "CLJ",
      "name": "Clojure"
    }
  ],
  "items2": [
    {
      "id": "DW",
      "name": "DataWeave"
    },
    {
      "id": "CLJ",
      "name": "Clojure"
    }
  ],
  "string": "abcdef"
}

Written with DataWeave 2.4.

DataWeave Delight: Counting items in array with predicate

The dw::core::Arrays module has some nice functions that works with arrays. One of these functions is countBy. The function takes an array as first argument and a predicate function that accepts an item from the array and should return true or false. The total number of items that adhere to the predicate is the result of the countBy function.

In the following examples we use the countBy function with different predicates:

Source

%dw 2.0

import countBy from dw::core::Arrays

var languages = ['DataWeave', 'Groovy', 'Java', 'Clojure']

output application/json
---
{ 
    // Count items that have more than 6 characters.
    sample1: languages countBy ((item) -> sizeOf(item) > 6), 

    // Count number of even numbers.
    sample2: 1 to 10 countBy (number) -> isEven(number),

    // Using shorthand notation $ to count upper case characters.
    sample3: ["AB", "cD", "Ef", "GH"] countBy dw::core::Strings::isUpperCase($)
}

Output

{
  "sample1": 2,
  "sample2": 5,
  "sample3": 2
}

Written with DataWeave 2.4.

February 15, 2022

DataWeave Delight: Trimming string values

In DataWeave we can use the trim function from the dw::Core module to remove any space character before and after a string value. The characters that are removed are the space, tab and newline characters. There is an overloaded function definition that will accept a null value and returns a null value.

In the following example we trim string values that start and end with spaces, tabs and newline characters:

Source

%dw 2.0

output application/json
---
{
    // Spaces at the start and end ar removed.
    spaces: trim("   mrhaki  "),

    // Also tabs are removed.
    tabs: trim("\t mrhaki"),

    // And newline characters are removed
    newline: trim("\tmrhaki \r\n"),

    // trim will return null when a null value is used.
    nullValue: trim(null),
}

Output

{
  "spaces": "mrhaki",
  "tabs": "mrhaki",
  "newline": "mrhaki",
  "nullValue": null
}

Written with DataWeave 2.4.

February 14, 2022

DataWeave Delight: Convert string value to a boolean

If we need to convert a string value "true" or "false" to a boolean we can use the toBoolean function from the dw::util::Coercions module. The function will return a boolean true if the string value is "true", mixed casing is allowed. And the function returns false for a mixed casing string value of "false". Any other string value will throw an exception and will not return a boolean value.

In the following example we coerce some string values to a boolean and also include an example where the input value cannot be coerced:

Source

%dw 2.0

import try from dw::Runtime
import toBoolean from dw::util::Coercions

output application/json
---
{
    // Coerce all string values to a boolean with value true.
    trueBooleans: ["TRUE", "true", "True", "trUE"] map (s) -> toBoolean(s),

    // Coerce all string value to a boolean with value false.
    falseBooleans: ["FALSE", "false", "False", "falSE"] map toBoolean($),

    // An exception is thrown when the string value cannot be coerced.
    invalidCoercion: try(() -> toBoolean("Yes"))
}

Output

{
  "trueBooleans": [
    true,
    true,
    true,
    true
  ],
  "falseBooleans": [
    false,
    false,
    false,
    false
  ],
  "invalidCoercion": {
    "success": false,
    "error": {
      "kind": "InvalidBooleanException",
      "message": "Cannot coerce String (Yes) to Boolean",
      "location": "\n16|     invalidCoercion: try(() -> toBoolean(\"Yes\"))\n
                                                                   ^^^^^",
      "stack": [
        "toBoolean (main:16:42)",
        "main (main:16:32)"
      ]
    }
  }
}

Written with DataWeave 2.4.