Search

Dark theme | Light theme

June 29, 2022

Groovy Goodness: Reading TOML Configuration

Since Groovy 4 we can parse TOML configuration data into a Map. Once the TOML data is transformed into the Map we can use all possibilities in Groovy to lookup keys and their values in maps. For example we can use GPath expressions to easily get the value of a (nested) key. To parse TOML configuration data we must use the TomlSlurper class that is in the groovy.toml package. We can use the parse method when we have a file, reader or stream with our configuration. To parse a String value with TOML configuration we use the parseText method.

In the following example we define our configuration using TOML syntax and use the parseText method to transform it into a Map. We use different ways with GPath expressions to read the configuration data:

import groovy.toml.TomlSlurper

// Configuration in TOML syntax.
def config = '''
application.name = "Groovy TOML"
application.version = "1.0.0"
application.ports = [80, 443]

# Set to true for debugging
debug.enabled = false

[build]
jdk = 'openjdk version "17.0.3" 2022-04-19'
time = "2022-06-29T07:32:00Z"

[[servers]]
name = "dev"
host = "localhost"
port = 8080

[[servers]]
name = "uat"
host = "cloud-acc"
port = 80
'''

// We get back a Map with all configuration.
def toml = new TomlSlurper().parseText(config)
// To read data from files, readers or streams we can use overloaded
// versions of the method TomlSlurper#parse.

// We can reference the properties using GPath expressions.
assert toml.application.name == "Groovy TOML"
assert toml.application.version == "1.0.0"
// TOML array is transformed to ArrayList.
assert toml.application.ports == [80, 443]
assert toml.application.ports.class == java.util.ArrayList
assert toml.application == [name: "Groovy TOML", version: "1.0.0", ports: [80, 443]]

// TOML boolean is transformed to boolean.
assert !toml.debug.enabled

assert toml.build.jdk == /openjdk version "17.0.3" 2022-04-19/
assert toml.build.time == "2022-06-29T07:32:00Z"
// Dates are not parsed, but we get them as String value.
assert toml.build.time.class == java.lang.String

// Array of tables in TOML are also supported by the TomlSlurper.
assert toml.servers.size() == 2

def developmentConfig = toml.servers.find { s -> s.name == "dev" }
assert developmentConfig.host == "localhost"
assert developmentConfig.port == 8080

def uatConfig = toml.servers.find { s -> s.name == "uat" }
assert uatConfig.host == "cloud-acc"
assert uatConfig.port == 80

assert toml.servers*.name == ["dev", "uat"]

Written with Groovy 4.0.3.

June 28, 2022

Groovy Goodness: Get Row Number In GINQ Result Set

GINQ (Groovy-INtegrated Query) is added since Groovy 4. With GINQ we can query in-memory collections with SQL like statements. If we want to get the row numbers for each row in the query result set we can use the implicit variable _rn. We must specify _rn in the select expression of our GINQ query. We can even use as to give it a meaningful name in the result set.

In the following example we have a basic GINQ query where use _rn to get the row number of each row in the query result set:

def letters = GQ {
    from letter in ['a', 'b', 'c']
    select _rn, letter
}.collect { item -> [item._rn, item.letter] }

assert letters == [[0, 'a'], [1, 'b'], [2, 'c']]

In the following example we first parse JSON and then use GINQ to query it. We see in the example that the row number is based on the result after the where expression is applied:

import groovy.json.JsonSlurper

def json = new JsonSlurper().parseText('''
[
  {
    "id": 1001,
    "language": "Groovy"
  },
  {
    "id": 1002,
    "language": "Clojure"
  }, 
  {
    "id": 1003,
    "language": "Java"
  }
]
''')

def languages = GQ {
  from l in json
  where l.language == "Groovy"
  // We add 1 to _rn to make it 1-based instead of 0-based. 
  // Also we use as rowNumber to give a meaningful name.
  select _rn + 1 as rowNumber, l.id as id, l.language as name
}.collect { item -> [row: item.rowNumber, name: item.name] }

// Also notice the row number is calculated based
// on the result after applying the where clause.
assert languages.first() == [row: 1, name: "Groovy"]

Written with Groovy 4.0.3.

Groovy Goodness: Using The Switch Expression

In a previous (old) post we learned how Groovy supports different classifiers for a switch case statement. Since Groovy 4 we can use switch also as an expression. This means the switch statement returns a value without having to use return. Instead of using a colon (:) and break we use the -> notation for a case. We specify the value that the switch expressions returns after ->. When we need a code block we simply put the code between curly braces ({...}).

In the following example we use the switch expression with different case statements:

def testSwitch(val) {
    switch (val) {
        case 52 -> 'Number value match'
        case "Groovy 4" -> 'String value match'
        case ~/^Switch.*Groovy$/ -> 'Pattern match'
        case BigInteger -> 'Class isInstance'
        case 60..90 -> 'Range contains'
        case [21, 'test', 9.12] -> 'List contains'
        case 42.056 -> 'Object equals'
        case { it instanceof Integer && it < 50 } -> 'Closure boolean'
        case [groovy: 'Rocks!', version: '1.7.6'] -> "Map contains key '$val'"
        default -> 'Default'
    } 
}

assert testSwitch(52) == 'Number value match'
assert testSwitch("Groovy 4") == 'String value match'
assert testSwitch("Switch to Groovy") == 'Pattern match'
assert testSwitch(42G) == 'Class isInstance'
assert testSwitch(70) == 'Range contains'
assert testSwitch('test') == 'List contains'
assert testSwitch(42.056) == 'Object equals'
assert testSwitch(20) == 'Closure boolean' 
assert testSwitch('groovy') == "Map contains key 'groovy'"
assert testSwitch('default') == 'Default'

Written with Groovy 4.0.3.

June 27, 2022

DataWeave Delight: Getting The Ordinal Value For A Number

The dw::core::Strings has a lot of functions to deal with strings. One of the functions is ordinalize that takes a number as argument and returns the ordinal value as string.

The following exmaple shows several numbers transformed to their ordinal values:

Source

%dw 2.0

import ordinalize from dw::core::Strings

output application/json
---
{
    "1": ordinalize(1),
    "2": ordinalize(2),
    "3": ordinalize(3),
    "10": ordinalize(10),
    "location": "The restaurant is at the " ++ ordinalize(4) ++ " floor."
}

Output

{
  "1": "1st",
  "2": "2nd",
  "3": "3rd",
  "10": "10th",
  "location": "The restaurant is at the 4th floor."
}

Written with DataWeave 2.4.

DataWeave Delight: Check Type Of Value

To check if a value is of a certain type in DataWeave we must use the is operator. We must specify the type after the is operator and the value before the is operator. For example to check if the value 42 is a Number we write 42 is Number. The result is a Boolean that is either true or false. In the previous example the result is true.

In the following examples we use is on several types:

Source

%dw 2.0

import every from dw::core::Arrays

// Closed object type with two properties alias and name
// that must have the String type.
type User = {|alias: String,name: String|}

// Helper function to check if argument value
// is an Array type and
// only contains elements of type String.
fun checkArrayString(value: Any): Boolean = 
    (value is Array) and
    ((value as Array) every ((element) -> element is String))

// Literal type.
type Language = "DataWeave" | "Java" | "Groovy" | "Clojure"

output application/json
---
{
    string: "DataWeave" is String,
    not_string: 42 is String,
    
    number: 42 is Number,

    // We can also use the Null type to check.
    null_value: null is Null,

    object: {alias: "mrhaki", name: "Hubert A. Klein Ikkink"} is Object,

    // We can check on our own object type.
    user: {alias: "mrhaki", name: "Hubert"} is User,

    // Only the alias property is valid.
    not_user1: {alias: "mrhaki", firstName: "Hubert", lastName: "Klein Ikkink"} is User,

    // User is a closed object so extra properties 
    // don't make it of type User anymore.
    not_user2: {alias: "mrhaki", name: "Hubert", location: "Tilburg"} is User,

    // Type of alias property is not correct.
    not_user3: {alias: 42, name: "Hubert"} is User,
    
    array: ["one", "two"] is Array,

    // To also check the types of the elements
    // we must use an extra check as defined
    // in the checkArrayString helper function.
    array_string: checkArrayString(["one", "two"]),
    not_array_string: checkArrayString([1, 2]),
    not_array: checkArrayString(42),

    // Literal types are supported.
    language: "DataWeave" is Language,
}

Output

{
  "string": true,
  "not_string": false,
  "number": true,
  "null_value": true,
  "object": true,
  "user": true,
  "not_user1": false,
  "not_user2": false,
  "not_user3": false,
  "array": true,
  "array_string": true,
  "not_array_string": false,
  "not_array": false,
  "language": true
}

Written with DataWeave 2.4.

June 26, 2022

DataWeave Delight: Calculating Remainder After Division

To calculate the modulo of two numbers in DataWeave we can use the mod function from the dw::Core module. We provide two arguments where the first argument is the number that needs to be divided by the number provided as second argument. The number that remains after a division of the input arguments is returned as a result.

In the following example we use different arguments for the mod function:

Source

%dw 2.0
output application/json
---
{
    result1: 6 mod 3,
    result2: 5 mod 3,
    result3: 5 mod -3,
    result4: -5 mod 3,
    result5: -5 mod -3,
    result6: 1.5 mod 1,
    result7: 5.095 mod 3
}

Output

{
  "result1": 0,
  "result2": 2,
  "result3": 2,
  "result4": -2,
  "result5": -2,
  "result6": 0.5,
  "result7": 2.095
}

Written with DataWave 2.4.

June 24, 2022

DataWeave Delight: Unzipping Arrays

In a previous blog post we learned about the zip function. DataWeave also gives us the unzip function that will do the opposite for an array with arrays. The input argument of the unzip function is an array where the elements are also arrays. This could be created by the zip function or just defined as data structure directly. The unzip function will take from each array the same index element and return it as an array with the index elements. For example with the input array [[1, "A"], [2, "B"]] will be unzipped to [[1, 2], ["A", "B"]]. When the number of elements in the arrays that need to unzipped are not equal, the unzip function will only return the elements from the index with the most elements.

In the following example we use the unzip function with different input arrays:

Source

%dw 2.0

var fruitPrices = [["Apple", 2.30], ["Pear", 1.82], ["Banana", 2.06]]

var fruitPricesIncomplete = [["Apple", 2.30], ["Pear", 1.82], [2.06]]

output application/json
---
{
    // unzip will break up each array into separate arrays.
    unzip: unzip(fruitPrices),

    // When the arrays to break up don't have the same 
    // number of elemnts unzip can only return elements
    // from the index that has the most elements.
    unzipIncomplete: unzip(fruitPricesIncomplete),

    // When the arrays to unzip have more than 2 elements
    // the results will be the number of arrays equal to the number of elements.
    // In this case we get 3 arrays as a result as the array to
    // unzip has 3 elements.
    unzipMoreElements: unzip([[1, "a", "A"], [2, "b", "B"]])
}

Output

{
  "unzip": [
    [
      "Apple",
      "Pear",
      "Banana"
    ],
    [
      2.30,
      1.82,
      2.06
    ]
  ],
  "unzipIncomplete": [
    [
      "Apple",
      "Pear",
      2.06
    ]
  ],
  "unzipMoreElements": [
    [
      1,
      2
    ],
    [
      "a",
      "b"
    ],
    [
      "A",
      "B"
    ]
  ]
}

Written with DataWeave 2.4.

June 21, 2022

DataWeave Delight: Zipping Arrays

DataWeave has a zip function in the dw::Core module. The function will merge two arrays into a new array. Each element in the new array is also an array and will have a value from the two original arrays from the same index grouped together. So for example we have an input array ["A", "B"] and another input array [1, 2]. The result of the zip function will be [["A", 1], ["B", 2]]. The size of the resulting array is the same as the minimal size of both input arrays. Any value from an array that cannot be merged is simply ignored and left out of the resulting array.

In the following code example we use the zip function for different arrays:

Source

%dw 2.0

import take from dw::core::Arrays

var fruit = ["Apple", "Pear", "Banana"]
var prices = [2.30, 1.82, 2.06]

output application/json
---
{
    // Create new array where each element is
    // an array with first element a fruit and
    // second element a price value. 
    zip: fruit zip prices,

    // When we have an array that contains arrays
    // of 2 items (a pair) we can easily turn it into an object
    // with key/value pairs using reduce.
    zipObj: fruit zip prices 
        reduce ((item, acc = {}) -> acc ++ {(item[0]): item[1]}),

    // The resulting array will have no more items
    // then the smallest array that is used with the zip function.
    // In the following example the second array only has 2 items
    // so the resulting array also has 2 items in total. 
    // The fruit "Banana" is now ignored.
    zipMinimal: fruit zip (prices take 2)
}

Source

{
  "zip": [
    [
      "Apple",
      2.30
    ],
    [
      "Pear",
      1.82
    ],
    [
      "Banana",
      2.06
    ]
  ],
  "zipObj": {
    "Apple": 2.30,
    "Pear": 1.82,
    "Banana": 2.06
  },
  "zipMinimal": [
    [
      "Apple",
      2.30
    ],
    [
      "Pear",
      1.82
    ]
  ]
}

Written with DataWeave 2.4.

June 14, 2022

DataWeave Delight: Measure Function Duration With time And duration Functions

To measure the time it takes to execute a function in DataWeave we can use the time and duration functions from the module dw::util::Timer. Both functions take a zero argument function that needs to be executed as argument (() -> T). But the output of the functions is different. The time function returns a TimeMeasurement object that has a result key containing the result of the function we passed as argument. We also get a start key that has the date and time value when the function gets executed. And finally we have the end key that stores the date and time value when the function is finished. To calculate the total duration time of the function we could use the start and end keys, but when we want the duration time we can better use the duration function. The duration function returns a DurationMeasurement object with also a key result that has the output of the function that is executed. The other key is time and contains the time it took for the function to be executed in milliseconds.

In the following example we use both timer functions together with the wait function. The wait function will wait for the given number of milliseconds before returning a value.

Source

%dw 2.0

import wait from dw::Runtime
import duration, time from dw::util::Timer

output application/json
---
{ 
    // time function returns a new object with
    // keys start, end and result.
    // Keys start and end contain the start and end datetime 
    // before and after the function is executed.
    // The result key has the value of the function.
    time: time(() -> 42 wait 1000),

    // duration function returns a new object with
    // keys time and result.
    // Key duration has the total duration for the
    // function execution in milliseconds.
    // The result key has the value of the function.
    duration: duration(() -> 42 wait 1000)
}

Output

{
  "time": {
    "start": "2022-06-14T04:39:21.582958Z",
    "result": 42,
    "end": "2022-06-14T04:39:22.583079Z"
  },
  "duration": {
    "time": 1000,
    "result": 42
  }
}

Applying the time or duration function in our code is intrusive as the result object is different from the result of our function we want to measure. We can write a helper function that uses the time or duration function, logs the output of the functions using log function and finally still returns the value from our input function by selecting the result key.

In the next example we create the wrapTime and wrapDuration functions that can be used to log the output of the time and duration functions and still return the result of the input function. This way we can introduce timing of function duration in a less intrusive way.

Source

%dw 2.0

import wait from dw::Runtime
import duration, time from dw::util::Timer

fun answer() = 42 wait 400

// We still want the original result from the answer function,
// but also log the output of the time function.
// We pass the output of the time function to the log function that
// will log the output and then return the output value. 
// And finally use the result key from the time function
// to get output from the answer function.
fun wrapTime(f) = log("Time", time(f)).result

// We still want the original result from the answer function,
// but also log the output of the duration function.
// We pass the output of the time function to the log function that
// will log the output and then return the output value. 
// And finally use the result key from the duration function output
// to get output from the answer function.    
fun wrapDuration(f) = log("Duration", duration(f)).result

output application/json
---
{ 
    // Simple invocation of the function answer to get the result.
    result: answer(),

    // Use wrapTime function to still get result, but also log time output.
    resultTimer: wrapTime(() -> answer()),

    // Use wrapDuration function to still get result, but also log duration output.
    resultDuration: wrapDuration(() -> answer())
} 

Output

{
  "result": 42,
  "resultTimer": 42,
  "resultDuration": 42
}

Log output

Time - { start: |2022-06-14T04:52:29.287724Z|, result: 42, end: |2022-06-14T04:52:29.687875Z| }
Duration - { time: 400, result: 42 }

Written with DataWeave 2.4.

March 16, 2022

DataWeave Delight: Partition An Array

In DataWeave we can partition the items in an array using a predicate function by using the partition function from the dw::core::Arrays module. The function takes an array as first argument and a predicate function as second argument. The predicate function should return true or false for each item of the array. The result is an object with the key success containing all items from the array that returned true for the predicate function and a key failure for the items that returned false.

In the following example code we use the partition function on an array:

Source

%dw 2.0

import partition from dw::core::Arrays

var items = ["language", "DataWeave", "username", "mrhaki", "age", 48]

output application/json 
---
{ 
    // Partition by item is of type String or not.
    example1: items partition ((item) -> typeOf(item) == String),

    // Partition by checking if item have value 1 or 4 or not
    // using shorthand notation.
    example2: (0 to 5) partition ([1, 4] contains $)
}

Output

{
  "example1": {
    "success": [
      "language",
      "DataWeave",
      "username",
      "mrhaki",
      "age"
    ],
    "failure": [
      48
    ]
  },
  "example2": {
    "success": [
      1,
      4
    ],
    "failure": [
      0,
      2,
      3,
      5
    ]
  }
}

Written with DataWeave 2.4.