jq Joy: Adding Elements In An Array Or Object

jq is a powerful tool to work with JSON from the command-line. The tool has a lot of functions that makes our live easier. One of the functions is add which adds all elements in an array or values in an object. The function has no arguments. The elements in an array are added together if they are numbers and concatenated if they are strings. If the input is an object then the values are added together. When the input is an empty array or object then null is returned.

In the following examples we see different usages of the add function:

$ echo '[1, 2, 3, 4]' | jq 'add'
10

$ echo '["a", "b", "c", "d"]' | jq 'add'
"abcd"

$ echo '{ "burger": 3.23, "drinks": 2.89 }' | jq 'add'
6.12

$ echo '[]' | jq 'add'
null

Written with jq 1.7.

jq Joy: Flatten Arrays

jq is a powerful tool to work with JSON from the command-line. The tool has a lot of functions that makes our live easier. We can use the flatten function to flatten nested arrays into a single array. If we use the function without an argument the arrays are flattened recursively, resulting in a flat array with all elements. But we can also set the depth we want to flatten the array by passing an integer argument to the flatten function.

In the following example we use the flatten function without arguments to recursively flatten the array:

$ echo '[1, [2, 3], [[4]], 5]' | jq 'flatten'
[
  1,
  2,
  3,
  4,
  5
]

We can pass an argument to the flatten function to indicate the depth to flatten the collection. In the following example we want to flatten only one level deep:

$ echo '[1, [2, 3], [[4]], 5]' | jq 'flatten(1)'
[
  1,
  2,
  3,
  [
    4
  ],
  5
]

Written with jq 1.7.

Awesome AssertJ: Using A Custom Representation For Objects

The assertion error messages from AssertJ will use the toString() method of an object to give more insight about why the assertion could have failed. If an object doesn’t override the toString() method the default implementation is Object#toString(). The default implementation will print out the class name and an hexadecimal value of hashCode separated by a @. This doesn’t give much information about the actual object. For classes we have control over we can always implement a toString() method, but for third party classes we may not be able to do that. In order to customize how an object is represented in assertion error messages, AssertJ allows us to provide a custom representation class for an object. The custom representation class must implement the org.assertj.core.presentation.Representation interface. The interface has one method String toStringOf(Object) that should return a String representation of the object. If we want to keep the default behavior for other classes and only override it for our own class, we can extend the StandardRepresentation class and override the method String fallbackToStringOf(Object).

In the following example we provide a custom representation for the User class that will print the username property in the error messages instead of the default toString() implementation:

package mrhaki;

import org.assertj.core.api.Assertions;
import org.assertj.core.presentation.StandardRepresentation;
import org.junit.jupiter.api.Test;

import static org.assertj.core.api.Assertions.assertThat;

class CustomRepresentation {

    // Helper class used in tests.
    private static class User {
        private final String username;

        private User(String username) {
            this.username = username;
        }
    }

    // Custom representation for the User class.
    // Here we can write how we want to represent the User class in
    // assertion error messages.
    private static class UserRepresentation extends StandardRepresentation {
        /**
         * The User class doesn't have a toString implementation, so we write
         * custom code so in assertion error message our object is nicely formatted.
         *
         * @param object the object to represent (never {@code null}
         * @return Custom string representation of User or standard representation for other objects.
         */
        @Override
        protected String fallbackToStringOf(Object object) {
            if (object instanceof User user) {
                return "User(username=%s)".formatted(user.username);
            }
            return super.fallbackToStringOf(object);
        }
    }

    @Test
    void assertUserObject() {
        final User mrhaki = new User("mrhaki");
        try {
            // assert will fail and the assertion error will contain
            // a default representation of the objects.
            assertThat(mrhaki).isEqualTo(new User("mrhaki42"));
        } catch (AssertionError e) {
            // expected: mrhaki.CustomRepresentation$User@126253fd
            // but was: mrhaki.CustomRepresentation$User@7eecb5b8
            assertThat(e).hasMessageContaining("expected: mrhaki.CustomRepresentation$User@")
                         .hasMessageContaining("but was: mrhaki.CustomRepresentation$User@");
        }
    }

    @Test
    void asserUserObjectWithCustomRepresentation() {
        final User mrhaki = new User("mrhaki");
        try {
            // assert will fail and the assertion error will contain
            // a custom representation of the objects only for this assert.
            assertThat(mrhaki).withRepresentation(new UserRepresentation()).isEqualTo(new User("mrhaki42"));
        } catch (AssertionError e) {
            // expected: User(username=mrhaki42)
            // but was: User(username=mrhaki)
            assertThat(e).hasMessageContaining("expected: User(username=mrhaki42)")
                         .hasMessageContaining("but was: User(username=mrhaki)");
        }
    }

    @Test
    void asserUserObjectWithCustomRepresentation2() {
        final User mrhaki = new User("mrhaki");
        try {
            // Set custom representation for User objects for all asserts.
            // This can also be defined at class level.
            Assertions.useRepresentation(new UserRepresentation());

            // assert will fail and the assertion error will contain
            // a custom representation of the objects.
            assertThat(mrhaki).isEqualTo(new User("mrhaki42"));
        } catch (AssertionError e) {
            // expected: User(username=mrhaki42)
            // but was: User(username=mrhaki)
            assertThat(e).hasMessageContaining("expected: User(username=mrhaki42)")
                         .hasMessageContaining("but was: User(username=mrhaki)");
        } finally {
            // Reset custom representation.
            Assertions.useDefaultRepresentation();
        }
    }

    @Test
    void asserUserObjectWithCustomRepresentationSAM() {
        final User mrhaki = new User("mrhaki");
        try {
            // assert will fail and the assertion error will contain
            // a custom representation of the objects for this assert only.
            assertThat(mrhaki).withRepresentation(new UserRepresentation()).isEqualTo(new User("mrhaki42"));
        } catch (AssertionError e) {
            // expected: User(username=mrhaki42)
            // but was: User(username=mrhaki)
            assertThat(e).hasMessageContaining("expected: User(username=mrhaki42)")
                         .hasMessageContaining("but was: User(username=mrhaki)");
        }
    }

    @Test
    void asserUserObjectWithCustomRepresentationSAM() {
        final User mrhaki = new User("mrhaki");
        try {
            // assert will fail and the assertion error will contain
            // a custom representation of the objects implemented
            // with the Representation interface as single abstract method (SAM).
            assertThat(mrhaki).withRepresentation(obj -> {
                if (obj instanceof User user) {
                    return "[User:username=%s]".formatted(user.username);
                } else {
                    return Objects.toIdentityString(obj);
                }
            }).isEqualTo(new User("mrhaki42"));
        } catch (AssertionError e) {
            // expected: [User:username=mrhaki42]
            // but was: [User:username=mrhaki]
            assertThat(e).hasMessageContaining("expected: [User:username=mrhaki42]")
                         .hasMessageContaining("but was: [User:username=mrhaki]");
        }
    }
}

Written with AssertJ 3.24.2

jq Joy: Deleting Keys From An Object

jq is a powerful tool to work with JSON from the command-line. The tool has a lot of functions that makes our live easier. If we want to delete keys from an object we can use the del function and pass the key name as argument. The argument is actually a path expression so we can refer to a key name with .key, but also use jq expressions.

$ echo '{
    "username": "mrhaki",
    "firstName": "Hubert",
    "lastName": "Klein Ikkink"
}' | jq 'del(.username)'
{
  "firstName": "Hubert",
  "lastName": "Klein Ikkink"
}

$ echo '{
    "username": "mrhaki",
    "firstName": "Hubert",
    "lastName": "Klein Ikkink"
}' | jq 'del(.username) | del(.lastName)'
{
  "firstName": "Hubert"
}

If we want to remove multiple keys at once we can pass multiple key names separated by comma to the del function:

$ echo '{
    "username": "mrhaki",
    "firstName": "Hubert",
    "lastName": "Klein Ikkink"
}' | jq 'del(.username, .lastName)'
{
  "firstName": "Hubert"
}

We can use also jq path expressions for example to delete keys by index instead of key name:

$ echo '{
    "username": "mrhaki",
    "firstName": "Hubert",
    "lastName": "Klein Ikkink"
    }' | jq 'del(.[keys_unsorted | .[0,2]])'
{
  "firstName": "Hubert"
}

Written with jq 1.7.

jq Joy: Getting Keys From Object And Indices From Array

jq is a powerful tool to work with JSON from the command-line. The tool has a lot of functions that makes our live easier. For example we can use the keys and keys_unsorted functions to get the keys from an object. The function keys will return the keys in sorted order while keys_unsorted will return them in the original order from the object. With the same functions we can also get the indices of the elements in an array, but there is no sorting involved, so both functions return the same output.

In the following examples we first use keys and then keys_unsorted to get the keys from an object:

$ echo '{
    "username": "mrhaki",
    "firstName": "Hubert",
    "lastName": "Klein Ikkink"
}' | jq keys
[
  "firstName",
  "lastName",
  "username"
]

$ echo '{
    "username": "mrhaki",
    "firstName": "Hubert",
    "lastName": "Klein Ikkink"
}' | jq keys_unsorted
[
  "username",
  "firstName",
  "lastName"
]

To get the indices of an array we can use the keys function as well:

$ echo '[1, 10, 100]' | jq keys
[
  0,
  1,
  2
]

Written with jq 1.7.

Java Joy: Using mapMulti Method Of The Stream API

Since Java 16 we can use the method mapMulti(BiConsumer) of the Stream API. This method allows us to map each element of the stream to multiple elements. We can also do that with the flatMap(Function) method, but if we want to map a limited set of elements, mapMulti is more convenient. Internally a shared stream is used and we don’t have the cost of creating a new stream for each element. Another use case is if the logic to map an element to multiple elements is complex and is hard to implement by returning a stream. Then mapMulti allows us to write that logic in a BiConsumer instead of a Function.

In the following code we use the mapMulti method in several examples:

package mrhaki.stream;

import javax.print.attribute.HashPrintServiceAttributeSet;
import java.util.Arrays;
import java.util.IntSummaryStatistics;
import java.util.List;
import java.util.Set;
import java.util.function.BiConsumer;
import java.util.function.Consumer;
import java.util.stream.Collectors;
import java.util.stream.Stream;

public class MapMulti {

    public static void main(String[] args) {
        // We want to return a stream of string values 
        // and the uppercase variant
        // if the original element has the letter o.
        assert Stream.of("Java", "Groovy", "Clojure")
                     .mapMulti((language, downstream) -> {
                         if (language.contains("o")) {
                             downstream.accept(language);
                             downstream.accept(language.toUpperCase());
                         }
                     })
                     .toList()
                     .equals(List.of("Groovy", "GROOVY", "Clojure", "CLOJURE"));

        // Same logic implemented with flatMap.
        assert Stream.of("Java", "Groovy", "Clojure")
                     .filter(language -> language.contains("o"))
                     .flatMap(language -> Stream.of(language, language.toUpperCase()))
                     .toList()
                     .equals(List.of("Groovy", "GROOVY", "Clojure", "CLOJURE"));


        // Helper record to store a name and set of language names.
        record Developer(String name, List<String> languages) {}

        // Number of sample developers that work with different languages.
        var hubert = new Developer("mrhaki", List.of("Java", "Groovy", "Clojure"));
        var java = new Developer("java", List.of("Java"));
        var clojure = new Developer("clojure", List.of("Clojure"));
        var groovy = new Developer("groovy", List.of("Groovy"));

        record Pair(String name, String language) {}

        // Let's find all developers that have Java in their
        // set of languages and return a new Pair
        // object with the name of the developer and a language.
        assert Stream.of(hubert, java, clojure, groovy)
                     // We can explicitly state the class that will be 
                     // in the downstream of the compiler cannot
                     // deduct it using a <...> syntax.
                     .<Pair>mapMulti((developer, downstream) -> {
                         var languages = developer.languages();
                         if (languages.contains("Java")) {
                             for (String language : developer.languages()) {
                                 downstream.accept(new Pair(developer.name(), language));
                             }
                         }
                     })
                     .toList()
                     .equals(List.of(new Pair("mrhaki", "Java"),
                                     new Pair("mrhaki", "Groovy"),
                                     new Pair("mrhaki", "Clojure"),
                                     new Pair("java", "Java")));

        // Same logic using filter and flatMap.
        assert Stream.of(hubert, java, clojure, groovy)
                     .filter(developer -> developer.languages().contains("Java"))
                     .flatMap(developer -> developer.languages()
                                                    .stream()
                                                    .map(language -> new Pair(developer.name(), language)))
                     .toList()
                     .equals(List.of(new Pair("mrhaki", "Java"),
                                     new Pair("mrhaki", "Groovy"),
                                     new Pair("mrhaki", "Clojure"),
                                     new Pair("java", "Java")));
        
        
        // We want to expand each number to itself and its square root value
        // and we muse mapMultiToInt here.
        var summaryStatistics = Stream.of(1, 2, 3)
                                      .mapMultiToInt((number, downstream) -> {
                                          downstream.accept(number);
                                          downstream.accept(number * number);
                                      })
                                      .summaryStatistics();

        assert summaryStatistics.getCount() == 6;
        assert summaryStatistics.getSum() == 20;
        assert summaryStatistics.getMin() == 1;
        assert summaryStatistics.getMax() == 9;
    }
}

Written with Java 20.

Awesome AssertJ: Writing Assertions For Optional

For a lot of types AssertJ has special assertion methods. Also for the type Optional. If we want to assert an Optional value we can use several methods that AssertJ provides. For example to check if an Optional is present we can use isPresent() or the alias isNotEmpty(). To check if the Optional is empty we can use isEmpty() or the alias isNotPresent(). Checking the value of an Optional (if it is indeed set) can be done with hasValue() or contains(). For more fine grained assertions on the value we can use hasValueSatisfying(Condition) or hasValueSatisfying(Consumer). With the map(Function) and flatMap(Function) methods we can map the Optional, if not empty, to another value and assert that value.

In the following example we see assertion methods for an Optional instance:

package mrhaki;

import org.assertj.core.api.Condition;
import org.assertj.core.api.InstanceOfAssertFactories;
import org.junit.jupiter.api.Test;

import java.util.Optional;

import static org.assertj.core.api.Assertions.assertThat;

class OptionalAssertions {
    
    @Test 
    void checkOptionalPresent(){
        assertThat(Optional.of("Awesome AssertJ"))
                // Check if Optional has a value.
                .isPresent()
                
                // Or we use the alias isNotEmpty().
                .isNotEmpty();
    }
    
    @Test 
    void checkOptionalNotPresent() {
        assertThat(Optional.empty())
                // Check if Optional is empty.
                .isEmpty()
                
                // Or we use the aliase isNotPresent().
                .isNotPresent();
    }
    
    @Test
    void checkValueOfOptional() {
        assertThat(Optional.of("Awesome AssertJ"))
                // We can check the value with hasValue(...).
                // We don't have to call get() ourselves.
                .hasValue("Awesome AssertJ")
                
                // Or we use the alias contains(...).
                .contains("Awesome AssertJ");
    }
    @Test
    void checkGetOfOptional() {
        // We can use get() to chain more assertions,
        // but we can only use generic Object assertions.
        assertThat(Optional.of("Awesome AssertJ"))
                .get()
                // We can only use Object assertions.
                .isEqualTo("Awesome AssertJ");

        // If we use get(InstanceOfAssertFactory) we get
        // back an object to use assertion methods for the 
        // given type.
        // The interface InstanceOfAssertFactories has a lot of 
        // useful constants to force the type returned by the Optional,
        // so we can use the assertions for that type.
        assertThat(Optional.of("Awesome AssertJ"))
                .get(InstanceOfAssertFactories.STRING)
                // Now we can use String assertions:
                .startsWith("Awesome")
                .endsWith("AssertJ");
    }

    @Test
    void checkValueSatisfyingOfConditional() {
        // We can use hasValueSatisfying(Consumer)
        // with the instance that is wrapped in the Optional
        // for more fine grained assertion.
        // Also, we can use assertions for the type wrapped
        // in the Optional.
        assertThat(Optional.of("Awesome AssertJ"))
                .hasValueSatisfying(s -> {
                    assertThat(s)
                            .startsWith("Awesome")
                            .endsWith("AssertJ");
                });
    }
    
    @Test
    void checkObjectValueOfOptionalWithCondition() {
        // Record to store properties of a framework.
        record Framework(String name, boolean awesome) {}

        // We can write a Condition that checks if the awesome
        // property is set to true.
        var isAwesomeFramework = new Condition<Framework>(Framework::awesome, "Framework is awesome!");

        // And with hasValueSatisfying we can use the Condition.
        assertThat(Optional.of(new Framework("AssertJ", true)))
                .hasValueSatisfying(isAwesomeFramework);
    }

    @Test
    void checkTransformedValueOfOptional() {
        // We can use map(...) to transform the value
        // of the optional (if present)
        // and write assertions for the transformed value.
        assertThat(Optional.of("Awesome AssertJ"))
                .map(String::toUpperCase)
                .hasValue("AWESOME ASSERTJ");

        // Record to store framework data with an optional type.
        record Framework(String name, Optional<Boolean> awesome) {}
        
        // With flatMap(...) we can transform the value
        // of the optional (if present) to another Optional
        // and write assertions for this transformed Optinal value.
        assertThat(Optional.of(new Framework("Awesome AssertJ", Optional.of(true))))
                .flatMap(Framework::awesome)
                .isPresent()
                .hasValue(true);
    }

    @Test
    void checkInstanceOfOptional() {
        // With containsInstanceOf we can write an assertion
        // to check the type that is contained in the Optional.
        assertThat(Optional.of("Awesome AssertJ"))
                .containsInstanceOf(String.class);
    }
}

Written with AssertJ 3.24.2.

Awesome AssertJ: Assertions For An URL Object

AssertJ has a lot of custom assertion methods for different types. For example to assert an URL object AssertJ gives us some specific methods. We can check for different components of the URL instance with different methods. For example we can check if the protocol is equal to the protocol we expect with hasProtocol(String). Similarly we can write assertions for the host, port, authority, path and anchor. To assert query parameters we can use hasQueryParameter(String) to check if query parameter is set and with hasQueryParameter(String, String) we can check if the query parameter has an expected value. To check the whole query string we can use hasQueryString(String).

Each of the assertion methods also has version to assert a component is not present. For example hasNoQuery() to assert a query is not defined for an URL.

In the following example we use the different assertion methods for an URL instance:

package mrhaki;

import org.junit.jupiter.api.Test;

import java.net.MalformedURLException;
import java.net.URL;

import static org.assertj.core.api.Assertions.assertThat;

class UrlChecks {

    @Test
    void checkUrl() throws MalformedURLException {
        assertThat(new URL("https://www.mrhaki.com:80/blogs/index.html#groovy-goodness"))

                // We can check the protocol.
                .hasProtocol("https")

                // We can check the host name.
                .hasHost("www.mrhaki.com")

                // We can check the port if it is part of the URL.
                .hasPort(80)

                // We can check the authority (host + port).
                .hasAuthority("www.mrhaki.com:80")

                // We can check the path part of the URL.
                .hasPath("/blogs/index.html")

                // We can check the anchor part.
                .hasAnchor("groovy-goodness");
    }

    @Test
    void checkQueryParameters() throws MalformedURLException {
        assertThat(new URL("https://www.mrhaki.com/blogs/?tag=Groovy&text=Goodness"))

                // We can check the full query string.
                .hasQuery("tag=Groovy&text=Goodness")

                // We can check if a query parameter is defined.
                .hasParameter("tag")
                .hasParameter("text")

                // We can check if a query parameter with a given value is set.
                .hasParameter("tag", "Groovy")
                .hasParameter("text", "Goodness")

                // We can verify with another URL after the query parameters are sorted.
                .isEqualToWithSortedQueryParameters(new URL("https://www.mrhaki.com/blogs/?text=Goodness&tag=Groovy"));
    }

    @Test
    void checkUserInfo() throws MalformedURLException {
        assertThat(new URL("https://user:password@www.mrhaki.com/"))

                // We can check the user info part of the URL.
                .hasUserInfo("user:password");
    }

    @Test
    void checkUrlNegated() throws MalformedURLException {
        // We can also check for the non-presence of components of the URL.
        assertThat(new URL("https://www.mrhaki.com"))
                .hasNoPort()
                .hasNoPath()
                .hasNoQuery()
                .hasNoParameters()
                .hasNoParameter("tag")
                .hasNoParameter("tag", "Groovy")
                .hasNoAnchor()
                .hasNoUserInfo();

        assertThat(new URL("file:///Users/mrhaki"))
                .hasNoHost();
    }
}

Written with AssertJ 3.24.2.

Awesome AssertJ: Check Base64 Encoded Strings

AssertJ has some nice methods to verify string values. If we want to verify a string value is Base64 encoded we can use the isBase64String() method. We can leave out the padding of the value as it is optional. With the method asBase64Decoded() we can decode the value and write our assertions for the decoded value. The method asBase64Decoded() returns a byte[] object and we can use the asString() to convert it into a string value again.

In the following example we use the isBase64() and asBase64Decoded() methods:

package mrhaki;

import org.junit.jupiter.api.Test;

import static org.assertj.core.api.Assertions.assertThat;
import static org.assertj.core.api.Assertions.assertThatExceptionOfType;

public class StringBase64 {

    @Test
    void checkValueIsBase64Encoded() {
        // We can simply check if the value is Base64 encoded
        // with the method isBase64().
        assertThat("QXNzZXJ0SiBiZSBBd2Vzb21lLCBtYXRleSE=").isBase64();

        // Padding (=) can be omitted from the Base64 encoded value.
        assertThat("QXNzZXJ0SiBiZSBBd2Vzb21lLCBtYXRleSE").isBase64();

        // If we want to check a value is NOT Base64 encoded we
        // need to check an AssertionError is thrown.
        assertThatExceptionOfType(AssertionError.class)
                .isThrownBy(() -> assertThat("AssertJ is Awesome").isBase64())
                .withMessageContaining("Expecting \"AssertJ is Awesome\" to be a valid Base64 encoded string");
    }

    @Test
    void checkBase64EncodedValue() {
        // We can use asBase64Decoded() to get a byte[] result
        // and apply our assertions.
        assertThat("QXNzZXJ0SiBiZSBBd2Vzb21lLCBtYXRleSE=")
                .asBase64Decoded().asString()
                .isEqualTo("AssertJ be Awesome, matey!");

        // Also here the padding (=) can be omitted.
        assertThat("QXNzZXJ0SiBiZSBBd2Vzb21lLCBtYXRleSE")
                .asBase64Decoded().asString()
                .isEqualTo("AssertJ be Awesome, matey!");

        assertThat("QXNzZXJ0SiBiZSBBd2Vzb21lLCBtYXRleSE=")
                .asBase64Decoded().asString()
                .isNotEqualTo("AssertJ is Awesome!");
    }
}

Written with AssertJ 3.24.2.

Awesome AssertJ: Use String Template To Verify String Value

To compare string values we can use the isEqualTo(String) method in AssertJ. But if we want to verify that a string contains a certain variable value we can use string templates. This makes the assertion more readable as we can see what value we expect in the string. To use string templates we must the method isEqualTo(String, Object…​). The first argument is the string template and the following arguments will be the actual values that should be used in the template. Actually the String.format(String, Object…​) method is used behind the scenes to format the string template, but we don’t have to clutter our assertions with that call.

In the following example we see how we can use string templates to verify string values:

package mrhaki;

import org.junit.jupiter.api.Test;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.ValueSource;

import static org.assertj.core.api.Assertions.assertThat;

public class StringIsEqualToTemplate {

    @Test
    void verifyStringIsEqualTo() {
        // We can use a template to define the format of the String.
        // If we do actually String.format() is used to create
        // our expected value.
        assertThat("AssertJ is awesome!").isEqualTo("%s is %s!", "AssertJ", "awesome");
    }

    // Using a template can be very useful for example in parameterized tests.
    @ParameterizedTest
    @ValueSource(strings = {"AssertJ", "Spock"})
    void verifyStringFormat(String libraryName) {
        // given
        record Library(String name) {
            @Override
            public String toString() {
                return name;
            }
        }
        var library = new Library(libraryName);

        // expect
        assertThat(library.name() + " is awesome").isEqualTo("%s is awesome", library);
    }

    @Test
    void verifyStringIsNotEqualTo() {
        // There is no method implementation for isNotEqualTo where we can define
        // the format as first argument. But we can still use String.format
        // ourselves to achieve the same functionality.
        assertThat("AssertJ is awesome!").isNotEqualTo(String.format("%s is %s!", "Spock", "awesome"));
    }
}

Written with AssertJ 3.24.2.