Search

Dark theme | Light theme
Showing posts with label GradleGoodness:Advanced. Show all posts
Showing posts with label GradleGoodness:Advanced. Show all posts

April 21, 2017

Gradle Goodness: Using Incremental Task Action

Gradle has incremental build support to speed up our builds. This means Gradle checks input and output for a task and if something changed the task is executed, otherwise the task is skipped. In previous posts we learned how to add incremental build support to our tasks with annotations and inputs and outputs property of a task. When we have a task that has an output file for an input file, like with transformations, we can have a more efficient task using an incremental task action. With an incremental task action we have extra information on the files that are handled by the task. We can have different actions based on if an input file is out of date or removed. This way we can handle only the input files that have changed or removed with incremental builds, instead of all the input files.

To create an incremental task action we must have a task action method (annotated with @TaskAction) that has a single argument of type IncrementalTaskInputs. The IncrementalTaskInputs class has the method outOfDate and removed. These methods take an action, that can be implemented with a closure, with an instance of InputFileDetails as argument. We can get to the input file via this instance and use that for our task logic. When an input file is out of date, because the file contents has changed or the output file has been removed, the action we defined for the outOfDate method is invoked. If the input file is removed the action for the method removed is invoked.

In the following example we have a task HtmlConverter with a task action that support incremental builds for input files. When an input file has changed it is processed, otherwise it is skipped. In the build file we create the task convert that uses the HtmlConverter task class:

// Create task to convert text to HTML.
task convert(type: HtmlConverter) {
    sourceDir = file('src/docs/text')
    outputDir = file("${buildDir}/html")
}

import groovy.xml.MarkupBuilder
import groovy.transform.CompileStatic
import groovy.transform.CompileDynamic

/**
 * Simple Gradle task that takes an text
 * input file and converts it to a HTML file.
 */
@CompileStatic
class HtmlConverter extends DefaultTask {

    @InputDirectory
    @PathSensitive(PathSensitivity.RELATIVE)
    File sourceDir

    @OutputDirectory
    File outputDir

    /**
     * Task action that will check if a source file is out of date
     * or removed. If out of date the source file is converted
     * to HTML. If source file is removed the generated 
     * HTML file is removed.
     *
     * @param inputs Used for incremental task action.
     */
    @TaskAction
    void convert(IncrementalTaskInputs inputs) {
        // If the user for example used --rerun-tasks
        // this task is not incremental. Only
        // inputs.outOfDate is executed, so we must first 
        // remove all output files.
        if (!inputs.incremental) {
            project.delete(outputDir.listFiles())
        }

        // Input file has changed, so we convert it.
        inputs.outOfDate { InputFileDetails outOfDate ->
            convertFile(outOfDate.file)
        }

        // Input file is removed, so we remove the
        // output file that was created for the input file.
        inputs.removed { InputFileDetails removed ->
            removeOutputFile(removed.file)
        }
    }

    /**
     * Convert text file to HTML.
     *
     * @param file Text file to convert to HTML.
     */
    private void convertFile(final File file) {
        logger.lifecycle 'Convert file {}', file.name
        final lines = file.readLines()
        final outputWriter = new FileWriter(new File(outputDir, outputFilename(file)))
        writeHtml(lines, outputWriter)
    }

    /**
     * Use first line as title for HTML, rest is body.
     *
     * @param lines Lines to transform to HTML.
     * @param writer Writer to write HTML to.
     */
    @CompileDynamic
    private void writeHtml(final List lines, final Writer writer) {
        final html = new MarkupBuilder(writer)
        html.html {
            head {
                title lines[0]
            }
            body {
                lines[2..-1].each { line ->
                    p line
                }
            }
        }
    }

    /**
     * Remove the output file thas was created for the
     * given input file.
     *
     * @param file Input file to remove output file for.
     */
    private void removeOutputFile(final File file) {
        logger.lifecycle 'Remove HTML for file {}', file.name
        new File(outputDir, outputFilename(file)).delete()
    }

    /**
     * Determine HTML output filename based on base of input filename.
     *
     * @param file Used to create HTML output file name.
     */
    private String outputFilename(final File file) {
        file.name[0..file.name.lastIndexOf('.')] + 'html'
    }

}

In our project we have 3 source files in the directory src/docs/text: sample1.txt, sample2.txt and hello.txt. We run the convert task for the first time and we see all input files are processed:

$ gradle convert
:convert
Convert file hello.txt
Convert file sample1.txt
Convert file sample2.txt

BUILD SUCCESSFUL

Total time: 0.948 secs

Next we change hello.txt and when re-run the task we see only our changed file is processed. If we rename it after the change we can see the hello.html is removed and the new file is processed:

$ echo "Gradle rocks" >> src/docs/text/hello.txt
$ gradle convert
:convert
Convert file hello.txt

BUILD SUCCESSFUL

Total time: 0.793 secs
$ mv src/docs/text/hello.txt src/docs/text/sample.txt
$ gradle convert
:convert
Convert file sample.txt
Remove HTML for file hello.txt

BUILD SUCCESSFUL

Total time: 0.76 secs
$

Written with Gradle 3.5.

February 19, 2016

Gradle Goodness: Using Nested Domain Object Containers

In a previous post we learned how to use the NamedDomainObjectContainer class. We could create new objects using a nice DSL in Gradle. But what if we want to use DSL syntax to create objects within objects? We can use the same mechanism to achieve this by nesting NamedDomainObjectContainer objects.

We want to support the following DSL to create a collection of Server objects, where each server can have multiple Node objects:

// File: build.gradle
apply plugin: com.mrhaki.gradle.DeploymentPlugin

deployments {
    aws {
        url = 'http://aws.address'

        nodes {
            node1 {
                port = 9000
            }
            node2 {
                port = 80
            }
        }
    }

    cf {
        url = 'http://cf.address'

        nodes {
            test {
                port = 10001
            }
            acceptanceTest {
                port = 10002
            }
        }
    }
}

This would create two Server objects with then names aws and cf. Each server has Node objects with names like node1, node2, test and acceptanceTest. Let's look at the Server class where we have added a nodes property of type NamedDomainObjectContainer as the nested object container. Also notice the nodes method so we can use the DSL syntax to create Node objects.

// File: buildSrc/src/main/groovy/com/mrhaki/gradle/Server.groovy
package com.mrhaki.gradle

import org.gradle.api.NamedDomainObjectContainer

class Server {

    /**
     * An instance is created in the plugin class, because
     * there we have access to the container() method
     * of the Project object.
     */
    NamedDomainObjectContainer<Node> nodes

    String url

    String name

    /**
     * We need this constructor so Gradle can create an instance
     * from the DSL.
     */
    Server(String name) {
        this.name = name
    }

    /**
     * Inside the DSL this method is invoked. We use
     * the configure method of the NamedDomainObjectContainer to
     * automatically create Node instances.
     * Notice this is a method not a property assignment.
     * <pre>
     * server1 {
     *     url = 'http://server1'
     *     nodes { // This is the nodes() method we define here.
     *         port = 9000
     *     }
     * }
     * </pre>
     */
    def nodes(final Closure configureClosure) {
        nodes.configure(configureClosure)
    }

}

And the Node class:

// File: buildSrc/src/main/groovy/com/mrhaki/gradle/Node.groovy
package com.mrhaki.gradle

class Node {

    String name

    Integer port

    /**
      * We need this constructor so Gradle can create an instance
      * from the DSL.
      */
    Node(String name) {
        this.name = name
    }
}

To make the DSL work we use a custom plugin so we can add the DSL for creating the objects to our project:

// File: buildSrc/src/main/groovy/com/mrhaki/gradle/DeploymentPlugin.groovy
package com.mrhaki.gradle

import org.gradle.api.Project
import org.gradle.api.Plugin
import org.gradle.api.NamedDomainObjectContainer

class DeploymentPlugin implements Plugin<Project> {

    public static final String EXTENSION_NAME = 'deployments'

    private static final String DEPLOY_TASK_PATTERN = 'deployOn%sTo%s'

    private static final String REPORTING_TASK_NAME = 'reportDeployments'

    private static final String TASK_GROUP_NAME = 'Deployment'

    void apply(final Project project) {
        setupExtension(project)
        createDeploymentTasks(project)
        createReportTask(project)
    }

    /**
     * Create extension on the project for handling the deployments
     * definition DSL with servers and nodes. This allows the following DSL
     * in our build script:
     * <pre>
     * deployments {
     *     server1 {
     *         url = 'http://server'
     *         nodes {
     *             node1 {
     *                 port = 9000
     *             }
     *         }
     *     }
     * }
     * </pre>
     */
    private void setupExtension(final Project project) {

        // Create NamedDomainObjectContainer for Server objects.
        // We must use the container() method of the Project class
        // to create an instance. New Server instances are
        // automatically created, because we have String argument
        // constructor that will get the name we use in the DSL.
        final NamedDomainObjectContainer<Server> servers =
            project.container(Server)

        servers.all {
            // Here we have access to the project object, so we
            // can use the container() method to create a
            // NamedDomainObjectContainer for Node objects.
            nodes = project.container(Node)
        }

        // Use deployments as name in the build script to define
        // servers and nodes.
        project.extensions.add(EXTENSION_NAME, servers)
    }

    /**
     * Create a new deployment task for each node.
     */
    private void createDeploymentTasks(final Project project) {
        def servers = project.extensions.getByName(EXTENSION_NAME)

        servers.all {
            // Actual Server instance is the delegate
            // of this closure. We assign it to a variable
            // so we can use it again inside the
            // closure for nodes.all() method.
            def serverInfo = delegate

            nodes.all {
                // Assign this closure's delegate to
                // variable so we can use it in the task
                // configuration closure.
                def nodeInfo = delegate

                // Make node and server names pretty
                // for use in task name.
                def taskName =
                    String.format(
                        DEPLOY_TASK_PATTERN,
                        name.capitalize(),
                        serverInfo.name.capitalize())

                // Create new task for this node.
                project.task(taskName, type: DeploymentTask) {
                    description = "Deploy to '${nodeInfo.name}' on '${serverInfo.name}'"
                    group = TASK_GROUP_NAME

                    server = serverInfo
                    node = nodeInfo
                }
            }
        }
    }

    /**
     * Add reporting task to project.
     */
    private void createReportTask(final Project project) {
        project.task(REPORTING_TASK_NAME, type: DeploymentReportTask) {
            description = 'Show configuration of servers and nodes'
            group = TASK_GROUP_NAME
        }
    }
}

We also have two custom tasks that use the Server and Node instances that are created by the DSL in our build file. The DeploymentTask is configured from the plugin where the server and node properties are set:

// File: buildSrc/src/main/groovy/com/mrhaki/gradle/DeploymentTask.groovy
package com.mrhaki.gradle

import org.gradle.api.tasks.TaskAction
import org.gradle.api.DefaultTask

class DeploymentTask extends DefaultTask {

    Server server

    Node node

    /**
     * Simple implementation to show we can
     * access the Server and Node instances created
     * from the DSL.
     */
    @TaskAction
    void deploy() {
        println "Deploying to ${server.url}:${node.port}"
    }

}

The DeploymentReportTask references the project extensions to get a hold of the Server and Node objects:

// File: buildSrc/src/main/groovy/com/mrhaki/gradle/DeploymentReportTask.groovy
package com.mrhaki.gradle

import org.gradle.api.tasks.TaskAction
import org.gradle.api.DefaultTask

class DeploymentReportTask extends DefaultTask {

    /**
     * Simple task to show we can access the
     * Server and Node instances also via the
     * project extension.
     */
    @TaskAction
    void report() {
        def servers = project.extensions.getByName(DeploymentPlugin.EXTENSION_NAME)

        servers.all {
            println "Server '${name}' with url '${url}':"

            nodes.all {
                println "\tNode '${name}' using port ${port}"
            }
        }
    }

}

Let's run the tasks task first to see which tasks are added by the plugin. Next we invoke some tasks:

$ gradle -q tasks
...
Deployment tasks
----------------
deployOnAcceptanceTestToCf - Deploy to 'acceptanceTest' on 'cf'
deployOnNode1ToAws - Deploy to 'node1' on 'aws'
deployOnNode2ToAws - Deploy to 'node2' on 'aws'
deployOnTestToCf - Deploy to 'test' on 'cf'
reportDeployments - Show configuration of servers and nodes
...
$ gradle -q deployOnNode2ToAws

Deploying to http://aws.address:80
$ gradle -q reportDeployments

Server 'aws' with url 'http://aws.address':
 Node 'node1' using port 9000
 Node 'node2' using port 80
Server 'cf' with url 'http://cf.address':
 Node 'acceptanceTest' using port 10002
 Node 'test' using port 10001
$

Written with Gradle 2.11.

Gradle Goodness: Create Objects Using DSL With Domain Object Containers

Gradle offers the NamedDomainObjectContainer class to create a collection of objects defined using a clean DSL. The only requirement for the objects we want to create is that they have a constructor that takes a String argument and a name property to identify the object. The value for the name property must be unique within the collection of objects. We create a new instance of a NamedDomainObjectContainer with the container method of the Gradle Project class. We can add the NamedDomainObjectContainer instance to the extensions property of our project, so we can use a DSL to create instances of objects that need to be in the NamedDomainObjectContainer object in our project.

The following code shows a simple build script in which we want to create a collection of Product objects. The creation of the NamedDomainObjectContainer object is done in a plugin so we only have to apply the plugin to use the DSL to create Product objects:

apply plugin: ProductsPlugin

// DSL to define new objects of type Product.
products {
    // Create Product with name pencil.
    pencil {
        price = 0.05
    }
    // Create Product with name crayon.
    crayon {
        price = 0.18
    }
}


class ProductsPlugin implements Plugin<Project> {
    void apply(final Project project) {
        // Create NamedDomainObjectContainer instance for
        // a collection of Product objects
        NamedDomainObjectContainer<Product> productContainer =
            project.container(Product)

        // Add the container instance to our project
        // with the name products.
        project.extensions.add('products', productContainer)

        // Simple task to show the Product objects
        // that are created by Gradle using
        // the DSL syntax in our build file.
        project.task('reportProducts') << {
            def products = project.extensions.getByName('products')

            products.all {
                // A Product instance is the delegate
                // for this closure.
                println "$name costs $price"
            }
        }
    }
}

class Product {
    // We need a name property
    // so the object can be created
    // by Gradle using a DSL.
    String name

    BigDecimal price

    Product(final String name) {
        this.name = name
    }
}

We can run the reportProducts task to see the name and price properties of the Product instances:

$ gradle -q reportProducts
crayon costs 0.18
pencil costs 0.05
$

Written with Gradle 2.11.