Groovy Goodness: Redirecting Print Methods In Scripts

To run external Groovy scripts in our Java or Groovy application is easy to do. For example we can use GroovyShell to evaluate Groovy code in our applications. If our script contains print methods like println we can redirect the output of these methods. The Script class, which is a base class to run script code, has an implementation for the print, printf and println methods. The implementation of the method is to look for a property out, either as part of a Script subclass or in the binding added to a Script class. If the property out is available than all calls to print, printf and println methods are delegated to the object assigned to the out property. When we use a PrintWriter instance we have such an object, but we could also write our own class with an implementation for the print methods. Without an assignment to the out property the fallback is to print on System.out.

In the following example we have a external script defined with the variable scriptText, but it could also be a file or other source with the contents of the script we want to run. We assign our own PrintWriter that encapsulates a StringWriter to capture all invocations to the print methods:

// Groovy script to execute.
def scriptText = '''
def s = "Groovy rocks!"

// Print value of s.
println s

// Use printf for formatted printing.
printf 'The answer is %X', 42
'''

// Assign new PrintWriter to "out"
// variable of binding object.
def stringWriter = new StringWriter()
def shellBinding = new Binding(out: new PrintWriter(stringWriter))

// Create GroovyShell to evaluate script.
def shell = new GroovyShell(shellBinding)

// Run the script.
shell.evaluate(scriptText)

// Check the output of print, println and printf methods.
assert stringWriter.toString() == 'Groovy rocks!\nThe answer is 2A'

Another option is to directory set the out property of a Script object:

def scriptText = '''
def s = "Groovy rocks!"

// Print value of s.
println s

// Use printf for formatted printing.
printf 'The answer is %X', 42
'''

def shell = new GroovyShell()

// Parse script text and return Script object.
def script = shell.parse(scriptText)

// Assign new PrintWriter to "out"
// variable of Script class.
def stringWriter = new StringWriter()
script.out = new PrintWriter(stringWriter)

// Run the script.
script.run()

// Check the output of print, println and printf methods.
assert stringWriter.toString() == 'Groovy rocks!\nThe answer is 2A'

Written with Groovy 2.4.10.

Groovy Goodness: BaseScript with Abstract Run Script Method

In a previous blog post we have seen how we can use a BaseScript AST transformation to set a base script class for running scripts. Since Groovy 2.3 we can apply the @BaseScript annotation on package and import statements. Also we can implement a run method in our Script class in which we call an abstract method. The abstract method will actually run the script, so we can execute code before and after the script code runs by implementing logic in the run method.

In the following sample we create a Script class CustomScript. We implement the run method and add the abstract method runCode:

// File: CustomScript.groovy
package com.mrhaki.groovy.blog

abstract class CustomScript extends Script {

    def run() {
        before()
        try {

            // Run actually script code.
            final result = runCode()

            println "Script says $result"
        } finally {
            println 'Script ended'
        }
    }

    private void before() {
        println 'Script starts'
    }

    // Abstract method as placeholder for
    // the actual script code to run.
    abstract def runCode()
}

Next we create a Groovy script where we use our new CustomScript class.

// File: Sample.groovy
// Since Groovy 2.3 we can apply the
// @BaseScript annotation on package
// and import statement.
@groovy.transform.BaseScript(com.mrhaki.groovy.blog.CustomScript)
package com.mrhaki.groovy.blog

// Script code:
final String value = 'Groovy rules'
assert value.size() == 12

// Return value
value

When we run our script we see the following output:

Before script runs
Script says Groovy rules
Script ended

Code written with Groovy 2.3.

Groovy Goodness: Define Compilation Customizers With Builder Syntax

Since Groovy 2.1 we can use a nice builder syntax to define customizers for a CompileConfiguration instance. We must use the static withConfig method of the class CompilerCustomizationBuilder in the package org.codehaus.groovy.control.customizers.builder. We pass a closure with the code to define and register the customizers. For all the different customizers like ImportCustomizer, SecureASTCustomizers and ASTTransformationCustomizer there is a nice compact syntax.

In the following sample we use this builder syntax to define different customizers for a CompileConfiguration instance:

package com.mrhaki.blog

import org.codehaus.groovy.control.customizers.ASTTransformationCustomizer
import org.codehaus.groovy.control.CompilerConfiguration
import org.codehaus.groovy.control.customizers.builder.CompilerCustomizationBuilder
import groovy.transform.*


def conf = new CompilerConfiguration()

// Define CompilerConfiguration using 
// builder syntax.
CompilerCustomizationBuilder.withConfig(conf) {
    ast(TupleConstructor)
    ast(ToString, includeNames: true, includePackage: false)
    
    imports {
        alias 'Inet', 'java.net.URL'
    }
    
    secureAst {
        methodDefinitionAllowed = false
    }
}


def shell = new GroovyShell(conf)
shell.evaluate '''
package com.mrhaki.blog

class User {
    String username, fullname
}

// TupleConstructor is added.
def user = new User('mrhaki', 'Hubert A. Klein Ikkink')

// toString() added by ToString transformation.
assert user.toString() == 'User(username:mrhaki, fullname:Hubert A. Klein Ikkink)'

// Use alias import.
def site = new Inet('http://www.mrhaki.com/')
assert site.text
'''

Code written with Groovy 2.2.2.

Groovy Goodness: Restricting Script Syntax With SecureASTCustomizer

Running Groovy scripts with GroovyShell is easy. We can for example incorporate a Domain Specific Language (DSL) in our application where the DSL is expressed in Groovy code and executed by GroovyShell. To limit the constructs that can be used in the DSL (which is Groovy code) we can apply a SecureASTCustomizer to the GroovyShell configuration. With the SecureASTCustomizer the Abstract Syntax Tree (AST) is inspected, we cannot define runtime checks here. We can for example disallow the definition of closures and methods in the DSL script. Or we can limit the tokens to be used to just a plus or minus token. To have even more control we can implement the StatementChecker and ExpressionChecker interface to determine if a specific statement or expression is allowed or not.

In the following sample we first use the properties of the SecureASTCustomizer class to define what is possible and not within the script:

package com.mrhaki.blog

import org.codehaus.groovy.control.customizers.SecureASTCustomizer
import org.codehaus.groovy.control.CompilerConfiguration
import org.codehaus.groovy.ast.stmt.*
import org.codehaus.groovy.ast.expr.*
import org.codehaus.groovy.control.MultipleCompilationErrorsException

import static org.codehaus.groovy.syntax.Types.PLUS
import static org.codehaus.groovy.syntax.Types.MINUS
import static org.codehaus.groovy.syntax.Types.EQUAL

// Define SecureASTCustomizer to limit allowed
// language syntax in scripts.
final SecureASTCustomizer astCustomizer = new SecureASTCustomizer(
    // Do not allow method creation.
    methodDefinitionAllowed: false,

    // Do not allow closure creation.
    closuresAllowed: false,

    // No package allowed.
    packageAllowed: false,

    // White or blacklists for imports.
    importsBlacklist: ['java.util.Date'],
    // or importsWhitelist
    staticImportsWhitelist: [],
    // or staticImportBlacklist
    staticStarImportsWhitelist: [],
    // or staticStarImportsBlacklist

    // Make sure indirect imports are restricted.
    indirectImportCheckEnabled: true,

    // Only allow plus and minus tokens.
    tokensWhitelist: [PLUS, MINUS, EQUAL],
    // or tokensBlacklist

    // Disallow constant types.
    constantTypesClassesWhiteList: [Integer, Object, String],
    // or constantTypesWhiteList
    // or constantTypesBlackList
    // or constantTypesClassesBlackList
    
    // Restrict method calls to whitelisted classes.
    // receiversClassesWhiteList: [],
    // or receiversWhiteList
    // or receiversClassesBlackList
    // or receiversBlackList

    // Ignore certain language statement by
    // whitelisting or blacklisting them.
    statementsBlacklist: [IfStatement],
    // or statementsWhitelist

    // Ignore certain language expressions by
    // whitelisting or blacklisting them.
    expressionsBlacklist: [MethodCallExpression]
    // or expresionsWhitelist
)

// Add SecureASTCustomizer to configuration for shell.
final conf = new CompilerConfiguration()
conf.addCompilationCustomizers(astCustomizer)

// Create shell with given configuration.
final shell = new GroovyShell(conf)

// All valid script.
final result = shell.evaluate '''
def s1 = 'Groovy'
def s2 = 'rocks'
"$s1 $s2!"
'''

assert result == 'Groovy rocks!'

// Some invalid scripts.
try {
    // Importing [java.util.Date] is not allowed
    shell.evaluate '''
    new Date() 
    '''
} catch (MultipleCompilationErrorsException e) {
    assert e.message.contains('Indirect import checks prevents usage of expression')
}


try {
    // MethodCallExpression not allowed
    shell.evaluate '''
    println "Groovy rocks!" 
    '''
} catch (MultipleCompilationErrorsException e) {
    assert e.message.contains('MethodCallExpressions are not allowed: this.println(Groovy rocks!)')
}

To have more fine-grained control on which statements and expression are allowed we can implement the StatementChecker and ExpressionChecker interfaces. These interfaces have one method isAuthorized with a boolean return type. We return true if a statement or expression is allowed and false if not.

package com.mrhaki.blog

import org.codehaus.groovy.control.customizers.SecureASTCustomizer
import org.codehaus.groovy.control.CompilerConfiguration
import org.codehaus.groovy.ast.stmt.*
import org.codehaus.groovy.ast.expr.*
import org.codehaus.groovy.control.MultipleCompilationErrorsException

import static org.codehaus.groovy.control.customizers.SecureASTCustomizer.ExpressionChecker
import static org.codehaus.groovy.control.customizers.SecureASTCustomizer.StatementChecker


// Define SecureASTCustomizer.
final SecureASTCustomizer astCustomizer = new SecureASTCustomizer()

// Define expression checker to deny 
// usage of variable names with length of 1.
def smallVariableNames = { expr ->
    if (expr instanceof VariableExpression) {
        expr.variable.size() > 1
    } else {
        true
    }
} as ExpressionChecker

astCustomizer.addExpressionCheckers smallVariableNames


// In for loops the collection name
// can only be 'names'.
def forCollectionNames = { statement ->
    if (statement instanceof ForStatement) {
        statement.collectionExpression.variable == 'names'
    } else {    
        true
    }
} as StatementChecker

astCustomizer.addStatementCheckers forCollectionNames


// Add SecureASTCustomizer to configuration for shell.
final CompilerConfiguration conf = new CompilerConfiguration()
conf.addCompilationCustomizers(astCustomizer)

// Create shell with given configuration.
final GroovyShell shell = new GroovyShell(conf)

// All valid script.
final result = shell.evaluate '''
def names = ['Groovy', 'Grails']
for (name in names) {
    print "$name rocks! " 
}

def s1 = 'Groovy'
def s2 = 'rocks'
"$s1 $s2!"
'''

assert result == 'Groovy rocks!'

// Some invalid scripts.
try {
    // Variable s has length 1, which is not allowed.
    shell.evaluate '''
    def s = 'Groovy rocks'
    s
    '''
} catch (MultipleCompilationErrorsException e) {
    assert e.message.contains('Expression [VariableExpression] is not allowed: s')
}


try {
    // Only names as collection expression is allowed.
    shell.evaluate '''
    def languages = ['Groovy', 'Grails']
    for (name in languages) {
        println "$name rocks!" 
    }
    '''
} catch (MultipleCompilationErrorsException e) {
    assert e.message.contains('Statement [ForStatement] is not allowed')
}

Code written with Groovy 2.2.2.

Groovy Goodness: Running Scripts from a JAR Archive

We can run Groovy scripts from the command-line with the groovy command followed by the name of Groovy script file. We could already run script with the http: protocol as described in this blog post. Since Groovy 2.1 we can even run scripts that are packaged in an archive with the jar: protocol.

Let's create first a simple Groovy script and package it in a sample.jar file.

// File: cookies.groovy

println 'Running from inside a JAR: '
def cookies = ['Cookie', 'Biscuit'].collect { it.toUpperCase() }.join(',')
println cookies

We can use the jar command to place the file cookies.groovy in sample.jar:

$ jar cvf sample.jar cookies.groovy
added manifest
adding: cookies.groovy(in = 153) (out= 126)(deflated 17%)
$ jar tvf sample.jar
     0 Fri Sep 06 14:50:24 CEST 2013 META-INF/
    68 Fri Sep 06 14:50:24 CEST 2013 META-INF/MANIFEST.MF
   153 Fri Sep 06 14:48:44 CEST 2013 cookies.groovy
$

We now have a single Groovy script, but we can place more scripts inside the JAR file if we want to. To run a specific Groovy script file we must use the jar: protocol. We must specify the location of the JAR file with the file: or http: protocols followed by !/. In some shells the ! symbol is a special character and we might need to escape it. For example on Mac OSX we need to invoke the following command to run our script placed in the JAR file:

$ groovy jar:file:sample.jar'!'/cookies.groovy
Running from inside a JAR:
COOKIE,BISCUIT

We can place the JAR file on a web server and use the jar:http://<address>/sample.jar!/cookies.groovy syntax to run the scripts remotely.

Code written with Groovy 2.1.6

Groovy Goodness: Using Groovy for Git Hooks

Git supports hooks, which are scripts that are fired when certain events happens. The scripts are simply shell scripts and we can use Groovy to run those scripts. We must make Groovy the script language with the hash-bang (#!) header in the Git hook script file. And then we are ready to go and use Groovy as the script language for the Git hooks.

Git hooks are placed in the .git/hooks directory of our project. We create an example script that will use growlnotify to create a notification message with information from the Git commit action. growlnotify is a command-line tool for Mac OSX to send out messages to Growl. Other operating systems also have tools to create notification message from the command-line.

We must create the file post-commit in the directory .git/hooks. The file must have execute rights: $ chmod +x post-commit. In the first line of the file we make sure Groovy is used. In the rest of the script we use Groovy to invoke git log and get information about the commit. Then we create a message and invoke growlnotify with the correct arguments so Growl can show the message.

#!/usr/bin/env groovy

// Arguments for git log command.
def logArgs = ['max-count': '1', 'pretty': 'format:%an commited %s {%h}']

// Invoke git log command.
def gitLog = logArgs.inject(['git', 'log']) { cmd, k, v -> 
    cmd << "--$k=$v" 
}.execute()

// Get git log message to be used as notification message.
def message = gitLog.text

// Set icon and title for message.
def iconPath = '/Users/mrhaki/Pictures/git-icon-black.png'
def title = 'Git commit'

// Notify user of commit with growlnotify.
def notifyArgs = [message: message, title: title, image: iconPath]
notifyArgs.inject(['growlnotify']) { cmd, k, v ->
    cmd << "--$k" << v
}.execute()

In the following screenshot we see the output shown with Growl:

Some script hook files take an argument. We can use the argument in our Groovy code and write code to handle it. In the following example we check if the commit message is empty. If the message is empty we return with an exit code that is not 0. Git will terminate the commit until the script return 0. We write a simple implementation of the commit-msg hook:

#!/usr/bin/env groovy

import static java.lang.System.*

// First argument is the name of the 
// temporary commit message file.
def msgFileName = args[0]

// Get the commit message file.
def msgFile = new File(msgFileName)

// Read commit message from file.
def commitMessage = msgFile.text

if (!commitMessage) {
    err.println 'Commit message is empty'
    exit 1
}

exit 0

(Written with Groovy 2.0.4 and Git 1.7.5.4)

Groovy Goodness: Variable Scope in Scripts

In Groovy scripts we can define variables with a type or with the def keyword. Or we can leave out any type definition or def keyword. This effects the scope of the variable. If we want to use the variable in a method we must not define it with a type or def. This way the variable is added to the script binding and can be accessed in a method.

String s = 'I am in local scope.'
def i = 42
counter = 0

def runIt() {
    try {
        assert 'I am in local scope.' == s
    } catch (e) {
        assert e instanceof MissingPropertyException
    }
    try {
        assert 42 == i
    } catch (e) {
        assert e instanceof MissingPropertyException
    }
    assert 0 == counter++
}

runIt()

assert 'I am in local scope.' == s
assert 42 == i
assert 1 == counter