In this blog post we learn how we can use the Google App Engine SDK API with Gaelyk and on the side we learn some Groovy syntax suggar. In a previous blog post we learned how easy it is to write and deploy a Groovlet with Gaelyk. We are going to extend this Groovlet with caching of the Yahoo! Pipe results. Each time a user requests the Groovlet, also the Yahoo! Pipe is requested. The results are transformed to a HTML page. Since we are using the URLFetchService from the Google App Engine, each request has a cost. Luckely we can make a lot of invocations on the URLFetchService, but still we have to take into account the number of invocations is limited.
To reduce the number of invocations on the URLFetchService and to speed things up, we will cache the Yahoo! Pipe results with the Google App Engine MemCacheService. Because the MemCacheService is part of the Google App Engine SDK API we are also limited to a number of invocations. But this number is much bigger than for the URLFetchService, so we can serve much more requests before we hit the limit.
Using the MemCacheService is very easy: the service is already injected into the Groovlet by Gaelyk and is available as memcacheService. The service itself is basically a hashmap we can store data in and retrieve data from. To save the Yahoo! Pipe results we only have to invoke the put(key, value) method. A downside of the Yahoo! Pipe request is that we don't get any response headers like Last-Modified or ETag. Normally we could have used this headers to determine if the results have changed and if we had to make a new request, or that we could use the cached value. But because we don't get this information we use a different caching mechanism: we store the results for five minutes in the cache. When a user requests the Groovlet again after five minutes we refresh the cache and make a new request for the Yahoo! Pipe. In a next blog post we see how we can add response headers Last-Modified and ETag to our Groovlet. Clients can use this information to use cached results if necessary. This will lessen the number of invocations on the URLFetchService and MemCacheService and increase the response time of the Groovlet.
Here is the code of the rewritten Groovlet with caching of the Yahoo! Pipe results:
import java.net.URL
import java.text.SimpleDateFormat
import java.text.ParseException
import org.codehaus.groovy.runtime.TimeCategory
import net.sf.json.groovy.JsonSlurper
import net.sf.json.JSONException
import com.ocpsoft.pretty.time.PrettyTime
/*
* Groovy Category to extend the Date and String class with
* the methods prettyTime and parseActivityDate.
* We use the <a href="http://ocpsoft.com/prettytime/">PrettyTime</a>
* library to get results like 9 minutes ago, 3 days ago.
*/
class ActivityDateTimeCategory {
static String prettyTime(Date date) {
new PrettyTime().format(date)
}
static Date parseActivityDate(String dateString) {
def result
// Remove the colon from the timezone, because otherwise we can't
// parse the date.
dateString = dateString.replaceAll(/(\d{2}):(\d{2})$/, '$1$2')
// The date formats of the different sources. We try each one
// to parse a date.
def parsers = [
new SimpleDateFormat("EEE, dd MMM yyyy HH:mm:ss Z", Locale.US),
new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss'Z'", Locale.US),
new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss.SSSZ", Locale.US)
]
for (SimpleDateFormat parser : parsers) {
try {
result = parser.parse(dateString)
break
} catch (ParseException e) {
continue
}
}
result
}
}
// If the request parameter refresh is set we always
// refresh the cache with the latest Yahoo! Pipe result.
refresh = params.refresh
// The number of minutes before we refresh the cache.
period = params.period ? Integer.valueOf(params.period) : 5
now = new Date()
lastUpdated = application.getAttribute('lastUpdated')
if (!lastUpdated) {
lastUpdated = now
application.setAttribute('lastUpdated', lastUpdated)
refresh = true
}
// Using the TimeCategory so we can use the cool ?.minutes syntax.
use (TimeCategory) {
checkDate = lastUpdated + period.minutes
if (now.after(checkDate)) {
lastUpdated = now
application.setAttribute('lastUpdated', lastUpdated)
refresh = true
}
}
def resultString
if (!refresh) {
resultString = memcacheService.get('result')
}
// If we have to refresh, or the resultString from the cache is null
// we make a new request.
if (!resultString) {
def pipeUrl = 'http://pipes.yahoo.com/pipes/pipe.run?_id=UtVVPkx83hGZ2wKUKX1_0w&_render=json'
def result = urlFetchService.fetch(new URL(pipeUrl))
resultString = new String(result.content)
memcacheService.put('result', resultString)
}
html.html {
head {
style (
"""
img {
width: 32px; height: 32px; float: left; padding: 4px 4px 0 0;
}
.item {
border-bottom: 1px solid #999;
padding: 5px;
}
#activities {
border-top: 1px solid #999;
}
"""
)
}
body {
h1 "Activities"
if (resultString) {
try {
def jsonReader = new JsonSlurper()
def json = jsonReader.parseText(resultString)
div(id: 'activities') {
json.value.items.each { item ->
div(class: 'item') {
img(src: 'http://localhost/images/socialicons/32x32/' + item.source + '.png', width: 32, height: 32, alt: item.source, title: item.source)
a(href: item.link, item.title)
br()
// Using our own Category and showing of ?. Groovy safe operator.
use (ActivityDateTimeCategory) {
span(class: 'date', item.pubDate?.parseActivityDate()?.prettyTime());
}
}
}
}
} catch (JSONException jsonException) {
p jsonException.getMessage()
pre resultString
}
} else {
p 'No results found. Try again later.'
}
}
}
