Gradle Spoon Console Plugin

As my blog followers know, I've been trying to get a Groovy Console into Pentaho Data Integration's (PDI's) Spoon UI for quite a while now.  I haven't put it into the Marketplace as we're wrestling with Groovy versions and I wanted to offer something that worked out-of-the-box versus something that needed extra setup/config.  For PDI 5.1 this should be a moot point...

...then I realized, I've been trying to bring the Groovy Console to PDI.  Why not bring PDI to the Groovy Console?  This idea was borne from a blog post by Mike Hugo with a tip of the hat to carlosgsouza who created a Gradle plugin from it.

The idea of the post (and thus mine) is that you can leverage Gradle to fetch your dependencies and keep track of your needed classpath, then start a Groovy Console with that classpath passed in. Thus you can access your project's dependencies (APIs, etc.) via the console, allowing for prototyping, rapid development, etc. of features.

For PDI, my GroovyConsoleSpoonPlugin project already had the necessary requirements: it had PDI/Kettle dependencies, it leveraged the Groovy Console, and it used Groovy to provide an internal Domain-Specific Language (DSL) for interacting with the PDI environment. After fixing a few bugs in my project related to working with the Groovy Console outside of Spoon, I'm happy to announce that the first version of the Gradle Console for PDI is ready to roll :)

To use, get Gradle and then just clone my GitHub project and run "gradle console" (or if you don't want/have Gradle, go to your clone and use "gradlew console"). You will be presented with a Groovy Console with PDI core dependencies, steps, etc. available, as well as my first attempt at an internal DSL for building transformations and jobs (see earlier blog posts). To change the PDI version, just edit the build.gradle file looking for the "project.ext.kettle_dependency_revision" property (yes, I know that's deprecated ;)

Here's an example of creating and running a (VERY!) basic transformation. I hope to implement a Builder soon to fill in metadata, etc. :

I'm hoping to make this DSL (and the whole experience) as easy yet powerful as possible, please let me know your ideas!

PDI memcached plugins

I've definitely been neglecting this blog :) but I've recently put a couple of plugins into the PDI marketplace to read and write from memcached instances. I hope this leads to more key/value store support (I'm working on Hazelcast plugins right now) for PDI.

Anyway the Memcached Input/Output plugins allow for the reading and writing of key/value pairs to memcached instances.  The Input step requires incoming key names, and the type of the values that will come out:

The Output step requires and value field names (it interrogates the types from the row metadata):

This is just an initial offering, eventually I'd like to add other features (I'm always open to suggestions for improvement!). But I figured I should get a foot in the door for key/value store support :)  I also have been working on a Map (key/value) type for PDI, that should be out in 5.1 or before if I can get some supporting changes into the 5.0.x code line.

Please feel free to install this from the PDI Marketplace and let me know how it works for you and/or how it could improve :) I only tested this on PDI 5.0, so I'm not sure it will work on PDI 4.x but I don't see any reason it shouldn't.

Cheers!

Groovy Memcached "client"

Ok so this post is not PDI related (yet, stay tuned :) but in my search for easy memcached client UIs I came up fairly shorthanded unless I wanted to buy something, write a Java client, or install a package.  All I needed to do for my test was to be able to set and get a couple of values and I didn't really want to start a new project, build a Java app, etc.

There's a very easy way to do this with Groovy Grapes and the spymemcached client (and the Groovy Console as a "UI"). Just @Grab the spymemcached JAR, connect a client, and off you go:

The script is as simple as this:

@Grab('net.spy:spymemcached:2.10.5')

import net.spy.memcached.MemcachedClient

def memcachedClient = new MemcachedClient( new InetSocketAddress('127.0.0.1', 11211 ) );

memcachedClient.set('myKey',3600, "Hello world!")

memcachedClient.set('intKey',3600, 45)

println "myKey = ${memcachedClient.get('myKey')}"

println "intKey = ${memcachedClient.get('intKey')}"

It's pretty easy to change this to a command-line interface (CLI) to take parameters for keys, expiration times, etc. Note I'm using the synchronous gets/sets but this is just for testing.

Anyway this came about while writing my Memcached Input/Output step plugins for PDI, keep your eye on the Marketplace, I hope to have them released this week. Cheers!