This week I started on the generic worker. Work under development here:

https://github.com/petemoore/generic-worker

So far, I’m able to publish tasks to the queue with a fake provisionerId, and then the generic worker can discover the task, and claim it. It is not yet running any tasks, so there is still some way to go.

It can currently handle the prioritisation workflow, and detect numerous failure conditions. It also receives all the task data in a structured format which it unmarshals into go types as a starting point for running a task.

First pass will assume good behaviour from a task, that it writes files only under a specified filepath top-level work directory. The worker will take care of cleaning up this directory path, and terminating processes.

http://petemoore.tumblr.com/post/113342091558