This week, using my new AMQP and HTTP taskcluster go client api library (see recent posts), I’ve written a service, running in heroku, to capture metrics for task cluster.

Five task cluster pulse queues are monitored to watch for completed tasks and completed task graphs. Then the taskcluster queue API is queried to get task data, such as worker payloads, in order to build up a view of durations taken for the tasks to progress through states until completion. This data is then collected, for pushing to influxdb for real-time query analysis. InfluxDB is similar to graphite.

In parallel, the hg pushlogs are monitored to get push times for hg changesets, so that we have a view in the end of how long it took from a change being received on the hg server, until a task graph is build, then until it is executed, and then until it completes.

The publish to InfluxDB is not quite done yet, hope to finish that off shortly.

The data is persisted using BoltDB during data collection and denormalisation.

I plan to wrap that up shortly, and then start work on …. the generic worker!

See bug 1136537 for more details.

Code lives here: https://github.com/petemoore/tc-gecko-metrics-go

It is auto-deployed to heroku using the https://github.com/kr/heroku-buildpack-go buildpack after a push to github.

http://petemoore.tumblr.com/post/112697565668