All posts by bigjools

Feeling productive?

As the lead on the Launchpad Soyuz project, I have to deal with a lot of different tasks at the same time.  I hate this as much as the next engineer, but burying my head in the sand and hoping things will just be all right simply won’t work.

At the end of the day I was feeling really unproductive and anxious about time wasted – I’d sit back and think, what did I do today?  I felt really busy but I can’t remember what the heck I’ve been doing! Even worse, I’d leave my office at the end of the day in a bad mood and the 30 seconds it takes to get back into the house is not enough to ease the tensions, which made it hard to interact with my wife and kids.  I want to reclaim that precious hour between finishing work and the kids going to bed!

So, what can I do about this?

Three things:

1. I made a conscious effort to finish my major work items 30 minutes before leaving the office and to spend those 30 minutes doing all the simple tasks that you know you need to sort out but never get around to doing.  This has had a dramatic affect on not only how I feel when I finish, but also I get the warm fuzzy feeling of having cleared out stuff that was niggling me at the back of my brain.

2. I have eschewed technology in favour of the classic pen and paper!  I used to keep TODO lists electronically, but I found this hard to keep up with as they were always hidden by other windows on another virtual desktop somewhere.  So now,  each week I start on a fresh piece of paper and write down all the things I know I need to do, with a little box next to them.  When I’ve done that task, I can put a great big satisfying tick next to it.  At the end of each day I can see visible progress.  This is very uplifting!  I start a new sheet each week so that the TODO list is not cluttered, and it also forces me to think of anything I missed.

3. Take regular breaks.  I know, this is easier said than done.  Michael Nelson put me on to the Pomodoro Technique which while I don’t follow religiously, does give some effective tips on how to time-box your activities.

If you’ve got any more tips, or used mine and found them useful, I’d love to hear about it!  Please leave a comment.

Success in Wellington

One of the  things that Launchpad is working on is the daily builds project, where we take a source branch, a recipe and turn it into a source package that can be built.

It turns out that this is quite hard because you need a secure environment to run the Bazaar recipe builder in.  Fortunately we already have the Launchpad Build Farm!  So, on January 11th the Soyuz and Codehosting teams, plus Launchpad Strategist Jono Lange and community contributor William Grant all met at the West Plaza Hotel in Wellington, New Zealand for a week-long coding sprint with the goal of getting a recipe build job through the build farm before the week was out.

The team hard at work

The amount of work was pretty sizeable, but we’d already done quite a lot of preparatory work before leaving so that non-Soyuz developers could get involved from day one.

We split the task into a few areas:

  • Writing the code that runs on the build slave itself
  • Job dispatching
  • Job collection
  • Job candidate selection
  • Creating a generalised build infrastructure to make it easy to plug in new job types

The team split into coding pairs that mostly lasted the week and there were no bloodied noses!  We had a strict rule that no pair could contain two people from the same Launchpad team, so that we could spread the knowledge around better.  This worked out really well.

The local coffee shop certainly had plenty of custom that week!  But by the end of it, on Friday afternoon at about 5:45pm, we finally got a recipe build job to run through a local Launchpad instance on William’s laptop and eventually turned into a binary package.  Success!

We’ll be building on this work in the next couple of months so that it has a UI that will allow a user to click on a branch and get it built and uploaded to their PPA.  It’s all very cool!

Finally, huge thanks to everyone involved, including the hotel.  It was the best wifi/internet anyone had ever had at any hotel, ever.

Bjorn, William and Michael
The team celebrates after a long week

Soyuz 3.0 review

The Soyuz team was very, very busy over the last year fixing lots of bugs and adding plenty of new features.  These are the highlights that I’d like to mention!

New features

  • Multiple PPAs per person – split your packages into different repositories without the hassle of creating new LP users
  • Karma for uploads! Distro and PPA uploaders (and the package creator if different) will be recognised for their work and get karma.
  • Massively improved webservice APIs to control various operations, such as copying packages, manipulating builds, inspecting PPAs etc.  Allows script-based control of many soyuz-related operations.
  • Hugely faster build farm scanner, builds are dispatched a LOT quicker now.  That means there’s less waiting around for your packages to get built.
  • Private PPA subscriptions / tokens.  People can now control who is able to download their PPA software.
  • Package sets and their upload ACLs implemented for Ubuntu.  Karmic and onwards will be using package sets for upload permissioning.
  • Security in Soyuz. Complete support for the Canonical security team to use a private PPA and directly unembargo packages from it

3.0 UI

We also did a lot of complete page redesigns for 3.0:

  • PPA page split into two pages; one user-focused, one developer-focused
  • New build page, with a cleaner look
  • New global /builders page; sortable table data and a time-based estimate of the queue sizes
  • New distribution source package release page; much better presentation of data
  • New distribution source package page; makes more use of upstream package description/logo etc.
  • Distro series source package release page gets a new layout
  • Builder +admin and builder +edit collapsed into a single page
  • New builder page

But the work doesn’t stop here.  We’re already thinking about 4.0!

Soyuz template conversions all done!

Thanks to the hard work from everyone in the Soyuz team, the Soyuz effort to convert all our templates to the new 3.0 UI standard is all done!  If you’re a beta tester, the new pages can be seen on on the edge service already.

Many of the changes were complete page re-designs based on better knowledge of use-cases that has been gathered over the years.  There was certainly a lot of dust on some of them!

There are still some minor tweaks to be made, but all feedback is most welcome.

A foray into virtual machines

One of the things that’s frustrating about being a Launchpad Developer is waiting three hours for the test suite to complete.  It’s always irked me that my quad-core sat there with three idle cores, and parallelising the tests would be tough, so I thought I’d try and partition the tests across some virtual machines.

I know pretty much nothing about VMs so I was pretty pleased to come across this JeOS VMBuilder resource!  I got my first VM built pretty quick with it, but then, what next?  The instructions don’t tell me how to actually run up my new VM.

So, I had a little chuckle to myself when I saw this blog post from Dustin Kirkland just appear!  I’m going to try his suggestions out now and see how it works out, but it looks like just what I need.

Build retrying and rescoring via the webservice

I recently added functionality to Launchpad so that you can retry and re-score builds via the webservice API.  This was the number one requested API feature from PPA users and Ubuntu packagers alike!

It’s trivial to use:

>>> ubuntu = launchpad.distributions['ubuntu']
>>> main_archive = ubuntu.main_archive
>>> sources = main_archive.getPublishedSources(source_name='my_package_name')
>>> builds = sources[0].getBuilds()
>>> for build in builds:
...     build.rescore(score=1000)
...     build.retry()

`rescore` requires that you have buildd admin privileges.   `retry` simply requires that you have permission to upload the source.

A more advanced example is to fetch all builds that are failed in a series:

>>> karmic = ubuntu.getSeries(name_or_version="karmic")
>>> failed_states = (
...     'Failed to build',
...     'Dependency wait',
...     'Chroot problem',
...     'Failed to upload',
...     )
>>> builds = []
>>> for build_state in failed_states:
...     builds.extend(
...         karmic.getBuildRecords(build_state=build_state))

This will potentially return builds that are superseded so they need to be filtered to see if the build’s source is published.  This is something we’re going to fix soon, so that you can specify that you only want builds with published sources in the getBuildRecords() call.