One of the  things that Launchpad is working on is the daily builds project, where we take a source branch, a recipe and turn it into a source package that can be built.

It turns out that this is quite hard because you need a secure environment to run the Bazaar recipe builder in.  Fortunately we already have the Launchpad Build Farm!  So, on January 11th the Soyuz and Codehosting teams, plus Launchpad Strategist Jono Lange and community contributor William Grant all met at the West Plaza Hotel in Wellington, New Zealand for a week-long coding sprint with the goal of getting a recipe build job through the build farm before the week was out.

The amount of work was pretty sizeable, but we’d already done quite a lot of preparatory work before leaving so that non-Soyuz developers could get involved from day one.

We split the task into a few areas:

• Writing the code that runs on the build slave itself
• Job dispatching
• Job collection
• Job candidate selection
• Creating a generalised build infrastructure to make it easy to plug in new job types

The team split into coding pairs that mostly lasted the week and there were no bloodied noses!  We had a strict rule that no pair could contain two people from the same Launchpad team, so that we could spread the knowledge around better.  This worked out really well.

The local coffee shop certainly had plenty of custom that week!  But by the end of it, on Friday afternoon at about 5:45pm, we finally got a recipe build job to run through a local Launchpad instance on William’s laptop and eventually turned into a binary package.  Success!

We’ll be building on this work in the next couple of months so that it has a UI that will allow a user to click on a branch and get it built and uploaded to their PPA.  It’s all very cool!

Finally, huge thanks to everyone involved, including the hotel.  It was the best wifi/internet anyone had ever had at any hotel, ever.

The Soyuz team was very, very busy over the last year fixing lots of bugs and adding plenty of new features.  These are the highlights that I’d like to mention!

New features

• Multiple PPAs per person – split your packages into different repositories without the hassle of creating new LP users
• Karma for uploads! Distro and PPA uploaders (and the package creator if different) will be recognised for their work and get karma.
• Massively improved webservice APIs to control various operations, such as copying packages, manipulating builds, inspecting PPAs etc.  Allows script-based control of many soyuz-related operations.
• Hugely faster build farm scanner, builds are dispatched a LOT quicker now.  That means there’s less waiting around for your packages to get built.
• Private PPA subscriptions / tokens.  People can now control who is able to download their PPA software.
• Package sets and their upload ACLs implemented for Ubuntu.  Karmic and onwards will be using package sets for upload permissioning.
• Security in Soyuz. Complete support for the Canonical security team to use a private PPA and directly unembargo packages from it

3.0 UI

We also did a lot of complete page redesigns for 3.0:

• PPA page split into two pages; one user-focused, one developer-focused
• New build page, with a cleaner look
• New global /builders page; sortable table data and a time-based estimate of the queue sizes
• New distribution source package release page; much better presentation of data
• New distribution source package page; makes more use of upstream package description/logo etc.
• Distro series source package release page gets a new layout
• Builder +admin and builder +edit collapsed into a single page
• New builder page

But the work doesn’t stop here.  We’re already thinking about 4.0!

Thanks to the hard work from everyone in the Soyuz team, the Soyuz effort to convert all our templates to the new 3.0 UI standard is all done!  If you’re a beta tester, the new pages can be seen on on the edge service already.

Many of the changes were complete page re-designs based on better knowledge of use-cases that has been gathered over the years.  There was certainly a lot of dust on some of them!

There are still some minor tweaks to be made, but all feedback is most welcome.

I recently added functionality to Launchpad so that you can retry and re-score builds via the webservice API.  This was the number one requested API feature from PPA users and Ubuntu packagers alike!

It’s trivial to use:

>>> ubuntu = launchpad.distributions['ubuntu']
>>> main_archive = ubuntu.main_archive
>>> sources = main_archive.getPublishedSources(source_name='my_package_name')
>>> builds = sources[0].getBuilds()
>>> for build in builds:
...     build.rescore(score=1000)
...     build.retry()

`rescore` requires that you have buildd admin privileges.   `retry` simply requires that you have permission to upload the source.

A more advanced example is to fetch all builds that are failed in a series:

>>> karmic = ubuntu.getSeries(name_or_version="karmic")
>>> failed_states = (
...     'Failed to build',
...     'Dependency wait',
...     'Chroot problem',
...     'Failed to upload',
...     )
>>> builds = []
>>> for build_state in failed_states:
...     builds.extend(
...         karmic.getBuildRecords(build_state=build_state))

This will potentially return builds that are superseded so they need to be filtered to see if the build’s source is published.  This is something we’re going to fix soon, so that you can specify that you only want builds with published sources in the getBuildRecords() call.