Revision 58 as of 2012-03-26 08:32:38

Clear message

This page contains general information about the Python BuildBot setup and issues.

Here are the current BuildBot results. For command-line querying of buidbot status (instead of the aforementioned Web page), you can use the bbreport tool.

We need to add more platforms (and compilers!) to the BuildBot farm. If you have one of these machines you are willing to run BuildBot on, please contact python-dev@python.org:

It might also be beneficial to run on older versions of operating systems or others not mentioned above. Feel free to contact us if you would like to offer another type of system.

Installing a buildslave

You need to install a recent version of Buildbot. If your OS has a packaging system (e.g. under Linux), it is probably available from the standard repositories. Otherwise, you'll have to build it from source (in which case you'll have to install Python and Twisted first).

A buildslave also needs the subversion client, and it needs to be able to build Python and some of its extension modules. It means you should have the standard development tools installed (compiler, linker), and the development headers for a couple of third-party libraries (such as zlib and OpenSSL).

Once all this is done, create a new user "buildbot" if it doesn't exist (your package manager might have done it for you). Then:

 % su - buildbot
 % mkdir buildarea
 % buildslave create-slave `pwd`/buildarea dinsdale.python.org:9020 slavename slavepasswd

If you have an older version of buildbot installed, the command will be "buildbot" instead of "buildslave". If this is the case, you should upgrade to a newer version of buildbot. If you can't find either, you should check to see that /usr/local/bin (the default installed location) is on your path.

You'll need to get someone to create the slavename/slavepasswd on dinsdale.python.org before doing this. Ask on python-dev@python.org once you're ready.

Then edit buildarea/info/admin and buildarea/info/host to set them appropriately.

Finally, start buildbot (still under the 'buildbot' user) with:

 % buildslave start ~/buildarea

Once the buildbot is running, don't forget to monitor the build results and solve any setup issues causing test failures.

Required ports

In order for the buildbot to operate properly, it needs access to various TCP ports on the internet. Below is a list of known ports and hosts; this list is not exhaustive and may evolve as new tests get written.

Port

Host

Description

20, 21

ftp.kernel.org, ftp.mirror.nl

test_urllib2net

53

your DNS server

test_socket (and others implicitly)

80

python.org

(several tests)

119

news.gmane.org

test_nntplib

443

(various)

test_ssl

465

smtp.gmail.com

test_smtpnet

9020

python.org

Buildbot connection

Many tests will also create local TCP sockets and connect to them (usually through "localhost" or "127.0.0.1"), so make sure this capability hasn't been disabled.

The following is a summary of a discussion that happened on python-dev regarding Buildbot security. No consensus came about, but the information is still helpful as a point of reference.

Required Resources

Based on [2]_, the recommended resource allocations for a Python buildbot are at least:

* 2 CPUs * 512 MB RAM * 30 GB free disk space

Note: the bigmem tests won't run in this configuration, since they require substantially more memory. However, these resources should be sufficient to ensure Python compiles correctly on the platform and can run the rest of test suite.

.. [2]: http://mail.python.org/pipermail/python-dev/2012-March/117978.html

Build Slaves and Security

Here are some important points of which you should be aware when running a build slave [1]_:

1. Anyone setting up a build slave should take care to invoke the build

2. Code runs differently as privileged vs. unprivileged users.

3. Access to root may provide access to slightly surprising resources,

.. [1] See the related python-dev thread <http://mail.python.org/pipermail/python-dev/2011-October/113935.html>_.

Unable to edit the page? See the FrontPage for instructions.