Differences between revisions 1 and 31 (spanning 30 versions)
Revision 1 as of 2003-08-09 12:47:24
Size: 957
Editor: modemcable046
Comment:
Revision 31 as of 2003-08-26 06:07:39
Size: 7358
Editor: p508E25B4
Comment:
Deletions are marked like this. Additions are marked like this.
Line 13: Line 13:
* determine disk usage patterns (ie. who're the hogs, what directories
  are biggest)
  * determine disk usage patterns (ie. which directories use up the most space, and which users
  are responsible for those directories)
  [[BR]][done: gward 2003/08/09]
Line 16: Line 17:
* browbeat disk space hogs into cleaning up   * browbeat top disk space users into cleaning up
  [[BR]][partly done: gward 2003/08/10; but some of the top users @python.net addresses bounce]
Line 18: Line 20:
* cleanup disks to prep for backup (DVD burn)
  * find out which users no longer have a valid @python.net email address, and then run the
  following loop:[[BR]]
  {{{
  for each user with no known forwarding address:
    if (user is "well-known" in the Python community) or
       (user has anything meaningful on starship):
      google to try to find a recent email address
      contact the user and ask:
        do they still need their starship account?
        what's their preferred email forwarding address?
        what's their SSH key (preferably SSHv2)?
    else:
      remove that user's account and all of their files
  }}}
  [[BR]][assigned: gward]
Line 23: Line 38:
* upgrade to Debian 'testing' (sarge)
  [done: sdrees 2003/08/07]
  * upgrade to Debian 'testing' (sarge)
  [[BR]][done: sdrees 2003/08/07]
Line 26: Line 41:
  * give the machine a proper hostname
  [[BR]](will have to be in python2.net domain for now; once DNS service is transferred to
  a new provider, we can give it a python.net name -- anything but starship.python.net,
  of course!) Proposed proper hostname: drydock.python2.net ;)
  [[BR]][done: sdrees 2003/08/10]


  * get email services working
  [[BR]](hmmm: testing this stuff will require have at least one real user on the system.
  I guess I'll create a temporary throwaway account rather than transfer real data over yet.)

    * install Exim 4 packages ('''critical path''')
    [[BR]](from unstable? or are they in testing yet?)
    [[BR]](tismer says: use exim-tls -- but is that necessary with the new exim4 packages?
    [[BR]][done: gward 2003/08/24; ended up getting 4.22-1 source package from unstable and
    building custom 4.22-gw1 packages]

    * configure/test Exim ('''critical path''')
    [[BR]][mostly done: gward 2003/08/24; still need to test with real users]

    * get elspy working with Debian Exim packages ('''critical path''')
    [[BR]](recompile necessary? or just fix elspy to work with dlopen() patch?)
    [[BR]][done: gward 2003/08/24]

    * install Courier ("semi-critical" for mwh--anyone else?)
    [[BR]](SSL daemon only) (IMAP only? or POP too?)
    [[BR]][assigned: gward]

    * configure/test Courier
    [[BR]][assigned: gward]

  * web services

    * install, configure, and test Apache ('''critical path''')
    [[BR]](with mod-proxy, mod-rewrite, and mod-ssl, please! Content data is already transfered by greg 2003/08/25)
    [[BR]][assigned: sdrees]

    * install, configure, and test Zope ('''not''' critical path)
    [[BR]](Zope 2.7 preferable; (really?) note that Debian testing has Zope 2.6.1 and a working plone on top ;) )
    [[BR]][assigned: sdrees]

  * install, configure, and test ProFTPd ('''critical path''')[[BR]][done: sdrees 2003/08/12]

  * install, configure, and test Mailman ('''critical path''')[[BR]]
  (it's installed, but not happy: the qrunner pegs at 99.9% CPU. Need to try the version from 'unstable',
  and then complain and/or figure out what's going wrong.)

  * Install Python and third party modules: [assigned: tbryan]
    * Motivation - One of Christian's goals in founding the Starship was to give developers a chance to play with the latest and greatest stuff and to show off Python's capabilities
    * History - We have built our own local copy of Python, with as many optional modules enabled as possible, since 1999.
    * Implementation - I was downloading the Python source, using Red Hat's RPM and our own custom Steup.local to build a custom binary RPM of Python for the starship
    * Third party modules - We have normally installed PIL, HTMLGen, and the mx* extensions for every Python release; we'd like to build more third party modules
    * Future - I'd like to reduce the amount of manual work involved in installing a new Python version. I'd like to tie in with the platform's package management system. I'd like to see nightly builds from CVS made available to crew members.
    * Plan - Install Python 2.2 and Python 2.3 with our normal third party modules as locally built software in /usr/local/. That will ensure some continuity in the short term when we move to the new Starship. After the move is complete, we can spend some time later to work out how we want to handle our local Python installations on a Debian machine. Then I'll remove the hand-built /usr/local/ installs and implement the new strategy.

  * compare /etc/{passwd,group,shadow} and friends on both ships, evaluate migration strategy of credential data ('''critical path''')[[BR]]user credential transfer will be greatly simplified by starship policy of using ssh keys for authentication. So the (shadowed) passwords will be a local secret ... greg wrote a little script already[[BR]][assigned: gward ;) ]




== Tasks for python.net DNS ==

  * find/select a good DNS provider (not critical path, but desirable)
  [[BR]](sdrees uses schlundtechnologies.com and has no complaints about it and has finalized transfer of python2.net to schlundtechnologies.com)
  [[BR]](gward uses dyndns.org and has no complaints about it)
  [[BR]](alternately, we could run the primary nameserver on starship, and use a commercial provider
  for the secondary nameserver)

  * transfer python.net domain to new DNS provider (not critical path, but desirable)
  [[BR]](currently the DNS servers are at zope.com/baymountain.com,
  and Starship admins cannot edit the python.net DNS records)
  [[BR]][assigned: tismer (he owns the domain!)]


== Detailed transfer plan ==

The transfer itself will start at 2003-08-xx hh:mm UTC.

  * disable logins on old starship
  * shut down Exim, Courier-IMAP on old starship
  * disable any other user processes that might update the disk
  (eg. CGI scripts that write state)
  * maybe: remount home read-only on old machine
  * rsync user directories (this assumes the main bulk transfer has
  already been done)
  * ensure ftpd running and working on new machine
  (need to change anything for name change?)
  * change canonical name of new machine to starship.python.net
  (including fixing reverse DNS resolution!)
  * edit Apache config for name change (drydock -> starship) and restart
  * edit Exim config for name change and restart
  * DNS change so "starship.python.net" resolves to the new machine
  * start Exim, Courier-IMAP on new machine
  * enable user logins on new machine






Starship Python is Moving!

On September 1, 2003, the machine that has been hosting Starship Python (courtesy of Zope Corporation) will be shutdown. Consequently, Starship is being transferred to a new machine, kindly provided by Stefan Drees. This page exists so the Starship admins can keep track of what tasks have to be done, what tasks have been done, and who's taking care of what. Also, the Starship user community can track the progress of the transfer here.

If you're a Starship admin, feel free to update this page. Otherwise, please email webmaster@python.net if you want to add stuff to our to-do list.

Tasks for old Starship

  • determine disk usage patterns (ie. which directories use up the most space, and which users are responsible for those directories)

    BR[done: gward 2003/08/09]

  • browbeat top disk space users into cleaning up

    BR[partly done: gward 2003/08/10; but some of the top users @python.net addresses bounce]

  • find out which users no longer have a valid @python.net email address, and then run the

    following loop:BR

      for each user with no known forwarding address:
        if (user is "well-known" in the Python community) or
           (user has anything meaningful on starship):
          google to try to find a recent email address
          contact the user and ask:
            do they still need their starship account?
            what's their preferred email forwarding address?
            what's their SSH key (preferably SSHv2)?
        else:
          remove that user's account and all of their files

    BR[assigned: gward]

Tasks for new Starship

  • upgrade to Debian 'testing' (sarge)

    BR[done: sdrees 2003/08/07]

  • give the machine a proper hostname

    BR(will have to be in python2.net domain for now; once DNS service is transferred to a new provider, we can give it a python.net name -- anything but starship.python.net, of course!) Proposed proper hostname: drydock.python2.net ;) BR[done: sdrees 2003/08/10]

  • get email services working

    BR(hmmm: testing this stuff will require have at least one real user on the system. I guess I'll create a temporary throwaway account rather than transfer real data over yet.)

    • install Exim 4 packages (critical path) BR(from unstable? or are they in testing yet?) BR(tismer says: use exim-tls -- but is that necessary with the new exim4 packages? BR[done: gward 2003/08/24; ended up getting 4.22-1 source package from unstable and building custom 4.22-gw1 packages]

    • configure/test Exim (critical path) BR[mostly done: gward 2003/08/24; still need to test with real users]

    • get elspy working with Debian Exim packages (critical path) BR(recompile necessary? or just fix elspy to work with dlopen() patch?) BR[done: gward 2003/08/24]

    • install Courier ("semi-critical" for mwh--anyone else?)

      BR(SSL daemon only) (IMAP only? or POP too?) BR[assigned: gward]

    • configure/test Courier

      BR[assigned: gward]

  • web services
    • install, configure, and test Apache (critical path) BR(with mod-proxy, mod-rewrite, and mod-ssl, please! Content data is already transfered by greg 2003/08/25) BR[assigned: sdrees]

    • install, configure, and test Zope (not critical path) BR(Zope 2.7 preferable; (really?) note that Debian testing has Zope 2.6.1 and a working plone on top ;) ) BR[assigned: sdrees]

  • install, configure, and test ProFTPd (critical path)BR[done: sdrees 2003/08/12]

  • install, configure, and test Mailman (critical path)BR (it's installed, but not happy: the qrunner pegs at 99.9% CPU. Need to try the version from 'unstable', and then complain and/or figure out what's going wrong.)

  • Install Python and third party modules: [assigned: tbryan]
    • Motivation - One of Christian's goals in founding the Starship was to give developers a chance to play with the latest and greatest stuff and to show off Python's capabilities
    • History - We have built our own local copy of Python, with as many optional modules enabled as possible, since 1999.
    • Implementation - I was downloading the Python source, using Red Hat's RPM and our own custom Steup.local to build a custom binary RPM of Python for the starship
    • Third party modules - We have normally installed PIL, HTMLGen, and the mx* extensions for every Python release; we'd like to build more third party modules
    • Future - I'd like to reduce the amount of manual work involved in installing a new Python version. I'd like to tie in with the platform's package management system. I'd like to see nightly builds from CVS made available to crew members.
    • Plan - Install Python 2.2 and Python 2.3 with our normal third party modules as locally built software in /usr/local/. That will ensure some continuity in the short term when we move to the new Starship. After the move is complete, we can spend some time later to work out how we want to handle our local Python installations on a Debian machine. Then I'll remove the hand-built /usr/local/ installs and implement the new strategy.
  • compare /etc/{passwd,group,shadow} and friends on both ships, evaluate migration strategy of credential data (critical path)BRuser credential transfer will be greatly simplified by starship policy of using ssh keys for authentication. So the (shadowed) passwords will be a local secret ... greg wrote a little script alreadyBR[assigned: gward ;) ]

Tasks for python.net DNS

  • find/select a good DNS provider (not critical path, but desirable)

    BR(sdrees uses schlundtechnologies.com and has no complaints about it and has finalized transfer of python2.net to schlundtechnologies.com) BR(gward uses dyndns.org and has no complaints about it) BR(alternately, we could run the primary nameserver on starship, and use a commercial provider for the secondary nameserver)

  • transfer python.net domain to new DNS provider (not critical path, but desirable)

    BR(currently the DNS servers are at zope.com/baymountain.com, and Starship admins cannot edit the python.net DNS records) BR[assigned: tismer (he owns the domain!)]

Detailed transfer plan

The transfer itself will start at 2003-08-xx hh:mm UTC.

  • disable logins on old starship
  • shut down Exim, Courier-IMAP on old starship
  • disable any other user processes that might update the disk (eg. CGI scripts that write state)
  • maybe: remount home read-only on old machine
  • rsync user directories (this assumes the main bulk transfer has already been done)
  • ensure ftpd running and working on new machine (need to change anything for name change?)
  • change canonical name of new machine to starship.python.net (including fixing reverse DNS resolution!)
  • edit Apache config for name change (drydock -> starship) and restart

  • edit Exim config for name change and restart
  • DNS change so "starship.python.net" resolves to the new machine
  • start Exim, Courier-IMAP on new machine
  • enable user logins on new machine

StarshipTransfer (last edited 2008-11-15 14:00:08 by localhost)

Unable to edit the page? See the FrontPage for instructions.