7981
Comment:
|
6311
markup fix
|
Deletions are marked like this. | Additions are marked like this. |
Line 7: | Line 7: |
* a guideline for implementing a PyPI-compatible server. | |
Line 12: | Line 11: |
Python package distribution since Python 2.4. Together with Distutils | Python package distribution since Python 2.4. Together with Distutils |
Line 14: | Line 13: |
releasing their packages in eggs, and are making them avaible in PyPI. | releasing their packages in eggs, and are making them available in PyPI. |
Line 16: | Line 15: |
Web frameworks such as Plone and the underlying framework, are now entirely distributed in eggs, and use PyPI as the main place to publish them. Some tools like zc.buildout automates the build of |
Web frameworks such as Plone and the underlying framework, are now entirely distributed in eggs, and use PyPI as the main place to publish them. Some tools like {{{zc.buildout}}} automate the build of |
Line 21: | Line 20: |
This central approach is great for Python advocacy, and for the cohesion of the community of developers, but brings a few issues: |
This central approach is great for Python advocacy, and for the cohesion of the community of developers, but brings a few issues. The ones addressed in this document are: |
Line 25: | Line 25: |
* when people from a sub-community, like Plone, want to promote their work, they need to shout it out in several places, to make sure it has reach people focused on the given framework. These places are acting a bit like the PyPI, so a common releasing standard make thing easier to automate. | * when people from a sub-community, like Plone, want to promote their work, they need to shout it out in several places, to make sure it has reach people focused on the given framework. These places are acting a bit like the PyPI, so a common releasing standard make things easier to automate. |
Line 27: | Line 27: |
This document provides a solution to transparently support several index servers. It is a very low risk and very simple to implement. It is also |
This document provides a solution to transparently support several index servers. It is a very low risk and very simple to implement. It is also |
Line 36: | Line 36: |
* making .pypirc supports multiple servers | * making {{{.pypirc}}} support multiple servers |
Line 38: | Line 38: |
* providing a base layer for PyPI-like servers implementation | |
Line 40: | Line 39: |
= Making .pypirc supports multiple servers = | = Making .pypirc support multiple servers = |
Line 42: | Line 41: |
Right now, the .pypirc file is intended to keep username and password | Right now, the {{{.pypirc}}} file is intended to keep username and password |
Line 49: | Line 48: |
Line 52: | Line 50: |
The default repository is PyPI, and when another repository has to be used, there are two options. |
The default repository is PyPI, and when another repository has to be used, there are two options. |
Line 56: | Line 54: |
Line 62: | Line 59: |
Either adding it in the .pypirc file: | or adding it in the {{{.pypirc}}} file: |
Line 71: | Line 68: |
In both cases, it is not possible when your username differs from a server to another, to keep a username/password for each server. Furthermore the realm |
In both cases, if your username differs from a server to another, it is not possible to keep a username/password for each server. Furthermore the realm |
Line 77: | Line 74: |
A simple way to enhance it, is to be able to add several sections in .pypirc. | A simple way to enhance it, is to be able to add several sections in {{{.pypirc}}}. The root section would be the {{{[distutils]}}} section, with a list of sections that represent a server. |
Line 79: | Line 78: |
For instance: | For example: |
Line 82: | Line 81: |
[server-login] | [distutils] index-servers = pypi my-other-server my-other-server-with-its-own-realm [pypi] repository:http://pypi.python.org/pypi/ |
Line 85: | Line 91: |
Line 98: | Line 105: |
This will allow backward-compatibility, and make register and upload able to interact with several servers when they are called. |
|
Line 101: | Line 106: |
== How sections in .pypirc are used == | When a user calls the {{{register}}} or the {{{upload}}} command, it will use the default server located in the {{{pypi}}} section, or the server given by the {{{-r}}} option: |
Line 103: | Line 110: |
When a user call the register or the upload command with such a file, it will loop through every section, and ask the used at the prompt if he wants to perform the action over the given server. For example: |
|
Line 110: | Line 112: |
$ python setup.py register sdist upload running mregister running egg_info ... Using PyPI login from /Users/tziade/.pypirc Do you want to register the package metadata at http://www.python.org/pypi (y/N)? n Do you want to register the package metadata at http://example.com/repository (y/N)? y Server at http://example.com/repository response (200): OK Do you want to register the package metadata at http://example2.com/repository (y/N)? n running sdist ... running upload Using PyPI login from /Users/tziade/.pypirc Do you want to upload the package at http://pypi.python.org/pypi (y/N)? n Do you want to upload the package at http://example.com/repository (y/N)? y Submitting dist/iw.releaser-0.1.tar.gz to http://example.com/repository Server response (200): OK Do you want to upload the package at http://example2.com/repository (y/N)? n }}} |
$ python setup.py register sdist upload -r http://example.com/repository # registering and uploading at example.com $ python setup.py register sdist upload # registering and uploading at PyPI }}} the {{{-r}}} option will also accept the name of the section: {{{ $ python setup.py register sdist upload -r my-other-server # registering and uploading at example.com $ python setup.py register sdist upload -r pypi # registering and uploading at PyPI }}} Default values in this file will be: * {{{distutils:index-servers}}} : {{{"pypi"}}} * {{{pypi:repository}}} : {{{http://pypi.python.org/pypi/}}} |
Line 131: | Line 129: |
XXX to be completed if if accepted, until then, iw.dist is a working prototype available at the cheeseshop. | Backward compatibility will be kept, and a file that uses the old format: |
Line 133: | Line 131: |
see: http://www.nabble.com/enhancing-.pypirc-file-for-multiple-servers-handling-to14504516.html#a14527503 | {{{ [server-login] username:tarek2 password:secret }}} ...will be translated as: {{{ [distutils] index-servers = pypi [pypi] username:tarek2 password:secret }}} |
Line 164: | Line 179: |
$ python setup.py register Do you want to register at PyPI (y/N) ? y |
$ python setup.py register # registering at PyPI |
Line 170: | Line 184: |
Do you want to register at ACME (y/N) ? y ... |
$ python setup.py register -r http://example.com/repository # registering at example.com |
Line 179: | Line 193: |
XXX code the patch for PyPI code if accepted = Providing a base layer for PyPI-like servers implementation = The PyPI current implementation is done in a cgi-like Python application, that works with a Postgres database. For example, when a package is uploaded, the request's form is checked, then the data are pushed into the database through an SQL query. Another server could have another storage strategy. So extracting an abstract layer from PyPI implementation could facilitate implementing any server. A module could be added to provide: * an abstract class for the storage, that works with a backend. * an abstract class for the API the web server has to provide. This module could be named interfaces.py and contain the exposed API. In this example, zope.interface is used to define these abstract class:: {{{ from zope.interface import Interface class IPyPIServer(Interface): """Interface for the web server""" def file_upload(response=True): """uploads a file""" def list_classifiers(): """returns list of classifiers""" ... more details ... class IPyPIStorage(Interface): """Defines a storage""" def get_classifiers(): """returns the classifiers""" ... more details ... }}} Then each server could implement these interfaces. With such a layer, it would facilitate the standardization of all PyPI-like servers, that could all used the same base. XXX to be completed if accepted |
Abstract
This document provides:
- a list of enhancements to make Distutils packaging system support several index servers;
- changes to be made in the central server index implementation, a.k.a PyPI;
Rationale
The PyPI is playing a very important role in the standardization of Python package distribution since Python 2.4. Together with Distutils and the third-party library Setuptools, most Python programmers are releasing their packages in eggs, and are making them available in PyPI.
Web frameworks such as Plone and the underlying framework, are now entirely distributed in eggs, and use PyPI as the main place to publish them. Some tools like zc.buildout automate the build of Python application by looking for eggs in PyPI and downloading them.
This central approach is great for Python advocacy, and for the cohesion of the community of developers, but brings a few issues. The ones addressed in this document are:
- when PyPI is unreachable, developers are unable to build their application, unless they have a local PyPI cache;
- when people from a sub-community, like Plone, want to promote their work, they need to shout it out in several places, to make sure it has reach people focused on the given framework. These places are acting a bit like the PyPI, so a common releasing standard make things easier to automate.
This document provides a solution to transparently support several index servers. It is a very low risk and very simple to implement. It is also finalizing a feature that was primarily intended by Distutils and PyPI, but not easy to use as-is: making it a central server, but allowing people to register and upload their package elsewhere.
The rest of the document presents the actions to take, and the work to be done for it:
making .pypirc support multiple servers
- making PyPI permissive for Trove classification
Making .pypirc support multiple servers
Right now, the .pypirc file is intended to keep username and password for registering a package. The file looks like
[server-login] username:tarek password:secret
The default repository is PyPI, and when another repository has to be used, there are two options.
Either adding the repository url in the command line:
$ python setup.py register -r http://example.com/repository
or adding it in the .pypirc file:
[server-login] username:tarek2 password:secret repository:http://example.com/repository
In both cases, if your username differs from a server to another, it is not possible to keep a username/password for each server. Furthermore the realm associated with the server is hardcoded to "pypi".
Several sections in .pypirc
A simple way to enhance it, is to be able to add several sections in .pypirc. The root section would be the [distutils] section, with a list of sections that represent a server.
For example:
[distutils] index-servers = pypi my-other-server my-other-server-with-its-own-realm [pypi] repository:http://pypi.python.org/pypi/ username:tarek2 password:secret [my-other-server] username:tarek2 password:secret repository:http://example.com/repository [my-other-server-with-its-own-realm] username:tarek3 password:secret3 repository:http://example2.com/repository realm:acme
When a user calls the register or the upload command, it will use the default server located in the pypi section, or the server given by the -r option:
$ python setup.py register sdist upload -r http://example.com/repository # registering and uploading at example.com $ python setup.py register sdist upload # registering and uploading at PyPI
the -r option will also accept the name of the section:
$ python setup.py register sdist upload -r my-other-server # registering and uploading at example.com $ python setup.py register sdist upload -r pypi # registering and uploading at PyPI
Default values in this file will be:
distutils:index-servers : "pypi"
pypi:repository : http://pypi.python.org/pypi/
Backward compatibility will be kept, and a file that uses the old format:
[server-login] username:tarek2 password:secret
...will be translated as:
[distutils] index-servers = pypi [pypi] username:tarek2 password:secret
Making PyPI permissive for Trove classification
PyPI is based on a Trove classification, see http://www.python.org/dev/peps/pep-0301/#distutils-trove-classification. Another server may have its own trove classification though, that differs from PyPI. This is intended because the server can provide a "package center" that has its own domain-specific categories.
For example, a package might have this classification:
Development Status :: 5 - Production/Stable Intended Audience :: Developers
But a slightly different one in another server. Let's say, the ACME company:
ACME :: Visibility :: Public Development Status :: 5 - Production/Stable Intended Audience :: Developers
A permissive Trove classification would allow the registering of the package in both servers, even if the categories does not exist. For each unkown category a warning is popped at the prompt. The unknown category is not created on the server: each server keeps its own classifiers.
When registering it to several servers, the expected output would be::
$ python setup.py register # registering at PyPI ... Warning "ACME :: Visibility :: Public" classifier not found on the server 200 - OK $ python setup.py register -r http://example.com/repository # registering at example.com Warning "Development Status :: 5 - Production/Stable" classifier not found on the server Warning "Intended Audience :: Developers" classifier not found on the server 200 - OK
This will allow visual checking when a typo is made. The package will then be available in each server, but only under the categories known to the server.