5079
Comment:
|
12534
Add information about the Test and Experimental instances
|
Deletions are marked like this. | Additions are marked like this. |
Line 1: | Line 1: |
<<TableOfContents>> |
|
Line 8: | Line 10: |
= Requirements = | == Requirements == |
Line 14: | Line 16: |
* A checkout of the python-dev instance from python's subversion repository. | * Installation of psycopg. * A checkout of the spambayes_integration. * A checkout of the python-dev instance from python's subversion repository. |
Line 18: | Line 24: |
For python-dev, we use a slighly modified version of roundup which has some features that are not yet in the core roundup. To keep track of |
For python-dev, we use a slightly modified version of Roundup which has some features that are not yet in the core Roundup. To keep track of |
Line 22: | Line 28: |
Check out /tracker/roundup-src from the python subversion | Check out [[http://svn.python.org/view/tracker/roundup-src|/tracker/roundup-src]] from the python subversion |
Line 53: | Line 59: |
any known problems. | any known problems. It has been reported that 8.2 and 8.3 work well. |
Line 87: | Line 93: |
== Installing psycopg or psycopg2 == You need to install psycopg or psycopg2. Download it from http://www.initd.org, as pre-packaged versions have been reported to cause problems. == Checkout spambayes_integration == Even if you don't intend to use the !SpamBayes detector you must still check out the {{{spambayes_integration}}} instance. {{{ svn co http://svn.python.org/projects/tracker/instances/spambayes_integration }}} If you do intend to use it, you will need to also [[#SpamBayesSetup | install SpamBayes proper and configure it appropriately]]. Otherwise, a [[#SpamBayesdummyconfiguration |dummy configuration]] should work. |
|
Line 89: | Line 108: |
The python-dev tracker instance is version-controlled with subversion, | The [[http://svn.python.org/view/tracker/instances/python-dev/ |python-dev]] tracker instance is version-controlled with subversion, |
Line 94: | Line 113: |
}}} Keep in mind that the python-dev instance has relative symlinks to {{{../../spambayes_integration/[...]}}}, so if your directory layout is different you have to update these links: {{{ extensions/spambayes.py -> ../../spambayes_integration/extensions/spambayes.py detectors/spambayes.py -> ../../spambayes_integration/detectors/spambayes.py |
|
Line 114: | Line 141: |
If you need to do development on the busybody or tellteam detectors, | If you need to do development on the '''busybody''' or '''tellteam''' detectors, |
Line 116: | Line 143: |
adjust the e-mail addresses accordingly. | adjust the e-mail addresses accordingly. <<Anchor(ciavc)>> The '''CIA.vc''' detector sends emails about issue changes and is on by default. Disable it by adding a {{{return}}} to its init() function (detectors/cia.py): {{{#!python def init(db): return # <-- db.issue.react('create', sendcia) db.issue.react('set', sendcia) }}} |
Line 133: | Line 168: |
get a roundup instance that looks just like http://bugs.python.org. | get a roundup instance that looks just like http://bugs.python.org, except for some missing new values for fields like Stages and Keywords. It's possible to replace '''initial_data.py''' with an [[attachment:initial_data_updated.py | updated version]] so that your fields will have values that match those currently present in the Python Tracker. The IDs for values in a given field might be different from those in the Python Tracker. This should only be a problem if you try to import CSV files from one tracker into the other. == SpamBayes Setup == If you intend to use the !SpamBayes detector to weed out spam from your tracker you will need to install it and start an instance of its {{{core_server}}} application. Briefly: * Download and install at least version 1.1a4 of the [[http://sourceforge.net/projects/spambayes|SpamBayes source]] from !SourceForge. If you are running on Windows '''do not''' use the binary installer! * Establish a basic configuration, e.g.: Devote a directory to it, e.g.: {{{ SBDIR=/usr/local/spambayes/core_server # or whatever... mkdir -p $SBDIR }}} Create an INI file: {{{ cd $SBDIR cat > bayescustomize.ini <<EOF [globals] verbose:False [Headers] include_evidence:True include_score:True [Tokenizer] record_header_absence:True summarize_email_prefixes:True summarize_email_suffixes:True mine_received_headers:True x-pick_apart_urls:True x-fancy_url_recognition:False x-lookup_ip:True lookup_ip_cache:$SBDIR/dnscache.pck max_image_size:100000 crack_image_cache:$SBDIR/imagecache.pck crack_images:True image_size:True ocr_engine:gocr [Classifier] use_bigrams:False [Categorization] ham_cutoff:0.2 spam_cutoff:0.85 [Storage] persistent_storage_file:$SBDIR/hammie.db persistent_use_database:pickle messageinfo_storage_file:$SBDIR/messageinfo.fs [html_ui] display_score:True EOF }}} Finally, run it: {{{ BAYESCUSTOMIZE=$SBDIR/bayescustomize.ini core_server.py -m XMLRPCPlugin }}} * If you are seeing image spam (this seems not to be a problem with wiki spam - yet), download and install [[http://jocr.sourceforge.net/|GOCR]]. == Common Problems == === SpamBayes dummy configuration === Incomplete/Missing !SpamBayes setup:: In your detectors/config.ini file set {{{spambayes_uri}}} to a syntactically valid, but nonexistent URI, e.g.: {{{ spambayes_uri = "http://localhost:9999/nobodyhere" }}} Check [[http://mail.python.org/pipermail/tracker-discuss/2008-February/001549.html |other available workarounds]]. === Misbehaving detectors === Problematic detectors may be disabled by [[#ciavc |returning from their init() function]], as was the case of detectors/audit2to3.py (no longer present). Note that this would be a temporary solution before a graceful fix became available. Reporting issues with local instances is the best way help solving the problem. = Resources for Tracker Development = |
Line 138: | Line 251: |
Since the Python Tracker is a slightly modified version of [[http://www.roundup-tracker.org |Roundup]], both Roundup's [[http://www.roundup-tracker.org/docs.html |Documentation]] and [[http://issues.roundup-tracker.org/ |issue tracker]] contain relevant information about how the Python Tracker works and problems one might find working with its code. == The Meta Tracker == A [[http://psf.upfronthosting.co.za/roundup/meta/ |Meta Tracker]] is available for handling bugs and features requests for the [[http://bugs.python.org/ |Python Tracker]]. == Learn more About the Importer(s) == http://svn.python.org/projects/tracker/importer/README.rst has some information about the importer(s) written to convert sourceforge data into roundup data. == The Test Tracker == Public test instance of the Python tracker: http://bot.bio.br/python-dev/ Instance that attempts to match the code used in http://bugs.python.org to allow testing, reproducing and verifying fixes for tracker bugs. Can also be used as a sandbox by users interested in learning about Developer, Coordinator or Admin tasks and features. Testing new content (e.g. Components or Statuses) is OK, but new features show be tested in the Experimental Tracker instead. Currently the email system is disabled (redirected to file), so people interested in having an account there to test new features should email [[http://mail.python.org/mailman/listinfo/tracker-discuss|tracker-discuss]] to get one. Maintained by DanielDiniz. == The Experimental Tracker == Modified version of the Python tracker: http://bot.bio.br/python-dev-exp/ It's an instance to showcase and test new features. New features (2009-04-18): * [[http://mail.python.org/pipermail/tracker-discuss/2009-April/002099.html|Issue tags]] * [[http://psf.upfronthosting.co.za/roundup/meta/issue249|Quiet properties]] * [[http://psf.upfronthosting.co.za/roundup/meta/issue267|Restore removed messages and files]] * [[http://psf.upfronthosting.co.za/roundup/meta/issue258|Claim ('assign to self') and add/remove self as nosy buttons]] * [[http://psf.upfronthosting.co.za/roundup/meta/issue266|Don't close issues with open dependencies]] * [[http://psf.upfronthosting.co.za/roundup/meta/issue258|Auto-add nosy users based on Components]] * [[http://psf.upfronthosting.co.za/roundup/meta/issue245|"Email me" buttons for messages and issues, "Reply by email"]] * [[http://psf.upfronthosting.co.za/roundup/meta/issue155|RSS feeds (per issue and global)]] * [[http://psf.upfronthosting.co.za/roundup/meta/issue246|Display selected issues in the index view]] * [[http://psf.upfronthosting.co.za/roundup/meta/issue248|Mass-update/batch-editing support]] Currently the email system is disabled (redirected to file), so people interested in having an account there to test new features should email [[http://mail.python.org/mailman/listinfo/tracker-discuss|tracker-discuss]] to get one. Maintained by DanielDiniz. |
Getting Your Own python-dev Tracker
This page describes how to setup a roundup tracker instance that works exactly as the tracker for python-dev on http://bugs.python.org. The intended audience is people that want to help out with the tracker, developing new features etc.
Requirements
- Roundup, from the vendor branch used for python-dev (see below).
- PostgreSQL for use as backend database.
- Installation of psycopg.
- A checkout of the spambayes_integration.
- A checkout of the python-dev instance from python's subversion repository.
Installing roundup
For python-dev, we use a slightly modified version of Roundup which has some features that are not yet in the core Roundup. To keep track of the changes, we keep a vendor branch of roundup.
Check out /tracker/roundup-src from the python subversion repository as documented on http://python.org/dev/tools/. For example, here's how to do anonymous checkout:
svn co http://svn.python.org/projects/tracker/roundup-src
Install this roundup somewhere by running setup.py
cd roundup-src python setup.py install --prefix /opt/tracker-roundup/
Install and configure PostgreSQL
For the python-dev tracker instance we use PostgreSQL as backend database. Roundup has support for many different databases for backend (anydbm, metakit, MySQL, PostgreSQL, SQLite...), but for the python-dev instance, PostgreSQL was selected as it has the best performance for large installations. Also, for performance reasons, there is at least one place in the python-dev instance that has a hardcoded dependency on the backend being an SQL-based one, so you do need PostgreSQL even for development work (another SQL-based backend might work, but this has not been tested). I recommend that you run postgreSQL to make your development environment behave exactly like the environment the real tracker runs in.
Details on how to install PostgreSQL is outside the scope of this Wiki Page. Both version 7.4 and 8.1 of PostgreSQL has been tested without any known problems. It has been reported that 8.2 and 8.3 work well.
Access configuration of postgresql can be tricky. You need to make sure that the user you're running roundup as (your unix/windows user) has access to the postgresql database you intend to use for roundup, as the postgresql user you've configured in roundup's configuration.
For a development environment with no demands on security, the easiest way to configure this is probably to allow all users to connect as any database user as long as the request originates from the local host. This can be accomplished by the following lines in pg_hba.conf:
local all all trust host all all 127.0.0.1 255.255.255.255 trust
PLEASE NOTE: This is not a secure configuration on a multi-user machine.
After the change in pg_hba.conf, reload your postgreSQL database to make sure it knows about your new access configuration. Something similar to /etc/init.d/postgresql reload should do the trick on Unix-like platforms.
Create a database user that is allowed to create databases. This user will be used when roundup connects to the database.
# psql -U postgres postgres=# create user roundup; postgres=# alter user roundup with createdb;
Installing psycopg or psycopg2
You need to install psycopg or psycopg2. Download it from http://www.initd.org, as pre-packaged versions have been reported to cause problems.
Checkout spambayes_integration
Even if you don't intend to use the SpamBayes detector you must still check out the spambayes_integration instance.
svn co http://svn.python.org/projects/tracker/instances/spambayes_integration
If you do intend to use it, you will need to also install SpamBayes proper and configure it appropriately. Otherwise, a dummy configuration should work.
Checkout and Configure your Development Roundup Instance
The python-dev tracker instance is version-controlled with subversion, and resides in the python subversion repository.
svn co http://svn.python.org/projects/tracker/instances/python-dev
Keep in mind that the python-dev instance has relative symlinks to ../../spambayes_integration/[...], so if your directory layout is different you have to update these links:
extensions/spambayes.py -> ../../spambayes_integration/extensions/spambayes.py detectors/spambayes.py -> ../../spambayes_integration/detectors/spambayes.py
Now enter the python-dev directory, and create the 'db' directory (not in svn as it's just to much fuzz to keep it there - every time you run roundup-admin init it is removed and recreated, confusing svn who wants to keep its .svn directory intact), as well as the db/backend_name file which decides which backend (i.e. database type) to use:
cd python-dev mkdir db echo postgresql > db/backend_name
Copy python-dev/config.ini.template into config.ini, and modify it according to your site. Pay special attention to settings marked with NO DEFAULT. However, if you've configure your PostgreSQL as above, most settings should be OK.
If you need to do development on the busybody or tellteam detectors, copy detectors/config.ini.template into detectors/config.ini, and adjust the e-mail addresses accordingly. The CIA.vc detector sends emails about issue changes and is on by default. Disable it by adding a return to its init() function (detectors/cia.py):
Initialize your roundup instance:
/opt/tracker-roundup/bin/roundup-admin -i <your python-dev directory> init
Provide a password for the 'admin' user when asked for.
Start your roundup instance:
/opt/tracker-roundup/bin/roundup-server -p 9999 python-dev=<your python-dev directory>
You should now be able to browse http://localhost:9999/python-dev/ and get a roundup instance that looks just like http://bugs.python.org, except for some missing new values for fields like Stages and Keywords. It's possible to replace initial_data.py with an updated version so that your fields will have values that match those currently present in the Python Tracker.
The IDs for values in a given field might be different from those in the Python Tracker. This should only be a problem if you try to import CSV files from one tracker into the other.
SpamBayes Setup
If you intend to use the SpamBayes detector to weed out spam from your tracker you will need to install it and start an instance of its core_server application. Briefly:
Download and install at least version 1.1a4 of the SpamBayes source from SourceForge. If you are running on Windows do not use the binary installer!
- Establish a basic configuration, e.g.:
Devote a directory to it, e.g.:
SBDIR=/usr/local/spambayes/core_server # or whatever... mkdir -p $SBDIR
Create an INI file:
cd $SBDIR cat > bayescustomize.ini <<EOF [globals] verbose:False [Headers] include_evidence:True include_score:True [Tokenizer] record_header_absence:True summarize_email_prefixes:True summarize_email_suffixes:True mine_received_headers:True x-pick_apart_urls:True x-fancy_url_recognition:False x-lookup_ip:True lookup_ip_cache:$SBDIR/dnscache.pck max_image_size:100000 crack_image_cache:$SBDIR/imagecache.pck crack_images:True image_size:True ocr_engine:gocr [Classifier] use_bigrams:False [Categorization] ham_cutoff:0.2 spam_cutoff:0.85 [Storage] persistent_storage_file:$SBDIR/hammie.db persistent_use_database:pickle messageinfo_storage_file:$SBDIR/messageinfo.fs [html_ui] display_score:True EOF
Finally, run it:
BAYESCUSTOMIZE=$SBDIR/bayescustomize.ini core_server.py -m XMLRPCPlugin
If you are seeing image spam (this seems not to be a problem with wiki spam - yet), download and install GOCR.
Common Problems
SpamBayes dummy configuration
Incomplete/Missing SpamBayes setup:: In your detectors/config.ini file set spambayes_uri to a syntactically valid, but nonexistent URI, e.g.:
spambayes_uri = "http://localhost:9999/nobodyhere"
Check other available workarounds.
Misbehaving detectors
Problematic detectors may be disabled by returning from their init() function, as was the case of detectors/audit2to3.py (no longer present). Note that this would be a temporary solution before a graceful fix became available. Reporting issues with local instances is the best way help solving the problem.
Resources for Tracker Development
Getting Help
Subscribe to and ask your question on the tracker-discuss mailing list. See http://mail.python.org/mailman/listinfo/tracker-discuss
Since the Python Tracker is a slightly modified version of Roundup, both Roundup's Documentation and issue tracker contain relevant information about how the Python Tracker works and problems one might find working with its code.
The Meta Tracker
A Meta Tracker is available for handling bugs and features requests for the Python Tracker.
Learn more About the Importer(s)
http://svn.python.org/projects/tracker/importer/README.rst has some information about the importer(s) written to convert sourceforge data into roundup data.
The Test Tracker
Public test instance of the Python tracker: http://bot.bio.br/python-dev/
Instance that attempts to match the code used in http://bugs.python.org to allow testing, reproducing and verifying fixes for tracker bugs. Can also be used as a sandbox by users interested in learning about Developer, Coordinator or Admin tasks and features. Testing new content (e.g. Components or Statuses) is OK, but new features show be tested in the Experimental Tracker instead.
Currently the email system is disabled (redirected to file), so people interested in having an account there to test new features should email tracker-discuss to get one. Maintained by DanielDiniz.
The Experimental Tracker
Modified version of the Python tracker: http://bot.bio.br/python-dev-exp/
It's an instance to showcase and test new features.
New features (2009-04-18):
Claim ('assign to self') and add/remove self as nosy buttons
"Email me" buttons for messages and issues, "Reply by email"
Currently the email system is disabled (redirected to file), so people interested in having an account there to test new features should email tracker-discuss to get one. Maintained by DanielDiniz.