things to do for piuparts
=========================

Please also check the BTS, especially the bugs with a severity higher than
wishlist!


for 0.5x:

- make piuparts-master + piuparts-slave packages work out-of-the-box:
  - piuparts-slave.postrm purge: clean up /var/cache/piuparts/tmp/
  - conf/piuparts.sudoers is only needed for slave package.
  - ship sane minimal sudoers in p-s.deb than can be enabled by default do we
    really need anything else than running piuparts and piuparts_slave_*
    as root? No, we don't.
  - the shipped crontabs are not suitable from cron.d/ due to missing user
    column - discuss this with DSA
  - verify that all scripts in $user/bin/ also work from master-slave packages
    eventually enable some conjobs in the packages
  - slave.postinst should setup ~piupartsm/.ssh/authorized_keys using triggers
    dpkg-trigger piuparts-master-please-install-the-slave-key
    + the current method only works if -slave is configured after -master
  - split /etc/piuparts/piuparts.conf in /etc/piuparts/master.conf and
    /etc/piuparts/slave.conf

- in maintainer view: sort buggy states up again

- more stats and graphs:
  - packages processed per day (in total and per slave)
    - master writes submissions.txt per section since 0.45
  - # of open bugs with tag piuparts
  - generate http://piuparts.debian.org/stable/states.png + testing.png from
    existing data

- check whether #660350 is really gone with a wheezy base system.

- add a sample config with all possible keys set to some useful value
  (like /usr/share/doc/apt/examples/configure-index.gz)

- look for a solution to use the global debian mirror for debian-backports,
  too, to avoid hardcoding a specific mirror in distros.conf

- maybe compress all logfiles

- notes from pejacevic setup:
  - the need for the symlink from /srv/piuparts.debian.org/etc/piuparts to
    /etc/piuparts is awkward when everything else is in 
    /srv/piuparts.debian.org
  - (likewise that the slave logins as master, even with a restricted key...)
  - use mirror via nfs (faster)
  - do not use eatmydata (we're using tmpfs in RAM anyway, so installing it
    just slows things down)

- re-read all documentation carefully and update were needed.

- move counts.txt from htdocs to master


for 0.6x:

- generate piuparts.1.txt automatically from piuparts.py - see this blog post
  for a nice howto:
  http://andialbrecht.wordpress.com/2009/03/17/creating-a-man-page-with-distutils-and-optparse/
  - though this seems pretty complicated... maybe rather grep for
    parser.add_option and help= in piuparts.py ?!
    - requires merging all the additional infomation in piuparts.1.txt into piuarts.py
    - parsing piuparts --help output may be easier than parsing piuparts.py

- rework known_problems:
  - split detect_well_known_errors
    - parsing the logfiles should stay there
    - generating the html should be integrated into piuparts-report
  - use a number prefix for sorting
  - add title information
  - piuparts-report: "discover" the available known_problems, dont hardcode the
    list
  - drop _issue/_error duplication, have flags inside to indicate thether to
    generate _issues.tpl (pass/) and/or _error.tpl (fail/ bugged/ affected/)
  - rework known problems to a python-friendlier format

- accept a PIUPARTS_CONF environment variable everywhere to point to a different
  piuparts.conf

- write reportbug-like wrapper for mass bug filing (start simple, make it more
  sophisticated later).

- rewrite piuparts-analyze to run over all sections and cache BTS responses

- "decorate" (strike-through) bug links generated by piuparts-analyze to
  indicate resolved state (take package version into account!)

* verify that find_default_debian_mirrors does something sane
- find_default_debian_mirrors: if parts[2] contains a / (think stable/updates
  for security.d.o), you can't ignore this, it will break later...
  - with distros.conf this may no longer be needed
  - check whether find_default_debian_mirrors produces something useful if
    sources.list does not exist (and sources.list.d/*.list is there instead)
  - maybe parse 'apt-cache policy' to get the mirror list instead

- make it possible to call aptitude (or similar) instead of apt-get and allow to
  override the commandline arguments.

- the templates used by update-reports.py and detect_well_known_errors should
  be taken from /etc/piuparts/templates/ and not be included in the python source

- check the logfiles (especially pass/) for
  - "Exception in thread"
  - java stacktraces
  - "Can't locate .* in @INC"

- test main without contrib/non-free in sources.list


for 0.7x and later:

- mounting /proc and perhaps others (usbfs, sysfs, /dev/pts, etc.) in the chroot
  might be a good idea because some packages might need this.

- report:
  - write stats about the reasons for failures, as its done with shell scripts
    now (piuparts-analyze.py is an existing "fragment".)
  - RSS feeds of logs
  - do more fancy R graphs, eg. also per state
  - sort packages on maintainer pages alphabetically (states first though)
  - link (and target) to piuparts.d.o configuration is static to pejacevic. should
    refer to the actual hosts configuration if running somewhere else

- a redirect of http://piuparts.d.o/foo to http://p.d.o/source/f/foo.html would
  be nice

- monitor: master.log grows to fast and there is no mechanism to stop it
  - use logrotate

- really support multiple architectures:
  - piuparts-report should have a list of available arch and list packages only
    available on untested archs in a new state
    "(depends-)not-available-on-tested-archs"
  - master should (per default) only schedule packages which are not available
    on the master arch to slaves of different archs ->
    "schedule-evenly-to-slaves = no"

- piuparts-master: keep track of to whom a reservation was given

- piuparts can't currently test upgrades of required packages.  (Because they
  cannot be removed, it assumes these are untestable, which is only true for
  removal tests...
  - all distupgrade tests implicitly tests these upgrades, although not
    individually per package

- not sure if it's a sensible thing to to, but provide a way to turn off
  debugging output for piuparts.py - see
http://docs.python.org/library/logging.html

- commandline-switches for all programms

- move shell cronjobs functionality into master, slave & report


1.0 should really have automated testing of piuparts itself...
-----------------------------------------------------------------

- create archive of broken packages to provide test cases for piuparts testing.

- create emacspeak-broken-dpkg-preconfigure package for broken repo. (then later
  put more broken packages in there and use that for testing piuparts)

