(large commit, see bazaar log)

[Bantracker]
 * Less spaces in README.txt
 * Remove mention of table 'users' in README.txt
 * Add more detail on how to create the bans database in README.txt
 * Add note about supybot-wizard creating the initial database in README.txt
 * Don't hard-code default values in config.py:configure()
 * Tweak config.py
 * Clean up bans.cgi a bit

[Bugtracker]
 * Comment-out obsolete "bug reporting" variables in config.py
 * Update README.txt and remove "bug reporting" stuff, also remove extraneous license info
 * Comment-out obsolete "bug reporting" code
 * Don't import imaplib

[Encyclopedia]
 * Don't hard-code default values in config.py:configure()
 * Check for 'owner' capability before checking if the hostmask is ignored in plugin.py:checkIgnored()
 * Clean up README.txt

[PackageInfo]
 * Don't hard-code default values in config.py:configure()
 * Update default distributions in config.py:configure()
 * Update defaultRelease in config.py 
 * Update README.txt

* Add a few docstrings to commoncgi.py
* Update COPYING
* Update README.txt
This commit is contained in:
Terence Simpson
2011-05-28 07:33:21 +01:00
parent 6fe55f7eb1
commit fd36bffcc0
16 changed files with 269 additions and 294 deletions

View File

@ -1,17 +1,8 @@
Copyright (c) 2005-2007, Dennis Kaarsemaker
This program is free software; you can redistribute it and/or modify
it under the terms of version 2 of the GNU General Public License as
published by the Free Software Foundation.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
Bug information reporting plugin, works with many well-known bugtrackers.
The syntax to add a tracker is weird, here are some examples:
@bugtracker add freedesktop bugzilla https://bugs.freedesktop.org Freedesktop
@bugtracker add malone malone https://launchpad.net/malone Malone
@bugtracker add launchpad launchpad https://launchpad.net/malone Launchpad
@bugtracker add debian debbugs http://bugs.debian.org Debian
@bugtracker add openoffice issuezilla http://www.openoffice.org/issues OpenOffice
@bugtracker add django trac http://code.djangoproject.com/ticket Django
@ -21,21 +12,23 @@ In general: @bugtracker add <name> <type> <baseurl> [description]
Bugtracker dialects (types) this plugin understands:
* Bugzilla
* Issuezilla (OpenOffice.org's tjernobyl transformation of bugzilla)
* Malone
* Launchpad (Including Ubuntu)
* Debbugs (debbugs sucks donkeyballs - please fix debbugs)
* Trac (with not-too-buggered-up templates, it needs to do screenscraping)
* Sourceforge (needs atid and group_id in the url!)
* Trac
* WikiForms (see bugs.gnewsense.org for an example)
* str.php from the CUPS project
* Mantis (http://www.mantisbt.org)
A notable exception is Sourceforge. Unfortunatly, it has no API or data export
feature to output bug information in a well-formed way.
To request a bug report, use this syntax:
bug 123
bug #123
supybot bug 123
launchpad bug 123
bug 123, 4, 5
bug 1, 3 and 89
bugs 1, 3 and 89
To rename a bugtracker:
@bugtracker rename old-name new-name
@ -45,47 +38,3 @@ existing tracker.
The bug snarfing (responding to bug numbers/urls) will only work in channels
where supybot.plugins.bugtracker.bugsnarfer is True.
Automatic reporting of new bugs is also possible for Malone (the launchpad
bugtracker). Enabling this is not a trivial process. First step is to set the
supybot.plugins.bugtracker.reportercache variable to a dir for this purpose. You
also need a mail account that supports the + hack (mail for foo+bar@baz.com is
automatically delivered to foo@baz.com while the Delivered-To: header is set to
foo+bar@baz.com) which is accessible via IMAP. I know this is a rather strong
requirement, but that's the way it works now. Patches to make it work in other
situations are appreciated.
Anyway, once that is all set up you're almost there. Let's assume the
mailaddress is bugreporter@yourdomain.com. Now pick a tag for your bugreports,
e.g. ubuntu (you can set a different tag per channel) and create a launchpad
account with address bugreporter+ubuntu@yourdomain.com. Activate that account
and make sure it gets bugmail for the product(s) you want to monitor.
Now set the supybot.plugins.bugtracker.bugreporter in the channels where bugs
are to be reported to the value of the tag for bugs to be reported there and
watch bugs flowing in.
To prevent old bugs from showing up when they change or a comment is being
added, you can manually fill the cache. Just touch files in the reporters cache
with the following name:
tag_here/malone/NN/MMMM where NN is int(bugid/1000) and MMMM is the bugid.
If your products already have many bugreports, consider doing some
screenscraping with the malone searchpages and sed/awk :)
A quick hack I use to get all launchpad bugids preseeded:
cd /home/ubugtu/data/bugmail # This is my cachedir
cd launchpad # This is the tag
mkdir malone
product=launchpad
amount=2000 # 2000 is the amount of bugs, chck this on
# launchpad under all bugs ever reported
# Download a summary of all bugs
for x in `seq 0 75 $amount`; do
wget "https://bugs.launchpad.net/$product/+bugs?search=Search&field.status=Unconfirmed&field.status=Confirmed&field.status=In+Progress&field.status=Needs+Info&field.status=Fix+Committed&field.status=Fix+Released&field.status=Rejected&field.omit_dupes.used=&start=$x" -O $x;
done
grep -h =.amount * | sed -e 's/.*>\(.*\)<.*/\1/' | awk '{print "malone/" int($1/1000)}' | sort -u | xargs mkdir -p
grep -h =.amount * | sed -e 's/.*>\(.*\)<.*/\1/' | awk '{print "malone/" int($1/1000) "/" $1}' | xargs -n100 touch

View File

@ -1,7 +1,7 @@
# -*- Encoding: utf-8 -*-
###
# Copyright (c) 2005-2007 Dennis Kaarsemaker
# Copyright (c) 2008-2010 Terence Simpson
# Copyright (c) 2008-2011 Terence Simpson
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of version 2 of the GNU General Public License as
@ -45,13 +45,14 @@ def configure(advanced):
else:
return repeatdelay
output("Each of the next 3 questions can be set per-channel with the '@Config channel' command")
bugSnarfer = yn("Enable detecting bugs numbers and URL in all channels?", default=Bugtracker.bugSnarfer._default)
cveSnarfer = yn("Enable detecting CVE numbers and URL in all channels?", default=Bugtracker.cveSnarfer._default)
oopsSnarfer = yn("Enable detecting Launchpad OOPS IDs in all channels?", default=Bugtracker.oopsSnarfer._default)
if advanced:
replyNoBugtracker = something("What should the bot reply with when a a user requests information from an unknown bug tracker?", default=Bugtracker.replyNoBugtracker._default)
snarfTarget = something("What should be the default bug tracker used when one isn't specified?", default=Bugtracker.snarfTarget._default)
replyWhenNotFound = yn("Respond when a bug is not found?", default=Bugtracker.replyWhenNotFound._default)
replyWhenNotFound = yn("Should the bot report when a bug is not found?", default=Bugtracker.replyWhenNotFound._default)
repeatdelay = getRepeatdelay()
else:
replyNoBugtracker = Bugtracker.replyNoBugtracker._default
@ -89,8 +90,8 @@ conf.registerChannelValue(Bugtracker, 'oopsSnarfer',
enabled, such that any OOPS ### seen in the channel
will have their information reported into the channel."""))
conf.registerChannelValue(Bugtracker, 'bugReporter',
registry.String('', """Report new bugs (experimental)"""))
#conf.registerChannelValue(Bugtracker, 'bugReporter',
# registry.String('', """Report new bugs (experimental)"""))
conf.registerChannelValue(Bugtracker, 'replyNoBugtracker',
registry.String('I don\'t have a bugtracker %s.', """Determines the phrase
@ -98,7 +99,7 @@ conf.registerChannelValue(Bugtracker, 'replyNoBugtracker',
bugtracker site."""))
conf.registerChannelValue(Bugtracker, 'snarfTarget',
registry.String('lp', """Determines the bugtracker to query when the
registry.String('launchpad', """Determines the bugtracker to query when the
snarf command is triggered"""))
conf.registerGlobalValue(Bugtracker, 'bugtrackers',
@ -117,18 +118,18 @@ conf.registerChannelValue(Bugtracker, 'showassignee',
conf.registerChannelValue(Bugtracker, 'extended',
registry.Boolean(False, "Show optional extneded bug information, specific to trackers"))
conf.registerGlobalValue(Bugtracker, 'reportercache',
registry.String('', """Name of the basedir for the bugreporter cache""", private=True))
#conf.registerGlobalValue(Bugtracker, 'reportercache',
# registry.String('', """Name of the basedir for the bugreporter cache""", private=True))
conf.registerGlobalValue(Bugtracker, 'imap_server',
registry.String('', """IMAP server for bugmail account""",private=True))
#conf.registerGlobalValue(Bugtracker, 'imap_server',
# registry.String('', """IMAP server for bugmail account""",private=True))
conf.registerGlobalValue(Bugtracker, 'imap_user',
registry.String('', """IMAP user for bugmail account""", private=True))
#conf.registerGlobalValue(Bugtracker, 'imap_user',
# registry.String('', """IMAP user for bugmail account""", private=True))
conf.registerGlobalValue(Bugtracker, 'imap_password',
registry.String('', """IMAP password for bugmail account""", private=True))
#conf.registerGlobalValue(Bugtracker, 'imap_password',
# registry.String('', """IMAP password for bugmail account""", private=True))
conf.registerGlobalValue(Bugtracker, 'imap_ssl',
registry.Boolean(False, """Use SSL for imap connections""", private=True))
#conf.registerGlobalValue(Bugtracker, 'imap_ssl',
# registry.Boolean(False, """Use SSL for imap connections""", private=True))

View File

@ -24,13 +24,15 @@ import supybot.registry as registry
import supybot.schedule as schedule
import supybot.log as supylog
import re, os, time, imaplib, commands
#import imaplib
import re, os, time, commands
import xml.dom.minidom as minidom
from htmlentitydefs import entitydefs as entities
import email.FeedParser
import SOAPpy
bad_words = ["fuck","fuk","fucking","fuking","fukin","fuckin","fucked","fuked","fucker","shit","cunt","bastard","nazi","nigger","nigga","cock","bitches","bitch"]
# All the words below will be censored when reporting bug information
bad_words = set(["fuck","fuk","fucking","fuking","fukin","fuckin","fucked","fuked","fucker","shit","cunt","bastard","nazi","nigger","nigga","cock","bitches","bitch"])
def makeClean(s):
words = s.split()
@ -95,7 +97,7 @@ class Bugtracker(callbacks.PluginRegexp):
def __init__(self, irc):
callbacks.PluginRegexp.__init__(self, irc)
self.db = ircutils.IrcDict()
events = []
# self.events = []
for name in self.registryValue('bugtrackers'):
registerBugtracker(name)
group = self.registryValue('bugtrackers.%s' % name.replace('.','\\.'), value=False)
@ -104,27 +106,28 @@ class Bugtracker(callbacks.PluginRegexp):
else:
self.log.warning("Bugtracker: Unknown trackertype: %s (%s)" % (group.trackertype(), name))
self.shorthand = utils.abbrev(self.db.keys())
# Schedule bug reporting
self.shown = {}
#TODO: Remove everything below this line
if self.registryValue('imap_server') and self.registryValue('reportercache'):
try:
schedule.removeEvent(self.name() + '.bugreporter')
except:
pass
schedule.addPeriodicEvent(lambda: self.reportnewbugs(irc), 60, name=self.name() + '.bugreporter')
self.events += [self.name() + '.bugreporter']
self.log.info('Bugtracker: Adding scheduled event "%s.bugreporter"' % self.name())
# # Schedule bug reporting
# #TODO: Remove everything below this line
# if self.registryValue('imap_server') and self.registryValue('reportercache'):
# try:
# schedule.removeEvent(self.name() + '.bugreporter')
# except:
# pass
# schedule.addPeriodicEvent(lambda: self.reportnewbugs(irc), 60, name=self.name() + '.bugreporter')
# self.events += [self.name() + '.bugreporter']
# self.log.info('Bugtracker: Adding scheduled event "%s.bugreporter"' % self.name())
def die(self): #TODO: Remove me
try:
for event in self.events:
self.log.info('Bugtracker: Removing scheduled event "%s"' % event)
schedule.removeEvent(event)
schedule.removeEvent(self.name())
except:
pass
pass
# try:
# for event in self.events:
# self.log.info('Bugtracker: Removing scheduled event "%s"' % event)
# schedule.removeEvent(event)
# schedule.removeEvent(self.name())
# except:
# pass
def is_ok(self, channel, tracker, bug):
'''Flood/repeat protection'''
@ -138,95 +141,97 @@ class Bugtracker(callbacks.PluginRegexp):
return False
def is_new(self, tracker, tag, id): #Depricated
bugreporter_base = self.registryValue('reportercache')
if not os.path.exists(os.path.join(bugreporter_base,tag,tracker.name,str(int(id/1000)),str(id))):
try:
os.makedirs(os.path.join(bugreporter_base,tag,tracker.name,str(int(id/1000))))
except:
pass
fd = open(os.path.join(bugreporter_base,tag,tracker.name,str(int(id/1000)),str(id)),'w')
fd.close()
return True
return False
pass
# bugreporter_base = self.registryValue('reportercache')
# if not os.path.exists(os.path.join(bugreporter_base,tag,tracker.name,str(int(id/1000)),str(id))):
# try:
# os.makedirs(os.path.join(bugreporter_base,tag,tracker.name,str(int(id/1000))))
# except:
# pass
# fd = open(os.path.join(bugreporter_base,tag,tracker.name,str(int(id/1000)),str(id)),'w')
# fd.close()
# return True
# return False
def reportnewbugs(self,irc): #Depricated
# Compile list of bugs
self.log.info("Bugtracker: Checking for new bugs")
bugs = {}
if self.registryValue('imap_ssl'):
sc = imaplib.IMAP4_SSL(self.registryValue('imap_server'))
else:
sc = imaplib.IMAP4(self.registryValue('imap_server'))
sc.login(self.registryValue('imap_user'), self.registryValue('imap_password'))
sc.select('INBOX')
new_mail = sc.search(None, '(UNSEEN)')[1][0].split()[:20]
# Read all new mail
for m in new_mail:
msg = sc.fetch(m, 'RFC822')[1][0][1]
fp = email.FeedParser.FeedParser()
sc.store(m, '+FLAGS', "(\Deleted)") # Mark message deleted so we don't have to process it again
fp.feed(msg)
bug = fp.close()
tag = None
if 'X-Launchpad-Bug' not in bug.keys():
self.log.info('Bugtracker: Ignoring e-mail with no detectable bug (Not from Launchpad)')
continue
else:
tag = bug['X-Launchpad-Bug']
if 'distribution=' not in tag and 'product=' not in tag:
self.log.info('Bugtracker: Ignoring e-mail with no detectable bug (no distro/product)')
continue
else:
tag = tag.split(';')[0].strip().replace("product=",'').replace("distribution=","")
if not tag:
self.log.info('Bugtracker: Ignoring e-mail with no detectible bug (bad tag)')
tag = tag[tag.find('+')+1:tag.find('@')]
if tag not in bugs:
bugs[tag] = {}
# Determine bugtracker type (currently only Launchpad is supported anyway)
if bug['X-Launchpad-Bug']:
tracker = self.db['launchpad']
id = int(bug['Reply-To'].split()[1])
subj = bug['Subject'];
if '[NEW]' not in subj: #Not a new bug
continue
if self.is_new(tracker, tag, id):
component = bug['X-Launchpad-Bug']
if 'component' in component:
component = component[component.find('component=')+10:]
component = component[:component.find(';')].replace('None','')
else:
component = ''
try:
if component:
bugs[tag][id] = self.get_bug('',tracker, id, False)[0].replace('"','(%s) "' % component, 1)
else:
bugs[tag][id] = self.get_bug('',tracker, id, False)[0]
if '[apport]' in bugs[tag][id]:
bugs[tag].pop(id)
except:
self.log.info("Bugtracker: Unable to get new bug %d" % id)
pass
else:
self.log.info('Bugtracker: Ignoring e-mail with no detectable bug')
reported_bugs = 0
for c in irc.state.channels:
tags = self.registryValue('bugReporter', channel=c)
if not tags:
continue
for tag in tags.split(','):
if not tag or tag not in bugs.keys():
continue
for b in sorted(bugs[tag].keys()):
irc.queueMsg(ircmsgs.privmsg(c,'New bug: #%s' % bugs[tag][b][bugs[tag][b].find('bug ')+4:]))
reported_bugs = reported_bugs+1
pass
# # Compile list of bugs
# self.log.info("Bugtracker: Checking for new bugs")
# bugs = {}
# if self.registryValue('imap_ssl'):
# sc = imaplib.IMAP4_SSL(self.registryValue('imap_server'))
# else:
# sc = imaplib.IMAP4(self.registryValue('imap_server'))
# sc.login(self.registryValue('imap_user'), self.registryValue('imap_password'))
# sc.select('INBOX')
# new_mail = sc.search(None, '(UNSEEN)')[1][0].split()[:20]
#
# # Read all new mail
# for m in new_mail:
# msg = sc.fetch(m, 'RFC822')[1][0][1]
# fp = email.FeedParser.FeedParser()
# sc.store(m, '+FLAGS', "(\Deleted)") # Mark message deleted so we don't have to process it again
# fp.feed(msg)
# bug = fp.close()
# tag = None
#
# if 'X-Launchpad-Bug' not in bug.keys():
# self.log.info('Bugtracker: Ignoring e-mail with no detectable bug (Not from Launchpad)')
# continue
# else:
# tag = bug['X-Launchpad-Bug']
# if 'distribution=' not in tag and 'product=' not in tag:
# self.log.info('Bugtracker: Ignoring e-mail with no detectable bug (no distro/product)')
# continue
# else:
# tag = tag.split(';')[0].strip().replace("product=",'').replace("distribution=","")
#
# if not tag:
# self.log.info('Bugtracker: Ignoring e-mail with no detectible bug (bad tag)')
#
# tag = tag[tag.find('+')+1:tag.find('@')]
# if tag not in bugs:
# bugs[tag] = {}
#
# # Determine bugtracker type (currently only Launchpad is supported anyway)
# if bug['X-Launchpad-Bug']:
# tracker = self.db['launchpad']
# id = int(bug['Reply-To'].split()[1])
# subj = bug['Subject'];
# if '[NEW]' not in subj: #Not a new bug
# continue
# if self.is_new(tracker, tag, id):
# component = bug['X-Launchpad-Bug']
# if 'component' in component:
# component = component[component.find('component=')+10:]
# component = component[:component.find(';')].replace('None','')
# else:
# component = ''
# try:
# if component:
# bugs[tag][id] = self.get_bug('',tracker, id, False)[0].replace('"','(%s) "' % component, 1)
# else:
# bugs[tag][id] = self.get_bug('',tracker, id, False)[0]
# if '[apport]' in bugs[tag][id]:
# bugs[tag].pop(id)
# except:
# self.log.info("Bugtracker: Unable to get new bug %d" % id)
# pass
# else:
# self.log.info('Bugtracker: Ignoring e-mail with no detectable bug')
#
# reported_bugs = 0
#
# for c in irc.state.channels:
# tags = self.registryValue('bugReporter', channel=c)
# if not tags:
# continue
# for tag in tags.split(','):
# if not tag or tag not in bugs.keys():
# continue
# for b in sorted(bugs[tag].keys()):
# irc.queueMsg(ircmsgs.privmsg(c,'New bug: #%s' % bugs[tag][b][bugs[tag][b].find('bug ')+4:]))
# reported_bugs = reported_bugs+1
def add(self, irc, msg, args, name, trackertype, url, description):
"""<name> <type> <url> [<description>]
@ -359,18 +364,18 @@ class Bugtracker(callbacks.PluginRegexp):
msg.tag('nbugs', nbugs + len(bugids))
bt = map(lambda x: x.lower(), match.group('bt').split())
# Strip off trailing ':' from the tracker name. Allows for (LP: #nnnnnn)
if len(bt) and bt[0].endswith(':'):
if bt and bt[0].endswith(':'):
bt[0] = bt[:-1]
name = ''
if len(bt) == 1 and not (bt[0] in ['bug','bugs']):
try:
name = bt[0].lower()
name = bt[0]
tracker = self.db[name]
except:
return
elif len(bt) == 2:
try:
name = bt[0].lower()
name = bt[0]
tracker = self.db[name]
except:
name = ''