val
/
irctest
mirror of https://github.com/progval/irctest.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Initial overhaul changes

This commit is contained in:
Daniel Oaks 2016-12-01 19:21:25 +10:00
parent 2f95675348
commit f359feb8e2
3 changed files with 37 additions and 117 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

88
README.md
View File

@ -1,56 +1,51 @@
# irctest # irctest
This project aims at testing interoperability of software using the This software intends to test IRC clients and servers, to ensure that they behave how they're expected to today.
IRC protocol, by running them against test suites and making different
software communicate with each other.
It is very young and does not contain a lot of test cases yet. The tests in this repo are based on [RFC 1459](https://tools.ietf.org/html/rfc1459), [RFC 2812](https://tools.ietf.org/html/rfc2812), the [Modern docs](http://modern.ircdocs.horse/) and generally-accepted software behaviour.
## The big picture This project is very young and doesn't contain a lot of test cases. However, it should still be useful and can highlight unexpected issues, particularly with newly-developed software.
This project contains:
* IRC protocol test cases ## Installing
* small wrappers around existing software to run tests on them
Wrappers run software in temporary directories, so running `irctest` should Clone the repo and install the relevant depdencies:
have no side effect, with [the exception of Sopel](https://github.com/sopel-irc/sopel/issues/946).
## Prerequisites
Install irctest and dependencies:
``` ```
git clone https://github.com/ProgVal/irctest.git git clone https://github.com/DanielOaks/irctest.git
cd irctest cd irctest
pip3 install --user -r requirements.txt pip3 install --user -r requirements.txt
python3 setup.py install --user
``` ```
Add `~/.local/bin/` to your `PATH` if it is not.
``` ## Running Tests
export PATH=$HOME/.local/bin/:$PATH
```
## Run tests For almost every client / server, all we require is that the software is installed and the binary is in the PATH.
To run (client) tests on Limnoria: Here are examples of how to install various software and run tests with them:
### Clients
To run tests on Limnoria:
``` ```
pip3 install --user limnoria pip3 install --user limnoria
python3 -m irctest irctest.controllers.limnoria python3 test.py irctest.controllers.limnoria
``` ```
To run (client) tests on Sopel: To run tests on Sopel:
``` ```
pip3 install --user sopel pip3 install --user sopel
mkdir ~/.sopel/ mkdir ~/.sopel/
python3 -m irctest irctest.controllers.sopel python3 test.py irctest.controllers.sopel
``` ```
To run (server) tests on InspIRCd:
### Servers
To run tests on InspIRCd:
``` ```
cd /tmp/ cd /tmp/
@ -59,17 +54,17 @@ cd inspircd
./configure --prefix=$HOME/.local/ --development ./configure --prefix=$HOME/.local/ --development
make -j 4 make -j 4
make install make install
python3 -m irctest irctest.controllers.inspircd python3 test.py irctest.controllers.inspircd
``` ```
To run (server) tests on Mammon: To run tests on Mammon:
``` ```
pip3 install --user git+https://github.com/mammon-ircd/mammon.git pip3 install --user git+https://github.com/mammon-ircd/mammon.git
python3 -m irctest irctest.controllers.mammon python3 test.py irctest.controllers.mammon
``` ```
To run (server) tests on Charybdis:: To run tests on Charybdis::
``` ```
cd /tmp/ cd /tmp/
@ -78,38 +73,17 @@ cd charybdis
./configure --prefix=$HOME/.local/ ./configure --prefix=$HOME/.local/
make -j 4 make -j 4
make install make install
python3 -m irctest irctest.controllers.charybdis python3 test.py irctest.controllers.charybdis
``` ```
## Full help
``` ## Program Help
usage: python3 -m irctest [-h] [--show-io] [-v] [-s SPECIFICATION] [-l] module
positional arguments: For more complete help, run `./test --help`
module The module used to run the tested program.
optional arguments:
-h, --help show this help message and exit
--show-io Show input/outputs with the tested program.
-v, --verbose Verbosity. Give this option multiple times to make it
even more verbose.
-s SPECIFICATION, --specification SPECIFICATION
The set of specifications to test the program with.
Valid values: RFC1459, RFC2812, IRCv3.1, IRCv3.2. Use
this option multiple times to test with multiple
specifications. If it is not given, defaults to all.
-l, --loose Disables strict checks of conformity to the
specification. Strict means the specification is
unclear, and the most restrictive interpretation is
choosen.
```
## What `irctest` is not ## What this is not
A formal proof that a given software follows any of the IRC specification, This isn't a formal proof that a given piece of IRC software follows the IRC specs to the letter, or is definitely going to be interoperable with other software.
or anything near that.
At best, `irctest` can help you find issues in your software, but it may At best, this can help you find issues with your software, but it may still have false-positives.
still have false positives (because it does not implement itself a
full-featured client/server, so it supports only “usual” behavior).

54
setup.py
View File

@ -1,54 +0,0 @@
#!/usr/bin/env python3
import os
import sys
from setuptools import setup
if sys.version_info < (3, 4, 0):
sys.stderr.write("This script requires Python 3.4 or newer.")
sys.stderr.write(os.linesep)
sys.exit(-1)
with open(os.path.join(os.path.dirname(__file__), 'requirements.txt')) as fd:
requirements = [x.split('#')[0]
for x in fd.readlines()]
setup(
name='irctest',
version='0.1',
author='Valentin Lorentz',
url='https://github.com/ProgVal/irctest/',
author_email='progval+irctest@progval.net',
description='A script to test interoperability of IRC software.',
platforms=['linux', 'linux2'],
long_description="""This script aims at testing interoperability of
software using the IRC protocol, by running them against test suites
and making different software communicate with each other.""",
classifiers = [
'Development Status :: 2 - Pre-Alpha',
'Environment :: Console',
'Intended Audience :: Developers',
'License :: OSI Approved :: MIT License',
'Natural Language :: English',
'Operating System :: POSIX',
'Programming Language :: Python :: 3.2',
'Programming Language :: Python :: 3.3',
'Programming Language :: Python :: 3.4',
'Programming Language :: Python :: 3.5',
'Programming Language :: Python :: 3 :: Only',
'Topic :: Communications :: Chat :: Internet Relay Chat',
'Topic :: Software Development :: Testing',
],
# Installation data
packages=[
'irctest',
'irctest.client_tests',
'irctest.controllers',
'irctest.irc_utils',
'irctest.server_tests',
],
install_requires=requirements,
)
# vim:set shiftwidth=4 softtabstop=4 expandtab textwidth=79:

12
irctest/__main__.py → test.py
View File

@ -4,10 +4,10 @@ import argparse
import unittest import unittest
import functools import functools
import importlib import importlib
from .cases import _IrcTestCase from irctest.cases import _IrcTestCase
from .runner import TextTestRunner from irctest.runner import TextTestRunner
from .specifications import Specifications from irctest.specifications import Specifications
from .basecontrollers import BaseClientController, BaseServerController from irctest.basecontrollers import BaseClientController, BaseServerController
def main(args): def main(args):
try: try:
@ -18,9 +18,9 @@ def main(args):
controller_class = module.get_irctest_controller_class() controller_class = module.get_irctest_controller_class()
if issubclass(controller_class, BaseClientController): if issubclass(controller_class, BaseClientController):
import irctest.client_tests as module from irctest import client_tests as module
elif issubclass(controller_class, BaseServerController): elif issubclass(controller_class, BaseServerController):
import irctest.server_tests as module from irctest import server_tests as module
else: else:
print(r'{}.Controller should be a subclass of ' print(r'{}.Controller should be a subclass of '
r'irctest.basecontroller.Base{{Client,Server}}Controller' r'irctest.basecontroller.Base{{Client,Server}}Controller'