mirror of https://github.com/progval/irctest.git
Initial overhaul changes
This commit is contained in:
parent
2f95675348
commit
f359feb8e2
88
README.md
88
README.md
|
@ -1,56 +1,51 @@
|
||||||
# irctest
|
# irctest
|
||||||
|
|
||||||
This project aims at testing interoperability of software using the
|
This software intends to test IRC clients and servers, to ensure that they behave how they're expected to today.
|
||||||
IRC protocol, by running them against test suites and making different
|
|
||||||
software communicate with each other.
|
|
||||||
|
|
||||||
It is very young and does not contain a lot of test cases yet.
|
The tests in this repo are based on [RFC 1459](https://tools.ietf.org/html/rfc1459), [RFC 2812](https://tools.ietf.org/html/rfc2812), the [Modern docs](http://modern.ircdocs.horse/) and generally-accepted software behaviour.
|
||||||
|
|
||||||
## The big picture
|
This project is very young and doesn't contain a lot of test cases. However, it should still be useful and can highlight unexpected issues, particularly with newly-developed software.
|
||||||
|
|
||||||
This project contains:
|
|
||||||
|
|
||||||
* IRC protocol test cases
|
## Installing
|
||||||
* small wrappers around existing software to run tests on them
|
|
||||||
|
|
||||||
Wrappers run software in temporary directories, so running `irctest` should
|
Clone the repo and install the relevant depdencies:
|
||||||
have no side effect, with [the exception of Sopel](https://github.com/sopel-irc/sopel/issues/946).
|
|
||||||
|
|
||||||
## Prerequisites
|
|
||||||
|
|
||||||
Install irctest and dependencies:
|
|
||||||
|
|
||||||
```
|
```
|
||||||
git clone https://github.com/ProgVal/irctest.git
|
git clone https://github.com/DanielOaks/irctest.git
|
||||||
cd irctest
|
cd irctest
|
||||||
pip3 install --user -r requirements.txt
|
pip3 install --user -r requirements.txt
|
||||||
python3 setup.py install --user
|
|
||||||
```
|
```
|
||||||
|
|
||||||
Add `~/.local/bin/` to your `PATH` if it is not.
|
|
||||||
|
|
||||||
```
|
## Running Tests
|
||||||
export PATH=$HOME/.local/bin/:$PATH
|
|
||||||
```
|
|
||||||
|
|
||||||
## Run tests
|
For almost every client / server, all we require is that the software is installed and the binary is in the PATH.
|
||||||
|
|
||||||
To run (client) tests on Limnoria:
|
Here are examples of how to install various software and run tests with them:
|
||||||
|
|
||||||
|
|
||||||
|
### Clients
|
||||||
|
|
||||||
|
To run tests on Limnoria:
|
||||||
|
|
||||||
```
|
```
|
||||||
pip3 install --user limnoria
|
pip3 install --user limnoria
|
||||||
python3 -m irctest irctest.controllers.limnoria
|
python3 test.py irctest.controllers.limnoria
|
||||||
```
|
```
|
||||||
|
|
||||||
To run (client) tests on Sopel:
|
To run tests on Sopel:
|
||||||
|
|
||||||
```
|
```
|
||||||
pip3 install --user sopel
|
pip3 install --user sopel
|
||||||
mkdir ~/.sopel/
|
mkdir ~/.sopel/
|
||||||
python3 -m irctest irctest.controllers.sopel
|
python3 test.py irctest.controllers.sopel
|
||||||
```
|
```
|
||||||
|
|
||||||
To run (server) tests on InspIRCd:
|
|
||||||
|
### Servers
|
||||||
|
|
||||||
|
To run tests on InspIRCd:
|
||||||
|
|
||||||
```
|
```
|
||||||
cd /tmp/
|
cd /tmp/
|
||||||
|
@ -59,17 +54,17 @@ cd inspircd
|
||||||
./configure --prefix=$HOME/.local/ --development
|
./configure --prefix=$HOME/.local/ --development
|
||||||
make -j 4
|
make -j 4
|
||||||
make install
|
make install
|
||||||
python3 -m irctest irctest.controllers.inspircd
|
python3 test.py irctest.controllers.inspircd
|
||||||
```
|
```
|
||||||
|
|
||||||
To run (server) tests on Mammon:
|
To run tests on Mammon:
|
||||||
|
|
||||||
```
|
```
|
||||||
pip3 install --user git+https://github.com/mammon-ircd/mammon.git
|
pip3 install --user git+https://github.com/mammon-ircd/mammon.git
|
||||||
python3 -m irctest irctest.controllers.mammon
|
python3 test.py irctest.controllers.mammon
|
||||||
```
|
```
|
||||||
|
|
||||||
To run (server) tests on Charybdis::
|
To run tests on Charybdis::
|
||||||
|
|
||||||
```
|
```
|
||||||
cd /tmp/
|
cd /tmp/
|
||||||
|
@ -78,38 +73,17 @@ cd charybdis
|
||||||
./configure --prefix=$HOME/.local/
|
./configure --prefix=$HOME/.local/
|
||||||
make -j 4
|
make -j 4
|
||||||
make install
|
make install
|
||||||
python3 -m irctest irctest.controllers.charybdis
|
python3 test.py irctest.controllers.charybdis
|
||||||
```
|
```
|
||||||
|
|
||||||
## Full help
|
|
||||||
|
|
||||||
```
|
## Program Help
|
||||||
usage: python3 -m irctest [-h] [--show-io] [-v] [-s SPECIFICATION] [-l] module
|
|
||||||
|
|
||||||
positional arguments:
|
For more complete help, run `./test --help`
|
||||||
module The module used to run the tested program.
|
|
||||||
|
|
||||||
optional arguments:
|
|
||||||
-h, --help show this help message and exit
|
|
||||||
--show-io Show input/outputs with the tested program.
|
|
||||||
-v, --verbose Verbosity. Give this option multiple times to make it
|
|
||||||
even more verbose.
|
|
||||||
-s SPECIFICATION, --specification SPECIFICATION
|
|
||||||
The set of specifications to test the program with.
|
|
||||||
Valid values: RFC1459, RFC2812, IRCv3.1, IRCv3.2. Use
|
|
||||||
this option multiple times to test with multiple
|
|
||||||
specifications. If it is not given, defaults to all.
|
|
||||||
-l, --loose Disables strict checks of conformity to the
|
|
||||||
specification. Strict means the specification is
|
|
||||||
unclear, and the most restrictive interpretation is
|
|
||||||
choosen.
|
|
||||||
```
|
|
||||||
|
|
||||||
## What `irctest` is not
|
## What this is not
|
||||||
|
|
||||||
A formal proof that a given software follows any of the IRC specification,
|
This isn't a formal proof that a given piece of IRC software follows the IRC specs to the letter, or is definitely going to be interoperable with other software.
|
||||||
or anything near that.
|
|
||||||
|
|
||||||
At best, `irctest` can help you find issues in your software, but it may
|
At best, this can help you find issues with your software, but it may still have false-positives.
|
||||||
still have false positives (because it does not implement itself a
|
|
||||||
full-featured client/server, so it supports only “usual” behavior).
|
|
||||||
|
|
54
setup.py
54
setup.py
|
@ -1,54 +0,0 @@
|
||||||
#!/usr/bin/env python3
|
|
||||||
|
|
||||||
import os
|
|
||||||
import sys
|
|
||||||
from setuptools import setup
|
|
||||||
|
|
||||||
if sys.version_info < (3, 4, 0):
|
|
||||||
sys.stderr.write("This script requires Python 3.4 or newer.")
|
|
||||||
sys.stderr.write(os.linesep)
|
|
||||||
sys.exit(-1)
|
|
||||||
|
|
||||||
with open(os.path.join(os.path.dirname(__file__), 'requirements.txt')) as fd:
|
|
||||||
requirements = [x.split('#')[0]
|
|
||||||
for x in fd.readlines()]
|
|
||||||
|
|
||||||
setup(
|
|
||||||
name='irctest',
|
|
||||||
version='0.1',
|
|
||||||
author='Valentin Lorentz',
|
|
||||||
url='https://github.com/ProgVal/irctest/',
|
|
||||||
author_email='progval+irctest@progval.net',
|
|
||||||
description='A script to test interoperability of IRC software.',
|
|
||||||
platforms=['linux', 'linux2'],
|
|
||||||
long_description="""This script aims at testing interoperability of
|
|
||||||
software using the IRC protocol, by running them against test suites
|
|
||||||
and making different software communicate with each other.""",
|
|
||||||
classifiers = [
|
|
||||||
'Development Status :: 2 - Pre-Alpha',
|
|
||||||
'Environment :: Console',
|
|
||||||
'Intended Audience :: Developers',
|
|
||||||
'License :: OSI Approved :: MIT License',
|
|
||||||
'Natural Language :: English',
|
|
||||||
'Operating System :: POSIX',
|
|
||||||
'Programming Language :: Python :: 3.2',
|
|
||||||
'Programming Language :: Python :: 3.3',
|
|
||||||
'Programming Language :: Python :: 3.4',
|
|
||||||
'Programming Language :: Python :: 3.5',
|
|
||||||
'Programming Language :: Python :: 3 :: Only',
|
|
||||||
'Topic :: Communications :: Chat :: Internet Relay Chat',
|
|
||||||
'Topic :: Software Development :: Testing',
|
|
||||||
],
|
|
||||||
|
|
||||||
# Installation data
|
|
||||||
packages=[
|
|
||||||
'irctest',
|
|
||||||
'irctest.client_tests',
|
|
||||||
'irctest.controllers',
|
|
||||||
'irctest.irc_utils',
|
|
||||||
'irctest.server_tests',
|
|
||||||
],
|
|
||||||
install_requires=requirements,
|
|
||||||
)
|
|
||||||
|
|
||||||
# vim:set shiftwidth=4 softtabstop=4 expandtab textwidth=79:
|
|
|
@ -4,10 +4,10 @@ import argparse
|
||||||
import unittest
|
import unittest
|
||||||
import functools
|
import functools
|
||||||
import importlib
|
import importlib
|
||||||
from .cases import _IrcTestCase
|
from irctest.cases import _IrcTestCase
|
||||||
from .runner import TextTestRunner
|
from irctest.runner import TextTestRunner
|
||||||
from .specifications import Specifications
|
from irctest.specifications import Specifications
|
||||||
from .basecontrollers import BaseClientController, BaseServerController
|
from irctest.basecontrollers import BaseClientController, BaseServerController
|
||||||
|
|
||||||
def main(args):
|
def main(args):
|
||||||
try:
|
try:
|
||||||
|
@ -18,9 +18,9 @@ def main(args):
|
||||||
|
|
||||||
controller_class = module.get_irctest_controller_class()
|
controller_class = module.get_irctest_controller_class()
|
||||||
if issubclass(controller_class, BaseClientController):
|
if issubclass(controller_class, BaseClientController):
|
||||||
import irctest.client_tests as module
|
from irctest import client_tests as module
|
||||||
elif issubclass(controller_class, BaseServerController):
|
elif issubclass(controller_class, BaseServerController):
|
||||||
import irctest.server_tests as module
|
from irctest import server_tests as module
|
||||||
else:
|
else:
|
||||||
print(r'{}.Controller should be a subclass of '
|
print(r'{}.Controller should be a subclass of '
|
||||||
r'irctest.basecontroller.Base{{Client,Server}}Controller'
|
r'irctest.basecontroller.Base{{Client,Server}}Controller'
|
Loading…
Reference in New Issue