diff --git a/README.md b/README.md index e0f278a..89d124f 100644 --- a/README.md +++ b/README.md @@ -3,16 +3,30 @@ This project aims at testing interoperability of software using the IRC protocol, by running them against common test suites. +It is also used while editing [the "Modern" specification](https://modern.ircdocs.horse/) +to check behavior of a large selection of servers at once. + ## The big picture This project contains: -* IRC protocol test cases -* small wrappers around existing software to run tests on them +* IRC protocol test cases, primarily checking conformance to + [the "Modern" specification](https://modern.ircdocs.horse/) and + [IRCv3 extensions](https://ircv3.net/irc/), but also + [RFC 1459](https://datatracker.ietf.org/doc/html/rfc1459) and + [RFC 2812](https://datatracker.ietf.org/doc/html/rfc2812). + Most of them are for servers but also some for clients. + Only the client-server protocol is tested; server-server protocols are out of scope. +* Small wrappers around existing software to run tests on them. + So far this is restricted to headless software (servers, service packages, + and clients bots). Wrappers run software in temporary directories, so running `irctest` should have no side effect. +Test results for the latest version of each supported software, and respective logs, +are [published daily](https://dashboard.irctest.limnoria.net/). + ## Prerequisites Install irctest and dependencies: @@ -20,7 +34,7 @@ Install irctest and dependencies: ``` sudo apt install faketime # Optional, but greatly speeds up irctest/server_tests/list.py cd ~ -git clone https://github.com/ProgVal/irctest.git +git clone https://github.com/progval/irctest.git cd irctest pip3 install --user -r requirements.txt ``` @@ -40,18 +54,23 @@ You can usually invoke it with `python3 -m pytest` command; which can often be called by the `pytest` or `pytest-3` commands (if not, alias them if you are planning to use them often). +After installing `pytest-xdist`, you can also pass `pytest` the `-n 10` option +to run `10` tests in parallel. + The rest of this README assumes `pytest` works. ## Test selection A major feature of pytest that irctest heavily relies on is test selection. Using the `-k` option, you can select and deselect tests based on their names -and/or markers (listed in `pytest.ini`). For example, you can run `LUSERS`-related tests with `-k lusers`. -Or only tests based on RFC1459 with `-k rfc1459`. + +Using the `-m` option, you can select and deselect and them based on their markers +(listed in `pytest.ini`). +For example, you can run only tests based on RFC1459 with `-m rfc1459`. By default, all tests run; even niche ones. So you probably always want to -use these options: `-k 'not Ergo and not deprecated and not strict`. +use these options: `-m 'not Ergo and not deprecated and not strict`. This excludes: * `Ergo`-specific tests (included as Ergo uses irctest as its official @@ -63,6 +82,10 @@ This excludes: ## Running tests +This list is non-exhaustive, see `workflows.yml` for software not listed here. +If software you want to test is not listed their either, please open an issue +or pull request to add support for it. + ### Servers #### Ergo: @@ -89,20 +112,6 @@ make install pytest --controller irctest.controllers.solanum -k 'not Ergo and not deprecated and not strict' ``` -#### Charybdis: - -``` -cd /tmp/ -git clone https://github.com/atheme/charybdis.git -cd charybdis -./autogen.sh -./configure --prefix=$HOME/.local/ -make -j 4 -make install -cd ~/irctest -pytest --controller irctest.controllers.charybdis -k 'not Ergo and not deprecated and not strict' -``` - #### InspIRCd: ``` @@ -123,14 +132,6 @@ cd ~/irctest pytest --controller irctest.controllers.inspircd -k 'not Ergo and not deprecated and not strict' ``` -#### Mammon: - -``` -pip3 install --user git+https://github.com/mammon-ircd/mammon.git -cd ~/irctest -pytest --controller irctest.controllers.mammon -k 'not Ergo and not deprecated and not strict' -``` - #### UnrealIRCd: ``` @@ -147,8 +148,8 @@ pytest --controller irctest.controllers.unreal -k 'not Ergo and not deprecated a ### Servers with services -Besides Ergo (that has built-in services), most server controllers can optionally run -service packages. +Besides Ergo (that has built-in services) and Sable (that ships its own services), +most server controllers can optionally run service packages. #### Atheme: