Semi-automated conformance checking for IRC implementations (RFC1459/RFC2812, the "modern spec" by ircdocs, and IRCv3)
Go to file
Mitchell Riley 321e254d15
Add SETNAME tests (#209)
* Add SETNAME tests

* fix race condition

* fix synchronization issue

sendLine does not synchronize by itself; call getMessage to synchronize
and test the message since we have it

* Update irctest/server_tests/

Co-authored-by: Val Lorentz <>


Co-authored-by: Shivaram Lingamneni <>
Co-authored-by: Val Lorentz <>
2023-06-04 17:06:53 -04:00
.github Bump Dlk version 2023-06-01 19:17:00 +02:00
data Add support for Nefarious (#151) 2022-04-10 11:37:35 +02:00
irctest Add SETNAME tests (#209) 2023-06-04 17:06:53 -04:00
patches bahamut: lower mainloop delay, and reduce parallelism to make tests less flaky 2022-06-18 20:26:53 +02:00
.gitignore add a test for INVITE 2018-12-23 16:41:54 -05:00
.pre-commit-config.yaml Bump linter versions (#188) 2023-03-04 10:51:40 +01:00 Split by command. 2021-08-11 20:46:10 +02:00
LICENSE Initial commit 2015-12-18 17:26:55 +01:00
Makefile Add support for The Lounge (#132) 2023-05-29 09:50:31 +02:00 inspircd: Use upstream mainloop hack when available (#200) 2023-05-20 20:06:59 +02:00 Add support for pytest 7.0.0 2022-02-04 20:48:28 +01:00 ci: Bump versions of actions we use (#199) 2023-05-20 13:32:42 +02:00
mypy.ini Bump linter versions (#188) 2023-03-04 10:51:40 +01:00
pyproject.toml Make black ignore irctest/scram/ 2021-11-12 22:34:48 +01:00
pytest.ini Add tests for draft/extended-monitor (#180) 2023-03-04 10:11:51 +01:00 Bump linter versions (#188) 2023-03-04 10:51:40 +01:00
requirements.txt basic server test for SCRAM-SHA-256 (#84) 2021-08-08 20:47:42 +02:00
setup.cfg exhaustive testing of Modern's WHOIS spec (#104) 2021-08-29 16:38:38 +02:00
workflows.yml Bump Dlk version 2023-06-01 19:17:00 +02:00


This project aims at testing interoperability of software using the IRC protocol, by running them against common test suites.

The big picture

This project contains:

  • IRC protocol test cases
  • small wrappers around existing software to run tests on them

Wrappers run software in temporary directories, so running irctest should have no side effect.


Install irctest and dependencies:

sudo apt install faketime  # Optional, but greatly speeds up irctest/server_tests/
cd ~
git clone
cd irctest
pip3 install --user -r requirements.txt

Add ~/.local/bin/ (and/or ~/go/bin/ for Ergo) to your PATH if it is not.

export PATH=$HOME/.local/bin/:$HOME/go/bin/:$PATH

Using pytest

irctest is invoked using the pytest test runner / CLI.

You can usually invoke it with python3 -m pytest command; which can often be called by the pytest or pytest-3 commands (if not, alias them if you are planning to use them often).

The rest of this README assumes pytest works.

Test selection

A major feature of pytest that irctest heavily relies on is test selection. Using the -k option, you can select and deselect tests based on their names and/or markers (listed in pytest.ini). For example, you can run LUSERS-related tests with -k lusers. Or only tests based on RFC1459 with -k rfc1459.

By default, all tests run; even niche ones. So you probably always want to use these options: -k 'not Ergo and not deprecated and not strict. This excludes:

  • Ergo-specific tests (included as Ergo uses irctest as its official integration test suite)
  • tests for deprecated specifications, such as the IRCv3 METADATA specification
  • tests that check for a strict interpretation of a specification, when the specification is ambiguous.

Running tests



cd /tmp/
git clone
cd ergo/
make install
cd ~/irctest
pytest --controller irctest.controllers.ergo -k 'not deprecated'


cd /tmp/
git clone
cd solanum
./configure --prefix=$HOME/.local/
make -j 4
make install
pytest --controller irctest.controllers.solanum -k 'not Ergo and not deprecated and not strict'


cd /tmp/
git clone
cd charybdis
./configure --prefix=$HOME/.local/
make -j 4
make install
cd ~/irctest
pytest --controller irctest.controllers.charybdis -k 'not Ergo and not deprecated and not strict'


cd /tmp/
git clone
cd inspircd

# Optional, makes tests run considerably faster. Pick one depending on the InspIRCd version:
# on Insp3 <= 3.16.0 and Insp4 <= 4.0.0a21:
patch src/inspircd.cpp < ~/irctest/patches/inspircd_mainloop.patch
# on Insp3 >= 3.17.0 and Insp4 >= 4.0.0a22:

./configure --prefix=$HOME/.local/ --development
make -j 4
make install
cd ~/irctest
pytest --controller irctest.controllers.inspircd -k 'not Ergo and not deprecated and not strict'


pip3 install --user git+
cd ~/irctest
pytest --controller irctest.controllers.mammon -k 'not Ergo and not deprecated and not strict'


cd /tmp/
git clone
cd unrealircd
./Config  # This will ask a few questions, answer them.
make -j 4
make install
cd ~/irctest
pytest --controller irctest.controllers.unreal -k 'not Ergo and not deprecated and not strict'

Servers with services

Besides Ergo (that has built-in services), most server controllers can optionally run service packages.


You can install it with

sudo apt install atheme-services

and add this to the pytest call:

--services-controller irctest.controllers.atheme_services


Build with:

cd /tmp/
git clone
cd anope
./Config  # This will ask a few questions, answer them.
make -C build -j 4
make -C build install

and add this to the pytest call:

--services-controller irctest.controllers.anope_services



pip3 install --user limnoria pyxmpp2-scram
cd ~/irctest
pytest --controller  irctest.controllers.limnoria


pip3 install --user sopel
mkdir ~/.sopel/
cd ~/irctest
pytest --controller irctest.controllers.sopel

What irctest is not

A formal proof that a given software follows any of the IRC specification, or anything near that.

At best, irctest can help you find issues in your software, but it may still have false positives (because it does not implement itself a full-featured client/server, so it supports only “usual” behavior). Bug reports for false positives are welcome.