in-source testing

Image of Author
March 9, 2023 (last updated April 28, 2023)

In-source testing is where the test code lives within the same file as the source code. There are two popular variants of this style of testing that I've come across: doc tests and regular tests.

regular tests

These are typical tests. They differ from alternative testing setups where you write your tests in a separate test file, and sometimes in a separate test folder.

I find the appeal of in-source testing in the sense that it keeps coupled content as close together as possible. The test subject is some entity within the source file. The absolute closest you can get (in a file-based paradigm) is the test within the file.

For all the upsides of in-source testing, I feel that there are some downsides. First is navigating large files can prove unwieldy. Test suites are often much larger in line count than the entities they test. This can be an unwieldy addition to a source file that increases confusion about what is and is not source code. Additionally, you will often want to import additional source files to achieve meaningful tests, adding these to source code could increase confusion. Finally, orchestrating integration tests do not feel like they should live in one of several files "under test", but should instead live outside them all. This means that independent test files will live somewhere, and therefore could easily host unit test files as well. The conclusion of all of this is that while I find in-source testing appealing to maximising adjacency/coupling, I think that in the end it proves unwieldy and I prefer the aesthetics of merely adjacent files, e.g., source.js and source.test.js.

doc tests

In the wild:

Doc tests are more docs than tests. They presume you are already doing "in-source documentation" as well. As you document, e.g., a function, you can then write small, light example cases of how to use the function. In some environments, you can actually test those code examples. This is fundamentally what a doc test is. It is a test of the documentation itself.

This is helpful because documentation is often a 2nd class citizen and goes stale quickly. A failing doctest can help you catch these errors, which often also imply that you forgot to update other parts of the documentation as well.