Combatting large data providers in unit testing

Question

I am performing validation tests for numerous fields on an object, each of which has numerous validation rules (required, integer, unique etc.). Rather than writing a test function for each field and rule, I have a data provider which is supplied to a generic test function.

A contrived example of the data provider:

public function validationErrorsProvider()
{
    return [
        [ 'field', 'input', 'error message' ],
        [ 'field', 'input', 'error message' ],
        [ 'field', 'input', 'error message' ],
    ];
}

The data provider is then used by a generic test method which asserts that for field with input it sees the error message.

/**
 * @test
 * @dataProvider validationErrorsProvider
 */
public function will_fail_with_errors_when_input_is_invalid($field, $input, $error)
{
    // do some stuff
}

This works fine but the data provider is becoming increasingly large as I add further fields and rules to be tested (20 - 30 elements in the validationErrorsProvider array). It appears more complicated that some of my tests.

Is this just something that happens, or is there an alternative? I have considered splitting the data provider into multiple providers, each holding just the data for a given field. This doesn't do anything other than make the data provider appear smaller.

I've been experimenting with using a CSV file adjacent to the test. In my provider, I'm reading that, polishing it up, and then kicking an array back as the return. So far it's working, I just haven't experimented with it enough to know whether or not I really like it or not. — eklingen, Aug 29 '22 at 12:53

score 1 · Accepted Answer · answered Jul 05 '21 at 21:21

This is one of these opinionated questions, i feel i can give a conclusion. All projects i have worked on, had some sort of testing and is the subject area I'm most passionated about.

In general software development, we try to make clever solutions to minimize code duplication and make it easier to maintain. Testing is the other side of the coin. Here i want a descriptive test, that if needed rather repeats itself, than being solved with that clever approach. There is a great post about it.

Keeping all your tests together gives you a blueprint for what the test accepts and what the results should be. Which is to be preferred. On the other side of it, how much value does testing validation actually gives? it's great for learning the testing ropes, but usually if not in an insanely high risk project, i would do a Happy Path test on validation, leave it be and wait for bugs or errors to be reported. Then add another test each time i have to revisit the validation.

So you should not be worried about if it is one or two classes, but if i don't at all think a 30 (or a 100 for that matter) line data provider class is a problem.

Thanks for the answer and linked article. So if I understand correctly, tests should ideally be descriptive and singular (test each possible path of a function independently), however, trivial tests such as validation can be consolidated? — themoog, Jul 06 '21 at 07:52
In testing there is no correctness, but does i would argue test that validation works and it fails. if testing email test that a@a.com works and notanemail fails leave it there and done :) but ofc working with dataproviders, you can easily add extra tests. Validation testing i would argue going deep in the rabbit hole does not provide value for the business and rather test core business logic. — mrhn, Jul 06 '21 at 08:26

Combatting large data providers in unit testing

1 Answers1