View RSS Feed

Petr Schreiber

Testing ThinBASIC: Starting at the end

Rating: 2 votes, 5.00 average.
In the previous blog post I presented areas in which thinBASIC ecosystem could be improved in order to harden the stability of both the core and modules, including those developed by you using ThinBASIC SDK.

Today, I would like to share my views on how looking at the end might give us ideas what we need to count with before we will start implementing first line of test supporting functionality.

What kind of information do we expect from tests?

We need to raise confidence thinBASIC works as we expected. ThinBASIC can be seen as a mix of core language functionality and supporting functions.

Just by having a quick peek at thinBasic help file, I can see over 2500 functions in default thinBASIC installation.

The functions are organized in modules and each module has some further logical structure.

To give example, even the Core module, providing the most elemental functionality, can be further divided to string functions, flow control and many others.

This hierarchical structure is calling for analogy in the test hierarchy.

During preparation of new thinBASIC iteration, we will need to have both new and old functionality tested.

Even if we would take just the functions in the account, we don't want to be exposed to results of 2500 functions and all of their tests at one time. The total test count will be easily in tens of thousands!

We need a structured information, not a list of 2500 test results for review.

What kind of test structure can help support it?

The natural idea in context of thinBASIC project is to structure the tests per module. I would also suggest to our futures ourselves to tightly bind the module code with module tests.

This model is already applied in the case of stringBuilder module repository, for example.

This approach has multiple advantages, each time you make the change to code:
- you can run the attached tests to check you did not break anything
- you can prepare a new tests in sync with the change on one place
- you can have the passing tests as the requirement to integrate code change

The last one is super useful, if you manage to setup your process in a way the tests are run before the change integration. And especially useful, if multiple persons collaborate on a single module.

Running tests before change integration will be possible for thinBASIC as for example GitHub Actions allow to spin virtual Windows agent to run user defined tests now, does not matter which framework.

As you can see, the module level structuring is a direction worth pursuing, and also something all the modules in thinBASIC can have in common.

Do we need to divide tests beyond the module level?

Simpler modules, such as StringBuilder, have a single purpose and no further divison might be needed.

On the other side, we have larger modules, such as Core or TBGL, which have more logical groups inside.

The number of logical groups and their nesting differs from module to module.

Having the finer control over test groups could allow us to review results more easily and also to run just specific parts of tests when needed - with great speed benefit, useful for example during development.


Looking at the current module situation, having the tests hierarchically structured seems reasonable for the following reasons:
- ability to break huge amount of tests to logical groups
- ability to run/evaluate only particular group of tests during development
- ability to quicky evaluate failed test origin once we run the complete test set

The example of test division, on example of Core module, could be:
Core -> String handling -> Mid$

...and then multiple tests in this nested group for each possible case of Mid$ usage.

The task to do in one of the next blog posts will be to think about how we can aid structuring on test definition and test result level.

Submit "Testing ThinBASIC: Starting at the end" to Facebook Submit "Testing ThinBASIC: Starting at the end" to Digg Submit "Testing ThinBASIC: Starting at the end" to Submit "Testing ThinBASIC: Starting at the end" to StumbleUpon Submit "Testing ThinBASIC: Starting at the end" to Google Submit "Testing ThinBASIC: Starting at the end" to Twitter