Page 1 of 2 12 LastLast
Results 1 to 10 of 12

Thread: Test Driven Development & Unit Testing for thinBASIC

  1. #1
    Super Moderator Petr Schreiber's Avatar
    Join Date
    Aug 2005
    Location
    Brno - Czech Republic
    Posts
    7,128
    Rep Power
    732

    Test Driven Development & Unit Testing for thinBASIC

    Thanks to my colleague Honza, I discovered the interesting world of Test Driven Development.

    It is a topic for fat book, but let's sum it up quickly:
    • test driven development is approach to write safe code with predicable output
    • it is based on idea of writing failing tests first and implementing functionality later, to remove the errors step by step


    While I hope we will be able to put together built in Unit Testing system with Eros (for ThinBASIC 2.x), let's experiment now!

    I prepaired simple unitTesting.tBasicU, which covers most of the needs for TDD beginner (bold statement, ain't it).

    It features engine for setting up the tests:
    • ut_Initialize
    • ut_LaunchTests
    • ut_SaveLog
    • ut_Release
    • ut_GetLastError


    Many pre-defined asserts for typical cases:
    • ut_assertEqual
    • ut_assertEqualRounded
    • ut_assertEqualEpsilon
    • ut_assertNotEqual
    • ut_assertEqualText
    • ut_assertNotEqualText
    • ut_assertIsGreater
    • ut_assertIsLess
    • ut_assertIsTrue
    • ut_assertIsFalse
    • ut_assertIsNull
    • ut_assertIsNotNull
    • ut_assertIsEmpty
    • ut_assertIsNotEmpty
    • ut_assertError


    And set of functions to retrieve the test results immediately:

    • ut_GetFailureCount
    • ut_GetFailureTestName
    • ut_GetFailureAssertType
    • ut_GetFailureDescription
    • ut_GetFailureComment


    The test functions should be prefixed with test_, with two specific cases:

    • test_SetUp - launched before each test
    • test_TearDown - launched after each test


    Confused? What does it mean? What is it good for?

    Let's have a look at example in my next post.


    Petr
    Attached Files Attached Files
    Last edited by Petr Schreiber; 14-06-2015 at 20:51.
    Learn 3D graphics with ThinBASIC, learn TBGL!
    Windows 10 64bit - Intel Core i5-3350P @ 3.1GHz - 16 GB RAM - NVIDIA GeForce GTX 1050 Ti 4GB

  2. #2
    Super Moderator Petr Schreiber's Avatar
    Join Date
    Aug 2005
    Location
    Brno - Czech Republic
    Posts
    7,128
    Rep Power
    732
    The unit testing is concept coming from TDD, and it allows you to test repeatedly your set of functions and check, whether it does not start to fall apart after few modifications.

    One of the key ideas of TDD is to write failing tests first, and try to fix the issue one by one, finally reaching fully reliable code (in ideal situation).

    Example
    Let's say we want to write factorial function on our own (note: thinBasic Math module already provides one)

    Factorial can be calculated as following:
    n! = (n) * (n-1) * (n - 2) * ... * 1

    It is known to be valid only for positive integeres, for zero it returns 1.

    To indicate wrong input somehow, let's agree it will return -1 for such a case.

    Writing the test
    So, we will write the test of the possible user cases first. It is good practice (and required by attached unit) to prefix the test functions with test_.

    You can see how the following code manages the testing engine.
    Uses "Console"
    
    ' -- Include file for unit testing support
    #INCLUDE "unitTesting.tBasicU"
    
    ' -- Main body of our testing engine
    Function TBMain()
      
      Print "Testing factorial..."
      ' -- Initialize testing engine
      ut_Initialize()
    
      ' -- Automatically launch all possible tests
      ut_LaunchTests()       
    
      ' -- Report failures found in report, if any
      Long nFailures = ut_GetFailureCount()
      Long i
      
      If nFailures Then    
        PrintL   
        PrintL                         
        
        For i = 1 To nFailures
          PrintL "#"+i
          PrintL "Test:        " + ut_GetFailureTestName(i)
          PrintL "Assert:      " + ut_GetFailureAssertType(i)
          PrintL "Description: " + ut_GetFailureDescription(i)
          PrintL "Comment:     " + ut_GetFailureComment(i)
          PrintL
        Next
      Else
        PrintL "PASSED"  
      End If              
      
      ' -- Save results to default file
      ut_SaveLog()
      
      ' -- Release engine 
      ut_Release()
      
      WaitKey
    
    End Function
    
    ' -- Our function
    Function Fact (ByVal aInput As Long) As Quad 
    
      Return 0
    
    End Function
    
    ' -- Custom tests
    Function test_Fact_NegativeInput()
                       
      ut_assertEqual( -1, fact(-1), "Test of factorial for negative value")                   
      
    End Function
    
    Function test_Fact_For0()
                       
      ut_assertEqual(  1, fact(0) , "Test of factorial for input equal to zero")
      
    End Function
    
    Function test_Fact_For1()
                       
      ut_assertEqual(  1, fact(1) , "Test of factorial for input equal to one")
      
    End Function
    
    Function test_Fact_ForGeneralPositiveValue()
                       
      ut_assertEqual(120, fact(5) , "Test of factorial for one of the possible positive values")
      
    End Function
    
    Function test_Fact_ForTooBigValue()
                       
      ut_assertEqual( -1, fact(21), "Test of factorial for value out of bounds, on the positive side")
      
    End Function
    
    The validation of returned values is handled via so called asserts. They indicate expected value and allow to add commentary. In this case, the following line:
      ut_assertEqual(120, fact(5) , "fact(5)")
    
    ...means: The value of 120 is expected result of fact(5) call. The "fact(5)" is just our commentary to be used as clue in case of failing test.

    If you run the code above, you will get the following flood of errors:
    #1
    Test: TEST_FACT_NEGATIVEINPUT
    Assert: UT_ASSERTEQUAL
    Description: Expected value=-1, found value=0
    Comment: Test of factorial for negative value

    #2
    Test: TEST_FACT_FORTOOBIGVALUE
    Assert: UT_ASSERTEQUAL
    Description: Expected value=-1, found value=0
    Comment: Test of factorial for value out of bounds, on the positive side

    #3
    Test: TEST_FACT_FORGENERALPOSITIVEVALUE
    Assert: UT_ASSERTEQUAL
    Description: Expected value=120, found value=0
    Comment: Test of factorial for one of the possible positive values

    #4
    Test: TEST_FACT_FOR0
    Assert: UT_ASSERTEQUAL
    Description: Expected value=1, found value=0
    Comment: Test of factorial for input equal to zero

    #5
    Test: TEST_FACT_FOR1
    Assert: UT_ASSERTEQUAL
    Description: Expected value=1, found value=0
    Comment: Test of factorial for input equal to one
    Note: The test test_Fact_ForTooBigValue was designed with limitations of QUAD(Int64) in mind. It is well known fact you can't go over 20! with it, because of overflow.

    Building it up
    Once we have a set of failing tests, we can start to fix the errors. Let's start with input validation.
    Thanks to using QUAD as datatype in this model example, we know the input must not be bigger than 20 and also that factorial for negative integers is not defined.

    Let's improve the Fact function this way:
    Function Fact (ByVal aInput As Long) As Quad 
    
      If Not Inside(aInput, 0, 20) Then
        Return -1
      End If
    
    End Function
    
    When we run the test again:
    #1
    Test: TEST_FACT_FORGENERALPOSITIVEVALUE
    Assert: UT_ASSERTEQUAL
    Description: Expected value=120, found value=0
    Comment: Test of factorial for one of the possible positive values

    #2
    Test: TEST_FACT_FOR0
    Assert: UT_ASSERTEQUAL
    Description: Expected value=1, found value=0
    Comment: Test of factorial for input equal to zero

    #3
    Test: TEST_FACT_FOR1
    Assert: UT_ASSERTEQUAL
    Description: Expected value=1, found value=0
    Comment: Test of factorial for input equal to one
    ...we can see we fixed two issues at once - test_Fact_NegativeInput and test_Fact_ForTooBigValue are gone. Hurray!

    Next step
    Then we can get rid of the special case of 0, by following modification:
    Function Fact (ByVal aInput As Long) As Quad 
    
      If Not Inside(aInput, 0, 20) Then
        Return -1
      End If     
      
      If (aInput = 0) Then
        Return 1
      End If
    
    End Function
    
    When we run the test, we can see test_Fact_For0 is really gone:
    #1
    Test: TEST_FACT_FORGENERALPOSITIVEVALUE
    Assert: UT_ASSERTEQUAL
    Description: Expected value=120, found value=0
    Comment: Test of factorial for one of the possible positive values

    #2
    Test: TEST_FACT_FOR1
    Assert: UT_ASSERTEQUAL
    Description: Expected value=1, found value=0
    Comment: Test of factorial for input equal to one
    Finish it!
    Well, what about making it actually calculating the factorial?
    Let's mod it to this:
    Function Fact (ByVal aInput As Long) As Quad 
    
      If Not Inside(aInput, 0, 20) Then
        Return -1
      End If     
      
      If (aInput = 0) Then
        Return 1
      End If     
      
      Long i
      Quad result = 1
      
      For i = 2 To aInput
        result *= i 
      Next
      
      Return result
    
    End Function
    
    Dramatic, isn't it. Let's run the test:
    Testing factorial...PASSED
    Oh yeah! So now we can be sure our Fact function works correctly for input in range 1..20.
    You don't have to be afraid to rewrite function Fact in other way, because in case of doubt, you can always automatically check it over and over again via the automated test.


    Petr

    P.S. The 3 phases of implementation are attached below, get the unitTesting.tBasicU from the first post
    Attached Files Attached Files
    Last edited by Petr Schreiber; 17-01-2014 at 13:32.
    Learn 3D graphics with ThinBASIC, learn TBGL!
    Windows 10 64bit - Intel Core i5-3350P @ 3.1GHz - 16 GB RAM - NVIDIA GeForce GTX 1050 Ti 4GB

  3. #3
    A great tool , Petr .. super !

    But in case "ut_assertEqual", is there something as " Equal within tollerance " , or can you set up such a construction ?
    Sometimes one starts with well known values without a well defined function , while knowing where it is defined and the knowledge of certain exact points of it, the calculated function in the end (while usable) may fail on exactly those points where you started from.
    (Also considering the fact p.e. that for floating points the output (let's consider singles) from oxygen is not the same as TB. some digits at the tail may vary , which make them close but not equal ).

    best, Rob

  4. #4
    @Petr

    Yes, unit testing is a great method for keeping projects consistent. I'm not sure if it's feasible to actually preface a project with a set of unit tests but once the project's core is set up, it is reasonable to get into the habit of running your unit tests whenever you add a new feature.

    As your project grows, however, the unit test code base becomes so huge (large projects even require a dedicated unit test programmer/maintainer/tester) that its output can hardly fit into a reasonable console buffer. OTOH scrollable list views don't permit individual word coloration. I'd suggest creating a helper GUI app with a scrollable richedit control where you can colorize and otherwise capitalize words however you wish. The richedit's text can also be streamed into a rich format text file.

    I have my own in-house implementation of a simple unit test system for FBSL's BASIC, for not only math but also data type casting, flow control, boolean logic, function parameter passing and function returns. The latter two are needed to keep handling of FBSL's Variant variables intact. Dynamic C has its own set of unit tests inherited from TCC. Dynamic Assembler is robust and modifications are seldom.

    @Rob

    What you call "tolerance" is usually referred to as "epsilon" in programming. And you are correct, epsilon is a must in FPU unit testing, especially for floating point evaluation of Singles at their 6th decimal place; Doubles, at decimal places after the 13th; and Extended floating-point values, at decimal places after the 15th.

    FPU precision is also largely language implementation dependent.

    As Eros said, all intermediate calc is done in TB via Extended quantities so the difference you're seeing between O2 and TB Singles is due to rounding errors on TB's side. Yes, gentlemen, these are TB's errors because O2 calculates its singles classically, i.e. directly on the FPU, as 32-bit values (DWORD-long values in asm terms) that don't accumulate intermediate calc errors in their least significant digits but rather round them up or down on the fly. There is a similar issue in FBSL where all FPU calc is historically done via Doubles. I wouldn't however call it a big problem in either (general-purpose) language if only one is aware of epsilon and allows for it in one's applied programming practice.
    Last edited by mike lobanovsky; 15-01-2014 at 07:20.
    Mike
    (3.6GHz i5 Core Quad w/ 16GB RAM, nVidia GTX 1060Ti w/ 6GB VRAM, x64 Windows 7 Ultimate Sp1)

  5. #5
    Super Moderator Petr Schreiber's Avatar
    Join Date
    Aug 2005
    Location
    Brno - Czech Republic
    Posts
    7,128
    Rep Power
    732
    Hi guys,

    thanks for the feedback!

    I think improved asserts comparing floats are good idea, maybe it could be also specified as "check for n-decimals" and so on. Something to think about, thanks!

    Mike - the output to console is secondary. The complete log is saved to file, so it can be read in any kind of text viewer. Of course, some advanced UI, like NUnit has, would be nice.
    Instead of rich text, I am considering XML. It would be structured way interpretable by custom GUIs.

    As for unit test becoming huge and complex - I am fine with this as long as I can sleep peacefully knowing the software does what it does. I think my friend Honza does it this way from project beginning.
    When designed well, it is just about adding new tests to existing pool, that is no problem.

    I will try to push this as far as possible via reflection, to have somehow user-tested approach, which could be used as base for integrated unit testing in ThinBASIC 2.x.


    Petr

    P.S. Discussion regarding precision is very interesting to me, but please open new thread for it. I would like to see failing examples, if possible.
    Last edited by Petr Schreiber; 15-01-2014 at 11:05.
    Learn 3D graphics with ThinBASIC, learn TBGL!
    Windows 10 64bit - Intel Core i5-3350P @ 3.1GHz - 16 GB RAM - NVIDIA GeForce GTX 1050 Ti 4GB

  6. #6

  7. #7
    I think he is trying to mount the honey in the red dress.
    ScriptBasic Project Manager
    Project Site
    support@scriptbasic.org

  8. #8
    Super Moderator Petr Schreiber's Avatar
    Join Date
    Aug 2005
    Location
    Brno - Czech Republic
    Posts
    7,128
    Rep Power
    732
    Hehe,

    okay, adding ut_assertRhino


    Petr
    Learn 3D graphics with ThinBASIC, learn TBGL!
    Windows 10 64bit - Intel Core i5-3350P @ 3.1GHz - 16 GB RAM - NVIDIA GeForce GTX 1050 Ti 4GB

  9. #9
    Super Moderator Petr Schreiber's Avatar
    Join Date
    Aug 2005
    Location
    Brno - Czech Republic
    Posts
    7,128
    Rep Power
    732
    Updated the header to version 1.3

    • Added support for optional setup routine for each group of tests launched via ut_LaunchTests


    In case a function named test_setup() or <prefix>_test_setup() is found, it is launched before each test.


    Petr
    Learn 3D graphics with ThinBASIC, learn TBGL!
    Windows 10 64bit - Intel Core i5-3350P @ 3.1GHz - 16 GB RAM - NVIDIA GeForce GTX 1050 Ti 4GB

  10. #10
    Super Moderator Petr Schreiber's Avatar
    Join Date
    Aug 2005
    Location
    Brno - Czech Republic
    Posts
    7,128
    Rep Power
    732
    Updated the header (and examples) to version 1.4
    • Added ut_assertEqualEpsilon
    • Added ut_assertEqualRounded
    • -
    • Added ut_GetFailureCount
    • Added ut_GetFailureTestName
    • Added ut_GetFailureAssertType
    • Added ut_GetFailureDescription
    • Added ut_GetFailureComment
    • -
    • ut_GetLastError (returns string of nonzero length in case you use ut_ functions inproperly)
    • -
    • Added support for test_TearDown function, launched after each test
    • -
    • Added ut_SaveLog to enable explicit saving
    • Changed log format to structured form:

    <header>
        <version>1.4</version>
        <date>01-17-2014</date>
        <time>12:37:44</time>
        <failCount>5</failCount>
    </header>
    <body>
        <fail>
            <id>1</id>
            <testName>TEST_FACT_NEGATIVEINPUT</testName>
            <assertType>UT_ASSERTEQUAL</assertType>
            <description>Expected value=-1, found value=0</description>
            <comment>Test of factorial for negative value</comment>
        </fail>
        <fail>
            <id>2</id>
            <testName>TEST_FACT_FORTOOBIGVALUE</testName>
            <assertType>UT_ASSERTEQUAL</assertType>
            <description>Expected value=-1, found value=0</description>
            <comment>Test of factorial for value out of bounds, on the positive side</comment>
        </fail>
        <fail>
            <id>3</id>
            <testName>TEST_FACT_FORGENERALPOSITIVEVALUE</testName>
            <assertType>UT_ASSERTEQUAL</assertType>
            <description>Expected value=120, found value=0</description>
            <comment>Test of factorial for one of the possible positive values</comment>
        </fail>
        <fail>
            <id>4</id>
            <testName>TEST_FACT_FOR0</testName>
            <assertType>UT_ASSERTEQUAL</assertType>
            <description>Expected value=1, found value=0</description>
            <comment>Test of factorial for input equal to zero</comment>
        </fail>
        <fail>
            <id>5</id>
            <testName>TEST_FACT_FOR1</testName>
            <assertType>UT_ASSERTEQUAL</assertType>
            <description>Expected value=1, found value=0</description>
            <comment>Test of factorial for input equal to one</comment>
        </fail>
    </body>
    

    Petr
    Last edited by Petr Schreiber; 17-01-2014 at 13:44.
    Learn 3D graphics with ThinBASIC, learn TBGL!
    Windows 10 64bit - Intel Core i5-3350P @ 3.1GHz - 16 GB RAM - NVIDIA GeForce GTX 1050 Ti 4GB

Page 1 of 2 12 LastLast

Similar Threads

  1. OpenB3D Engine Wrapper for ThinBasic (open testing)
    By Artemka in forum TBGL Scripts and Projects
    Replies: 4
    Last Post: 12-03-2013, 23:38
  2. Astronomical Unit (module) simple example for testing :)
    By largo_winch in forum thinBasic vaporware
    Replies: 1
    Last Post: 24-12-2011, 11:21
  3. Simple unit for UDT list in ThinBASIC
    By Petr Schreiber in forum Sources, Templates, Code Snippets, Tips and Tricks, Do you know ...
    Replies: 3
    Last Post: 10-07-2010, 21:21
  4. Development for thinBasic
    By kryton9 in forum Other languages
    Replies: 12
    Last Post: 06-08-2008, 17:40
  5. Speed2, a comparative speed test through different development environments
    By RobertoBianchi in forum Suggestions/Ideas discussions
    Replies: 28
    Last Post: 21-02-2008, 22:43

Members who have read this thread: 1

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •