Results 1 to 2 of 2

Thread: calculator bug?

  1. #1

    calculator bug?

    I've been playing around with the two calculator examples and I think I have found a bug in the code. Do this:
    1 press the 6 button
    2 press the 1/x button
    answer: .166666666666667
    3 press the 1/x button
    answer: 5.99999999999999

    I'm not too good at math but this seems to give the wrong answer. Could someone let me know why it gives the above result?
    I've tried this in the Microsoft calculator and it gives a different answer. Pressing the 1/x button the second time gives an answer of 6.

    Thanks
    Sandy

  2. #2
    thinBasic author ErosOlmi's Avatar
    Join Date
    Sep 2004
    Location
    Milan - Italy
    Age
    57
    Posts
    8,782
    Rep Power
    10

    Re: calculator bug?

    Sandy,

    you are right. The problem are the number of decimal places used when converting from number to string and the other way round. thinBasic is limited with the limit of the compiler used. Max number of decimals in converting number to strings are 18 (in calculator example 15 are used). Internal calculations are done with the precision of EXTENDED numbers (so very high) but continuous conversions /string/number/string introduce a lot of roundings.

    Microsoft Calculator uses much more decimal places to perform conversions.

    I will see what I can do because I do not like that too.

    Ciao
    Eros
    www.thinbasic.com | www.thinbasic.com/community/ | help.thinbasic.com
    Windows 10 Pro for Workstations 64bit - 32 GB - Intel(R) Xeon(R) W-10855M CPU @ 2.80GHz - NVIDIA Quadro RTX 3000

Similar Threads

  1. Power supply calculator
    By Michael Clease in forum General
    Replies: 8
    Last Post: 12-04-2008, 08:36

Members who have read this thread: 0

There are no members to list at the moment.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •