Virtual issue
issueid=436 16-08-2013 12:55
thinBasic MVPs
Number of reported issues by ReneMiner: 83
Virtual issue
Heap is loosing data somehow...I thought at first....

I made some tests and found out that resizing a virtual array-overlay to a smaller size affects the stored data - it simply gets lost. (= overwritten with 0-bytes)

Example fills in data for 3 elements. Put some overlay there to read out.
Redim the overlay to another (smaller) size will delete the leftover elements not just virtually. The position where to redim At does not make a difference, can be 0 or any other pointer. Especially when having more than one heap and the overlay has to be moved from a larger to a smaller one the large one looses it's data. That's especially leading to enormous leaks when these heaps just store pointers - they're all lost...

(script reads out values twice for re-check)

Uses "console"
' setup some type
Type t_Type
  R As Byte
  G As Byte
  B As Byte
End Type  

DWord foo, i
String sTemp
    
  For i = 1 To 3
  ' create some data 
    sTemp += MKBYT$(i, 2 * i , 3 * i)   
  Next    
    
  foo = HEAP_AllocByStr(sTemp) 
  
  ' check what we have:
  Dim vArray(HEAP_Size(foo)/SizeOf(t_type)) As t_Type At foo 

  For i = 1 To UBound(vArray)
    PrintL "foo("+i+").R = " + vArray(i).R
    PrintL "foo("+i+").G = " + vArray(i).G
    PrintL "foo("+i+").B = " + vArray(i).B
    PrintL     
  Next 
' resize the virtual overlay deletes heap-content!
  ReDim vArray(1) At 0 ' or even foo - no matter at where
  
  PrintL $CRLF + Repeat$(50,"-") + $CRLF
  PrintL "key to continue" + $CRLF
  WaitKey  
  
  ' check again:
  ReDim vArray(HEAP_Size(foo)/SizeOf(t_type)) At foo 

  For i = 1 To UBound(vArray)
    PrintL "foo("+i+").R = " + vArray(i).R
    PrintL "foo("+i+").G = " + vArray(i).G
    PrintL "foo("+i+").B = " + vArray(i).B
    PrintL     
  Next 
  

  PrintL $CRLF + Repeat$(50,"-") + $CRLF
  PrintL "key to continue" + $CRLF
  WaitKey  
 
  ' check once again peeking bytes directly so it's not just virtual display-issue:
  For i = 1 To HEAP_Size(foo)/SizeOf(t_Type)
    PrintL "foo("+i+").R = " + Peek(foo + (i-1) * 3)
    PrintL "foo("+i+").G = " + Peek(foo + (i-1) * 3 + 1)
    PrintL "foo("+i+").B = " + Peek(foo + (i-1) * 3 + 2)
    PrintL     
  Next
  
PrintL
PrintL "key to end" + $CRLF
WaitKey
Issue Details
Issue Number 436
Issue Type Bug
Project thinBasic
Category Core engine (thinCore.dll)
Status Fixed
Priority 1 - Highest
Affected Version 1.9.7
Fixed Version 1.9.8
Milestone (none)
Users able to reproduce bug 0
Users unable to reproduce bug 0
Attachments 0
Assigned Users (none)
Tags (none)




16-08-2013 13:42
thinBasic author
Try REDIM PRESERVE ...
Reply Reply  

16-08-2013 13:50
thinBasic MVPs
that's not working. (posted image removerd)
Reply Reply  

16-08-2013 16:46
Super Moderator
Hi guys!,

Petr's testing code:
Uses "Console"

Long nNumbers(3)

nNumbers(1) = 1, 2, 3

PrintL "Original:"
PrintL Join$(nNumbers, ", ")

PrintL "After REDIM to 1 element:"
ReDim nNumbers(1)
PrintL Join$(nNumbers, ", ")

PrintL 

PrintL "Original:"        
ReDim nNumbers(3)
nNumbers(1) = 1, 2, 3
PrintL Join$(nNumbers, ", ")

PrintL "After REDIM PRESERVE to 1 element:"
ReDim Preserve nNumbers(1)
PrintL Join$(nNumbers, ", ")

' -- Now virtual...                  
PrintL 

ReDim nNumbers(3)
nNumbers(1) = 1, 2, 3
Long nVirtual(3) At VarPtr(nNumbers(1))

PrintL "Original virtual (should be 1, 2, 3):"
PrintL Join$(nNumbers, ", ")

PrintL "After REDIM virtual to 1 element (should be 0 and 1, 0, 0):"
ReDim nVirtual(1)
PrintL Join$(nVirtual, ", ") + "<-- this should be zero"
PrintL Join$(nNumbers, ", ")

PrintL 

PrintL "Original virtual (should be 1, 2, 3):"        
ReDim nVirtual(3)
nVirtual(1) = 1, 2, 3
PrintL Join$(nVirtual, ", ")

PrintL "After REDIM PRESERVE to 1 element (should be 1 and 1, 0, 0):"
ReDim Preserve nVirtual(1) ' -- Not allowed
PrintL Join$(nVirtual, ", ")
PrintL Join$(nNumbers, ", ")               

WaitKey
To me, the REDIM for virtual array behaves like REDIM PRESERVE, while REDIM PRESERVE for virtual is not possible to use - this could use some fine tuning.

But!

To allow what Renne needs, we would need another keyword, because leaving the memory as-was is not something default. I would propose for example:
REDIM OVERLAY myArray(newDimension)
This way it would behave like REDIM PRESERVE + it would leave the "released" memory untouched - no zeroing.


Petr
Reply Reply  

16-08-2013 17:20
thinBasic MVPs
If you do "Overlay" I suggest to do it as least as good or even better as oxygen does:

oxygen-syntax:
Dim vArray as myType At StrPtr(myString)
which automatically creates an array of myType at this stringpointer with as much as elements fit into length of this string...

desired tB-synatx:
Dim vArray As myType [Overlay] At StrPtr(myString)
using DIM here should create an array vArray() with as much elements of myType as fit onto that string - always start with first element of course- alike current syntax:
Dim vArray(StrPtrLen(StrPtr(myString))\SizeOf(myType)) as myType At StrPtr(myString)
now:
Dim vArray As myType Overlay At Heap(myHeap)

Redim vArray Overlay At Dictionary(myDict, myKey) ' should not affect the old overlays content
Redim vArray Overlay At LL_Item(myLL, myItemhandle)
perhaps it can be done by making Overlay + At to OverlayAt - I would not care...

while just this:
Local vElement As myType At StrPtr(myString)
should not make an array - no DIM-keyword here...

But however- it's very important that the elements on the previous overlay stay untouched when putting the overlay in different size somewhere else. Otherwise the overlay works only once per function and it would need a virtual variable for each - problem here is- i think there is no "multidimensional" overlay possible as
Dim vArray(1, DataLen/SizeOf(myType)) At myPtr(1)
Dim vArray(2, DataLen/SizeOf(myType)) At myPtr(2) '?
maybe just without Dim:
Overlay_String varName() At myStrPtr As myType
Overlay_Heap varName() At hPtr As MyType
Overlay_Dictionary varName() At  myDictPtr As myType
Overlay_LL varname() At myLLPtr As myType
even without those parens ()
for simple variables there's still SetAt...
Reply Reply  

16-08-2013 17:53
Super Moderator
I think it should be two different things - not every time you need to overlay memory completely, not every time you need to let the memory untouched...
But generally I support both - the smart REDIM could be faster than the zeroing one, and the complete coverage one could be useful in many cases.


Petr
Reply Reply  

16-08-2013 17:57
thinBasic MVPs
Of course. But when you're already work at the root...

The current behaviour is kind of stupid. Why does it delete memory-content of the old layover-area when moving the virtual layover in smaller size somewhere else?
It's supposed to be virtual...it doesnt resize the Heap though so does not really ReDim anyway... therefor one could use a dozen other methods to overwrite the unwanted data with 0... virtual layover as it is now is very unsafe and leads to leaking memory and losing pointers. It needs a method to place some layover in any size anywhere - of course the initial type has to stay the same.
But it must not delete the data at the "old place" when moving to another place. If ReDim at same place that would be maybe OK, but not when it's placed onto another position.
-if I do a simple variable using SetAt it doesn't assign the value (equals array-bound or dimensions size: would be the same if it would cut a few bits from a simple variable) from new position at the old position- does it?
I don't care if I had to use another keyword nor the syntax - I just care about that priceless, most important, unpayable and irrecoverable user-data when user sits hours and hours there -and all of a sudden:
All is gone...

And another virtual problem is 
long x at myPtr
X = 1
when myPtr is 0 the X=1 produces an error. I would prefer if a variable at 0 just would ignore any try to assign something
Reply Reply  

17-08-2013 12:37
thinBasic author
Fixed, next version REDIM of a virtual arrays will never clear memory area but just change its size and optionally its memory position.
Reply Reply  

17-08-2013 14:33
thinBasic MVPs
Great :)
Reply Reply  

17-08-2013 17:10
thinBasic author
Beta of betas :)

http://www.thinbasic.biz/projects/th...ic_1.9.8.0.zip

Few changes comparing to 1.9.7 but more in coming days.
Reply Reply  

17-08-2013 17:53
thinBasic MVPs
Now it works - I am happy and you earned yourself some cool drink
Reply Reply  

+ Reply