Page 1 of 2 12 LastLast
Results 1 to 10 of 15

Thread: Storing massive amounts of data in INI files

  1. #1
    Member marcuslee's Avatar
    Join Date
    Sep 2008
    Location
    Kansas, USA
    Age
    42
    Posts
    222
    Rep Power
    38

    Storing massive amounts of data in INI files

    It would seem to me that one can store massive amounts of data in an ini file. Plus, if the data is similar, it could be stored in the same field, which can easily be parsed or split into an array.

    The following code doesn't even begin to show what an INI file could do (methinks), but it is a start.

    Uses "INI"
    
    Dim x, n As String
    Dim z(10), d(10) As String
    Dim y,v,b As Byte
    Dim iFile As String = APP_ScriptPath + "file.ini"
    
    INI_SetKey(iFile, "1", "1", "23|26|65|76|87|43|21|35|43|99")
    x = INI_GetKey(iFile, "1", "1")           
    
    y = Parse(x,z,"|") 
    v = Split(x,"|",d)
    
    n = d(3) + z(2)
            
    b = MsgBox (0,n)
    
    Now, I see a possible problem ... hopefully there isn't one. Is there a limit to the amount of data that an INI file can or should hold?

    I have an idea for a non-graphical game that might require a lot of variables/game data to be stored in a file. When I say a lot, I mean hundreds if not thousands. Much of the data would be related to each other (earlier versions of the same piece of data, for example), so they could be stored together as I have shown in the above example.

    Is an INI file a good way to do this? Are there other ways? Pros and cons?




    Mark

  2. #2
    thinBasic MVPs
    Join Date
    May 2007
    Location
    UK
    Posts
    1,427
    Rep Power
    159
    Marcus an ini file is just a text file so it's no worse or better that any other method it does have one advantage that you can edit it in notepad,wordpad,word... and doesn't require a program specially written.

    If you keep the format of the file in some sort of order you could overlay an array of UDT'S which makes it more like a record system.
    Home Desktop : Windows 7 - Intel Pentium (D) - 3.0 Ghz - 2GB - Geforce 6800GS
    Home Laptop : WinXP Pro SP3 - Intel Centrino Duo - 1.73 Ghz - 2 GB - Intel GMA 950
    Home Laptop : Windows 10 - Intel(R) Core(TM) i5-4210U CPU @ 1.70GHz, 2401 Mhz, 2 Core(s), 4 Logical Processor(s) - 4 GB - Intel HD 4400
    Work Desktop : Windows 10 - Intel I7 - 4 Ghz - 8GB - Quadro Fx 370

  3. #3
    Member marcuslee's Avatar
    Join Date
    Sep 2008
    Location
    Kansas, USA
    Age
    42
    Posts
    222
    Rep Power
    38
    Quote Originally Posted by Michael Clease View Post
    If you keep the format of the file in some sort of order you could overlay an array of UDT'S which makes it more like a record system.
    How would an overlay work? A simple example?



    Mark

  4. #4
    thinBasic MVPs
    Join Date
    May 2007
    Location
    UK
    Posts
    1,427
    Rep Power
    159

    Hows this

    Uses "File"
    
    Type tRecord
      one    As Byte
      two    As Byte
      three  As Byte
      four   As Byte
      Txt    As String*10
    End Type
    
    Dim FileBuffer As String
    Dim FileName   As String
    Dim sMsg       As String
    Dim FirstValue As Byte
    
      FileBuffer = FILE_Load(FileName)
    
    '   This must be called after filling the buffer or you can use REDIM
    Dim MyRecord(10) As tRecord At StrPtr(FileBuffer)
    
     FirstValue = MyRecord(1).One
    
    Forgot the array
    Home Desktop : Windows 7 - Intel Pentium (D) - 3.0 Ghz - 2GB - Geforce 6800GS
    Home Laptop : WinXP Pro SP3 - Intel Centrino Duo - 1.73 Ghz - 2 GB - Intel GMA 950
    Home Laptop : Windows 10 - Intel(R) Core(TM) i5-4210U CPU @ 1.70GHz, 2401 Mhz, 2 Core(s), 4 Logical Processor(s) - 4 GB - Intel HD 4400
    Work Desktop : Windows 10 - Intel I7 - 4 Ghz - 8GB - Quadro Fx 370

  5. #5
    Member marcuslee's Avatar
    Join Date
    Sep 2008
    Location
    Kansas, USA
    Age
    42
    Posts
    222
    Rep Power
    38

    A correct extension?

    Extending your example, would you use something like the following line?

    INI_SetKey(FileName, "1", "1", MyRecord)


    Of course, the name of the section and the record could be different in the file. Those are just labels that don't mean anything to the program ... just organizing for the programmer, correct?



    Mark

  6. #6
    thinBasic MVPs
    Join Date
    May 2007
    Location
    UK
    Posts
    1,427
    Rep Power
    159
    Sorry I forgot the INI thing and just thought of the way I would do it

    If you use INI you will be bashing the HDD and performance for lots of entries would be quite slow.

    Uses "File","INI"
    
    Type tRecord
      one    As Byte
      two    As Byte
      three  As Byte
      four   As Byte
      Txt    As String*10
    End Type
    
    Dim FileBuffer As String
    Dim FileName   As String = APP_ScriptPath + "file.ini"
    Dim sMsg       As String
    Dim FirstValue As Byte
    
    '  FileBuffer = FILE_Load(FileName)
      FileBuffer = String$(SizeOf(tRecord)*10, $SPC)  
    
    '   This must be called after filling the buffer or you can use REDIM
    Dim MyRecord(10) As tRecord At StrPtr(FileBuffer)
    
     MyRecord(1).One   = 54
     MyRecord(1).two   = 55
     MyRecord(1).three = 56
     MyRecord(1).four  = 57
     MyRecord(1).txt   = "Record 1"  
     MyRecord(2).txt   = "Record 2"  
     
     
     INI_SetKey(FileName, "1",   "1", MyRecord(1).One)
     INI_SetKey(FileName, "1",   "2", MyRecord(1).two)
     INI_SetKey(FileName, "1",   "3", MyRecord(1).three)
     INI_SetKey(FileName, "1",   "4", MyRecord(1).four)
     INI_SetKey(FileName, "1", "Txt", MyRecord(1).txt)
    
     INI_SetKey(FileName, "FullString", "String", FileBuffer)
    
    As you can see its a lot of work for not much gain, as I said I would just do it without ini its already formatted by the udt you just need to watch the filebuffer is large enough for the array.
    Home Desktop : Windows 7 - Intel Pentium (D) - 3.0 Ghz - 2GB - Geforce 6800GS
    Home Laptop : WinXP Pro SP3 - Intel Centrino Duo - 1.73 Ghz - 2 GB - Intel GMA 950
    Home Laptop : Windows 10 - Intel(R) Core(TM) i5-4210U CPU @ 1.70GHz, 2401 Mhz, 2 Core(s), 4 Logical Processor(s) - 4 GB - Intel HD 4400
    Work Desktop : Windows 10 - Intel I7 - 4 Ghz - 8GB - Quadro Fx 370

  7. #7
    Member marcuslee's Avatar
    Join Date
    Sep 2008
    Location
    Kansas, USA
    Age
    42
    Posts
    222
    Rep Power
    38
    Quote Originally Posted by Michael Clease View Post
    If you use INI you will be bashing the HDD and performance for lots of entries would be quite slow.

    ...

    As you can see its a lot of work for not much gain, as I said I would just do it without ini its already formatted by the udt you just need to watch the filebuffer is large enough for the array.
    I thought that might be the case ... slow performance for lots of data. What would be a good way to store a massive amount of data in a file for later retrieval, the next time the game is played?


    Mark

  8. #8
    thinBasic MVPs
    Join Date
    May 2007
    Location
    UK
    Posts
    1,427
    Rep Power
    159
    What will the data be? What sort of format is the data?

    If its a repeating format then my first example would work quite happily and you could save the FileBuffer at the end of the program which reduces bashing.
    Home Desktop : Windows 7 - Intel Pentium (D) - 3.0 Ghz - 2GB - Geforce 6800GS
    Home Laptop : WinXP Pro SP3 - Intel Centrino Duo - 1.73 Ghz - 2 GB - Intel GMA 950
    Home Laptop : Windows 10 - Intel(R) Core(TM) i5-4210U CPU @ 1.70GHz, 2401 Mhz, 2 Core(s), 4 Logical Processor(s) - 4 GB - Intel HD 4400
    Work Desktop : Windows 10 - Intel I7 - 4 Ghz - 8GB - Quadro Fx 370

  9. #9
    Member marcuslee's Avatar
    Join Date
    Sep 2008
    Location
    Kansas, USA
    Age
    42
    Posts
    222
    Rep Power
    38
    Quote Originally Posted by Michael Clease View Post
    What will the data be? What sort of format is the data?
    The data and format will vary across the board: numbers and strings, long and short, precise (floating point) and integer.

    If its a repeating format then my first example would work quite happily and you could save the FileBuffer at the end of the program which reduces bashing.
    A little confused here.

    Your first example didn't include INI, right? If so, what type of file would it be?

    How does knowing the size of the file speed up the program? I'm sorry if I misunderstand.




    Mark

  10. #10
    thinBasic MVPs
    Join Date
    May 2007
    Location
    UK
    Posts
    1,427
    Rep Power
    159
    The file type would be custom it would be your format

    Where did I say the size of the file would speed it up ?

    you say "The data and format will vary across the board: numbers and strings, long and short, precise (floating point) and integer."

    but will this data be same for each record?

    Because I load the all of the records from one file in one lump and then edit that information in memory I only access the hdd ONCE to read and then when I finish with it at the end of the program I could SAVE it then.
    Home Desktop : Windows 7 - Intel Pentium (D) - 3.0 Ghz - 2GB - Geforce 6800GS
    Home Laptop : WinXP Pro SP3 - Intel Centrino Duo - 1.73 Ghz - 2 GB - Intel GMA 950
    Home Laptop : Windows 10 - Intel(R) Core(TM) i5-4210U CPU @ 1.70GHz, 2401 Mhz, 2 Core(s), 4 Logical Processor(s) - 4 GB - Intel HD 4400
    Work Desktop : Windows 10 - Intel I7 - 4 Ghz - 8GB - Quadro Fx 370

Page 1 of 2 12 LastLast

Similar Threads

  1. Storing not allows
    By peter in forum General
    Replies: 15
    Last Post: 13-10-2008, 21:08

Members who have read this thread: 0

There are no members to list at the moment.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •