execption when inserting record in DISKLESS mode
Brought to you by:
knizhnik
Menu
▾
▴
1 Attachments
#7 execption when inserting record in DISKLESS mode
fastdb version: 3.73
env: windows 8, visual c++ 2010 sp1
desc:
I wrote a short program which using fastdb in DISKLESS mode (#define DISKLESS_CONFIGURATION 1), which created a db instance with initialsize = 1000000000(1GB), and repeat insert a small struct to db. fastdb throw a FileLimitExeeded execption when the No.198181 record is inserting.
I've checked the call stack, the execption throwed in function inline void dbDatabase::extend(offs_t size), where the parameter size = 0xfff82000, maybe something wrong?
the program is attached.
In diskless mode FastDB is not able to reallocate database , so initial
database size is actually maximal database size.
Please notice that specified size of database determines size of virtual
memory region and doesn't cause allocation of correspondent amount of
physical memory. So you can specify very larger initial database size in
diskless mode - much more than for example physical memory at your
system. It will not cause inefficient usage of system resources.
On 09/24/2013 01:35 PM, yymailb wrote:
Related
Bugs: #7
I've tried some different initial size, such as 100MB, 1GB, and 2GB. in each condition, the program always throw exception at No. 198181 record. the computer has 16GB phisical memory. I noticed that the fastdb call function dbDatabase::extend(offs_t size) with parameter size = 0xfff82000(4.29GB), I'm not sure if this value exceed some limit for a 32bit process.
Are you using 32-bit or 64-bit version of Windows?
Did you build FastDB and you applications as 64-bit application?
Fro 32-bit application size of database is naturally limited by 4Gb.
On 09/24/2013 05:03 PM, yymailb wrote:
Related
Bugs: #7
I'm using a 64bit windows 8. fastdb and my program are compiled as 32bit. I know that the 4gb limit for 32bit process. In this case, the struct size is 48 byte, 198181 records means about 9.5MB, much less than initial db size, why dbDatabase::extend try resize to 4.2GB?
I find out a problem - it is caused by my recent attempt to avoid too
deep recursion in memory allocator.
For diskless mode which is not able to extend database this approach
doesn't work.
I have committed the fix in SVN repository at SourceForge.
On 09/24/2013 05:43 PM, yymailb wrote:
Related
Bugs: #7
I've checked out the newest version, it's works well. Now I can insert 255826 records in a 100MB db before exception, each record cost 390 bytes while the record struct size is 48 bytes. Is this normal?
Last edit: yymailb 2013-09-25
On 09/25/2013 05:42 AM, yymailb wrote:
Sorry, I can not reproduce the problem. In my case performance is about
10 seconds for inserting all objects.
Please check that you are still using diskless mode.
In any case - transaction commit after each insert is not so efficient
even in diskless mode (and in normal mode it will be extremely sow
because each transaction commit will require synchronous write to the
disk). You can reduce insert time by doing all inserts within one
transaction or using delayed commit policy or at least perform commit
after each N inserts.
Related
Bugs: #7
I've checked out the newest version, it's works well. Now I can insert 255826 records in a 100MB db before exception, each record cost 390 bytes while the record struct size is 48 bytes. Is this normal?
On 09/25/2013 10:38 AM, yymailb wrote:
Estimated average size of DbMarginRate object indatabase is 64 bytes (8
+ 4 + 8*4 + 4 + 10) align on 16 byte quantum.
Also there 3 indexes - about 24 extra bytes per object. Plus two
OID->offset maps - up tp 16 bytes per object.
Large number of short transactions (frequent commits) requires
allocation of huge number of shadow page copies. Them are deallocated on
commit, but it leads to fragmentation and also can cause growth of
database size.
If you are building FastDB with LARGE_DATABASE_SUPPORT ( I am not sure
that it happen i you case because you are using 32-bit mode),
then FastDB supports up tp 1Tb database and reserve about 100 space for
bitmap pages.
What is the size of empty database in your case (when you create
database and just insert 1 object)?
Related
Bugs: #7
I created database and just insert 1 object, db.getDatabaseSize() return 1000341504, db.getUsedDatabaseSize() return 4337664.
Last edit: yymailb 2013-09-25
I created database and just insert 1 object, db.getDatabaseSize() return 1000341504, db.getUsedDatabaseSize() return 4337664.