1. Summary
  2. Files
  3. Support
  4. Report Spam
  5. Create account
  6. Log in

Ticket #217 (closed defect: fixed)

Opened 2 years ago

Last modified 17 months ago

BTreeCounters does not track bytes released.

Reported by: thompsonbry Owned by: thompsonbry
Priority: major Milestone:
Component: B+Tree Version:
Keywords: counters Cc: martyncutcher

Description

The BTreeCounters class tracks the #of bytes written on each index, but it does not track the #of bytes associated with records which have been deleted. Thus, it is not possible to compute the #of bytes for only the current records on the B+Tree. This means that estimates of the average page size based on the current #of nodes and leaves and bytes written are incorrect, at least for the RWStore.

Add a counter for bytesReleased and update logic which is using bytesWritten under the assumption that it reflects the WORM architecture, which does not recycle, and the scale-out architecture, which recycles by deleting entire files.

Change History

Changed 2 years ago by thompsonbry

There is also interest in:

- Making the counters restart safe (save them with the B+Tree checkpoint).

- Tracking the eviction rate for the index.

- Using the load size of the index nodes and leaves (from their persistent addresses) and possibly metrics about eviction rates to dynamically control the size of the write retention queue.

Changed 2 years ago by thompsonbry

BTree#removeAll() has an optimization for the RWStore. The performance counters need to be updated in that optimized code path as well.

Changed 2 years ago by thompsonbry

Once we have restart safe performance counters for the B+Tree we can rapidly report on the statistics for the different indices. DumpJournal? and DumpIndex? currently coordinate to do this, but this process involves a page scan of the index. With restart safe performance counters which correctly track the bytes utilized for nodes and leaves we could produce the same table "instantly".

Changed 17 months ago by thompsonbry

  • status changed from new to closed
  • resolution set to fixed

Change sets for RWStore history and persistent memory leaks following code review with MCutcher.

Committed Revision r5809 (against 1.0.x maintenance branch and TERMS_REFACTOR_BRANCH)

@see https://sourceforge.net/apps/trac/bigdata/ticket/217 (BTreeCounters does not track bytes released)
@see https://sourceforge.net/apps/trac/bigdata/ticket/329 (B+Tree should delete bloom filter when it is disabled)
@see https://sourceforge.net/apps/trac/bigdata/ticket/372 (RWStore does not prune commit record index).
@see https://sourceforge.net/apps/trac/bigdata/ticket/375 (Persistent memory leaks (RWStore/DISK))

Note: See TracTickets for help on using tickets.