Uploaded image for project: 'OpenDJ'
  1. OpenDJ
  2. OPENDJ-3315

Reduce memory usage of PDB Exchange Buffers

    Details

    • Type: Improvement
    • Status: Done
    • Priority: Minor
    • Resolution: Won't Fix
    • Affects Version/s: 4.0.0
    • Fix Version/s: Not applicable
    • Component/s: backends
    • Labels:
      None
    • Support Ticket IDs:

      Description

      The changes made in OPENDJ-2719 meant that PDB Exchange buffers which exceeded 4MB are not retained in the heap.

      However these buffers can still contain a significant amount of data each (up to 4MB).

      Since the number of Exchanges used is a function of worker threads * backend containers, the amount of total heap used by these objects can vary significantly. With just a moderate increase in typical default numbers it is possible to see a very high number, for example 64 worker threads and 3 backends with 15 containers is 64*3*15 = 2880 exchange objects.

      If each object can retain up to 4MB this can use up a lot of memory very quickly, e.g if 25% the buffers end up at 2MB values that's still 1.4GB of memory, and this is far from the worst case scenario.

      If possible it might be useful to decide whether to retain Exchange buffers dynamically based on how much memory is already being used/available and perhaps relative to backend cache sizes. This would keep the memory usage lower and more predictable.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                matthew Matthew Swift
                Reporter:
                ian.packer Ian Packer [X] (Inactive)
                Dev Assignee:
                Matthew Swift
              • Votes:
                1 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: