Uploaded image for project: 'OpenDJ'
  1. OpenDJ
  2. OPENDJ-7476

Add a histogram monitoring metric which records operations which target large entries

    XMLWordPrintable

    Details

      Description

      Operations which very frequently access large entries, especially those modifying them, are likely to have a high resource impact:

      • increased CPU costs decoding and encoding the large entries at the DB and network layer
      • increased network costs transferring the large entries (if they are unfiltered) over the network
      • increased disk costs reading and writing the entries to the DB.

      The last point is particularly important.

      This issue can be closed once we have monitoring metrics similar to these (we may want different thresholds):

      • ds-mon-updates-larger-than-10kb
      • ds-mon-updates-larger-than-100kb
      • ds-mon-updates-larger-than-1mb
      • ds-mon-updates-larger-than-10mb
      • ds-mon-reads-larger-than-10kb
      • ds-mon-reads-larger-than-100kb
      • ds-mon-reads-larger-than-1mb
      • ds-mon-reads-larger-than-10mb

      The attributes would contain a histogram metric similar to our other operation based metrics:

      {"count":0,"total":0.000,"mean_rate":0.000,"m1_rate":0.000,"m5_rate":0.000,"m15_rate":0.000,"mean":0.000,"min":0.000,"max":0.000,"stddev":0.000,"p50":0.000,"p75":0.000,"p95":0.000,"p98":0.000,"p99":0.000,"p999":0.000,"p9999":0.000,"p99999":0.000}
      

      Questions:

      • which component are the metrics associated with? The connection handlers? Backends? Global?
      • how do we measure the size? The best candidate is likely to be at the DB layer. We could record the size at the point where we read/write id2entry. However, abstraction layers may make it hard to propagate this information up the call-stack so that it can be reported correctly. Yannick Lecaillez has suggested systematically adding a virtual attribute at the DB layer. It would bypass the virtual attribute rule config, but perhaps it doesn't matter.

        Attachments

          Issue Links

            Activity

              People

              ondrej.fuchsik Ondrej Fuchsik
              matthew Matthew Swift
              Yannick Lecaillez Yannick Lecaillez
              Ondrej Fuchsik Ondrej Fuchsik
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: