Uploaded image for project: 'OpenDJ'
  1. OpenDJ
  2. OPENDJ-4027

Contention in compressed schema

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 5.5.0, 4.0.0, 3.5.2, 3.5.1
    • Fix Version/s: 5.5.0
    • Component/s: backends
    • Support Ticket IDs:
    • Sprint:
      OpenDJ Sprint 106

      Description

      There is a significant performance drop between 2.4.6/2.6.4 and 3.5.x/4.0.0

      Using the following type of searchrate, performance drops off from 2.x to 3.x/4.x.

      • Using template generated dc=example,dc=com data the drop off is not as significant.
      • Using customer supplied data, the drop of is significant.
      ./searchrate -h localhost --port 1389 -D "cn=Directory Manager" -w password -s sub --baseDN "ou=People,dc=example,dc=com" --numConnections 50 --targetThroughput 50 --maxDuration 60 "(objectClass=*)" frguid xxxx xxxx
      

      Template example data + default schema:

      Testing 2.4.6 - dc=example,dc=com
      -------------------------------------------------------------------------------
           Throughput                            Response Time                       
         (ops/second)                           (milliseconds)                       
      recent  average  recent  average  99.9%  99.99%  99.999%  err/sec  Entries/Srch
      -------------------------------------------------------------------------------
        59.9     60.0  213.725  213.725  815.00  815.00   815.00      0.0        1116.0
        50.1     55.0  67.080  147.068  815.00  815.00   815.00      0.0        1116.0
        50.2     53.4  49.403  116.548  815.00  815.00   815.00      0.0        1116.0
        50.0     52.6  57.052  102.382  809.00  815.00   815.00      0.0        1116.0
        50.0     52.0  39.565   90.302  809.00  815.00   815.00      0.0        1116.0
        50.0     51.7  37.105   81.722  809.00  815.00   815.00      0.0        1116.0
        50.0     51.5  39.890   75.912  809.00  815.00   815.00      0.0        1116.0
        50.0     51.3  37.833   71.268  796.00  815.00   815.00      0.0        1116.0
        50.0     51.1  36.809   67.522  796.00  815.00   815.00      0.0        1116.0
        50.0     51.0  48.167   65.625  796.00  815.00   815.00      0.0        1116.0
        50.0     50.9  35.133   62.902  796.00  815.00   815.00      0.0        1116.0
        50.0     50.8  35.919   60.691  795.00  815.00   815.00      0.0        1116.0
      
      Testing 2.6.4 - dc=example,dc=com
      -------------------------------------------------------------------------------
           Throughput                            Response Time                       
         (ops/second)                           (milliseconds)                       
      recent  average  recent  average  99.9%  99.99%  99.999%  err/sec  Entries/Srch
      -------------------------------------------------------------------------------
        60.0     60.0  241.351  241.351  1200.00  1200.00  1200.00      0.0        1116.0
        50.1     55.1  36.543  148.256  1200.00  1200.00  1200.00      0.0        1116.0
        50.0     53.4  35.930  113.154  1200.00  1200.00  1200.00      0.0        1116.0
        50.0     52.5  36.556   94.916  1100.00  1200.00  1200.00      0.0        1116.0
        50.0     52.0  33.249   83.057  1100.00  1200.00  1200.00      0.0        1116.0
        50.0     51.7  33.971   75.140  1100.00  1200.00  1200.00      0.0        1116.0
        50.0     51.4  38.435   70.042  1100.00  1200.00  1200.00      0.0        1116.0
        50.0     51.3  58.438   68.627  1100.00  1200.00  1200.00      0.0        1116.0
        50.0     51.1  30.663   64.500  1100.00  1200.00  1200.00      0.0        1116.0
        50.0     51.0  36.475   61.753  1100.00  1200.00  1200.00      0.0        1116.0
        50.0     50.9  31.922   59.089  1100.00  1200.00  1200.00      0.0        1116.0
        49.0     50.7  31.542   56.873  1000.00  1200.00  1200.00      0.0        1116.0
      
      Testing 3.5.1 - dc=example,dc=com
      -------------------------------------------------------------------------------
           Throughput                            Response Time                       
         (ops/second)                           (milliseconds)                       
      recent  average  recent  average  99.9%  99.99%  99.999%  err/sec  Entries/Srch
      -------------------------------------------------------------------------------
        52.6     52.6  551.075  551.075  2000.00  2000.00  2000.00      0.0        1139.1
        50.5     51.5  65.423  312.974  2000.00  2000.00  2000.00      0.0        1118.0
        50.7     51.2  74.168  234.202  2000.00  2000.00  2000.00      0.0        1095.7
        49.4     50.8  58.965  191.516  2000.00  2000.00  2000.00      0.0        1130.1
        50.6     50.7  65.233  166.299  2000.00  2000.00  2000.00      0.0        1106.8
        49.6     50.6  59.444  148.807  2000.00  2000.00  2000.00      0.0        1111.4
        49.9     50.5  57.740  135.953  2000.00  2000.00  2000.00      0.0        1130.4
        50.4     50.5  63.210  126.860  1900.00  2000.00  2000.00      0.0        1103.1
        49.0     50.3  64.297  120.080  1900.00  2000.00  2000.00      0.0        1131.6
        50.6     50.3  61.744  114.210  1900.00  2000.00  2000.00      0.0        1111.8
        49.6     50.3  59.491  109.297  1900.00  2000.00  2000.00      0.0        1116.2
        50.0     50.2  56.375  104.904  1900.00  2000.00  2000.00      0.0        1097.6
      
      Testing 3.5.2 - dc=example,dc=com
      -------------------------------------------------------------------------------
           Throughput                            Response Time                       
         (ops/second)                           (milliseconds)                       
      recent  average  recent  average  99.9%  99.99%  99.999%  err/sec  Entries/Srch
      -------------------------------------------------------------------------------
        55.2     55.2  456.463  456.463  1900.00  1900.00  1900.00      0.0        1122.4
        50.7     53.0  48.369  260.918  1900.00  1900.00  1900.00      0.0        1109.1
        49.6     51.8  48.682  193.090  1900.00  1900.00  1900.00      0.0        1124.4
        49.8     51.3  54.660  159.462  1700.00  1900.00  1900.00      0.0        1113.4
        50.2     51.1  50.198  137.969  1700.00  1900.00  1900.00      0.0        1115.6
        50.0     50.9  50.536  123.645  1700.00  1900.00  1900.00      0.0        1117.8
        49.2     50.7  52.106  113.713  1700.00  1900.00  1900.00      0.0        1123.4
        50.7     50.7  67.760  107.972  1600.00  1900.00  1900.00      0.0        1110.1
        49.9     50.6  62.969  103.044  1600.00  1900.00  1900.00      0.0        1117.2
        50.2     50.6  48.344   97.607  1600.00  1900.00  1900.00      0.0        1109.7
        49.8     50.5  49.567   93.295  1600.00  1900.00  1900.00      0.0        1123.9
        50.0     50.4  48.900   89.624  1600.00  1900.00  1900.00      0.0        1104.8
      
      Testing 4.0.0 - dc=example,dc=com
      -------------------------------------------------------------------------------
           Throughput                            Response Time                       
         (ops/second)                           (milliseconds)                       
      recent  average  recent  average  99.9%  99.99%  99.999%  err/sec  Entries/Srch
      -------------------------------------------------------------------------------
        34.8     34.8  869.910  869.910  2300.00  2300.00  2300.00      0.0        1157.6
        56.0     45.4  213.164  465.424  2300.00  2300.00  2300.00      0.0        1090.7
        49.4     46.8  75.084  327.690  2300.00  2300.00  2300.00      0.0        1137.0
        49.4     47.4  63.967  258.905  2300.00  2300.00  2300.00      0.0        1099.2
        48.4     47.6  59.151  218.248  2300.00  2300.00  2300.00      0.0        1121.8
        49.2     47.9  60.005  191.121  2300.00  2300.00  2300.00      0.0        1108.8
        48.7     48.0  70.331  173.629  2300.00  2300.00  2300.00      0.0        1117.4
        48.0     48.0  75.450  161.344  2300.00  2300.00  2300.00      0.0        1123.9
        49.2     48.1  72.540  151.249  2200.00  2300.00  2300.00      0.0        1107.7
        48.6     48.2  62.571  142.296  2200.00  2300.00  2300.00      0.0        1119.4
        49.0     48.3  60.091  134.702  2200.00  2300.00  2300.00      0.0        1112.6
        49.0     48.3  59.394  128.333  2200.00  2300.00  2300.00      0.0        1111.7
      

      Customer data + custom schema

      Testing 2.4.6 - o=customer.com
      -------------------------------------------------------------------------------
           Throughput                            Response Time                       
         (ops/second)                           (milliseconds)                       
      recent  average  recent  average  99.9%  99.99%  99.999%  err/sec  Entries/Srch
      -------------------------------------------------------------------------------
        55.3     55.4  403.110  403.110  1200.00  1200.00  1200.00      0.0        1164.6
        49.4     52.4  165.898  291.081  1200.00  1200.00  1200.00      0.0        1072.4
        49.1     51.3  156.727  248.221  1200.00  1200.00  1200.00      0.0        1110.5
        51.3     51.3  167.820  228.120  1000.00  1200.00  1200.00      0.0        1107.5
        50.9     51.2  162.616  215.101  1000.00  1200.00  1200.00      0.0        1117.4
        48.2     50.7  135.795  202.519  1000.00  1200.00  1200.00      0.0        1114.6
        50.2     50.6  110.161  189.422  1000.00  1200.00  1200.00      0.0        1124.1
        51.4     50.7  193.844  189.983  1000.00  1200.00  1200.00      0.0        1088.1
        48.4     50.5  107.304  181.164  1000.00  1200.00  1200.00      0.0        1150.9
        49.8     50.4  149.551  178.038  1000.00  1200.00  1200.00      0.0        1093.1
        50.9     50.4  99.774  170.867  1000.00  1200.00  1200.00      0.0        1109.8
        50.0     50.4  111.343  165.943  1000.00  1200.00  1200.00      0.0        1099.0
      
      Testing 2.6.4 - o=customer.com
      -------------------------------------------------------------------------------
           Throughput                            Response Time                       
         (ops/second)                           (milliseconds)                       
      recent  average  recent  average  99.9%  99.99%  99.999%  err/sec  Entries/Srch
      -------------------------------------------------------------------------------
        51.1     51.2  656.023  656.023  2100.00  2100.00  2100.00      0.0        1117.4
        52.0     51.6  92.878  372.259  2100.00  2100.00  2100.00      0.0        1118.6
        50.0     51.0  62.837  271.009  2100.00  2100.00  2100.00      0.0        1115.7
        50.0     50.8  62.029  219.485  2000.00  2100.00  2100.00      0.0        1108.6
        50.0     50.6  57.274  187.402  2000.00  2100.00  2100.00      0.0        1116.5
        49.4     50.4  61.876  166.883  2000.00  2100.00  2100.00      0.0        1112.4
        50.6     50.4  49.887  150.103  2000.00  2100.00  2100.00      0.0        1114.2
        50.0     50.4  62.300  139.204  1900.00  2100.00  2100.00      0.0        1109.0
        50.0     50.3  49.891  129.341  1900.00  2100.00  2100.00      0.0        1111.5
        50.1     50.3  65.008  122.944  1900.00  2100.00  2100.00      0.0        1117.4
        50.0     50.3  48.658  116.225  1900.00  2100.00  2100.00      0.0        1110.2
        49.6     50.2  50.015  110.773  1900.00  2100.00  2100.00      0.0        1104.2
      
      Testing 3.5.1 - o=customer.com
      -------------------------------------------------------------------------------
           Throughput                            Response Time                       
         (ops/second)                           (milliseconds)                       
      recent  average  recent  average  99.9%  99.99%  99.999%  err/sec  Entries/Srch
      -------------------------------------------------------------------------------
        28.9     28.9  1115.517  1115.517  2800.00  2800.00  2800.00      0.0        1187.5
        47.9     38.4  641.997  820.497  2800.00  2800.00  2800.00      0.0        1095.5
        46.5     41.1  481.675  692.817  2800.00  2800.00  2800.00      0.0        1122.3
        46.7     42.5  422.233  618.294  2800.00  2800.00  2800.00      0.0        1106.2
        46.4     43.3  399.413  571.347  2800.00  2800.00  2800.00      0.0        1117.2
        44.3     43.4  436.385  548.369  2800.00  2800.00  2800.00      0.0        1109.4
        44.1     43.5  474.727  537.696  2800.00  2800.00  2800.00      0.0        1116.2
        40.7     43.2  556.421  539.905  2800.00  2800.00  2800.00      0.0        1109.6
        41.4     43.0  554.246  541.438  2800.00  2800.00  2800.00      0.0        1129.5
        44.9     43.2  504.968  537.640  2500.00  2800.00  2800.00      0.0        1108.5
        43.7     43.2  480.237  532.357  2500.00  2800.00  2800.00      0.0        1111.2
        44.6     43.3  479.260  527.808  2500.00  2800.00  2800.00      0.0        1111.7
      
      Testing 3.5.2 - o=customer.com
      -------------------------------------------------------------------------------
           Throughput                            Response Time                       
         (ops/second)                           (milliseconds)                       
      recent  average  recent  average  99.9%  99.99%  99.999%  err/sec  Entries/Srch
      -------------------------------------------------------------------------------
        33.3     33.3  963.651  963.651  2400.00  2400.00  2400.00      0.0        1154.0
        45.3     39.3  595.995  752.084  2400.00  2400.00  2400.00      0.0        1139.4
        46.5     41.7  450.907  639.928  2400.00  2400.00  2400.00      0.0        1091.6
        44.0     42.3  479.117  597.961  2400.00  2400.00  2400.00      0.0        1113.1
        44.7     42.8  471.404  571.486  2300.00  2400.00  2400.00      0.0        1122.2
        42.3     42.7  542.511  566.698  2300.00  2400.00  2400.00      0.0        1094.0
        41.5     42.5  597.881  571.048  2300.00  2400.00  2400.00      0.0        1126.3
        42.6     42.5  594.464  573.975  2300.00  2400.00  2400.00      0.0        1108.2
        42.6     42.5  557.176  572.109  2300.00  2400.00  2400.00      0.0        1126.8
        43.8     42.7  533.697  568.170  2100.00  2400.00  2400.00      0.0        1107.8
        42.4     42.6  527.145  564.466  2100.00  2400.00  2400.00      0.0        1104.7
        43.5     42.7  558.824  563.988  2100.00  2400.00  2400.00      0.0        1095.1
      
      Testing 4.0.0 - o=customer.com
      -------------------------------------------------------------------------------
           Throughput                            Response Time                       
         (ops/second)                           (milliseconds)                       
      recent  average  recent  average  99.9%  99.99%  99.999%  err/sec  Entries/Srch
      -------------------------------------------------------------------------------
        19.7     19.7  1601.816  1601.816  2900.00  2900.00  2900.00      0.0        1252.6
        40.0     29.9  767.379  1041.792  2900.00  2900.00  2900.00      0.0        1090.7
        40.0     33.2  725.764  915.253  2900.00  2900.00  2900.00      0.0        1101.8
        36.0     33.9  715.565  862.160  2900.00  2900.00  2900.00      0.0        1121.0
        37.0     34.5  645.297  815.816  2900.00  2900.00  2900.00      0.0        1121.6
        38.1     35.1  640.741  784.166  2900.00  2900.00  2900.00      0.0        1091.0
        36.7     35.4  648.818  764.094  2900.00  2900.00  2900.00      0.0        1126.7
        37.2     35.6  656.705  750.093  2900.00  2900.00  2900.00      0.0        1113.2
        37.1     35.7  621.160  735.223  2900.00  2900.00  2900.00      0.0        1113.4
        36.9     35.9  631.186  724.516  2900.00  2900.00  2900.00      0.0        1104.1
        35.4     35.8  656.812  718.418  2900.00  2900.00  2900.00      0.0        1117.9
        38.2     36.0  598.356  707.782  2700.00  2900.00  2900.00      0.0        1094.3
      

      Example testcase.

      1. Create a basic dc=example,dc=com instance with 1113 entries and default schema
      2. Test using the above searchrate

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                matthew Matthew Swift
                Reporter:
                lee.trujillo Lee Trujillo
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: