[OPENDJ-922] Replication window size is too small on high latency networks Created: 21/May/13  Updated: 08/Nov/19  Resolved: 21/May/13

Status: Done
Project: OpenDJ
Component/s: replication
Affects Version/s: 2.6.0
Fix Version/s: 2.6.0

Type: Bug Priority: Major
Reporter: Matthew Swift Assignee: Matthew Swift
Resolution: Fixed Votes: 0
Labels: release-notes

Issue Links:
Relates
is related to OPENDJ-4988 Topology wide inter process deadlock ... Done
is related to OPENDJ-1354 replication threads BLOCKED in pendin... Done
is related to OPENDJ-934 Changes to RS window-size property re... Done
Dev Assignee: Matthew Swift

 Description   

By default it is 100 which is too small on high latency networks, e.g. WAN where latency is 100ms.

In fact, the purpose of this configuration is unclear. It looks like something inherited from Sun DSEE which had a very different LDAP based protocol as well as supporting functionality like prioritized replication. In addition, network bandwidth has improved, as have network protocol stack implementations (e.g. adaptive TCP buffering), so it is unclear whether this functionality is even needed any more (note that DJ does not support prioritized replication).

For 2.6 we should consider at the very least increasing the window size and/or deprecating it.

Example #1:

Given a modrate of of 1000/s
Given a network latency of 100ms
Given a window size of 100

  • publisher is sending 1 update / ms
  • after 100ms the receive receives the first update and the publisher blocks (as it has sent 100 updates)
  • after 150ms the receiver sends an ack
  • after 200ms the receiver has received all 100 updates
  • after 250ms the publisher receives the ack for the first 50 updates.

>>> means they are spending 150ms out of 250ms doing nothing.

Example #2:

Given a modrate of of 1000/s (same as before)
Given a network latency of 100ms (same as before)
Given a window size of 1000

  • publisher is sending 1 update / ms
  • after 100ms the receive receives the first update
  • after 600ms the receiver sends an ack
  • after 700ms the publisher receives the ack for the first 500 updates (but still has a 300 remaining in its window) and resets its window to 800 (300+500).

>>> publisher not blocked.



 Comments   
Comment by Matthew Swift [ 21/May/13 ]

Here's two equations which express the relationship:

  W = 4LT
  T = W / 4L

Where:

  T = required write throughput (operations/s)
  W = window size
  L = network latency (s)
Comment by Matthew Swift [ 21/May/13 ]

Fixed:

  • set the default window size to 100K
  • indicate that the properties may be deprecated in future releases
  • tag properties as advanced.

A default window size of 100K will permit a throughput of 25K updates/s on an extremely slow 1s latency network.

Comment by Matthew Swift [ 07/Nov/19 ]

Moved to closed state because the fixVersion has already been released.

Generated at Tue Nov 24 21:07:40 UTC 2020 using Jira 7.13.12#713012-sha1:6e07c38070d5191bbf7353952ed38f111754533a.