The use of msgQueue and lateQueue originally sounded like a useful optimization: When the RS is fast enough following what the DS does, it stays in memory, and when it becomes late it reads from the lateQueue, after populating it from cursors.
If the flow of messages coming from the DS is small, then the RS will always be following. However, in such case, the I/O savings are not worth the added complexity because the RS is not under big load from this DS. In the case where the flow of messages coming from the DS is very big, the RS cannot follow and is resorting to the lateQueue which incurs cursoring anyway.
However, with the file based changelog it is less than ideal to repeatedly have to open and close several cursors for reading 100 entries then only each time. The disk seeks are a big toll to pay to maintain a complex design, even more when the system is under heavy load.
I think it might be better to replace the msgQueue and lateQueue by cursors into the changelogDB that would always remain open. This way, we would rely on the OS file system caching and virtual pages to speed up linear reads (changelogDB is append only).