Info
The window stage estimates the memory Y that a window frame of size X will need using linear regression and the formula Y = aX + b. It computes the coefficients a and b incrementally sampling the documents that are added in the window. Every time a new document is added in the window the estimated memory is computed and compared against a threshold.
The problem exists only in the case when the window is not removable. In this case, the samples consider the size of the frame (i.e the total number of documents) but the memory of the last seen document only. When the memory of the new documents is smaller than the memory of the previous documents then the memory and the window frame size are seen to be reverse proportional, i.e. the window frame increases and the memory decreases, and the linear regression formula gives negative results.
The implementation of the memory estimation should be updated to correctly consider the memory in the case of the non removable windows.
Top User Comments
xgen-internal-githook commented on Thu, 18 Apr 2024 15:11:19 +0000:
Author:
{'name': 'fotiniAlvanaki', 'email': '78495594+fotiniAlvanaki@users.noreply.github.com', 'username': 'fotiniAlvanaki'}
Message: SERVER-89408 Do not estimate state memory for non-removable windows (#21205)
GitOrigin-RevId: 90a5f9e0569db7b2c518024533b1172062415a15
Branch: v8.0
https://github.com/mongodb/mongo/commit/ff0622e01439bc65a54c09ef07bb3e9260401225
xgen-internal-githook commented on Wed, 17 Apr 2024 15:38:50 +0000:
Author:
{'name': 'fotiniAlvanaki', 'email': '78495594+fotiniAlvanaki@users.noreply.github.com', 'username': 'fotiniAlvanaki'}
Message: SERVER-89408 Do not estimate state memory for non-removable windows (#21205)
GitOrigin-RevId: b3768279ece273b15edd569badb8a80ec822ccb5
Branch: master
https://github.com/mongodb/mongo/commit/3d4f2abcd70e2b903a78f87ce8ca1db5a7c2a590
Steps to Reproduce
Query
[
{
$setWindowFields: {
sortBy: {
"num": -1
},
output: {
"minStr": {
$min: "$str"
}
}
}
}
]
Data
[
{_id: 0, "num": 10, "str": "ABCDEFGHIJK" },
{_id: 1, "num": 3, "str": "ABCDE"},
{_id: 2, "num": 5, "str": "AB"},
]
