Skip to content

Commit

Permalink
CASMCMS-9225: Do not use itertools.batched, because it is not availab…
Browse files Browse the repository at this point in the history
…le in this Python version

(cherry picked from commit 1039bbe)
  • Loading branch information
mharding-hpe committed Dec 19, 2024
1 parent ae41bfd commit 592d069
Show file tree
Hide file tree
Showing 2 changed files with 8 additions and 3 deletions.
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased]

## [2.0.49] - 2024-12-19
### Fixed
- Do not use `itertools.batched`, because it is not available in this Python version.

## [2.0.48] - 2024-12-18
### Changed
- Improve performance of large `GET` components requests.
Expand Down
7 changes: 4 additions & 3 deletions src/bos/server/redis_db_utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,6 @@
# ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
# OTHER DEALINGS IN THE SOFTWARE.
#
from itertools import batched
import json
import logging
from typing import Callable
Expand Down Expand Up @@ -111,9 +110,11 @@ def iter_values(self):
"""
Iterate through every item in the database. Parse each item as JSON and yield it.
"""
for next_keys in batched(self.client.scan_iter(), 500):
for datastr in self.client.mget(next_keys):
all_keys = list(self.client.scan_iter())
while all_keys:
for datastr in self.client.mget(all_keys[:500]):
yield json.loads(datastr) if datastr else None
all_keys = all_keys[500:]

def put(self, key, new_data):
"""Put data in to the database, replacing any old data."""
Expand Down

0 comments on commit 592d069

Please sign in to comment.