Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

@W-13729777 - handle closed index in index merger #605

Merged
merged 1 commit into from
Aug 16, 2023
Merged

@W-13729777 - handle closed index in index merger #605

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 18 additions & 6 deletions ...-java/timbermill-local/src/main/java/com/datorama/oss/timbermill/ElasticsearchClient.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -983,20 +983,31 @@ private List<String> getIndicesToBeMoved(String alias) throws IOException {
List<String> indicesToMove = new ArrayList<>();
asJsonObject.forEach(jsonElement -> {
JsonObject jsonObject = jsonElement.getAsJsonObject();
String storeSize = jsonObject.get("store.size").getAsString();
String status = jsonObject.get("status").toString();
String index = jsonObject.get("index").getAsString();
if (status.equalsIgnoreCase("close")) {
try {
deleteIndex(index);
return;
} catch (Exception e) {
// ignore error
return;
}
}
String storeSize = jsonObject.get("store.size").getAsString();

if (indexNotToMerge.contains(index)) {
return;
}
if (isSizeLowerThen(storeSize, (maxIndexSizeInGB * indexMergePercentage/100))) { // 10% of max index size
if (isSizeLowerThen(storeSize, (maxIndexSizeInGB * indexMergePercentage / 100))) { // 10% of max index size
indicesToMove.add(index);
LOG.info("Index {} has a size of {}. Will be moved to another index", index, storeSize);
}
});
LOG.info("Number of indices to be moved: {}", indicesToMove.size());
return indicesToMove;
} catch (Exception e) {
LOG.error("Failed to get indices to be moved", e);
LOG.error("Failed to get indices to be moved for alias {}",alias, e);
return Collections.emptyList();
} finally {
if (content != null) {
Expand Down Expand Up @@ -1074,7 +1085,7 @@ private boolean isReindexSuccess(TaskId taskId) {
LOG.info("Reindex task completed. Created {} documents. Num of failures {}", created, numOfFailures);
return numOfFailures == 0;
} else {
Thread.sleep(10000);
Thread.sleep(10000);
}
}
} catch (Exception e) {
Expand All @@ -1091,7 +1102,7 @@ private boolean isReindexSuccess(TaskId taskId) {
return false;
}

private void deleteIndex(String index) {
private void deleteIndex(String index) throws IOException {
try {
LOG.info("Deleting index {}", index);
DeleteIndexRequest request = new DeleteIndexRequest(index);
Expand All @@ -1101,8 +1112,9 @@ private void deleteIndex(String index) {
} else {
LOG.info("Index {} deleted", index);
}
} catch (IOException e) {
} catch (Exception e) {
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

instead of rethrow, you can just not cache the exception here, and handle the catch in caller method (logging)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i need to figure some thing about the delete
i will do it in a separate PR

LOG.error("Failed to delete index {}", index, e);
throw e;
}
}
}
Expand Down
Loading