Update operation causes duplicate entries #1092
Comments
|
From your code it is not clear how you have configured the database or what does your entity looks like. Could you please share the nitrite specific code involved? A reproducible code would be greatly appreciated. |
|
On a closer look on your code, are you trying to call the same function recursively while holding the lock? If my understanding is correct, you are creating a deadlock itself in your code, which has nothing to do with nitrite itself. Remove the lock and try once. |
|
Nitrite config: protected Nitrite build(Configs configs) throws Exception {
var path = configs
.paths()
.database()
.toAbsolutePath();
FileUtils.createParentDirectories(path.toFile());
MVStoreModule storeModule = MVStoreModule.withConfig()
.filePath(path.toFile())
.compress(true)
.build();
JacksonMapper jacksonMapper = new JacksonMapper() {
@Override protected <Target> Target convertFromDocument(Document source, Class<Target> type) {
try {
return super.convertFromDocument(source, type);
} catch (IllegalArgumentException iae) {
log.warn("Unable to convert from document, source: {}, type {}", source, type, iae);
return null;
}
}
};
for (Module jacksonModule : List.of(
new ParameterNamesModule(),
new Jdk8Module(),
new JavaTimeModule())) {
jacksonMapper.registerJacksonModule(jacksonModule);
}
return Nitrite.builder()
.loadModule(storeModule)
.loadModule(NitriteModule.module(jacksonMapper))
.openOrCreate();
}This piece of code can get called concurrently to update the tags. I'm locking this on the sku/id because multiple things can change/add tags for the entity at the same time. public void updateUserItemTags(SKU sku, TagGenerator generator) {
Lock lock = userItemsMutex.get(sku);
lock.lock();
try {
appManager.database().items().get(sku).ifPresent(UserItemEntity item -> {
updateUserItemTags(item, generator);
});
} finally {
lock.unlock();
}
}
public void updateUserItemTags(UserItemEntity item, TagGenerator generator) {
Lock lock = userItemsMutex.get(item.getSku());
lock.lock();
try {
item.setTags(generator.generateTags(this, item));
appManager.database().items().upsert(item);
} finally {
lock.unlock();
}
}I'm using wrapper classes for db collections, in this case the key is a string and the public void upsert(OBJ obj) {
repository.update(obj, true);
}@Entity("UserItems")
@Data
@AllArgsConstructor
@NoArgsConstructor
public class UserItemEntity implements TagQuery, SkuEntity {
@Id
private String id;
private List<Tag> tags = new ObjectArrayList<>();
}
I'm not, I have implemented this because of some race condition with the update of the tags that I had. As for replicating the issue, I don't know what causes this, the duplication happens after some time seemingly at random, or perhaps when I initialize a load on the db. |
|
I don't see any problem with the nitrite configuration on a quick look. If the sole purpose of the lock is to update nitrite only, I'll say remove the lock and test, because nitrite has its own locking mechanism to prevent race conditions. Also, in your code you are using the same mutex for both methods. The lock in second method is unnecessary and is trying to acquire lock from an already locked mutex. Either use different mutex for two methods or better don't use any lock if you are just updating nitrite. |
|
I have looked into the database debug logs, and I found this
1735151350020 -> Wed Dec 25 19:29:10 CET 2024 |
|
From my testing, I think that it's db corruption that causes the duplicates. So I ended up switching to rocks db, if I see the same behavior I will report it here. |
|
I have not observed the same issue I had with mv store, but I have found another. There is a race condition that can happen when you clear a collection and update it right after. This is probably because of this |
|
Also while looking thought the code I found that you're using concurrent maps, but are not using them in a thread safe way, making them useless. Is this intentional? |
|
Could you please raise a separate issue for this and provide some reproducible code? |
|
It's not a problem, but im still wondering if i should make 2 issues or just for the race condition? |
|
For the race condition please. |
The entity is using the nitrite id annotation, in this case the object is not getting removed, and its not updated concurrently. The tags get updated with a lock on the id.
The update op insert if absent is used.
I am also unable to remove the duplicated objects
I don't have a way of replicating this since it happens randomly.
Version: 4.3.0
Java: 21-jbr
Database: MvStore
The text was updated successfully, but these errors were encountered: