Unable to reach quorum #168
Comments
|
Downgrading to version 4 for redlock and ioredis solved my issue completely, I guess there's a bug with the v5 implementation. |
|
having this same issue too. |
|
I have the same problem, is there any solution? |
|
@jskorlol try downgrading to version 4 of this package and ioredis |
|
yeah, the problem is i would really like the lock auto extend functionality. |
|
I'm having this problem too, we use redlock v4 to great success so I'm downgrading for now. |
|
any update on this, I am having this problem too |
|
Hey, we also encountered this issue on both Failing code: // redis and redlock setup is omitted
const wait = async (ms: number) => {
return new Promise((resolve) => setTimeout(() => resolve(ms), ms));
};
const main = async () => {
try {
const lock = await redlock.acquire(['a'], 1000);
await wait(1500);
await lock.release();
} catch (e) {
console.error(e);
throw e;
}
};
main();Error: working code: // redis, redlock and wait setup is omitted
const main = async () => {
try {
const lock = await redlock.acquire(['a'], 1000);
await lock.extend(1600);
await wait(1500);
await lock.release();
} catch (e) {
console.error(e);
throw e;
}
};
main();for us switching to I think the best for the library would be to make |
|
We are still seeing this intermittently even with |
|
I was also running into this on version With the call to using looking like: await redlock.using([lockId], 5000, async (signal) => { ... my code ... });The problem seems to occur when the same resource is trying to be locked multiple times in close succession. Increasing the |
|
Got bitten by this as well |
|
Hi folks, I faced this problem, but was a mistake in my code. I was trying to aquire and make a set in the same key, as: const main = async () => { But this way is wrong because we need to aquire a resourceKey like "myLokedResouce:myKey" and then make changes in our key (is tihis exemple "myKey") and after release the lock on "myLokedResouce:myKey". Now my code search by "resourceKey" to know if can makes changes in my "key" like: const key = "project" async function lockAndSet(resource , lockTtl) { try { } catch (e) { lockAndSet(resource, lockTtl) |
|
I have issue too! I have no idea. I read code and find |
|
I believe this is a bug in the v5 implementation. When you attempt to set a lock using |
|
I was facing same issue but it was because of me passing redis client as |
|
We are also hitting this issue as soon as 2 process are trying to lock the same resource. This is 100% reproduceable: Result when running twice this process with p1 and p2 name: Edit: This seems to happen 100% of the time if the acquire lock time is shorter than the time the function takes to complete, but seems to work when the acquire time is plenty more. This is not an issue if the function releases properly the lock, it might be if it crashes and doesn't though.
|
Can you specify concrete version? |
|
Any status update on the fix? It looks like a few PRs were made (I saw another github thread). where does this stand? I am also having this issue where I do not know the TTL upfront so I need the autoextending and it is giving me problems. |
This resolved my issues. Be sure to add a resource key ^5.0.0-beta.2 |
|
Code: const sleep = (ms) => new Promise((res) => setTimeout(res, ms));
for (let i = 0; i < 30; i++) {
redlock
.using(['test'], 10_000, async () => {
await sleep(200);
console.log('OK');
})
.catch((e) => console.error(e.message));
} |
|
Hey everyone! 👋 |
|
I also faced the same problem, but the solution is around the duration of lock which you mentioned. |
|
So currently, with what is available, to prevent errors, you have to give a timelock beyond the duration of your service. That is if your service will run for 2 seconds, make it 5s to be safe, so as long the service does not crash, it will notify Redis when it is done for another service to acquire the lock. Or you can use the |
|
I write this because there's maybe someone who's mistaken like me. First, I wrote my code like but I faced error and after I read lib codes, so now, I know my case needs to be fixed like |
|
Just stumbled on this now. Downgrading to 4.2.0 was the only solution. As far as I can tell it works with the latest version of |
|
I think it is a bug If you have any existing lock Line 561 in 5138813 and after this place Line 597 in 5138813 return error because lock key is not implemented ( BUT after return error from here Line 465 in 5138813 |
|
for lock error vote === 'against' and after we always return error qurum because skip by this line Line 446 in 5138813 |
This worked for me. Thank you. Summary: |
This was also the solution for my case. The DOCS need updating, I've done so on this PR: #291 |
|
Solved it using a check for the lock expiration: const now = new Date().getTime();
if (lock && lock.expiration > now) {
await lock.release();
}If the lock is expired (TTL), it is no longer in the cache, so don't release it. |
I encountered an error while debugging library code a few days ago. I tried my best to find solutions, but I couldn’t. Do you have a solution now? |
I used "ioredis" version "^4.2.0" and "redlock" version "^4.2.0". But it cannot lock the resource. I got the error message: |
|
In case somebody made my mistake: i got the same error as OP ( const redlock = new Redlock([redis], {
retryCount: 15,
retryDelay: 50,
});
const [result1, result2] = await Promise.all([
redlock.using(['qwerty'], 3000, async () => {
await sleep(1500);
return 1;
}),
redlock.using(['qwerty'], 3000, async () => {
await sleep(1500);
return 2;
}),
]);
console.log(result1, result2);In the example above second lock will always fail since it cannot aquire lock, locked for 3000ms, within 15 * 50ms (750ms) |
|
Was there a resolution for this ? |
|
Make sure the code control still holds the redis lock before giving up the lock, the issue is losing the lock and then asking redlock to release the lock(which is already released automatically using timer). |
|
Using above suggestion I have resolved this issue with these changes. The key is to overwrite import Redlock from 'redlock'
import { redis } from '../storage/redis'
export const redlock = new Redlock([redis], {
// The expected clock drift; for more details see:
// http://redis.io/topics/distlock
driftFactor: 0.01, // multiplied by lock ttl to determine drift time
// The max number of times Redlock will attempt to lock a resource
// before erroring.
retryCount: 30,
// the time in ms between attempts
retryDelay: 2000, // time in ms
// the max time in ms randomly added to retries
// to improve performance under high contention
// see https://www.awsarchitectureblog.com/2015/03/backoff.html
retryJitter: 200, // time in ms
// The minimum remaining time on a lock before an extension is automatically
// attempted with the `using` API.
automaticExtensionThreshold: 500, // time in ms
})
// Save the original acquire method
const originalAcquire = Redlock.prototype.acquire
// Override the release method for all lock instances
Redlock.prototype.acquire = async function (...args) {
const duration = args[1] // this is a duration value
// use the duration to create additional settings
args[2] = {
retryCount: Math.ceil((duration / 2_000) * 1.5),
retryDelay: 2_000,
...args[2],
}
return originalAcquire.apply(this, args) // Call the original release method
}
// Save the original release method
const originalRelease = Redlock.prototype.release
// Override the release method for all lock instances
Redlock.prototype.release = async function (...args) {
const now = new Date().getTime()
if (args[0] && args[0].expiration > now) {
// Check if the lock still exists
return originalRelease.apply(this, args) // Call the original release method
}
return {
attempts: [],
}
} |
You are right bro, thank u |
|
It appears that this may have been addressed in the following PRs: These are linked to the issue #169 which seems to be the same issue seen here. These PRs were merged to Someone appears to have published a "temporary" package with that fix (and others) here https://www.npmjs.com/package/redlock-temp-fix The company I'm working for is using this package and currently seeing this error quite a lot, so we're going to publish a private package based on Edit: This did not seem to fix the issue we're seeing. D'oh. |
and others
I'm having some trouble using this library. I have an endpoint which gets called twice upon page load and I'm trying to use a lock to perform sensitive database operations safely.
I'm using this library and I keep getting the following error in unusual ways which I haven't been able to figure out (maybe I have the library misconfigured?). Here's the code I'm using to initialize redlock:
here's the code which does the sensitive database work:
here's the full output I'm getting:
The time between the
acquired lockandreleasing locklog lines is around 1 second but for some reason, even with the generous retry policy I'm seeing redlock fail consistently.The top stack trace includes line 143 which is the line where I release the lock. The bottom stack trace include line 126 which is where I acquire the lock.
I've tried using
redlock.usingas well but had the same results. I've downloaded a redis gui to inspect my local redis instance to see if there are any spurious keys but haven't been able to find any.Here's my package.json:
The text was updated successfully, but these errors were encountered: