Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Provide method(s) to evict connections currently in pool #6

Open
ttddyy opened this issue Apr 10, 2019 · 11 comments
Open

Provide method(s) to evict connections currently in pool #6

ttddyy opened this issue Apr 10, 2019 · 11 comments

Comments

@ttddyy
Copy link
Member

ttddyy commented Apr 10, 2019

Method to evict(invalidate/purge) connections currently in pool.
In-use connections need to be invalidated after it is released. Or user should be able to define the behavior by passing parameter or provide separate methods for different behavior.

@nishant-1984
Copy link

Is this being implemented? There are serious connection leak issues with library

@mp911de
Copy link
Member

mp911de commented Mar 13, 2020

There are no proper methods to signal a full eviction to the pool. We should file a ticket in https://github.com/reactor/reactor-pool/issues.

@mbanaszkiewicz-vonage
Copy link

mbanaszkiewicz-vonage commented Jul 1, 2020

Probably related to this issue.

Sometimes I'm seeing the following warning log from netty, after enabling debug it looks like its preceded by connection release:
{"timestamp":"2020-07-01T13:14:36.654+0000","level":"DEBUG","thread":"reactor-tcp-epoll-1","message":"Releasing R2DBC Connection [PooledConnection[dev.miku.r2dbc.mysql.MySqlConnection@1c185c05]] after transaction","correlationId":"c46aa7b3-79b3-5080-ad3c-ee494705c4b8"} **{"timestamp":"2020-07-01T13:14:45.064+0000","level":"WARN","thread":"reactor-tcp-epoll-1","message":"Connection has been closed by peer","correlationId":"c46aa7b3-79b3-5080-ad3c-ee494705c4b8"}**

Our r2dbc-pool settings are as follows:

          max-size: 20
          initial-size: 1
          max-life-time: 280 seconds
          max-idle-time: 240 seconds
          max-acquire-time: 30 seconds

Our Mysql database has 300 seconds max connection lifetime. We checked connection list on mysql side, and turns out r2dbc-pool connections are sometimes not closed properly on pool side after 280s, and mysql server has to clean them up after 300s. We are using spring boot 2.3.0, spring-boot-starter-data-r2dbc 2.3.0, r2dbc-pool 0.8.2, reactor-pool 0.1.3, r2dbc-mysql 0.8.1 - also tried bumping up r2dbc-pool & reactor-pool to newest - didn't change anything.

Is there any ongoing work which might improve the situation?

@mbanaszkiewicz-vonage
Copy link

I've prepared a simple spring boot app which reproduces the problem and makes "connection closed by peer" logs visible:
https://github.com/mbanaszkiewicz-vonage/r2dbc-testing

README.md contains very short instruction what to do.

@mp911de
Copy link
Member

mp911de commented Jul 3, 2020

We're using Reactor Pool to provide general object pooling functionality. Eviction of connections only happens if there's activity on the pool. There's no background thread that evicts connections actively. Instead, the application must either borrow or return connections. Can you recheck whether this works for you if your application has pool activity if you exceed 280 seconds?

@mbanaszkiewicz-vonage
Copy link

mbanaszkiewicz-vonage commented Jul 3, 2020

All right, thanks for clarification. I've verified this by generating constant load, indeed pool activity causes the connections to be evicted properly. @mp911de there are no plans to change this behavior and introducing something automatic?

@mp911de
Copy link
Member

mp911de commented Jul 3, 2020

It would make sense to ask reactor-pool to provide background eviction so that TTLs can be considered properly. I think we cannot do anything about automated eviction from a R2DBC pool perspective.

@petromir
Copy link
Contributor

petromir commented May 13, 2021

Isn't this covered with #115?

@mp911de
Copy link
Member

mp911de commented May 14, 2021

Actually, not, because reactor.pool.Pool doesn't expose an evictAll method.

@ojh3636
Copy link

ojh3636 commented Nov 16, 2021

@petromir
I think that's because, r2dbc pool set backgroundEvictionInterval of reactor pool as same as maxIdleTime when it created.
So, when creating reactor pool, it start's eviction and then do next eviction after maxIdleTime (in your case 240s).
But your first connections are acquired after the reactor pool initialized, your first connections has no chance to properly evicted by reactor pool.

0s -> 1s -> 240s -> 300s ->
(pool created, eviction) -> first connection acquired -> first eviction run (but no evict), -> connection closed by db

So I recommend to set (backgroundEvictionInterval + maxIdleTime) < wait_time of db, then you can solve your problem

@restrepo86Me
Copy link

Hi, I am currently having this problem, could you put an example of configuration based on what you just mentioned
(backgroundEvictionInterval + maxIdleTime) considering that wait_time of db is 20 minutes ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

7 participants