Skip to content

Commit

Permalink
bumping codec limit to 20mb (#258)
Browse files Browse the repository at this point in the history
## Summary
this is a permanent solution to resolve 

```bash
org.springframework.core.io.buffer.DataBufferLimitException: Exceeded limit on max bytes to buffer : 10240000
```

which can occur on large listing APIs such as `/v1/databases` which
resolves up to 18.5k tables at most which exceeds client side http
request of 10MB

## Changes

- [ ] Client-facing API Changes
- [ ] Internal API Changes
- [X] Bug Fixes
- [ ] New Features
- [ ] Performance Improvements
- [ ] Code Style
- [ ] Refactoring
- [ ] Documentation
- [ ] Tests

## Testing Done
<!--- Check any relevant boxes with "x" -->

- [ ] Manually Tested on local docker setup. Please include commands
ran, and their output.
- [ ] Added new tests for the changes made.
- [ ] Updated existing tests to reflect the changes made.
- [ ] No tests added or updated. Please explain why. If unsure, please
feel free to ask for help.
- [X] Some other form of testing like staging or soak time in
production. Please explain.

will monitor the acceptance tests

# Additional Information

- [ ] Breaking Changes
- [ ] Deprecations
- [ ] Large PR broken into smaller PRs, and PR plan linked in the
description.
  • Loading branch information
cbb330 authored Nov 25, 2024
1 parent b7a38eb commit 3f041c5
Show file tree
Hide file tree
Showing 5 changed files with 5 additions and 5 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ public abstract class WebClientFactory {
private static final String SESSION_ID = "session-id";

public static final String HTTP_HEADER_CLIENT_NAME = "X-Client-Name";
private static final int IN_MEMORY_BUFFER_SIZE = 10 * 1000 * 1024;
private static final int IN_MEMORY_BUFFER_SIZE = 20 * 1024 * 1024;
// The maximum number of connections per connection pool
private static final int MAX_CONNECTION_POOL_SIZE = 500;
// Max time to keep requests in pending queue before acquiring a connection
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
spring.codec.max-in-memory-size=10MB
spring.codec.max-in-memory-size=20MB
springdoc.packages-to-scan=com.linkedin.openhouse, com.linkedin.openhouse.housetables
springdoc.swagger-ui.disable-swagger-default-url=true
springdoc.swagger-ui.filter=true
Expand Down
2 changes: 1 addition & 1 deletion services/jobs/src/main/resources/application.properties
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
spring.codec.max-in-memory-size=10MB
spring.codec.max-in-memory-size=20MB
springdoc.packages-to-scan=com.linkedin.openhouse.jobs
springdoc.swagger-ui.disable-swagger-default-url=true
springdoc.swagger-ui.filter=true
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@
public class MainApplicationConfig extends BaseApplicationConfig {
public static final String APP_NAME = "tables";
private static final Pattern VERSION_PART_PATTERN = Pattern.compile("v[0-9]+");
private static final int IN_MEMORY_BUFFER_SIZE = 10 * 1000 * 1024;
private static final int IN_MEMORY_BUFFER_SIZE = 20 * 1024 * 1024;

private static final int DNS_QUERY_TIMEOUT_SECONDS = 10;

Expand Down
2 changes: 1 addition & 1 deletion services/tables/src/main/resources/application.properties
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
spring.mvc.throw-exception-if-no-handler-found=true
spring.mvc.static-path-pattern=favicon.ico
spring.web.resources.static-locations=classpath:/static
spring.codec.max-in-memory-size=10MB
spring.codec.max-in-memory-size=20MB
springdoc.packages-to-scan=com.linkedin.openhouse.tables
springdoc.swagger-ui.disable-swagger-default-url=true
springdoc.swagger-ui.filter=true
Expand Down

0 comments on commit 3f041c5

Please sign in to comment.