Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

srcache_store response hangs with request body when used with ngx.req.get_body_data() #84

Open
suyogdilipkale opened this issue Apr 25, 2020 · 0 comments

Comments

@suyogdilipkale
Copy link

Openresty srcache_store doesn't work if there is payload in request body. and ngx.req.get_body_data() is used

Using openresty lua-nginx-module to cache and serve web-pages. (reverse proxy to caching)
https://github.com/openresty/lua-nginx-module
Openresty 1.15.8.3
ngx_lua 0.10.16

ISSUE: When try request with request body, it hangs the response. . same is also observed using redis2_query and redis_pass.

Below is my default.conf script:


server {
    # Listen on port 80.
    listen 80 default_server;
    listen [::]:80 default_server;
    # The document root.
    root /usr/local/openresty/nginx/html/default;
    # Add index.php if you are using PHP.
    index index.html index.htm;

    # The server name, which isn't relevant in this case, because we only have one.
    server_name _;

    access_log /var/log/openresty/nginx_cache.log custom_cache_log ;

    location / {
     default_type application/json;
     client_max_body_size 100k;
     client_body_buffer_size 100k;
     set $key "";
     access_by_lua_block {
        ngx.req.read_body()
        local data = ngx.req.get_body_data()
        ngx.var.key = "es:" .. ngx.md5(ngx.var.request_uri .. (data or ""))
        ngx.log(ngx.ERR, "key: ", ngx.var.key)
    }

     srcache_fetch GET /redis-fetch key=$key;
     srcache_store PUT /redis-store key=$key&exptime=3600;

     proxy_set_header Host $host;
     proxy_set_header X-Real-IP $remote_addr;
     proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
     proxy_set_header X-Forwarded-Proto $scheme;
     proxy_pass http://ElasticSearch_SERVER:9200;
    }

    location = /redis-fetch {
        internal;
        content_by_lua_block {
                local redis = require "resty.redis"
                local red = redis:new()
                red:set_timeouts(10000, 10000, 10000)
                local ok, err = red:connect("Redis_Server",16318)
                if not ok then
                    ngx.say("failed to connect: ", err)
                    return
                end
                ngx.say(ok)
        }
    }

    location = /redis-store {
    internal;
    content_by_lua_block {
               local redis = require "resty.redis"
               local red = redis:new()
               red:set_timeouts(10000, 10000, 10000)
               local ok, err = red:connect("Redis_Server",16318)
               if not ok then
                   ngx.say("failed to connect: ", err)
                   return
               end
               ok, err = red:set(ngx.var.arg_key,ngx.var.echo_request_body)
               if not ok then
                   ngx.say("failed to set key: ", err)
                   return
               end
               ok, err = red:expire(ngx.var.arg_key,ngx.var.arg_exptime)
     }
   }   
}

Following request works fine (first time cache is not exists so served from upstream server and cache is created in redis. next time same page request is served from cache redis)

  1. Curl request, GET method:
    curl -u "username:Password" "http://Proxy_SERVER:80/index/_search?typed_keys=true&ignore_unavailable=true&expand_wildcards=open&allow_no_indices=true&search_type=query_then_fetch&batched_reduce_size=512"
    Result => works as expected.

  2. Curl to without any explicit HTTP method defined but small body parameters:
    curl -u username:password "http://Reverse_proxy:80/_search?pretty" -H 'Content-Type: application/json' -d '{"query":{"match_all":{}}}'

Result => response hangs

  1. Curl to without any explicit HTTP method defined but large body parameters:

Note that we can explicitly send HTTP method in every request as its coming from different client. Can we override the HTTP method in default.conf ?? or is there any other settings need to done to solve this issue?

We already tried following options but no luck:

  1. set srcache_methods POST GET PUT HEAD
  2. proxy_method POST
  3. increase proxy buffer
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant