-
Notifications
You must be signed in to change notification settings - Fork 81
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
yet another - too big subrequest response while sending to client - issue #26
Comments
fyi i switched to fetch API and I no longer have this issue. But is this the correct way instead of subrequest ?
|
@yogeshgadge if you don't need to put some cache, using However if you need to have some cache (because your upstream API is rate limited for example) you will need to use # /etc/nginx/conf.d/default.conf
proxy_cache_path /tmp/nginx-cache/ levels=1:2 keys_zone=cachezone:100m max_size=1g inactive=30d use_temp_path=off;
js_path "/etc/nginx/njs/";
js_import main from search.js;
server {
server_name _;
### example cache configuration
proxy_cache cachezone;
proxy_cache_revalidate on;
proxy_cache_min_uses 1;
proxy_cache_background_update on;
proxy_cache_lock on;
proxy_cache_key $scheme$proxy_host$request_uri;
proxy_cache_use_stale error updating timeout http_500 http_502 http_503 http_504;
proxy_cache_valid 301 1h;
proxy_cache_valid 404 1m;
### end of cache configuration
# public route
location = /search {
js_content main.search;
}
# internal route
location /_internal_search {
internal; # important so nobody can use the route from outside
subrequest_output_buffer_size 500m; # still have a random value here
proxy_pass https://redacted-domain/api/search;
proxy_cache_valid 200 302 5m;
}
} With // /etc/nginx/njs/search.js
async function search(r) {
const reply = await r.subrequest('/_internal_search');
const json = JSON.parse(reply.responseText);
// do things with your JSON
r.return(200, JSON.stringify(json));
}
export default { search }; Note that if you use let reply = await ngx.fetch('https://redacted-domain/api/search', { verify: false }); |
Well as a followup, seems using subrequest with caching doesn't work as expected. On cache stale NJS explodes with
so stick with the |
I need to transform response body so I am using
r.subrequest
which runs intotoo big subrequest response while sending to client
. Several issues are there to this effect and the solution seems to be to useIn my case I kept increasing this value until
4M
until it worked by trial and error.My question is what should be this value - does this depend on the size of the response.
If it is then how can this value to be set when we may not the size upfront.
I understand that setting this value too big is safe for it to work but at the same time unsafe for resources etc.
I tried using responseBuffer instead of responseBody but of no use.
Wondering if I should/should use subrequest in the first place ?
Wondering if this has anything to do with
internal;
in the subrequest location ?and here is my njs
and the errors
curl -I http://localhost/search
The text was updated successfully, but these errors were encountered: