Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

query may timeout if next is not called in time. #118

Open
youngsofun opened this issue Dec 29, 2023 · 9 comments
Open

query may timeout if next is not called in time. #118

youngsofun opened this issue Dec 29, 2023 · 9 comments

Comments

@youngsofun
Copy link
Member

currently, next_uri is called only when iter.next() is called, client may do some time-consuming tasks between 2 calls of next,
this may lead to timeout of the query in the server side.

since the results is keeped in memory now, the server does need a way to clear the queries that are not polled for long time.

one solution

  1. client get timeout time from the server, from the first resp
  2. start a thread to send the heartbeat request
    3. optionally, there can be a global thread to send all query-ids in use together, it never fails: client return query-ids finished, and the heartbeater simpliy not send them any more
@youngsofun
Copy link
Member Author

youngsofun commented Dec 29, 2023

@hantmac
Copy link
Member

hantmac commented Dec 29, 2023

@youngsofun So that is mean we can't call next_uri concurrently?

@youngsofun
Copy link
Member Author

@youngsofun So that is mean we can't call next_uri concurrently?

not for same query_id.

@hantmac
Copy link
Member

hantmac commented Jan 3, 2024

The timeout was caused by memory leak which reason is presign stmt not closed correctly. And the bug was fixed the timeout disappeard.
I will close this issue.

@hantmac hantmac closed this as completed Jan 3, 2024
@youngsofun
Copy link
Member Author

youngsofun commented Jan 5, 2024

I think this is still a problem, but not very easy to produce.
because most of time results is returned in one page (10000 rows/page)
I guess this will fail:

  1. select numbers(20000) ,
  2. call next. // this will fetch 10000 rows
  3. sleep(5 sec)
  4. call next again entil end (timeout!) // first consuming the 10000 rows already fetched, get timeout when fetching more

updated @hantmac

@youngsofun
Copy link
Member Author

where is the reopen button?

@hantmac hantmac reopened this Jan 5, 2024
@hantmac
Copy link
Member

hantmac commented Jan 5, 2024

I think this is still a problem, but not very easy to produce. because most of time results is returned in one page (10000 rows/page) I guess this will fail:

  1. select numbers(20000) ,
  2. call next
  3. sleep(3 sec)
  4. call next again (timeout!)

I can't reproduce it.
image

I sleep 10s while call next but it works well.

@youngsofun
Copy link
Member Author

@hantmac

I think this is still a problem, but not very easy to produce. because most of time results is returned in one page (10000 rows/page) I guess this will fail:

  1. select numbers(20000) ,
  2. call next. // this will fetch 10000 rows
  3. sleep(5 sec)
  4. call next again entil end (timeout!) // first consuming the 10000 rows already fetched, get timeout when fetching more
    updated @hantmac

@wubx
Copy link
Member

wubx commented Jan 31, 2024

Now for many case I have change http_handler_result_timeout_secs=300

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants