Uvicorn cannot be shutdown programmatically #1103
Replies: 26 comments 21 replies
-
Hi, Not documented indeed, but a multithreaded approach should do… import contextlib
import time
import threading
import uvicorn
class Server(uvicorn.Server):
def install_signal_handlers(self):
pass
@contextlib.contextmanager
def run_in_thread(self):
thread = threading.Thread(target=self.run)
thread.start()
try:
while not self.started:
time.sleep(1e-3)
yield
finally:
self.should_exit = True
thread.join()
config = Config("example:app", host="127.0.0.1", port=5000, log_level="info")
server = Server(config=config)
with server.run_in_thread():
# Server started.
...
# Server stopped. Very handy to run a live test server locally using a pytest fixture… # conftest.py
import pytest
@pytest.fixture(scope="session")
def server():
server = ...
with server.run_in_thread():
yield |
Beta Was this translation helpful? Give feedback.
-
I do appreciate the reply/code, but: Why is this not built in and documented? It's a webserver. Run_in_thread() and shutdown() are core functionality...am I wrong? |
Beta Was this translation helpful? Give feedback.
-
I'm following @florimondmanca s approach to run pact tests on the provider side. However when using this i get the following error from uvicorn's Server.run method:
Any idea on that? Happens with python 3.6.9 as well as 3.7.7. another edit: I got it to work using
|
Beta Was this translation helpful? Give feedback.
-
Fully agreed, this question of how to run and stop unicorn pops up on stack overflow as well, it's quite obscure as it is now |
Beta Was this translation helpful? Give feedback.
-
I appreciate the feedback here, but then this brings the question — what would folks expect the usage API to be for something like this? I think a nice approach for allowing a programmatic shutdown of Uvicorn would be to exit the space of But then there's the question of A/ waiting for the server to startup (without having to reinvent the wheel each time), and B/ triggering the server shutdown and have it clean up gracefully (straight up cancelling a task is very... brutal). Personally, I'd love this kind of API: async with open_task_group() as tg:
tg.start_soon(server.serve)
await server.wait_started()
# ...
# Cause serve() to terminate soon, allowing the task
# to finish cleanly when exiting the context manager.
server.shutdown_soon(). Here @asynccontextmanager
async def start_concurrently(async_fn):
task = asyncio.create_task(async_fn())
try:
yield
finally:
await task It could be used like this: async with start_concurrently(server.serve):
await server.wait_started()
# ...
server.shutdown_soon() I'm also happy to discuss a threaded equivalent API, for use cases where an event loop isn't readily available or practical (for example, when testing the server using a sync HTTP client). Then we need to figure out how to make these APIs come to life. :-) Maybe this means we'll need sync/async equivalents, like |
Beta Was this translation helpful? Give feedback.
-
That's quite an elaborate pondering, thank you for that! I'd be willing to use all these APIs:) Just for curiosityIf I understand correctly, having the
|
Beta Was this translation helpful? Give feedback.
-
Well, yes, if we add a And then a Now that we know this is the kind of thing we want to have eventually, anyone that'd like to figure out internals and ways to implement this is very much welcome to. :) I'll reopen since I think this is indeed a valuable thing to add. |
Beta Was this translation helpful? Give feedback.
-
For people that stop by here and want to try out florimondmanca's multithreaded approach and are a bit new to uvicorn, Config in the code comes from uvicorn. Configuration below also includes plazmakeks' addition in the case of running into a runtime error described by them.
Thanks for this discussion all. |
Beta Was this translation helpful? Give feedback.
-
I also encountered this RuntimeError and I did not want to force I think it would help everyone if the following code was implemented for import asyncio
+ import threading
import uvloop
def uvloop_setup():
- asyncio.set_event_loop_policy(uvloop.EventLoopPolicy())
+ if (threading.current_thread() is threading.main_thread()):
+ asyncio.set_event_loop_policy(uvloop.EventLoopPolicy())
+ else:
+ asyncio.set_event_loop(uvloop.new_event_loop()) |
Beta Was this translation helpful? Give feedback.
-
@florimondmanca I used your code and found the code to be ever running in case self.started was never set. In my case , a port was already in use. But the program does not stop on exception here. Added some output Log below :-
And this line was never met. Hence I have modified the code as below . Since I am not familiar with the internals of uvicorn server, I may not know if the below code has some edge case or anything that I am missing. Let me know if I am missing anything !! class Server(uvicorn.Server):
def install_signal_handlers(self):
pass
@contextlib.contextmanager
def running(self):
thread = threading.Thread(target=self.run)
thread.start()
try:
while not self.started and thread.is_alive(): # Added a condition for checking if thread is alive
time.sleep(1e-3)
yield
finally:
self.should_exit = True
thread.join() |
Beta Was this translation helpful? Give feedback.
-
I am very confused why this requires some crazy-hoops. I checked the def run(app, **kwargs):
runner = create_runner(app, kwargs)
runner.run()
def create_runner(app, **kwargs):
config = Config(app, **kwargs)
server = Server(config=config)
if (config.reload or config.workers > 1) and not isinstance(app, str):
logger = logging.getLogger("uvicorn.error")
logger.warning(
"You must pass the application as an import string to enable 'reload' or "
"'workers'."
)
sys.exit(1)
if config.should_reload:
sock = config.bind_socket()
supervisor = ChangeReload(config, target=server.run, sockets=[sock])
return supervisor
elif config.workers > 1:
sock = config.bind_socket()
supervisor = Multiprocess(config, target=server.run, sockets=[sock])
return supervisor
else:
return server This way someone can still do |
Beta Was this translation helpful? Give feedback.
-
Anyone care to review #1011? It doesn't solve this issue completely, but it would make the solution a little simpler. |
Beta Was this translation helpful? Give feedback.
-
I have not had success in triggering this to shut down from within a FastAPI route. Could anyone shed light as to how this can be made to work without resorting to psutil process-killing?
|
Beta Was this translation helpful? Give feedback.
-
migrating this as a discussion following this comment from @rmorshea let's come up with a design and we see if we add it or not, I still dont get why subclassing the server is not sufficient, but I admit I didn't spend much time on this, nor have the will to do it, so someone will have to step up and eli5 me or other maintainers the ins and outs. just for the record, the whole test suite was using this run in a thread pattern and it was just ugly, flaky, full of undesirable and different side-effects on the various platforms we support, so we removed it and so far the experience is way better without it. |
Beta Was this translation helpful? Give feedback.
-
I would like to suggest an easier solution: import multiprocessing
from time import sleep
from fastapi import FastAPI
from uvicorn import Config, Server
class UvicornServer(multiprocessing.Process):
def __init__(self, config: Config):
super().__init__()
self.config = config
def stop(self):
self.terminate()
def run(self, *args, **kwargs):
server = Server(config=self.config)
server.run()
app = FastAPI()
def main(): # POC
u = UvicornServer(config=Config(app=app))
u.start()
sleep(5) # Start timeout
u.stop()
sleep(5) # Stop timeout & Prevent EOF termination
if __name__ == '__main__':
main() |
Beta Was this translation helpful? Give feedback.
-
I don't see why you need to inherit from In any case, there's a more normal way to cancel async tasks: async task cancellation. Just run
[Edit: I just noticed that the original request doesn't even mention threads. If you just want to run a uvicorn server in a separate asyncio task and then stop it later, you can just use Cancellation of asyncio tasks from other threads is a bit of faff but that's more asyncio's fault than uvicorn's. You need to create a task and capture the asyncio loop so you can call the task's cancel method in the other thread. For example, it could look like this (assuming this is part of a wider class but I've not bothered to fill it all in):
You would stop the server like this:
Assuming you start a thread with Gracefully waiting for startup to complete, if that's important, is harder. It's a pity that the best suggestion is to loop waiting for |
Beta Was this translation helpful? Give feedback.
-
some good suggestions here https://stackoverflow.com/q/57412825/1032286 |
Beta Was this translation helpful? Give feedback.
-
To bad his is in discussion mode and not handled as an issue any more. It would be great to hide the complexity behind this especially since the multiprocessing suggestion doesn't cut it in case there is non-pickable stuff. |
Beta Was this translation helpful? Give feedback.
-
Apologies if i am missing the obvious, but is there a reason we cannot add a For example, I plan on using (i.e. have not tested) the following to make this work. class Config(uvicorn.Config):
def __init__(self, should_exit_timeout: Optional[float] = None, *args, **kwargs) -> None:
super().__init__(*args, **kwargs)
self.should_exit_timeout = should_exit_timeout
class Server(uvicorn.Server):
async def shutdown(self, sockets: Optional[List[socket.socket]] = None) -> None:
logger = logging.getLogger("uvicorn.error")
logger.info("Shutting down")
should_exit_deadline = None
if self.config.should_exit_timeout is not None:
should_exit_deadline = time.time() + self.config.should_exit_timeout
# Stop accepting new connections.
for server in self.servers:
server.close()
for sock in sockets or []:
sock.close()
for server in self.servers:
await server.wait_closed()
# Request shutdown on all existing connections.
for connection in list(self.server_state.connections):
connection.shutdown()
await asyncio.sleep(0.1)
# Wait for existing connections to finish sending responses.
if self.server_state.connections and not self.force_exit:
msg = "Waiting for connections to close. (CTRL+C to force quit)"
logger.info(msg)
while self.server_state.connections and not self.force_exit and \
(should_exit_deadline is None or time.time() < should_exit_deadline):
await asyncio.sleep(0.1)
# Wait for existing tasks to complete.
if self.server_state.tasks and not self.force_exit:
msg = "Waiting for background tasks to complete. (CTRL+C to force quit)"
uvicorn.logger.info(msg)
while self.server_state.tasks and not self.force_exit:
await asyncio.sleep(0.1)
# Send the lifespan shutdown event, and wait for application shutdown.
if not self.force_exit:
await self.lifespan.shutdown()
uvicorn.Server.shutdown = shutdown |
Beta Was this translation helpful? Give feedback.
-
I may have missed something, but it's really easy to do this extending the server as @euri10 said previously Naive example: class UvicornServer(uvicorn.Server):
serve_task: Task
did_start: Event
did_close: Event
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self.did_start = Event()
self.did_close = Event()
async def start(self):
self.serve_task = asyncio.create_task(self.serve())
self.serve_task.add_done_callback(lambda _: self.did_close.set())
await self.did_start.wait()
async def startup(self, sockets: list = None) -> None:
await super().startup(sockets)
self.did_start.set()
async def shutdown(self):
await super().shutdown()
self.serve_task.cancel()
await self.did_close.wait() |
Beta Was this translation helpful? Give feedback.
-
|
Beta Was this translation helpful? Give feedback.
-
If you are fine with running Uvicorn in a separate thread, it is possible to start/stop without relying on async: class ThreadedUvicorn:
def __init__(self, config: Config):
self.server = uvicorn.Server(config)
self.thread = threading.Thread(daemon=True, target=self.server.run)
def start(self):
self.thread.start()
asyncio.run(self.wait_for_started())
async def wait_for_started(self):
while not self.server.started:
await asyncio.sleep(0.1)
def stop(self):
if self.thread.is_alive():
self.server.should_exit = True
while self.thread.is_alive():
continue
server = ThreadedUvicorn(app, host=host, port=port)
server.start()
server.stop() @rodja this should also work cross-platform. Might need to check if the platform is |
Beta Was this translation helpful? Give feedback.
-
Okey, I have a one-line solution, please provide explenation why this is also a bad idea:
(the need to import |
Beta Was this translation helpful? Give feedback.
-
see also zauberzeug/nicegui#1956 |
Beta Was this translation helpful? Give feedback.
-
I'm surprised nobody suggested something like signal.raise_signal(signal.SIGINT) If you're running in the main thread, then uvicorn already has signal handlers set up for you, so all we need is to emulate a signal to shutdown. |
Beta Was this translation helpful? Give feedback.
-
On windows, go to the task manager (ctrl + shift + esc) and kill the cmd process |
Beta Was this translation helpful? Give feedback.
-
There is no documented way to shutdown uvicorn in python:
ex:
How do we shutdown uvicorn?
Beta Was this translation helpful? Give feedback.
All reactions