-
Notifications
You must be signed in to change notification settings - Fork 1
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
* Doc prototype * f * Soft dependency for temporalio * Creating_application page * Docs deployment
- Loading branch information
1 parent
ff4b2f5
commit e1c6006
Showing
11 changed files
with
827 additions
and
9 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,16 @@ | ||
name: Documentation deployment | ||
|
||
on: | ||
push: | ||
branches: | ||
- main | ||
jobs: | ||
deploy: | ||
runs-on: ubuntu-latest | ||
steps: | ||
- uses: actions/checkout@v3 | ||
- uses: actions/setup-python@v3 | ||
with: | ||
python-version: 3.x | ||
- run: pip install mkdocs-material | ||
- run: mkdocs gh-deploy --force |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,184 @@ | ||
# Creating application | ||
|
||
## Base code example | ||
|
||
This\`s the base code snippet to start working with the framework. Create `BoostApp` object, set configuration for an app and run it. | ||
|
||
```python | ||
# Creating `BoostApp` object | ||
|
||
from temporal_boost import BoostApp, BoostLoggerConfig, BoostOTLPConfig | ||
|
||
app: BoostApp = BoostApp( | ||
name="BoostApp example", # Name of the service | ||
temporal_endpoint="localhost:7233", # Temporal endpoint | ||
temporal_namespace="default", # Temporal namespace | ||
logger_config=BoostLoggerConfig( | ||
json=True, | ||
bind_extra={"foo": "bar",}, | ||
level="DEBUG", | ||
multiprocess_safe=True | ||
), | ||
otlp_config=BoostOTLPConfig( | ||
otlp_endpoint="otlp-collector", | ||
service_name="example-app" | ||
), | ||
use_pydantic=True # Use special JSON serializer inside Temporal SDK | ||
) | ||
``` | ||
### BoostLoggerConfig | ||
|
||
BoostLoggerConfig is an object with setting for internal logger, based on loguru. There are the description: | ||
|
||
Logs output. Can be io object of filename | ||
> ```sink: typing.TextIO = sys.stdout``` | ||
Log severity | ||
> ```level: str = "DEBUG" ``` | ||
Internal seriallizer for logs | ||
> ```json: bool = True ``` | ||
We use internal formatter, but ypu can provide anyone | ||
> ```formatter: typing.Callable | str = _default_json_formatter``` | ||
Enqueue mode of loguru. Use it in async and muliprocess apps | ||
> ```multiprocess_safe: bool = True``` | ||
Dict, that you can fill with anythind you want. It\`ll be added to `extra` field of log format | ||
> ```bind_extra: dict | None = None``` | ||
|
||
### BoostOTLPConfig | ||
|
||
Similar to BoostLoggerConfig, BoostOTLPConfig is an object with setting for OTLP tracer. This tracer will be provided for original Tempral SDK tracer too. | ||
|
||
OTLP collector endpoint | ||
> ```otlp_endpoint: str``` | ||
OTLP service name | ||
> ```service_name: str | None = None``` | ||
## Adding temporal workers | ||
|
||
For adding worker to the app, you should use `add_worker` method. There are arguments for it: | ||
|
||
```python | ||
def add_worker( | ||
self, | ||
worker_name: str, | ||
task_queue: str, | ||
workflows: list = [], | ||
activities: list = [], | ||
cron_schedule: str | None = None, | ||
cron_runner: typing.Coroutine | None = None, | ||
metrics_endpoint: str | None = None, | ||
description: str = "", | ||
) -> None: | ||
``` | ||
Name of the worker. Should be unique for the app and non-matching with TemporalBoost system worker names | ||
|
||
|
||
|
||
!!! warning annotate "There some prohibited worker name, which are using for system purposes" | ||
|
||
* *all* | ||
* *internal* | ||
|
||
> ```worker_name: str``` | ||
Task queue, there activities and workflows of this worker will`be registered. | ||
> ```task_queue: str``` | ||
List of workflows for this worker, defaults to [] | ||
> ```workflows: list = []``` | ||
List of activities for this worker, defaults to [] | ||
> ```activities: list = []``` | ||
Cron schedule string, if you want to create cron worker, ex `* * * * *` | ||
> ```cron_schedule: str | None = None``` | ||
Workflow `run` method, which will be executed, if you created cron worker. Required, if `cron_schedule` is not `None` | ||
> ```cron_runner: typing.Coroutine | None = None``` | ||
Prometheus metrics endpoint for this worker. Should looks like `0.0.0.0:9000` | ||
> ```metrics_endpoint: str | None = None``` | ||
Non-nessesary description for the worker | ||
> ```description: str = ""``` | ||
|
||
### Examples | ||
```python | ||
app.add_worker( | ||
"my_worker_1", | ||
"my_queue_1", | ||
activities=[my_activity], | ||
metrics_endpoint="0.0.0.0:9000", | ||
description="This workers serves activity my_activity on my_queue_1 and metrics endpoint" | ||
) | ||
app.add_worker( | ||
"worker_2", | ||
"my_queue_2", | ||
workflows=[MyWorkflow], | ||
description="This workers serves workflow MyWorkflow on my_queue_2" | ||
) | ||
app.add_worker( | ||
"worker_3", | ||
"my_queue_3", | ||
workflows=[MyWorkflow2], | ||
activities=[my_activity2], | ||
description="This workers serves workflow MyWorkflow3 and activity my_activity2 on my_queue_3" | ||
) | ||
|
||
``` | ||
|
||
## Adding CRON workers | ||
|
||
To execute some workflow with cron schedule, create cron worker like this: | ||
```python | ||
app.add_worker( | ||
"worker_4", | ||
"task_q_4", | ||
workflows=[MyWorkflow], | ||
cron_runner=MyWorkflow.run, | ||
cron_schedule="* * * * *" | ||
) | ||
``` | ||
!!! note annotate "About arguments" | ||
Here, `cron_runner` is a corutine, which will be started with `cron_schedule` | ||
|
||
## Adding internal worker | ||
|
||
Now, internal worker can be used for autogenerated documentation for Temporal objects | ||
|
||
In future, opportunities of this object will be expanded | ||
|
||
```python | ||
app.add_internal_worker("0.0.0.0", 8888, doc_endpoint="/doc") | ||
``` | ||
|
||
Now, go to [http://localhost:8888/doc](http://localhost:8888/doc) and use autogenerated documentation for workers, workflows, activities and other temporal objects. | ||
|
||
!!! warning annotate "This documentation as a prototype in Alpha version" | ||
|
||
![doc screenshot](img/doc_v0_screen.png) | ||
|
||
## Adding ASGI workers | ||
|
||
To add [FastAPI](https://github.com/fastapi/fastapi) application as an ASGI worker to your application at first you should add something like this: | ||
|
||
```python | ||
from fastapi import FastAPI, Response | ||
|
||
fastapi_app: FastAPI = FastAPI(docs_url="/doc") | ||
``` | ||
|
||
Now, you can add it to Temporal-boost application: | ||
|
||
```python | ||
app.add_asgi_worker("asgi_worker", fastapi_app, "0.0.0.0", 8000) | ||
``` | ||
|
||
This application will be executed with [Hypercorn](https://github.com/pgjones/hypercorn) + [Trio](https://github.com/python-trio/trio) in asyncio runtime |
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,106 @@ | ||
# Getting started | ||
![](https://socialify.git.ci/northpowered/temporal-boost/image?description=1&font=Source%20Code%20Pro&logo=https%3A%2F%2Fraw.githubusercontent.com%2Ftemporalio%2Fdocumentation%2Fmain%2Fstatic%2Fimg%2Ffavicon.svg&name=1&owner=1&pattern=Brick%20Wall&theme=Light) | ||
|
||
## About the framework | ||
|
||
Temporal-boost is a small framework, focused on fast and comfortable developing and deploying Temporal-based microservices. It based on standart Temporal SDK for Python, but offers FastAPI code organisation for developers. | ||
|
||
### Main dependencies | ||
|
||
- [x] [Temporal SDK (python)](https://github.com/temporalio/sdk-python) | ||
- [x] [Pydantic - for serialization](https://github.com/pydantic/pydantic) | ||
- [x] [Typer - for comportable CLI interface](https://github.com/fastapi/typer) | ||
- [x] [Loguru - for extending stardart logger](https://github.com/Delgan/loguru) | ||
- [x] [Hypercorn - for running ASGI (read `FastAPI`) applications](https://github.com/pgjones/hypercorn) | ||
- [x] [Robyn - for service purposes (auto-generated documentation)](https://github.com/sparckles/Robyn) | ||
|
||
### Main features | ||
|
||
- [x] FastAPI-style application with ability to add workers (like routers) | ||
- [x] Autogenerated documentation for workflows,activities, etc. | ||
- [x] Сentralized logging and tracing management | ||
- [x] Simple usage of CRON workflows | ||
- [x] Ability to add external applications (ex. FastAPI) and deploy it in a separate container/pod | ||
|
||
## Installation | ||
|
||
```bash | ||
poetry add temporal-boost | ||
``` | ||
|
||
or | ||
|
||
```bash | ||
pip install temporal-boost | ||
``` | ||
|
||
## Quick start | ||
|
||
### Code example | ||
> main.py | ||
```python | ||
from datetime import timedelta | ||
from temporalio import activity, workflow | ||
|
||
# Import `BoostApp` class | ||
from temporal_boost import BoostApp, BoostLoggerConfig | ||
|
||
# Create `BoostApp` object | ||
app: BoostApp = BoostApp( | ||
logger_config=BoostLoggerConfig( | ||
json=True, | ||
bind_extra={"logging_extra_data": "foo_bar"}, | ||
level="DEBUG"), | ||
use_pydantic=True, | ||
) | ||
|
||
|
||
# Describe your activities/workflows | ||
@activity.defn(name="my_activity") | ||
async def my_activity(name: str) -> str: | ||
return f"Hello, {name}!" | ||
|
||
@workflow.defn(sandboxed=False, name="MyWorkflow") | ||
class MyWorkflow: | ||
@workflow.run | ||
async def run(self, name: str) -> str: | ||
return await workflow.execute_activity( | ||
my_activity, | ||
name, | ||
task_queue="my_queue_1", | ||
start_to_close_timeout=timedelta(minutes=1), | ||
) | ||
|
||
# Add async workers to your app | ||
app.add_worker( | ||
"my_worker_1", | ||
"my_queue_1", | ||
activities=[my_activity], | ||
metrics_endpoint="0.0.0.0:9000", | ||
description="This workers serves activity my_activity on my_queue_1", | ||
) | ||
app.add_worker( | ||
"worker_2", | ||
"my_queue_2", | ||
workflows=[MyWorkflow] | ||
) | ||
|
||
app.add_internal_worker("0.0.0.0",8000, doc_endpoint="/doc") | ||
|
||
app.run() | ||
``` | ||
### Start example application | ||
|
||
Starting all workers at once | ||
|
||
```bash | ||
python3 main.py run all | ||
``` | ||
|
||
Starting workers one by one | ||
|
||
```bash | ||
python3 main.py run worker_1 | ||
python3 main.py run worker_2 | ||
``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,59 @@ | ||
# Running application | ||
|
||
Here the example of our app: | ||
|
||
```python | ||
# main.py | ||
from temporal_boost import BoostApp | ||
|
||
app: BoostApp = BoostApp() | ||
|
||
# Some workflows or activities | ||
# ... | ||
# Some FastAPI app | ||
# ... | ||
|
||
# Temporal workers | ||
app.add_worker( | ||
"worker_1", "task_q_1", activities=[test_boost_activity_1, test_boost_activity_3], | ||
) | ||
app.add_worker("worker_2", "task_q_2", activities=[test_boost_activity_2]) | ||
|
||
app.add_worker("worker_3", "task_q_3", workflows=[MyWorkflow]) | ||
|
||
# Cron worker | ||
app.add_worker("worker_4", "task_q_4", workflows=[MyWorkflow], cron_runner=MyWorkflow.run, cron_schedule="* * * * *") | ||
|
||
# FastAPI worker | ||
app.add_asgi_worker("asgi_worker", fastapi_app, "0.0.0.0", 8000) | ||
|
||
# Internal worker | ||
app.add_internal_worker("0.0.0.0", 8888, doc_endpoint="/doc") | ||
|
||
app.run() | ||
``` | ||
|
||
## Running in dev mode | ||
|
||
In `dev` mode all workers (except crons) can be started with one command in multiprocess mode: | ||
|
||
```bash | ||
python3 main.py run all | ||
``` | ||
!!! warning annotate "Never use this way in production!" | ||
|
||
## Running workers one-by-one | ||
|
||
Also, you can run each worker by name: | ||
|
||
```bash | ||
python3 main.py run worker_1 | ||
|
||
python3 main.py run asgi_worker | ||
``` | ||
|
||
To run cron worker, use this command: | ||
|
||
```bash | ||
python3 main.py cron worker_4 | ||
``` |
Oops, something went wrong.