Skip to content

Commit

Permalink
Documentation on gh pages (#23)
Browse files Browse the repository at this point in the history
* Doc prototype

* f

* Soft dependency for temporalio

* Creating_application page

* Docs deployment
  • Loading branch information
northpowered authored Aug 13, 2024
1 parent ff4b2f5 commit e1c6006
Show file tree
Hide file tree
Showing 11 changed files with 827 additions and 9 deletions.
16 changes: 16 additions & 0 deletions .github/workflows/documentation.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
name: Documentation deployment

on:
push:
branches:
- main
jobs:
deploy:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/setup-python@v3
with:
python-version: 3.x
- run: pip install mkdocs-material
- run: mkdocs gh-deploy --force
5 changes: 3 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,11 @@
# Temporal-boost
> Under active development

> Documentation in progress
![social_preview](https://socialify.git.ci/northpowered/temporal-boost/image?description=1&font=Source%20Code%20Pro&logo=https%3A%2F%2Fraw.githubusercontent.com%2Ftemporalio%2Fdocumentation%2Fmain%2Fstatic%2Fimg%2Ffavicon.svg&name=1&owner=1&pattern=Brick%20Wall&theme=Light)


Small framework based on [temporalio/sdk-python](https://github.com/temporalio/sdk-python) - create [Temporal](https://temporal.io/) microservices as fast as you can


# Features

* Create Temporal workers with FastAPI-style
Expand Down
184 changes: 184 additions & 0 deletions docs/creating_application.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,184 @@
# Creating application

## Base code example

This\`s the base code snippet to start working with the framework. Create `BoostApp` object, set configuration for an app and run it.

```python
# Creating `BoostApp` object

from temporal_boost import BoostApp, BoostLoggerConfig, BoostOTLPConfig

app: BoostApp = BoostApp(
name="BoostApp example", # Name of the service
temporal_endpoint="localhost:7233", # Temporal endpoint
temporal_namespace="default", # Temporal namespace
logger_config=BoostLoggerConfig(
json=True,
bind_extra={"foo": "bar",},
level="DEBUG",
multiprocess_safe=True
),
otlp_config=BoostOTLPConfig(
otlp_endpoint="otlp-collector",
service_name="example-app"
),
use_pydantic=True # Use special JSON serializer inside Temporal SDK
)
```
### BoostLoggerConfig

BoostLoggerConfig is an object with setting for internal logger, based on loguru. There are the description:

Logs output. Can be io object of filename
> ```sink: typing.TextIO = sys.stdout```
Log severity
> ```level: str = "DEBUG" ```
Internal seriallizer for logs
> ```json: bool = True ```
We use internal formatter, but ypu can provide anyone
> ```formatter: typing.Callable | str = _default_json_formatter```
Enqueue mode of loguru. Use it in async and muliprocess apps
> ```multiprocess_safe: bool = True```
Dict, that you can fill with anythind you want. It\`ll be added to `extra` field of log format
> ```bind_extra: dict | None = None```

### BoostOTLPConfig

Similar to BoostLoggerConfig, BoostOTLPConfig is an object with setting for OTLP tracer. This tracer will be provided for original Tempral SDK tracer too.

OTLP collector endpoint
> ```otlp_endpoint: str```
OTLP service name
> ```service_name: str | None = None```
## Adding temporal workers

For adding worker to the app, you should use `add_worker` method. There are arguments for it:

```python
def add_worker(
self,
worker_name: str,
task_queue: str,
workflows: list = [],
activities: list = [],
cron_schedule: str | None = None,
cron_runner: typing.Coroutine | None = None,
metrics_endpoint: str | None = None,
description: str = "",
) -> None:
```
Name of the worker. Should be unique for the app and non-matching with TemporalBoost system worker names



!!! warning annotate "There some prohibited worker name, which are using for system purposes"

* *all*
* *internal*

> ```worker_name: str```
Task queue, there activities and workflows of this worker will`be registered.
> ```task_queue: str```
List of workflows for this worker, defaults to []
> ```workflows: list = []```
List of activities for this worker, defaults to []
> ```activities: list = []```
Cron schedule string, if you want to create cron worker, ex `* * * * *`
> ```cron_schedule: str | None = None```
Workflow `run` method, which will be executed, if you created cron worker. Required, if `cron_schedule` is not `None`
> ```cron_runner: typing.Coroutine | None = None```
Prometheus metrics endpoint for this worker. Should looks like `0.0.0.0:9000`
> ```metrics_endpoint: str | None = None```
Non-nessesary description for the worker
> ```description: str = ""```

### Examples
```python
app.add_worker(
"my_worker_1",
"my_queue_1",
activities=[my_activity],
metrics_endpoint="0.0.0.0:9000",
description="This workers serves activity my_activity on my_queue_1 and metrics endpoint"
)
app.add_worker(
"worker_2",
"my_queue_2",
workflows=[MyWorkflow],
description="This workers serves workflow MyWorkflow on my_queue_2"
)
app.add_worker(
"worker_3",
"my_queue_3",
workflows=[MyWorkflow2],
activities=[my_activity2],
description="This workers serves workflow MyWorkflow3 and activity my_activity2 on my_queue_3"
)

```

## Adding CRON workers

To execute some workflow with cron schedule, create cron worker like this:
```python
app.add_worker(
"worker_4",
"task_q_4",
workflows=[MyWorkflow],
cron_runner=MyWorkflow.run,
cron_schedule="* * * * *"
)
```
!!! note annotate "About arguments"
Here, `cron_runner` is a corutine, which will be started with `cron_schedule`

## Adding internal worker

Now, internal worker can be used for autogenerated documentation for Temporal objects

In future, opportunities of this object will be expanded

```python
app.add_internal_worker("0.0.0.0", 8888, doc_endpoint="/doc")
```

Now, go to [http://localhost:8888/doc](http://localhost:8888/doc) and use autogenerated documentation for workers, workflows, activities and other temporal objects.

!!! warning annotate "This documentation as a prototype in Alpha version"

![doc screenshot](img/doc_v0_screen.png)

## Adding ASGI workers

To add [FastAPI](https://github.com/fastapi/fastapi) application as an ASGI worker to your application at first you should add something like this:

```python
from fastapi import FastAPI, Response

fastapi_app: FastAPI = FastAPI(docs_url="/doc")
```

Now, you can add it to Temporal-boost application:

```python
app.add_asgi_worker("asgi_worker", fastapi_app, "0.0.0.0", 8000)
```

This application will be executed with [Hypercorn](https://github.com/pgjones/hypercorn) + [Trio](https://github.com/python-trio/trio) in asyncio runtime
Binary file added docs/img/doc_v0_screen.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
106 changes: 106 additions & 0 deletions docs/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,106 @@
# Getting started
![](https://socialify.git.ci/northpowered/temporal-boost/image?description=1&font=Source%20Code%20Pro&logo=https%3A%2F%2Fraw.githubusercontent.com%2Ftemporalio%2Fdocumentation%2Fmain%2Fstatic%2Fimg%2Ffavicon.svg&name=1&owner=1&pattern=Brick%20Wall&theme=Light)

## About the framework

Temporal-boost is a small framework, focused on fast and comfortable developing and deploying Temporal-based microservices. It based on standart Temporal SDK for Python, but offers FastAPI code organisation for developers.

### Main dependencies

- [x] [Temporal SDK (python)](https://github.com/temporalio/sdk-python)
- [x] [Pydantic - for serialization](https://github.com/pydantic/pydantic)
- [x] [Typer - for comportable CLI interface](https://github.com/fastapi/typer)
- [x] [Loguru - for extending stardart logger](https://github.com/Delgan/loguru)
- [x] [Hypercorn - for running ASGI (read `FastAPI`) applications](https://github.com/pgjones/hypercorn)
- [x] [Robyn - for service purposes (auto-generated documentation)](https://github.com/sparckles/Robyn)

### Main features

- [x] FastAPI-style application with ability to add workers (like routers)
- [x] Autogenerated documentation for workflows,activities, etc.
- [x] Сentralized logging and tracing management
- [x] Simple usage of CRON workflows
- [x] Ability to add external applications (ex. FastAPI) and deploy it in a separate container/pod

## Installation

```bash
poetry add temporal-boost
```

or

```bash
pip install temporal-boost
```

## Quick start

### Code example
> main.py
```python
from datetime import timedelta
from temporalio import activity, workflow

# Import `BoostApp` class
from temporal_boost import BoostApp, BoostLoggerConfig

# Create `BoostApp` object
app: BoostApp = BoostApp(
logger_config=BoostLoggerConfig(
json=True,
bind_extra={"logging_extra_data": "foo_bar"},
level="DEBUG"),
use_pydantic=True,
)


# Describe your activities/workflows
@activity.defn(name="my_activity")
async def my_activity(name: str) -> str:
return f"Hello, {name}!"

@workflow.defn(sandboxed=False, name="MyWorkflow")
class MyWorkflow:
@workflow.run
async def run(self, name: str) -> str:
return await workflow.execute_activity(
my_activity,
name,
task_queue="my_queue_1",
start_to_close_timeout=timedelta(minutes=1),
)

# Add async workers to your app
app.add_worker(
"my_worker_1",
"my_queue_1",
activities=[my_activity],
metrics_endpoint="0.0.0.0:9000",
description="This workers serves activity my_activity on my_queue_1",
)
app.add_worker(
"worker_2",
"my_queue_2",
workflows=[MyWorkflow]
)

app.add_internal_worker("0.0.0.0",8000, doc_endpoint="/doc")

app.run()
```
### Start example application

Starting all workers at once

```bash
python3 main.py run all
```

Starting workers one by one

```bash
python3 main.py run worker_1
python3 main.py run worker_2
```
59 changes: 59 additions & 0 deletions docs/runnig_application.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
# Running application

Here the example of our app:

```python
# main.py
from temporal_boost import BoostApp

app: BoostApp = BoostApp()

# Some workflows or activities
# ...
# Some FastAPI app
# ...

# Temporal workers
app.add_worker(
"worker_1", "task_q_1", activities=[test_boost_activity_1, test_boost_activity_3],
)
app.add_worker("worker_2", "task_q_2", activities=[test_boost_activity_2])

app.add_worker("worker_3", "task_q_3", workflows=[MyWorkflow])

# Cron worker
app.add_worker("worker_4", "task_q_4", workflows=[MyWorkflow], cron_runner=MyWorkflow.run, cron_schedule="* * * * *")

# FastAPI worker
app.add_asgi_worker("asgi_worker", fastapi_app, "0.0.0.0", 8000)

# Internal worker
app.add_internal_worker("0.0.0.0", 8888, doc_endpoint="/doc")

app.run()
```

## Running in dev mode

In `dev` mode all workers (except crons) can be started with one command in multiprocess mode:

```bash
python3 main.py run all
```
!!! warning annotate "Never use this way in production!"

## Running workers one-by-one

Also, you can run each worker by name:

```bash
python3 main.py run worker_1

python3 main.py run asgi_worker
```

To run cron worker, use this command:

```bash
python3 main.py cron worker_4
```
Loading

0 comments on commit e1c6006

Please sign in to comment.