Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature Request] There should not be error-level logs on startup #633

Open
dandavison opened this issue Aug 9, 2024 · 1 comment
Open
Labels
enhancement New feature or request

Comments

@dandavison
Copy link
Contributor

When starting up with no previous DB contents, there should not be any errors (or warnings).

$ /opt/homebrew/bin/temporal server start-dev
time=2024-08-09T12:36:38.477 level=ERROR msg="service failures" operation=GetSystemInfo error="Frontend is not healthy yet"
time=2024-08-09T12:36:38.477 level=WARN msg="error creating sdk client" service=worker error="failed reaching server: Frontend is not healthy yet"
CLI 1.0.0 (Server 1.24.2, UI 2.28.0)

Server:  localhost:7233
UI:      http://localhost:8233
Metrics: http://localhost:50972/metrics

v1.0

@dandavison dandavison added the enhancement New feature or request label Aug 9, 2024
@dandavison
Copy link
Contributor Author

Another one seen (on recent server) is

time=2024-08-09T13:12:52.750 level=ERROR msg="Skipping notification for new tasks, processor not registered" shard-id=1 address=127.0.0.1:52909 component=history-engine queue-task-category-id=7
time=2024-08-09T13:12:52.756 level=WARN msg="error in prometheus reporter" error="a previously registered descriptor with the same fully-qualified name as Desc{fqName: \"workflow_context_cleared\", help: \"workflow_context_cleared counter\", constLabels: {}, variableLabels: {service_name,operation,cache_type,namespace_id}} has different label names or a different help string"

stephanos added a commit that referenced this issue Aug 9, 2024
<!--- Note to EXTERNAL Contributors -->
<!-- Thanks for opening a PR! 
If it is a significant code change, please **make sure there is an open
issue** for this.
We work best with you when we have accepted the idea first before you
code. -->

<!--- For ALL Contributors 👇 -->

## What was changed
<!-- Describe what has changed in this PR -->

Use static membership assignment for server services.

## Why?
<!-- Tell your future self why have you made these changes -->

Fixes #632 and
#633.

## Checklist
<!--- add/delete as needed --->

1. Closes <!-- add issue number here -->

2. How was this tested:
<!--- Please describe how you tested your changes/how we can test them
-->

**(1) used a script to verify the behavior**

Before change:
```
go build ./cmd/temporal
Iteration 1/10
Error: 'Frontend is not healthy yet' found.
Killing the Temporal server with PID 21433...
```

```
go build ./cmd/temporal
Iteration 1/10
Killing the Temporal server with PID 21600...
Iteration 2/10
Error: 'Not enough hosts to serve the request' found.
Killing the Temporal server with PID 21613...
```

After change:
```
go build ./cmd/temporal
Iteration 1/10
Killing the Temporal server with PID 21190...
Iteration 2/10
Killing the Temporal server with PID 21198...
Iteration 3/10
Killing the Temporal server with PID 21206...
Iteration 4/10
Killing the Temporal server with PID 21214...
Iteration 5/10
Killing the Temporal server with PID 21222...
Iteration 6/10
Killing the Temporal server with PID 21230...
Iteration 7/10
Killing the Temporal server with PID 21237...
Iteration 8/10
Killing the Temporal server with PID 21243...
Iteration 9/10
Killing the Temporal server with PID 21251...
Iteration 10/10
Killing the Temporal server with PID 21257...
```

<details>
<summary>Script</summary>

```bash
#!/bin/bash

make build

SERVER_PID=0
kill_server() {
    if [ $SERVER_PID -ne 0 ]; then
        echo "Killing the Temporal server with PID $SERVER_PID..."
        kill $SERVER_PID
        wait $SERVER_PID 2>/dev/null
        SERVER_PID=0
        rm -rf server_output.log
    fi
}
trap kill_server EXIT

# Loop to run the start-stop sequence 10 times
for i in {1..10}; do
    echo "Iteration $i/10"

    ./temporal server start-dev --db-filename=/tmp/db > server_output.log 2>&1 &
    SERVER_PID=$!

    sleep 2

    if grep -q "Not enough hosts to serve the request" server_output.log; then
        echo "Error: 'Not enough hosts to serve the request' found."
        kill $SERVER_PID
        exit 1
    fi

    if grep -q "Frontend is not healthy yet" server_output.log; then
        echo "Error: 'Frontend is not healthy yet' found."
        kill $SERVER_PID
        exit 1
    fi

    kill_server

    sleep 1
done
```
</details>

**(2) ran sdk-go integration tests against dev server**

3. Any docs updates needed?
<!--- update README if applicable
      or point out where to update docs.temporal.io -->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant