Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deployment method using AWS Console doesn't work #732

Closed
tetianakravchenko opened this issue Jul 1, 2024 · 5 comments
Closed

Deployment method using AWS Console doesn't work #732

tetianakravchenko opened this issue Jul 1, 2024 · 5 comments
Assignees

Comments

@tetianakravchenko
Copy link

tetianakravchenko commented Jul 1, 2024

I followed documentation - https://www.elastic.co/guide/en/esf/current/aws-deploy-elastic-serverless-forwarder.html#aws-serverless-forwarder-deploy-console

during installation - I've set only ElasticServerlessForwarderS3ConfigFile in format s3://<config-bucket-name>/config.yaml

inputs:
  - type: "cloudwatch-logs"
    id: "arn:aws:logs:...:...:log-group:/aws/lambda/name-test:*"

    outputs:
      - type: "elasticsearch"
        args:
          # either elasticsearch_url or cloud_id, elasticsearch_url takes precedence if both are included
          elasticsearch_url: "https://..."
          # cloud_id: "cloud_id:..."
          # either api_key or username/password, username/password takes precedence if both are included
          # api_key: "..."
          username: "..."
          password: "..."
          es_datastream_name: "logs-generic-default"

and installation is failing:
Screenshot 2024-07-01 at 15 35 26

View stack event:
Screenshot 2024-07-01 at 15 57 33

cc @constanca-m

@constanca-m
Copy link
Contributor

I have tried to deploy it. I updated the needed variables:

  • The cloudwatch logs group here:
    image
    In my case, the value is arn:aws:logs:eu-north-1:627286350134:log-group:constanca-test-esf:*

  • And the S3 bucket to the config file:
    image

In my case, it would be: s3://constanca-test-esf/config.yaml, and this bucket looks like this:
image

The config.yaml looks like this:

"inputs":
- "id": "arn:aws:logs:eu-north-1:627286350134:log-group:constanca-test-esf:*"
  "outputs":
  - "args":
      "api_key": "..."
      "elasticsearch_url": "https://terraform-8b3bac.es.eu-central-1.aws.cloud.es.io"
      "es_datastream_name": "logs-esf.cloudwatch-default"
    "type": "elasticsearch"
  "type": "cloudwatch-logs"

Then I sent an event in my cloudwatch logs group:
image

And if I look in Discover I can see it there:
image

@tetianakravchenko
Copy link
Author

tetianakravchenko commented Jul 4, 2024

So the issue is related to fact that I was trying to deploy lambda application in the region where it was already deployed.

To sum up - we need to update documentation with the information:

  • all arns must be added
  • limitation that only 1 lambda application can be deployed per region

@tetianakravchenko
Copy link
Author

Also to check: Is it possible to install multiple installations per region? maybe it is possible to fix somehow?

@girodav
Copy link
Contributor

girodav commented Jul 22, 2024

Also to check: Is it possible to install multiple installations per region? maybe it is possible to fix somehow?

It's never been possible (with SAR) due to limitations on the AWS side (full context from Andrea: #150 (comment) ). Things might have changed since the last time either myself or Andrea looked at this, so it's worth having a look.

@kaiyan-sheng
Copy link
Contributor

Doc update: #767

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants