Skip to content

Commit

Permalink
Refresh plugin metadata 2023-11-06
Browse files Browse the repository at this point in the history
  • Loading branch information
pnadolny13 authored Nov 6, 2023
1 parent 6c2080a commit e96d230
Show file tree
Hide file tree
Showing 26 changed files with 773 additions and 196 deletions.
6 changes: 3 additions & 3 deletions _data/meltano/extractors/tap-cassandra/datarts-tech.yml
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ settings:
name: keyspace
- description: The local_dc parameter should be the name of the datacenter.
kind: string
label: Local Datacenter
label: Local Dc
name: local_dc
- description: Should be a total number of attempts to be made before giving up.
kind: integer
Expand All @@ -76,7 +76,7 @@ settings:
kind: password
label: Password
name: password
- description: The server-side port to open connections to. Defaults to 9042..
- description: The server-side port to open connections to. Defaults to 9042. .
kind: integer
label: Port
name: port
Expand All @@ -91,7 +91,7 @@ settings:
label: Reconnect Delay
name: reconnect_delay
value: 60
- description: Request timeout used when not overridden in Session.execute().
- description: Request timeout used when not overridden in Session. Execute().
kind: integer
label: Request Timeout
name: request_timeout
Expand Down
1 change: 1 addition & 0 deletions _data/meltano/extractors/tap-db2/danielptv.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ capabilities:
- about
- catalog
- discover
- state
- stream-maps
description: IBM DB2 Relational Database
domain_url: https://www.ibm.com/products/db2
Expand Down
2 changes: 1 addition & 1 deletion _data/meltano/extractors/tap-dbt/meltanolabs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,7 @@ settings:
kind: string
label: User Agent
name: user_agent
value: tap-dbt/0.8.3.dev9+eff3995 Singer tap for the dbt Cloud API.
value: tap-dbt/0.9.1.dev6+8b49746 Singer tap for the dbt Cloud API.
settings_group_validation:
- - account_ids
- api_key
Expand Down
2 changes: 1 addition & 1 deletion _data/meltano/extractors/tap-gmail/kgpayne.yml
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ settings:
value: false
- description: Only return messages matching the specified query. Supports the same
query format as the Gmail search box. For example, "from:[email protected]
rfc822msgid:<somemsgid@example.com> is:unread". Parameter cannot be used when
rfc822msgid:<somemsgid@example. Com> is:unread". Parameter cannot be used when
accessing the api using the gmail. Metadata scope. Https://developers. Google.
Com/gmail/api/reference/rest/v1/users. Messages/list#query-parameters
kind: string
Expand Down
2 changes: 1 addition & 1 deletion _data/meltano/extractors/tap-gmail/meltanolabs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ settings:
value: false
- description: Only return messages matching the specified query. Supports the same
query format as the Gmail search box. For example, "from:[email protected]
rfc822msgid:<somemsgid@example.com> is:unread". Parameter cannot be used when
rfc822msgid:<somemsgid@example. Com> is:unread". Parameter cannot be used when
accessing the api using the gmail. Metadata scope. Https://developers. Google.
Com/gmail/api/reference/rest/v1/users. Messages/list#query-parameters
kind: string
Expand Down
87 changes: 87 additions & 0 deletions _data/meltano/extractors/tap-google-analytics/connorflyn.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
capabilities:
- about
- catalog
- discover
- schema-flattening
- state
- stream-maps
description: App and website analytics platform hosted by Google (GA4)
domain_url: https://developers.google.com/analytics/devguides/reporting/data/v1
executable: tap-google-analytics
hidden: true
keywords:
- meltano_sdk
label: Google Analytics (GA4)
logo_url: /assets/logos/extractors/ga4.png
maintenance_status: active
name: tap-ga4
namespace: tap_ga4
next_steps: ''
pip_url: git+https://github.com/connorflyn/tap-google-analytics-v4.git
quality: bronze
repo: https://github.com/connorflyn/tap-google-analytics-v4
settings:
- description: Google Analytics Client Secrets Dictionary
kind: object
label: Client Secrets
name: client_secrets
- description: The last record date to sync
kind: date_iso8601
label: End Date
name: end_date
- description: "'True' to enable schema flattening and automatically expand nested
properties."
kind: boolean
label: Flattening Enabled
name: flattening_enabled
- description: The max depth to flatten schemas.
kind: integer
label: Flattening Max Depth
name: flattening_max_depth
- description: File Path to Google Analytics Client Secrets
kind: password
label: Key File Location
name: key_file_location
- description: Google Analytics Access Token
kind: password
label: OAuth Credentials Access Token
name: oauth_credentials.access_token
- description: Google Analytics Client ID
kind: password
label: OAuth Credentials Client ID
name: oauth_credentials.client_id
- description: Google Analytics Client Secret
kind: password
label: OAuth Credentials Client Secret
name: oauth_credentials.client_secret
- description: Google Analytics Refresh Token
kind: password
label: OAuth Credentials Refresh Token
name: oauth_credentials.refresh_token
- description: Google Analytics Property ID
kind: password
label: Property ID
name: property_id
- description: Google Analytics Reports Definition
kind: string
label: Reports
name: reports
- description: The earliest record date to sync
kind: date_iso8601
label: Start Date
name: start_date
- description: User-defined config values to be used within map expressions.
kind: object
label: Stream Map Config
name: stream_map_config
- description: Config object for stream maps capability. For more information check
out [Stream Maps](https://sdk.meltano.com/en/latest/stream_maps.html).
kind: object
label: Stream Maps
name: stream_maps
settings_group_validation:
- - property_id
- start_date
settings_preamble: ''
usage: ''
variant: connorflyn
165 changes: 45 additions & 120 deletions _data/meltano/extractors/tap-google-analytics/meltanolabs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,27 +5,27 @@ capabilities:
- schema-flattening
- state
- stream-maps
description: App and website analytics platform hosted by Google
domain_url: https://developers.google.com/analytics/devguides/reporting/core/v4/
description: App and website analytics platform hosted by Google (GA4)
domain_url: https://developers.google.com/analytics/devguides/reporting/data/v1
executable: tap-google-analytics
keywords:
- api
- meltano_sdk
label: Google Analytics (Universal Analytics API - Deprecated)
logo_url: /assets/logos/extractors/google-analytics.png
label: Google Analytics (GA4)
logo_url: /assets/logos/extractors/ga4.png
maintenance_status: active
name: tap-google-analytics
namespace: tap_google_analytics
pip_url: git+https://github.com/MeltanoLabs/tap-google-analytics.git@main_deprecated_ua
name: tap-ga4
namespace: tap_ga4
next_steps: ''
pip_url: git+https://github.com/MeltanoLabs/tap-google-analytics.git
quality: gold
repo: https://github.com/MeltanoLabs/tap-google-analytics
settings:
- description: |
Follow the above steps for [Key File Location](#key-file-location) but instead of providing a path you can provide the serialized json directly. This can be useful for ephemeral runtime environments where its easier to provide an environment variable instead of a file.
- description: Google Analytics Client Secrets Dictionary
kind: object
label: Client Secrets
name: client_secrets
placeholder: Ex. client_secrets.json
- description: Date up to when historical data will be extracted.
- description: The last record date to sync
kind: date_iso8601
label: End Date
name: end_date
Expand All @@ -38,94 +38,37 @@ settings:
kind: integer
label: Flattening Max Depth
name: flattening_max_depth
- description: |
#### How to get
Follow the steps below if you don't already have a valid `client_secrets.json` to upload. The process below can take over 10 minutes, but it's a one-time setup that's well worth it.
This extractor supports service account based authorization, where an administrator manually creates a service account with the appropriate permissions to view the account, property, and view you wish to fetch data from.
To access your Google Analytics data, the "Analytics Reporting API" and "Analytics API" both need to be enabled. These need to be enabled for a project inside the same organization as your Google Analytics account.
##### Step 1: Creating Service Account Credentials
As a first step, you need to create a new project in Google Cloud Platform or use an existing one:
1. Sign in to the Google Account you are using for managing Google Analytics (you must have Manage Users permission at the account, property, or view level).
2. Open the [Service accounts page](https://console.developers.google.com/iam-admin/serviceaccounts). If prompted, select a project or create a new one to use for accessing Google Analytics.
![Screenshot of Google Service Accounts page](/assets/images/tap-google-analytics/02-ga-service-account-configuration-create-new-account.png)
3. Click "Create service account"
In the Create service account window, type a name for the service account, and click `Create`.
We do not need to provide any additional permissions for this account, so click `Continue` in the `Service account permissions` configuration page.
We also do not need to grant access to any users for this service account, as we only need the key.
![Screenshot of Google Service Account Configuration for new Account](/assets/images/tap-google-analytics/02-ga-service-account-configuration-new-account.png)
Click `Create Key`, select `JSON` as the key type and create a new private key. Then click `Save` and store it locally as `client_secrets.json`.
Meltano will use the private key in this `client_secrets.json` file to connect with the Google Analytics API.
##### Step 2: Linking Credentials to Google Analytics
The newly created service account will have an email address that looks similar to:
```
[email protected]
```
To grant this service account access to your Google Analytics data, add the email address as a [new user](https://support.google.com/analytics/answer/1009702) to your Google Analytics account, property or view through the "Admin > User Management" page.
Only the [Read & Analyze permissions](https://support.google.com/analytics/answer/2884495) are needed as Meltano only extracts data to generate reports.
![Screenshot of Google Analytics Add User](/assets/images/tap-google-analytics/03-ga-add-user.png)
##### Step 3: Enabling the APIs
1. Visit the [Google Analytics Reporting API](https://console.developers.google.com/apis/api/analyticsreporting.googleapis.com/overview) dashboard and make sure that the project you used in the previous step is selected.
Now enable the API using the button at the top, so that the button will say "Disable API" instead:
![Screenshot of Google Analytics Reporting API](/assets/images/tap-google-analytics/04-ga-reporting-api.png)
2. Next, visit the [Google Analytics API](https://console.developers.google.com/apis/api/analytics.googleapis.com/overview) dashboard, make sure that the project you used in the previous step is selected, and enable this API as well.
![Screenshot of Google Analytics API](/assets/images/tap-google-analytics/05-ga-api.png)
- description: File Path to Google Analytics Client Secrets
kind: password
label: Key File Location
name: key_file_location
placeholder: Ex. $MELTANO_PROJECT_ROOT/client_secrets.json
- description: Google Analytics Access Token. See
https://developers.google.com/analytics/devguides/reporting/core/v4/authorization#OAuth2Authorizing.
- description: Google Analytics Access Token
kind: password
label: OAuth Credentials Access Token
name: oauth_credentials.access_token
- description: |
See <https://developers.google.com/analytics/devguides/reporting/core/v4/authorization#OAuth2Authorizing>.
Takes precedence over [Key File Location](#key-file-location) if both are specified.
- description: Google Analytics Client ID
kind: password
label: OAuth Credentials Client ID
name: oauth_credentials.client_id
- description: Google Analytics Client Secret. See
https://developers.google.com/analytics/devguides/reporting/core/v4/authorization#OAuth2Authorizing.
- description: Google Analytics Client Secret
kind: password
label: OAuth Credentials Client Secret
name: oauth_credentials.client_secret
- description: Google Analytics Refresh Token. See
https://developers.google.com/analytics/devguides/reporting/core/v4/authorization#OAuth2Authorizing.
- description: Google Analytics Refresh Token
kind: password
label: OAuth Credentials Refresh Token
name: oauth_credentials.refresh_token
- description: Google Analytics Property ID
kind: password
label: Property ID
name: property_id
- description: |
Project-relative path to JSON file with the definition of the reports to be generated.
Google Analytics Reports Definition.
The tap uses the [default reports definition](https://github.com/MeltanoLabs/tap-google-analytics/blob/main/tap_google_analytics/defaults/default_report_definition.json)
if this field is not provided.
A project-relative path to JSON file with the definition of the reports to be generated.
See <https://ga-dev-tools.appspot.com/dimensions-metrics-explorer/> for valid dimensions and metrics.
See <https://ga-dev-tools.google/ga4/dimensions-metrics-explorer/> for valid dimensions and metrics.
The JSON structure expected is as follows:
Expand Down Expand Up @@ -156,25 +99,25 @@ settings:
{ "name" : "users_per_day",
"dimensions" :
[
"ga:date"
"date"
],
"metrics" :
[
"ga:users",
"ga:newUsers"
"newUsers",
"active1DayUsers"
]
},
{ "name" : "sessions_per_country_day",
"dimensions" :
[
"ga:date",
"ga:country"
"date",
"country"
],
"metrics" :
[
"ga:sessions",
"ga:sessionsPerUser",
"ga:avgSessionDuration"
"sessions",
"sessionsPerUser",
"avgSessionDuration"
]
}
]
Expand All @@ -183,9 +126,11 @@ settings:
label: Reports
name: reports
placeholder: Ex. my_report_definition.json
- description: This property determines how much historical data will be extracted.
Please be aware that the larger the time period and amount of data, the longer
the initial extraction can be expected to take.
- description: List of Google Analytics Reports Definitions
kind: array
label: Reports List
name: reports_list
- description: The earliest record date to sync
kind: date_iso8601
label: Start Date
name: start_date
Expand All @@ -198,39 +143,19 @@ settings:
kind: object
label: Stream Maps
name: stream_maps
- description: |
The ID for the view to fetch data from.
#### How to get
To get your View ID:
1. Visit Google Analytics: <https://analytics.google.com/>
2. Log in if you haven't already.
3. Open the account/property/view selector in the top left corner
![Screenshot of closed account selector](/assets/images/tap-google-analytics/account-selector-closed.png)
3. Select the account, property, and view that you would like to connect with Meltano
![Screenshot of open account selector](/assets/images/tap-google-analytics/account-selector-open.png)
4. You will see the View ID displayed inside the selector below the name of the view (e.g. "All Web Site Data"): `188274549`
kind: password
label: View ID
name: view_id
placeholder: Ex. 198343027
settings_group_validation:
- - key_file_location
- start_date
- view_id
- - client_secrets
- property_id
- start_date
- - key_file_location
- property_id
- start_date
- view_id
- - oauth_credentials.access_token
- oauth_credentials.client_id
- oauth_credentials.client_secret
- oauth_credentials.refresh_token
- property_id
- start_date
- view_id
settings_preamble: ''
usage: ''
variant: meltanolabs
Loading

0 comments on commit e96d230

Please sign in to comment.