Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: [Hub Bot] Refresh metadata 2024-12-09 #1907

Merged
merged 3 commits into from
Dec 9, 2024

Conversation

MeltyBot
Copy link
Contributor

@MeltyBot MeltyBot commented Dec 9, 2024

Updates Plugin Definitions

Copy link

netlify bot commented Dec 9, 2024

Deploy Preview for meltano-hub ready!

Name Link
🔨 Latest commit 569c0e5
🔍 Latest deploy log https://app.netlify.com/sites/meltano-hub/deploys/675717d64e672e00085569d5
😎 Deploy Preview https://deploy-preview-1907--meltano-hub.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

Copy link

github-actions bot commented Dec 9, 2024

Testing plugin tap-athena (meltanolabs variant):

Starting test job...
...
Job completed.

Auto-generated README.md

tap-athena

Athena tap class.

Built with the Meltano Singer SDK.

Capabilities

  • catalog
  • state
  • discover
  • about
  • stream-maps
  • schema-flattening
  • batch

Supported Python Versions

  • 3.9
  • 3.10
  • 3.11
  • 3.12
  • 3.13

Settings

Setting Required Default Description
aws_access_key_id True None AWS access key ID
aws_secret_access_key True None AWS secret access key
aws_region True None The AWS region
s3_staging_dir True None The S3 staging directory where output is written.
schema_name True None Athena schema name
stream_maps False None Config object for stream maps capability. For more information check out Stream Maps.
stream_map_config False None User-defined config values to be used within map expressions.
faker_config False None Config for the Faker instance variable fake used within map expressions. Only applicable if the plugin specifies faker as an addtional dependency (through the singer-sdk faker extra or directly).
faker_config.seed False None Value to seed the Faker generator for deterministic output: https://faker.readthedocs.io/en/master/#seeding-the-generator
faker_config.locale False None One or more LCID locale strings to produce localized output for: https://faker.readthedocs.io/en/master/#localization
flattening_enabled False None 'True' to enable schema flattening and automatically expand nested properties.
flattening_max_depth False None The max depth to flatten schemas.
batch_config False None Configuration for BATCH message capabilities.
batch_config.encoding False None Specifies the format and compression of the batch files.
batch_config.encoding.format False None Format to use for batch files.
batch_config.encoding.compression False None Compression format to use for batch files.
batch_config.storage False None Defines the storage layer to use when writing batch files
batch_config.storage.root False None Root path to use when writing batch files.
batch_config.storage.prefix False None Prefix to use when writing batch files.

A full list of supported settings and capabilities is available by running: tap-athena --about

Version info
tap-athena v1.1.0, Meltano SDK v0.42.1
Usage info
melty-bot % tap-athena --help
Usage: tap-athena [OPTIONS]

  Execute the Singer tap.

Options:
  --version                 Display the package version.
  --about                   Display package metadata and settings.
  --format [json|markdown]  Specify output style for --about
  --config TEXT             Configuration file location or 'ENV' to use
                            environment variables.
  --discover                Run the tap in discovery mode.
  --test TEXT               Use --test to sync a single record for each
                            stream. Use --test=schema to test schema output
                            without syncing records.
  --catalog PATH            Use a Singer catalog file with the tap.
  --state PATH              Use a bookmarks file for incremental replication.
  --help                    Show this message and exit.
Detected capabilities
  • ✅ 'discover'
  • ✅ 'catalog'
  • ✅ 'state'
  • ✅ 'about'
JSON Metadata
{
  "name": "tap-athena",
  "description": "Athena tap class.",
  "version": "1.1.0",
  "sdk_version": "0.42.1",
  "supported_python_versions": [
    "3.9",
    "3.10",
    "3.11",
    "3.12",
    "3.13"
  ],
  "capabilities": [
    "catalog",
    "state",
    "discover",
    "about",
    "stream-maps",
    "schema-flattening",
    "batch"
  ],
  "settings": {
    "type": "object",
    "properties": {
      "aws_access_key_id": {
        "type": [
          "string"
        ],
        "title": "AWS Access Key ID",
        "description": "AWS access key ID",
        "secret": true,
        "writeOnly": true
      },
      "aws_secret_access_key": {
        "type": [
          "string"
        ],
        "title": "AWS Secret Access Key",
        "description": "AWS secret access key",
        "secret": true,
        "writeOnly": true
      },
      "aws_region": {
        "type": [
          "string"
        ],
        "title": "AWS region",
        "description": "The AWS region"
      },
      "s3_staging_dir": {
        "type": [
          "string"
        ],
        "title": "S3 staging directory",
        "description": "The S3 staging directory where output is written."
      },
      "schema_name": {
        "type": [
          "string"
        ],
        "title": "Schema Name",
        "description": "Athena schema name"
      },
      "stream_maps": {
        "type": [
          "object",
          "null"
        ],
        "properties": {},
        "title": "Stream Maps",
        "description": "Config object for stream maps capability. For more information check out [Stream Maps](https://sdk.meltano.com/en/latest/stream_maps.html)."
      },
      "stream_map_config": {
        "type": [
          "object",
          "null"
        ],
        "properties": {},
        "title": "User Stream Map Configuration",
        "description": "User-defined config values to be used within map expressions."
      },
      "faker_config": {
        "type": [
          "object",
          "null"
        ],
        "properties": {
          "seed": {
            "oneOf": [
              {
                "type": [
                  "number"
                ]
              },
              {
                "type": [
                  "string"
                ]
              },
              {
                "type": [
                  "boolean"
                ]
              },
              {
                "type": "null"
              }
            ],
            "title": "Faker Seed",
            "description": "Value to seed the Faker generator for deterministic output: https://faker.readthedocs.io/en/master/#seeding-the-generator"
          },
          "locale": {
            "oneOf": [
              {
                "type": [
                  "string"
                ]
              },
              {
                "type": "array",
                "items": {
                  "type": [
                    "string"
                  ]
                }
              },
              {
                "type": "null"
              }
            ],
            "title": "Faker Locale",
            "description": "One or more LCID locale strings to produce localized output for: https://faker.readthedocs.io/en/master/#localization"
          }
        },
        "title": "Faker Configuration",
        "description": "Config for the [`Faker`](https://faker.readthedocs.io/en/master/) instance variable `fake` used within map expressions. Only applicable if the plugin specifies `faker` as an addtional dependency (through the `singer-sdk` `faker` extra or directly)."
      },
      "flattening_enabled": {
        "type": [
          "boolean",
          "null"
        ],
        "title": "Enable Schema Flattening",
        "description": "'True' to enable schema flattening and automatically expand nested properties."
      },
      "flattening_max_depth": {
        "type": [
          "integer",
          "null"
        ],
        "title": "Max Flattening Depth",
        "description": "The max depth to flatten schemas."
      },
      "batch_config": {
        "type": [
          "object",
          "null"
        ],
        "properties": {
          "encoding": {
            "type": [
              "object",
              "null"
            ],
            "properties": {
              "format": {
                "type": [
                  "string",
                  "null"
                ],
                "title": "Batch Encoding Format",
                "description": "Format to use for batch files.",
                "enum": [
                  "jsonl",
                  "parquet"
                ]
              },
              "compression": {
                "type": [
                  "string",
                  "null"
                ],
                "title": "Batch Compression Format",
                "description": "Compression format to use for batch files.",
                "enum": [
                  "gzip",
                  "none"
                ]
              }
            },
            "title": "Batch Encoding Configuration",
            "description": "Specifies the format and compression of the batch files."
          },
          "storage": {
            "type": [
              "object",
              "null"
            ],
            "properties": {
              "root": {
                "type": [
                  "string",
                  "null"
                ],
                "title": "Batch Storage Root",
                "description": "Root path to use when writing batch files."
              },
              "prefix": {
                "type": [
                  "string",
                  "null"
                ],
                "title": "Batch Storage Prefix",
                "description": "Prefix to use when writing batch files."
              }
            },
            "title": "Batch Storage Configuration",
            "description": "Defines the storage layer to use when writing batch files"
          }
        },
        "title": "Batch Configuration",
        "description": "Configuration for BATCH message capabilities."
      }
    },
    "required": [
      "aws_access_key_id",
      "aws_secret_access_key",
      "aws_region",
      "s3_staging_dir",
      "schema_name"
    ]
  }
}

Copy link

github-actions bot commented Dec 9, 2024

Testing plugin tap-db2 (danielptv variant):

Starting test job...
...
Job completed.

Auto-generated README.md

tap-db2

Tap-DB2 is a Singer tap for IBM DB2 data sources.

Built with the Meltano Singer SDK.

Capabilities

  • about
  • stream-maps
  • catalog
  • state
  • discover

Supported Python Versions

  • 3.9
  • 3.10
  • 3.11
  • 3.12
  • 3.13

Settings

Setting Required Default Description
host True localhost The DB2 hostname.
port True 50000 The DB2 port.
database True None The DB2 database.
schema False None The DB2 schema.
user True None The DB2 username.
password True None The DB2 password.
encryption False None Encryption settings for the DB2 connection. Setting this to an empty object will append 'SECURITY=SSL' to the connection string. For more information check out python-ibmdb.
encryption.ssl_server_certificate False None The path to the SSL server certificate.
encryption.ssl_client_key_store_db False None
encryption.ssl_client_key_store_db.database True None The full path to the client keystore database.
encryption.ssl_client_key_store_db.password False None The keystore password.
encryption.ssl_client_key_store_db.key_stash False None The full path to the client key stash.
connection_parameters False None Additional parameters to be appended to the connection string. This is an objects containing key-value pairs.
sqlalchemy_execution_options False None Additional execution options to be passed to SQLAlchemy. This is an objects containing key-value pairs.
query_partition False None Partition query into smaller subsets. Useful when working with DB2 that has set strict resource limits per query. Only works for streams with numeric primary keys.
filter False None Apply a custom WHERE condition per stream. Unlike the filter available in stream_maps, this will be evaluated BEFORE extracting the data.
ignore_supplied_tables False 1 Ignore DB2-supplied user tables. For more info check out Db2-supplied user tables.
ignore_views False 0 Ignore views.
stream_maps False None Config object for stream maps capability. For more information check out Stream Maps.
stream_map_config False None User-defined config values to be used within map expressions.
faker_config False None Config for the Faker instance variable fake used within map expressions. Only applicable if the plugin specifies faker as an addtional dependency (through the singer-sdk faker extra or directly).
faker_config.seed False None Value to seed the Faker generator for deterministic output: https://faker.readthedocs.io/en/master/#seeding-the-generator
faker_config.locale False None One or more LCID locale strings to produce localized output for: https://faker.readthedocs.io/en/master/#localization
flattening_enabled False None 'True' to enable schema flattening and automatically expand nested properties.
flattening_max_depth False None The max depth to flatten schemas.

A full list of supported settings and capabilities is available by running: tap-db2 --about

Version info
tap-db2 v0.11.0.post1+9013b82, Meltano SDK v0.42.1
Usage info
melty-bot % tap-db2 --help
Usage: tap-db2 [OPTIONS]

  Execute the Singer tap.

Options:
  --version                 Display the package version.
  --about                   Display package metadata and settings.
  --format [json|markdown]  Specify output style for --about
  --config TEXT             Configuration file location or 'ENV' to use
                            environment variables.
  --discover                Run the tap in discovery mode.
  --test TEXT               Use --test to sync a single record for each
                            stream. Use --test=schema to test schema output
                            without syncing records.
  --catalog PATH            Use a Singer catalog file with the tap.
  --state PATH              Use a bookmarks file for incremental replication.
  --help                    Show this message and exit.
Detected capabilities
  • ✅ 'discover'
  • ✅ 'catalog'
  • ✅ 'state'
  • ✅ 'about'
JSON Metadata
{
  "name": "tap-db2",
  "description": "`Tap-DB2` is a Singer tap for IBM DB2 data sources.",
  "version": "0.11.0.post1+9013b82",
  "sdk_version": "0.42.1",
  "supported_python_versions": [
    "3.9",
    "3.10",
    "3.11",
    "3.12",
    "3.13"
  ],
  "capabilities": [
    "about",
    "stream-maps",
    "catalog",
    "state",
    "discover"
  ],
  "settings": {
    "type": "object",
    "properties": {
      "host": {
        "type": [
          "string"
        ],
        "format": "hostname",
        "default": "localhost",
        "description": "The DB2 hostname."
      },
      "port": {
        "type": [
          "integer"
        ],
        "default": 50000,
        "description": "The DB2 port."
      },
      "database": {
        "type": [
          "string"
        ],
        "description": "The DB2 database."
      },
      "schema": {
        "type": [
          "string",
          "null"
        ],
        "description": "The DB2 schema."
      },
      "user": {
        "type": [
          "string"
        ],
        "description": "The DB2 username."
      },
      "password": {
        "type": [
          "string"
        ],
        "description": "The DB2 password.",
        "secret": true,
        "writeOnly": true
      },
      "encryption": {
        "type": [
          "object",
          "null"
        ],
        "properties": {
          "ssl_server_certificate": {
            "type": [
              "string",
              "null"
            ],
            "description": "The path to the SSL server certificate."
          },
          "ssl_client_key_store_db": {
            "type": [
              "object",
              "null"
            ],
            "properties": {
              "database": {
                "type": [
                  "string"
                ],
                "description": "The full path to the client keystore database."
              },
              "password": {
                "type": [
                  "string",
                  "null"
                ],
                "description": "The keystore password.",
                "secret": true,
                "writeOnly": true
              },
              "key_stash": {
                "type": [
                  "string",
                  "null"
                ],
                "description": "The full path to the client key stash."
              }
            },
            "required": [
              "database"
            ]
          }
        },
        "description": "Encryption settings for the DB2 connection. Setting this to an empty object will append 'SECURITY=SSL' to the connection string. For more information check out [python-ibmdb](https://github.com/ibmdb/python-ibmdb#example-of-ssl-connection-string)."
      },
      "connection_parameters": {
        "type": [
          "object",
          "null"
        ],
        "properties": {},
        "description": "Additional parameters to be appended to the connection string. This is an objects containing key-value pairs."
      },
      "sqlalchemy_execution_options": {
        "type": [
          "object",
          "null"
        ],
        "properties": {},
        "description": "Additional execution options to be passed to SQLAlchemy. This is an objects containing key-value pairs."
      },
      "query_partition": {
        "type": [
          "object",
          "null"
        ],
        "properties": {},
        "additionalProperties": {
          "type": [
            "object",
            "null"
          ],
          "properties": {
            "partition_key": {
              "type": [
                "string"
              ]
            },
            "partition_size": {
              "type": [
                "integer"
              ]
            }
          }
        },
        "description": "Partition query into smaller subsets. Useful when working with DB2 that has set strict resource limits per query. Only works for streams with numeric primary keys."
      },
      "filter": {
        "type": [
          "object",
          "null"
        ],
        "properties": {},
        "additionalProperties": {
          "type": [
            "object",
            "null"
          ],
          "properties": {
            "where": {
              "type": [
                "string"
              ]
            }
          }
        },
        "description": "Apply a custom WHERE condition per stream. Unlike the filter available in stream_maps, this will be evaluated BEFORE extracting the data."
      },
      "ignore_supplied_tables": {
        "type": [
          "boolean",
          "null"
        ],
        "default": true,
        "description": "Ignore DB2-supplied user tables. For more info check out [Db2-supplied user tables](https://www.ibm.com/docs/en/db2-for-zos/12?topic=db2-supplied-user-tables)."
      },
      "ignore_views": {
        "type": [
          "boolean",
          "null"
        ],
        "default": false,
        "description": "Ignore views."
      },
      "stream_maps": {
        "type": [
          "object",
          "null"
        ],
        "properties": {},
        "additionalProperties": {
          "type": [
            "object",
            "string",
            "null"
          ],
          "properties": {
            "__filter__": {
              "type": [
                "string",
                "null"
              ]
            },
            "__source__": {
              "type": [
                "string",
                "null"
              ]
            },
            "__alias__": {
              "type": [
                "string",
                "null"
              ]
            },
            "__else__": {
              "type": [
                "string",
                "null"
              ],
              "enum": [
                null,
                "__NULL__"
              ]
            },
            "__key_properties__": {
              "type": [
                "array",
                "null"
              ],
              "items": {
                "type": "string"
              }
            }
          },
          "additionalProperties": {
            "type": [
              "string",
              "null"
            ]
          }
        },
        "description": "Config object for stream maps capability. For more information check out [Stream Maps](https://sdk.meltano.com/en/latest/stream_maps.html)."
      },
      "stream_map_config": {
        "type": [
          "object",
          "null"
        ],
        "properties": {},
        "description": "User-defined config values to be used within map expressions."
      },
      "faker_config": {
        "type": [
          "object",
          "null"
        ],
        "properties": {
          "seed": {
            "oneOf": [
              {
                "type": [
                  "number"
                ]
              },
              {
                "type": [
                  "string"
                ]
              },
              {
                "type": [
                  "boolean"
                ]
              },
              {
                "type": "null"
              }
            ],
            "title": "Faker Seed",
            "description": "Value to seed the Faker generator for deterministic output: https://faker.readthedocs.io/en/master/#seeding-the-generator"
          },
          "locale": {
            "oneOf": [
              {
                "type": [
                  "string"
                ]
              },
              {
                "type": "array",
                "items": {
                  "type": [
                    "string"
                  ]
                }
              },
              {
                "type": "null"
              }
            ],
            "title": "Faker Locale",
            "description": "One or more LCID locale strings to produce localized output for: https://faker.readthedocs.io/en/master/#localization"
          }
        },
        "title": "Faker Configuration",
        "description": "Config for the [`Faker`](https://faker.readthedocs.io/en/master/) instance variable `fake` used within map expressions. Only applicable if the plugin specifies `faker` as an addtional dependency (through the `singer-sdk` `faker` extra or directly)."
      },
      "flattening_enabled": {
        "type": [
          "boolean",
          "null"
        ],
        "title": "Enable Schema Flattening",
        "description": "'True' to enable schema flattening and automatically expand nested properties."
      },
      "flattening_max_depth": {
        "type": [
          "integer",
          "null"
        ],
        "title": "Max Flattening Depth",
        "description": "The max depth to flatten schemas."
      }
    },
    "required": [
      "host",
      "port",
      "database",
      "user",
      "password"
    ]
  }
}

Copy link

github-actions bot commented Dec 9, 2024

Testing plugin tap-clickhouse (akurdyukov variant):

Starting test job...
...
Job completed.

Auto-generated README.md

tap-clickhouse

ClickHouse tap class.

Built with the Meltano Singer SDK.

Capabilities

  • catalog
  • state
  • discover
  • about
  • stream-maps
  • schema-flattening
  • batch

Settings

Setting Required Default Description
driver False http Driver type
username False default Database user
password True None Username password
host False localhost Database host
port False 8123 Database connection port
database False default Database name
secure False 0 Should the connection be secure
verify False 1 Should secure connection need to verify SSL/TLS
stream_maps False None Config object for stream maps capability. For more information check out Stream Maps.
stream_map_config False None User-defined config values to be used within map expressions.
faker_config False None Config for the Faker instance variable fake used within map expressions. Only applicable if the plugin specifies faker as an addtional dependency (through the singer-sdk faker extra or directly).
faker_config.seed False None Value to seed the Faker generator for deterministic output: https://faker.readthedocs.io/en/master/#seeding-the-generator
faker_config.locale False None One or more LCID locale strings to produce localized output for: https://faker.readthedocs.io/en/master/#localization
flattening_enabled False None 'True' to enable schema flattening and automatically expand nested properties.
flattening_max_depth False None The max depth to flatten schemas.
batch_config False None Configuration for BATCH message capabilities.
batch_config.encoding False None Specifies the format and compression of the batch files.
batch_config.encoding.format False None Format to use for batch files.
batch_config.encoding.compression False None Compression format to use for batch files.
batch_config.storage False None Defines the storage layer to use when writing batch files
batch_config.storage.root False None Root path to use when writing batch files.
batch_config.storage.prefix False None Prefix to use when writing batch files.

A full list of supported settings and capabilities is available by running: tap-clickhouse --about

Version info
tap-clickhouse v[could not be detected], Meltano SDK v0.42.1
Usage info
melty-bot % tap-clickhouse --help
Usage: tap-clickhouse [OPTIONS]

  Execute the Singer tap.

Options:
  --version                 Display the package version.
  --about                   Display package metadata and settings.
  --format [json|markdown]  Specify output style for --about
  --config TEXT             Configuration file location or 'ENV' to use
                            environment variables.
  --discover                Run the tap in discovery mode.
  --test TEXT               Use --test to sync a single record for each
                            stream. Use --test=schema to test schema output
                            without syncing records.
  --catalog PATH            Use a Singer catalog file with the tap.
  --state PATH              Use a bookmarks file for incremental replication.
  --help                    Show this message and exit.
Detected capabilities
  • ✅ 'discover'
  • ✅ 'catalog'
  • ✅ 'state'
  • ✅ 'about'
JSON Metadata
{
  "name": "tap-clickhouse",
  "description": "ClickHouse tap class.",
  "version": "[could not be detected]",
  "sdk_version": "0.42.1",
  "supported_python_versions": null,
  "capabilities": [
    "catalog",
    "state",
    "discover",
    "about",
    "stream-maps",
    "schema-flattening",
    "batch"
  ],
  "settings": {
    "type": "object",
    "properties": {
      "driver": {
        "type": [
          "string",
          "null"
        ],
        "default": "http",
        "description": "Driver type",
        "enum": [
          "http",
          "native",
          "asynch"
        ]
      },
      "username": {
        "type": [
          "string",
          "null"
        ],
        "default": "default",
        "description": "Database user"
      },
      "password": {
        "type": [
          "string"
        ],
        "description": "Username password",
        "secret": true,
        "writeOnly": true
      },
      "host": {
        "type": [
          "string",
          "null"
        ],
        "default": "localhost",
        "description": "Database host"
      },
      "port": {
        "type": [
          "integer",
          "null"
        ],
        "default": 8123,
        "description": "Database connection port"
      },
      "database": {
        "type": [
          "string",
          "null"
        ],
        "default": "default",
        "description": "Database name"
      },
      "secure": {
        "type": [
          "boolean",
          "null"
        ],
        "default": false,
        "description": "Should the connection be secure"
      },
      "verify": {
        "type": [
          "boolean",
          "null"
        ],
        "default": true,
        "description": "Should secure connection need to verify SSL/TLS"
      },
      "stream_maps": {
        "type": [
          "object",
          "null"
        ],
        "properties": {},
        "title": "Stream Maps",
        "description": "Config object for stream maps capability. For more information check out [Stream Maps](https://sdk.meltano.com/en/latest/stream_maps.html)."
      },
      "stream_map_config": {
        "type": [
          "object",
          "null"
        ],
        "properties": {},
        "title": "User Stream Map Configuration",
        "description": "User-defined config values to be used within map expressions."
      },
      "faker_config": {
        "type": [
          "object",
          "null"
        ],
        "properties": {
          "seed": {
            "oneOf": [
              {
                "type": [
                  "number"
                ]
              },
              {
                "type": [
                  "string"
                ]
              },
              {
                "type": [
                  "boolean"
                ]
              },
              {
                "type": "null"
              }
            ],
            "title": "Faker Seed",
            "description": "Value to seed the Faker generator for deterministic output: https://faker.readthedocs.io/en/master/#seeding-the-generator"
          },
          "locale": {
            "oneOf": [
              {
                "type": [
                  "string"
                ]
              },
              {
                "type": "array",
                "items": {
                  "type": [
                    "string"
                  ]
                }
              },
              {
                "type": "null"
              }
            ],
            "title": "Faker Locale",
            "description": "One or more LCID locale strings to produce localized output for: https://faker.readthedocs.io/en/master/#localization"
          }
        },
        "title": "Faker Configuration",
        "description": "Config for the [`Faker`](https://faker.readthedocs.io/en/master/) instance variable `fake` used within map expressions. Only applicable if the plugin specifies `faker` as an addtional dependency (through the `singer-sdk` `faker` extra or directly)."
      },
      "flattening_enabled": {
        "type": [
          "boolean",
          "null"
        ],
        "title": "Enable Schema Flattening",
        "description": "'True' to enable schema flattening and automatically expand nested properties."
      },
      "flattening_max_depth": {
        "type": [
          "integer",
          "null"
        ],
        "title": "Max Flattening Depth",
        "description": "The max depth to flatten schemas."
      },
      "batch_config": {
        "type": [
          "object",
          "null"
        ],
        "properties": {
          "encoding": {
            "type": [
              "object",
              "null"
            ],
            "properties": {
              "format": {
                "type": [
                  "string",
                  "null"
                ],
                "title": "Batch Encoding Format",
                "description": "Format to use for batch files.",
                "enum": [
                  "jsonl",
                  "parquet"
                ]
              },
              "compression": {
                "type": [
                  "string",
                  "null"
                ],
                "title": "Batch Compression Format",
                "description": "Compression format to use for batch files.",
                "enum": [
                  "gzip",
                  "none"
                ]
              }
            },
            "title": "Batch Encoding Configuration",
            "description": "Specifies the format and compression of the batch files."
          },
          "storage": {
            "type": [
              "object",
              "null"
            ],
            "properties": {
              "root": {
                "type": [
                  "string",
                  "null"
                ],
                "title": "Batch Storage Root",
                "description": "Root path to use when writing batch files."
              },
              "prefix": {
                "type": [
                  "string",
                  "null"
                ],
                "title": "Batch Storage Prefix",
                "description": "Prefix to use when writing batch files."
              }
            },
            "title": "Batch Storage Configuration",
            "description": "Defines the storage layer to use when writing batch files"
          }
        },
        "title": "Batch Configuration",
        "description": "Configuration for BATCH message capabilities."
      }
    },
    "required": [
      "password"
    ]
  }
}

Copy link

github-actions bot commented Dec 9, 2024

Testing plugin tap-mailchimp (acarter24 variant):

Starting test job...
...
Job completed.

Auto-generated README.md

tap-mailchimp

Mailchimp tap class.

Built with the Meltano Singer SDK.

Capabilities

  • catalog
  • state
  • discover
  • about
  • stream-maps
  • schema-flattening

Settings

Setting Required Default Description
api_key True None The token to authenticate against the API service
start_date False None The earliest record date to sync
dc True None Your Mailchimp DC
stream_maps False None Config object for stream maps capability. For more information check out Stream Maps.
stream_map_config False None User-defined config values to be used within map expressions.
flattening_enabled False None 'True' to enable schema flattening and automatically expand nested properties.
flattening_max_depth False None The max depth to flatten schemas.

A full list of supported settings and capabilities is available by running: tap-mailchimp --about

Version info
tap-mailchimp v0.0.1, Meltano SDK v0.24.0
Usage info
melty-bot % tap-mailchimp --help
Usage: tap-mailchimp [OPTIONS]

  Execute the Singer tap.

Options:
  --state PATH              Use a bookmarks file for incremental replication.
  --catalog PATH            Use a Singer catalog file with the tap.
  --test TEXT               Use --test to sync a single record for each
                            stream. Use --test=schema to test schema output
                            without syncing records.
  --discover                Run the tap in discovery mode.
  --config TEXT             Configuration file location or 'ENV' to use
                            environment variables.
  --format [json|markdown]  Specify output style for --about
  --about                   Display package metadata and settings.
  --version                 Display the package version.
  --help                    Show this message and exit.
Detected capabilities
  • ✅ 'discover'
  • ✅ 'catalog'
  • ✅ 'state'
  • ✅ 'about'
JSON Metadata
{
  "name": "tap-mailchimp",
  "description": "Mailchimp tap class.",
  "version": "0.0.1",
  "sdk_version": "0.24.0",
  "capabilities": [
    "catalog",
    "state",
    "discover",
    "about",
    "stream-maps",
    "schema-flattening"
  ],
  "settings": {
    "type": "object",
    "properties": {
      "api_key": {
        "type": [
          "string"
        ],
        "description": "The token to authenticate against the API service",
        "secret": true,
        "writeOnly": true
      },
      "start_date": {
        "type": [
          "string",
          "null"
        ],
        "format": "date-time",
        "description": "The earliest record date to sync"
      },
      "dc": {
        "type": [
          "string"
        ],
        "description": "Your Mailchimp DC"
      },
      "stream_maps": {
        "type": [
          "object",
          "null"
        ],
        "properties": {},
        "description": "Config object for stream maps capability. For more information check out [Stream Maps](https://sdk.meltano.com/en/latest/stream_maps.html)."
      },
      "stream_map_config": {
        "type": [
          "object",
          "null"
        ],
        "properties": {},
        "description": "User-defined config values to be used within map expressions."
      },
      "flattening_enabled": {
        "type": [
          "boolean",
          "null"
        ],
        "description": "'True' to enable schema flattening and automatically expand nested properties."
      },
      "flattening_max_depth": {
        "type": [
          "integer",
          "null"
        ],
        "description": "The max depth to flatten schemas."
      }
    },
    "required": [
      "api_key",
      "dc"
    ]
  }
}
Discovered streams
  • campaigns
  • lists
  • lists_members
  • reports_email_activity
  • reports_sent_to
  • reports_unsubscribes

Copy link

github-actions bot commented Dec 9, 2024

Testing plugin target-redshift (ticketswap variant):

Starting test job...
...
Job completed.

Auto-generated README.md

target-redshift

Target for Redshift.

Built with the Meltano Singer SDK.

Capabilities

  • about
  • stream-maps
  • schema-flattening
  • validate-records
  • target-schema
  • hard-delete

Supported Python Versions

  • 3.9
  • 3.10
  • 3.11
  • 3.12

Settings

Setting Required Default Description
host False None Hostname for redshift instance.
port False 5432 The port on which redshift is awaiting connection.
enable_iam_authentication False None If true, use temporary credentials (https://docs.aws.amazon.com/redshift/latest/mgmt/generating-iam-credentials-cli-api.html).
cluster_identifier False None Redshift cluster identifier. Note if sqlalchemy_url is set or enable_iam_authentication is false this will be ignored.
user False None User name used to authenticate. Note if sqlalchemy_url is set this will be ignored.
password False None Password used to authenticate. Note if sqlalchemy_url is set this will be ignored.
dbname False None Database name. Note if sqlalchemy_url is set this will be ignored.
aws_redshift_copy_role_arn True None Redshift copy role arn to use for the COPY command from s3
s3_bucket True None S3 bucket to save staging files before using COPY command
s3_region False None AWS region for S3 bucket. If not specified, region will be detected by boto config resolution. See https://boto3.amazonaws.com/v1/documentation/api/latest/guide/configuration.html
s3_key_prefix False S3 key prefix to save staging files before using COPY command
remove_s3_files False 0 If you want to remove staging files in S3
temp_dir False temp Where you want to store your temp data files.
default_target_schema False None Redshift schema to send data to, example: tap-clickup
activate_version False 0 If set to false, the tap will ignore activate version messages. If set to true, add_record_metadata must be set to true as well.
hard_delete False 0 When activate version is sent from a tap this specefies if we should delete the records that don't match, or mark them with a date in the _sdc_deleted_at column. This config option is ignored if activate_version is set to false.
add_record_metadata False 0 Note that this must be enabled for activate_version to work!This adds _sdc_extracted_at, _sdc_batched_at, and more to every table. See https://sdk.meltano.com/en/latest/implementation/record_metadata.html for more information.
ssl_enable False 0 Whether or not to use ssl to verify the server's identity. Use ssl_certificate_authority and ssl_mode for further customization. To use a client certificate to authenticate yourself to the server, use ssl_client_certificate_enable instead.
ssl_mode False verify-full SSL Protection method, see [redshift documentation](https://docs.aws.amazon.com/redshift/latest/mgmt/connecting-ssl-support.html for more information. Must be one of disable, allow, prefer, require, verify-ca, or verify-full.
load_method False append-only The method to use when loading data into the destination. append-only will always write all input records whether that records already exists or not. upsert will update existing records and insert new records. overwrite will delete all existing records and insert all input records.
batch_size_rows False None Maximum number of rows in each batch.
validate_records False 1 Whether to validate the schema of the incoming streams.
stream_maps False None Config object for stream maps capability. For more information check out Stream Maps.
stream_map_config False None User-defined config values to be used within map expressions.
faker_config False None Config for the Faker instance variable fake used within map expressions. Only applicable if the plugin specifies faker as an addtional dependency (through the singer-sdk faker extra or directly).
faker_config.seed False None Value to seed the Faker generator for deterministic output: https://faker.readthedocs.io/en/master/#seeding-the-generator
faker_config.locale False None One or more LCID locale strings to produce localized output for: https://faker.readthedocs.io/en/master/#localization
flattening_enabled False None 'True' to enable schema flattening and automatically expand nested properties.
flattening_max_depth False None The max depth to flatten schemas.

A full list of supported settings and capabilities is available by running: target-redshift --about

Version info
target-redshift v0.1.0, Meltano SDK v0.42.1
Usage info
melty-bot % target-redshift --help
Usage: target-redshift [OPTIONS]

  Execute the Singer target.

Options:
  --version                 Display the package version.
  --about                   Display package metadata and settings.
  --format [json|markdown]  Specify output style for --about
  --config TEXT             Configuration file location or 'ENV' to use
                            environment variables.
  --input FILENAME          A path to read messages from instead of from
                            standard in.
  --help                    Show this message and exit.
Detected capabilities
  • ❌ 'discover'
  • ❌ 'catalog'
  • ❌ 'properties'
  • ❌ 'state'
  • ✅ 'about'
JSON Metadata
{
  "name": "target-redshift",
  "description": "Target for Redshift.",
  "version": "0.1.0",
  "sdk_version": "0.42.1",
  "supported_python_versions": [
    "3.9",
    "3.10",
    "3.11",
    "3.12"
  ],
  "capabilities": [
    "about",
    "stream-maps",
    "schema-flattening",
    "validate-records",
    "target-schema",
    "hard-delete"
  ],
  "settings": {
    "type": "object",
    "properties": {
      "host": {
        "type": [
          "string",
          "null"
        ],
        "description": "Hostname for redshift instance."
      },
      "port": {
        "type": [
          "string",
          "null"
        ],
        "default": "5432",
        "description": "The port on which redshift is awaiting connection."
      },
      "enable_iam_authentication": {
        "type": [
          "boolean",
          "null"
        ],
        "title": "Enable IAM Authentication",
        "description": "If true, use temporary credentials (https://docs.aws.amazon.com/redshift/latest/mgmt/generating-iam-credentials-cli-api.html)."
      },
      "cluster_identifier": {
        "type": [
          "string",
          "null"
        ],
        "description": "Redshift cluster identifier. Note if sqlalchemy_url is set or enable_iam_authentication is false this will be ignored."
      },
      "user": {
        "type": [
          "string",
          "null"
        ],
        "description": "User name used to authenticate. Note if sqlalchemy_url is set this will be ignored."
      },
      "password": {
        "type": [
          "string",
          "null"
        ],
        "description": "Password used to authenticate. Note if sqlalchemy_url is set this will be ignored."
      },
      "dbname": {
        "type": [
          "string",
          "null"
        ],
        "title": "Database Name",
        "description": "Database name. Note if sqlalchemy_url is set this will be ignored."
      },
      "aws_redshift_copy_role_arn": {
        "type": [
          "string"
        ],
        "title": "AWS Redshift Copy Role ARN",
        "description": "Redshift copy role arn to use for the COPY command from s3",
        "secret": true,
        "writeOnly": true
      },
      "s3_bucket": {
        "type": [
          "string"
        ],
        "description": "S3 bucket to save staging files before using COPY command"
      },
      "s3_region": {
        "type": [
          "string",
          "null"
        ],
        "description": "AWS region for S3 bucket. If not specified, region will be detected by boto config resolution. See https://boto3.amazonaws.com/v1/documentation/api/latest/guide/configuration.html"
      },
      "s3_key_prefix": {
        "type": [
          "string",
          "null"
        ],
        "default": "",
        "description": "S3 key prefix to save staging files before using COPY command"
      },
      "remove_s3_files": {
        "type": [
          "boolean",
          "null"
        ],
        "default": false,
        "description": "If you want to remove staging files in S3"
      },
      "temp_dir": {
        "type": [
          "string",
          "null"
        ],
        "default": "temp",
        "description": "Where you want to store your temp data files."
      },
      "default_target_schema": {
        "type": [
          "string",
          "null"
        ],
        "description": "Redshift schema to send data to, example: tap-clickup"
      },
      "activate_version": {
        "type": [
          "boolean",
          "null"
        ],
        "default": false,
        "description": "If set to false, the tap will ignore activate version messages. If set to true, add_record_metadata must be set to true as well."
      },
      "hard_delete": {
        "type": [
          "boolean",
          "null"
        ],
        "default": false,
        "description": "When activate version is sent from a tap this specefies if we should delete the records that don't match, or mark them with a date in the `_sdc_deleted_at` column. This config option is ignored if `activate_version` is set to false."
      },
      "add_record_metadata": {
        "type": [
          "boolean",
          "null"
        ],
        "default": false,
        "description": "Note that this must be enabled for activate_version to work!This adds _sdc_extracted_at, _sdc_batched_at, and more to every table. See https://sdk.meltano.com/en/latest/implementation/record_metadata.html for more information."
      },
      "ssl_enable": {
        "type": [
          "boolean",
          "null"
        ],
        "default": false,
        "description": "Whether or not to use ssl to verify the server's identity. Use ssl_certificate_authority and ssl_mode for further customization. To use a client certificate to authenticate yourself to the server, use ssl_client_certificate_enable instead."
      },
      "ssl_mode": {
        "type": [
          "string",
          "null"
        ],
        "default": "verify-full",
        "description": "SSL Protection method, see [redshift documentation](https://docs.aws.amazon.com/redshift/latest/mgmt/connecting-ssl-support.html for more information. Must be one of disable, allow, prefer, require, verify-ca, or verify-full."
      },
      "load_method": {
        "type": [
          "string",
          "null"
        ],
        "default": "append-only",
        "description": "The method to use when loading data into the destination. `append-only` will always write all input records whether that records already exists or not. `upsert` will update existing records and insert new records. `overwrite` will delete all existing records and insert all input records.",
        "enum": [
          "append-only",
          "upsert",
          "overwrite"
        ]
      },
      "batch_size_rows": {
        "type": [
          "integer",
          "null"
        ],
        "title": "Batch Size Rows",
        "description": "Maximum number of rows in each batch."
      },
      "validate_records": {
        "type": [
          "boolean",
          "null"
        ],
        "title": "Validate Records",
        "default": true,
        "description": "Whether to validate the schema of the incoming streams."
      },
      "stream_maps": {
        "type": [
          "object",
          "null"
        ],
        "properties": {},
        "title": "Stream Maps",
        "description": "Config object for stream maps capability. For more information check out [Stream Maps](https://sdk.meltano.com/en/latest/stream_maps.html)."
      },
      "stream_map_config": {
        "type": [
          "object",
          "null"
        ],
        "properties": {},
        "title": "User Stream Map Configuration",
        "description": "User-defined config values to be used within map expressions."
      },
      "faker_config": {
        "type": [
          "object",
          "null"
        ],
        "properties": {
          "seed": {
            "oneOf": [
              {
                "type": [
                  "number"
                ]
              },
              {
                "type": [
                  "string"
                ]
              },
              {
                "type": [
                  "boolean"
                ]
              },
              {
                "type": "null"
              }
            ],
            "title": "Faker Seed",
            "description": "Value to seed the Faker generator for deterministic output: https://faker.readthedocs.io/en/master/#seeding-the-generator"
          },
          "locale": {
            "oneOf": [
              {
                "type": [
                  "string"
                ]
              },
              {
                "type": "array",
                "items": {
                  "type": [
                    "string"
                  ]
                }
              },
              {
                "type": "null"
              }
            ],
            "title": "Faker Locale",
            "description": "One or more LCID locale strings to produce localized output for: https://faker.readthedocs.io/en/master/#localization"
          }
        },
        "title": "Faker Configuration",
        "description": "Config for the [`Faker`](https://faker.readthedocs.io/en/master/) instance variable `fake` used within map expressions. Only applicable if the plugin specifies `faker` as an addtional dependency (through the `singer-sdk` `faker` extra or directly)."
      },
      "flattening_enabled": {
        "type": [
          "boolean",
          "null"
        ],
        "title": "Enable Schema Flattening",
        "description": "'True' to enable schema flattening and automatically expand nested properties."
      },
      "flattening_max_depth": {
        "type": [
          "integer",
          "null"
        ],
        "title": "Max Flattening Depth",
        "description": "The max depth to flatten schemas."
      }
    },
    "required": [
      "aws_redshift_copy_role_arn",
      "s3_bucket"
    ]
  }
}

Copy link

github-actions bot commented Dec 9, 2024

Testing plugin tap-totango (edsoncezar16 variant):

Starting test job...
...
Job completed.

Version info
tap-totango v0.5.0, Meltano SDK v0.27.0
Usage info
melty-bot % tap-totango --help
Usage: tap-totango [OPTIONS]

  Execute the Singer tap.

Options:
  --state PATH              Use a bookmarks file for incremental replication.
  --catalog PATH            Use a Singer catalog file with the tap.
  --test TEXT               Use --test to sync a single record for each
                            stream. Use --test=schema to test schema output
                            without syncing records.
  --discover                Run the tap in discovery mode.
  --config TEXT             Configuration file location or 'ENV' to use
                            environment variables.
  --format [json|markdown]  Specify output style for --about
  --about                   Display package metadata and settings.
  --version                 Display the package version.
  --help                    Show this message and exit.
Detected capabilities
  • ✅ 'discover'
  • ✅ 'catalog'
  • ✅ 'state'
  • ✅ 'about'
JSON Metadata
{
  "name": "tap-totango",
  "description": "totango tap class.",
  "version": "0.5.0",
  "sdk_version": "0.27.0",
  "capabilities": [
    "catalog",
    "state",
    "discover",
    "about",
    "stream-maps",
    "schema-flattening"
  ],
  "settings": {
    "type": "object",
    "properties": {
      "api_url": {
        "type": [
          "string"
        ],
        "default": "https://api.totango.com",
        "description": "The url for the API services. https://api.totango.com is for US services, whereas https://api-eu1.totango.com is for EU services.",
        "enum": [
          "https://api.totango.com",
          "https://api-eu1.totango.com "
        ]
      },
      "auth_token": {
        "type": [
          "string"
        ],
        "description": "The token to authenticate against the API service",
        "secret": true,
        "writeOnly": true
      },
      "events_terms": {
        "type": "array",
        "items": {
          "type": "object",
          "properties": {
            "type": {
              "type": [
                "string"
              ]
            }
          },
          "required": [
            "type"
          ],
          "additionalProperties": true
        },
        "default": [],
        "description": "An array containing filter conditions to use for the events stream search.",
        "examples": [
          [
            {
              "type": "event_property",
              "name": "event_type",
              "eq": "note"
            }
          ],
          [
            {
              "type": "or",
              "or": [
                {
                  "type": "event_property",
                  "name": "event_type",
                  "eq": "note"
                },
                {
                  "type": "event_property",
                  "name": "event_type",
                  "eq": "campaign_touch"
                }
              ]
            }
          ],
          [
            {
              "type": "date",
              "term": "date",
              "joker": "yesterday"
            },
            {
              "type": "or",
              "or": [
                {
                  "type": "event_property",
                  "name": "event_type",
                  "eq": "note"
                },
                {
                  "type": "event_property",
                  "name": "event_type",
                  "eq": "campaign_touch"
                }
              ]
            }
          ],
          [
            {
              "type": "date",
              "term": "date",
              "gte": 1587859200000
            },
            {
              "type": "event_property",
              "name": "event_type",
              "eq": "note"
            }
          ]
        ]
      },
      "events_count": {
        "type": [
          "integer"
        ],
        "default": 1000,
        "description": "The maximum number of accounts to return in the events result set. The max. value for count is 1000."
      },
      "events_offset": {
        "type": [
          "integer"
        ],
        "default": 0,
        "description": "Page number (0 is the 1st-page)."
      },
      "account_id": {
        "type": [
          "string",
          "null"
        ],
        "description": "Filter the events stream results for a specific account."
      },
      "accounts_terms": {
        "type": "array",
        "items": {
          "type": "object",
          "properties": {
            "type": {
              "type": [
                "string"
              ]
            }
          },
          "required": [
            "type"
          ],
          "additionalProperties": true
        },
        "default": [],
        "description": "An array containing filter conditions to use for the accounts stream search.",
        "examples": [
          [
            {
              "type": "string",
              "term": "status_group",
              "in_list": [
                "paying"
              ]
            }
          ]
        ]
      },
      "accounts_fields": {
        "type": "array",
        "items": {
          "type": "object",
          "properties": {
            "type": {
              "type": [
                "string"
              ]
            }
          },
          "required": [
            "type"
          ],
          "additionalProperties": true
        },
        "default": [],
        "description": "List of fields to return as results. Note that the account name and account-id are always returned as well.",
        "examples": [
          [
            {
              "type": "string",
              "term": "health",
              "field_display_name": "Health rank "
            },
            {
              "type": "health_trend",
              "field_display_name": "Health last change "
            },
            {
              "type": "string_attribute",
              "attribute": "Success Manager",
              "field_display_name": "Success Manager"
            }
          ]
        ]
      },
      "accounts_count": {
        "type": [
          "integer",
          "null"
        ],
        "default": 100,
        "description": "The maximum number of accounts to return in the accounts result set. The max. value for count is 1000."
      },
      "accounts_offset": {
        "type": [
          "integer",
          "null"
        ],
        "default": 0,
        "description": "Record number (0 states \"start at record 0\"). The record size can be defined using the count parameter (and limited to 1000). Tip: To page through results, ask for 1000 records (count: 1000). If you receive 1000 records, assume there\u2019s more, in which case you want to pull the next 1000 records (offset: 1000\u2026then 2000\u2026etc.). Repeat paging until the number of records returned is less than 1000."
      },
      "accounts_sort_by": {
        "type": [
          "string",
          "null"
        ],
        "default": "display_name",
        "description": "Field name to sort the accounts stream results set by."
      },
      "accounts_sort_order": {
        "type": [
          "string",
          "null"
        ],
        "enum": [
          "ASC",
          "DESC"
        ],
        "default": "ASC",
        "description": "Order to sort the accounts stream results set by."
      },
      "users_terms": {
        "type": "array",
        "items": {
          "type": "object",
          "properties": {
            "type": {
              "type": [
                "string"
              ]
            }
          },
          "required": [
            "type"
          ],
          "additionalProperties": true
        },
        "default": [],
        "description": "An array containing filter conditions to use for the users stream search.",
        "examples": [
          [
            {
              "type": "parent_account",
              "terms": [
                {
                  "type": "string",
                  "term": "status_group",
                  "in_list": [
                    "paying"
                  ]
                }
              ]
            }
          ]
        ]
      },
      "users_fields": {
        "type": "array",
        "items": {
          "type": "object",
          "properties": {
            "type": {
              "type": [
                "string"
              ]
            }
          },
          "required": [
            "type"
          ],
          "additionalProperties": true
        },
        "default": [],
        "description": "List of fields to return as results. Note that the user name and id along with account name and account-id are always returned as well.",
        "examples": [
          [
            {
              "type": "date",
              "term": "last_activity_time",
              "field_display_name": "Last activity",
              "desc": true
            },
            {
              "type": "named_aggregation",
              "aggregation": "total_activities",
              "duration": 14,
              "field_display_name": "Activities (14d)"
            }
          ]
        ]
      },
      "users_count": {
        "type": [
          "integer",
          "null"
        ],
        "default": 1000,
        "description": "The maximum number of users to return in the users result set. The max. value for count is 1000."
      },
      "users_offset": {
        "type": [
          "integer",
          "null"
        ],
        "default": 0,
        "description": "Record number (0 states \"start at record 0\"). The record size can be defined using the count parameter (and limited to 1000). Tip: To page through results, ask for 1000 records (count: 1000). If you receive 1000 records, assume there\u2019s more, in which case you want to pull the next 1000 records (offset: 1000\u2026then 2000\u2026etc.). Repeat paging until the number of records returned is less than 1000."
      },
      "users_sort_by": {
        "type": [
          "string",
          "null"
        ],
        "default": "display_name",
        "description": "Field name to sort the users stream results set by."
      },
      "users_sort_order": {
        "type": [
          "string",
          "null"
        ],
        "enum": [
          "ASC",
          "DESC"
        ],
        "default": "ASC",
        "description": "Order to sort the users stream results set by."
      },
      "stream_maps": {
        "type": [
          "object",
          "null"
        ],
        "properties": {},
        "description": "Config object for stream maps capability. For more information check out [Stream Maps](https://sdk.meltano.com/en/latest/stream_maps.html)."
      },
      "stream_map_config": {
        "type": [
          "object",
          "null"
        ],
        "properties": {},
        "description": "User-defined config values to be used within map expressions."
      },
      "flattening_enabled": {
        "type": [
          "boolean",
          "null"
        ],
        "description": "'True' to enable schema flattening and automatically expand nested properties."
      },
      "flattening_max_depth": {
        "type": [
          "integer",
          "null"
        ],
        "description": "The max depth to flatten schemas."
      }
    },
    "required": [
      "api_url",
      "auth_token",
      "events_terms",
      "events_count",
      "events_offset",
      "accounts_terms",
      "accounts_fields",
      "users_terms",
      "users_fields"
    ]
  }
}
Discovered streams
  • accounts
  • events
  • users

Copy link

github-actions bot commented Dec 9, 2024

Testing plugin tap-postgres (meltanolabs variant):

Starting test job...
...
Job completed.

Auto-generated README.md

tap-postgres

Singer tap for Postgres.

Built with the Meltano Singer SDK.

Capabilities

  • catalog
  • state
  • discover
  • about
  • stream-maps
  • schema-flattening
  • batch

Settings

Setting Required Default Description
replication_slot_name False tappostgres Name of the replication slot to use for logical replication. Must be unique for parallel extractions. Only applicable when replication_method is LOG_BASED.- Contain only letters, numbers, and underscores. - Be less than or equal to 63 characters. - Not start with 'pg_'.
host False None Hostname for postgres instance. Note if sqlalchemy_url is set this will be ignored.
port False 5432 The port on which postgres is awaiting connection. Note if sqlalchemy_url is set this will be ignored.
user False None User name used to authenticate. Note if sqlalchemy_url is set this will be ignored.
password False None Password used to authenticate. Note if sqlalchemy_url is set this will be ignored.
database False None Database name. Note if sqlalchemy_url is set this will be ignored.
max_record_count False None Optional. The maximum number of records to return in a single stream.
sqlalchemy_url False None Example postgresql://[username]:[password]@localhost:5432/[db_name]
filter_schemas False None If an array of schema names is provided, the tap will only process the specified Postgres schemas and ignore others. If left blank, the tap automatically determines ALL available Postgres schemas.
dates_as_string False 0 Defaults to false, if true, date, and timestamp fields will be Strings. If you see ValueError: Year is out of range, try setting this to True.
json_as_object False 0 Defaults to false, if true, json and jsonb fields will be Objects.
ssh_tunnel False None SSH Tunnel Configuration, this is a json object
ssh_tunnel.enable False 0 Enable an ssh tunnel (also known as bastion server), see the other ssh_tunnel.* properties for more details
ssh_tunnel.host False None Host of the bastion server, this is the host we'll connect to via ssh
ssh_tunnel.username False None Username to connect to bastion server
ssh_tunnel.port False 22 Port to connect to bastion server
ssh_tunnel.private_key False None Private Key for authentication to the bastion server
ssh_tunnel.private_key_password False None Private Key Password, leave None if no password is set
ssl_enable False 0 Whether or not to use ssl to verify the server's identity. Use ssl_certificate_authority and ssl_mode for further customization. To use a client certificate to authenticate yourself to the server, use ssl_client_certificate_enable instead. Note if sqlalchemy_url is set this will be ignored.
ssl_client_certificate_enable False 0 Whether or not to provide client-side certificates as a method of authentication to the server. Use ssl_client_certificate and ssl_client_private_key for further customization. To use SSL to verify the server's identity, use ssl_enable instead. Note if sqlalchemy_url is set this will be ignored.
ssl_mode False verify-full SSL Protection method, see postgres documentation for more information. Must be one of disable, allow, prefer, require, verify-ca, or verify-full. Note if sqlalchemy_url is set this will be ignored.
ssl_certificate_authority False ~/.postgresql/root.crl The certificate authority that should be used to verify the server's identity. Can be provided either as the certificate itself (in .env) or as a filepath to the certificate. Note if sqlalchemy_url is set this will be ignored.
ssl_client_certificate False ~/.postgresql/postgresql.crt The certificate that should be used to verify your identity to the server. Can be provided either as the certificate itself (in .env) or as a filepath to the certificate. Note if sqlalchemy_url is set this will be ignored.
ssl_client_private_key False ~/.postgresql/postgresql.key The private key for the certificate you provided. Can be provided either as the certificate itself (in .env) or as a filepath to the certificate. Note if sqlalchemy_url is set this will be ignored.
ssl_storage_directory False .secrets The folder in which to store SSL certificates provided as raw values. When a certificate/key is provided as a raw value instead of as a filepath, it must be written to a file before it can be used. This configuration option determines where that file is created.
default_replication_method False FULL_TABLE Replication method to use if there is not a catalog entry to override this choice. One of FULL_TABLE, INCREMENTAL, or LOG_BASED.
stream_maps False None Config object for stream maps capability. For more information check out Stream Maps.
stream_map_config False None User-defined config values to be used within map expressions.
faker_config False None Config for the Faker instance variable fake used within map expressions. Only applicable if the plugin specifies faker as an addtional dependency (through the singer-sdk faker extra or directly).
faker_config.seed False None Value to seed the Faker generator for deterministic output: https://faker.readthedocs.io/en/master/#seeding-the-generator
faker_config.locale False None One or more LCID locale strings to produce localized output for: https://faker.readthedocs.io/en/master/#localization
flattening_enabled False None 'True' to enable schema flattening and automatically expand nested properties.
flattening_max_depth False None The max depth to flatten schemas.
batch_config False None Configuration for BATCH message capabilities.
batch_config.encoding False None Specifies the format and compression of the batch files.
batch_config.encoding.format False None Format to use for batch files.
batch_config.encoding.compression False None Compression format to use for batch files.
batch_config.storage False None Defines the storage layer to use when writing batch files
batch_config.storage.root False None Root path to use when writing batch files.
batch_config.storage.prefix False None Prefix to use when writing batch files.

A full list of supported settings and capabilities is available by running: tap-postgres --about

Version info
tap-postgres v[could not be detected], Meltano SDK v0.42.1
Usage info
melty-bot % tap-postgres --help
Usage: tap-postgres [OPTIONS]

  Execute the Singer tap.

Options:
  --version                 Display the package version.
  --about                   Display package metadata and settings.
  --format [json|markdown]  Specify output style for --about
  --config TEXT             Configuration file location or 'ENV' to use
                            environment variables.
  --discover                Run the tap in discovery mode.
  --test TEXT               Use --test to sync a single record for each
                            stream. Use --test=schema to test schema output
                            without syncing records.
  --catalog PATH            Use a Singer catalog file with the tap.
  --state PATH              Use a bookmarks file for incremental replication.
  --help                    Show this message and exit.
Detected capabilities
  • ✅ 'discover'
  • ✅ 'catalog'
  • ✅ 'state'
  • ✅ 'about'
JSON Metadata
{
  "name": "tap-postgres",
  "description": "Singer tap for Postgres.",
  "version": "[could not be detected]",
  "sdk_version": "0.42.1",
  "supported_python_versions": null,
  "capabilities": [
    "catalog",
    "state",
    "discover",
    "about",
    "stream-maps",
    "schema-flattening",
    "batch"
  ],
  "settings": {
    "type": "object",
    "properties": {
      "replication_slot_name": {
        "type": [
          "string",
          "null"
        ],
        "pattern": "^(?!pg_)[A-Za-z0-9_]{1,63}$",
        "default": "tappostgres",
        "description": "Name of the replication slot to use for logical replication. Must be unique for parallel extractions. Only applicable when replication_method is LOG_BASED.- Contain only letters, numbers, and underscores. - Be less than or equal to 63 characters. - Not start with 'pg_'."
      },
      "host": {
        "type": [
          "string",
          "null"
        ],
        "description": "Hostname for postgres instance. Note if sqlalchemy_url is set this will be ignored."
      },
      "port": {
        "type": [
          "integer",
          "null"
        ],
        "default": 5432,
        "description": "The port on which postgres is awaiting connection. Note if sqlalchemy_url is set this will be ignored."
      },
      "user": {
        "type": [
          "string",
          "null"
        ],
        "description": "User name used to authenticate. Note if sqlalchemy_url is set this will be ignored."
      },
      "password": {
        "type": [
          "string",
          "null"
        ],
        "description": "Password used to authenticate. Note if sqlalchemy_url is set this will be ignored.",
        "secret": true,
        "writeOnly": true
      },
      "database": {
        "type": [
          "string",
          "null"
        ],
        "description": "Database name. Note if sqlalchemy_url is set this will be ignored."
      },
      "max_record_count": {
        "type": [
          "integer",
          "null"
        ],
        "description": "Optional. The maximum number of records to return in a single stream."
      },
      "sqlalchemy_url": {
        "type": [
          "string",
          "null"
        ],
        "description": "Example postgresql://[username]:[password]@localhost:5432/[db_name]",
        "secret": true,
        "writeOnly": true
      },
      "filter_schemas": {
        "type": [
          "array",
          "null"
        ],
        "items": {
          "type": [
            "string"
          ]
        },
        "description": "If an array of schema names is provided, the tap will only process the specified Postgres schemas and ignore others. If left blank, the tap automatically determines ALL available Postgres schemas."
      },
      "dates_as_string": {
        "type": [
          "boolean",
          "null"
        ],
        "default": false,
        "description": "Defaults to false, if true, date, and timestamp fields will be Strings. If you see ValueError: Year is out of range, try setting this to True."
      },
      "json_as_object": {
        "type": [
          "boolean",
          "null"
        ],
        "default": false,
        "description": "Defaults to false, if true, json and jsonb fields will be Objects."
      },
      "ssh_tunnel": {
        "type": [
          "object",
          "null"
        ],
        "properties": {
          "enable": {
            "type": [
              "boolean",
              "null"
            ],
            "default": false,
            "description": "Enable an ssh tunnel (also known as bastion server), see the other ssh_tunnel.* properties for more details"
          },
          "host": {
            "type": [
              "string",
              "null"
            ],
            "description": "Host of the bastion server, this is the host we'll connect to via ssh"
          },
          "username": {
            "type": [
              "string",
              "null"
            ],
            "description": "Username to connect to bastion server"
          },
          "port": {
            "type": [
              "integer",
              "null"
            ],
            "default": 22,
            "description": "Port to connect to bastion server"
          },
          "private_key": {
            "type": [
              "string",
              "null"
            ],
            "description": "Private Key for authentication to the bastion server",
            "secret": true,
            "writeOnly": true
          },
          "private_key_password": {
            "type": [
              "string",
              "null"
            ],
            "description": "Private Key Password, leave None if no password is set",
            "secret": true,
            "writeOnly": true
          }
        },
        "description": "SSH Tunnel Configuration, this is a json object"
      },
      "ssl_enable": {
        "type": [
          "boolean",
          "null"
        ],
        "default": false,
        "description": "Whether or not to use ssl to verify the server's identity. Use ssl_certificate_authority and ssl_mode for further customization. To use a client certificate to authenticate yourself to the server, use ssl_client_certificate_enable instead. Note if sqlalchemy_url is set this will be ignored."
      },
      "ssl_client_certificate_enable": {
        "type": [
          "boolean",
          "null"
        ],
        "default": false,
        "description": "Whether or not to provide client-side certificates as a method of authentication to the server. Use ssl_client_certificate and ssl_client_private_key for further customization. To use SSL to verify the server's identity, use ssl_enable instead. Note if sqlalchemy_url is set this will be ignored."
      },
      "ssl_mode": {
        "type": [
          "string",
          "null"
        ],
        "default": "verify-full",
        "description": "SSL Protection method, see [postgres documentation](https://www.postgresql.org/docs/current/libpq-ssl.html#LIBPQ-SSL-PROTECTION) for more information. Must be one of disable, allow, prefer, require, verify-ca, or verify-full. Note if sqlalchemy_url is set this will be ignored."
      },
      "ssl_certificate_authority": {
        "type": [
          "string",
          "null"
        ],
        "default": "~/.postgresql/root.crl",
        "description": "The certificate authority that should be used to verify the server's identity. Can be provided either as the certificate itself (in .env) or as a filepath to the certificate. Note if sqlalchemy_url is set this will be ignored."
      },
      "ssl_client_certificate": {
        "type": [
          "string",
          "null"
        ],
        "default": "~/.postgresql/postgresql.crt",
        "description": "The certificate that should be used to verify your identity to the server. Can be provided either as the certificate itself (in .env) or as a filepath to the certificate. Note if sqlalchemy_url is set this will be ignored."
      },
      "ssl_client_private_key": {
        "type": [
          "string",
          "null"
        ],
        "default": "~/.postgresql/postgresql.key",
        "description": "The private key for the certificate you provided. Can be provided either as the certificate itself (in .env) or as a filepath to the certificate. Note if sqlalchemy_url is set this will be ignored."
      },
      "ssl_storage_directory": {
        "type": [
          "string",
          "null"
        ],
        "default": ".secrets",
        "description": "The folder in which to store SSL certificates provided as raw values. When a certificate/key is provided as a raw value instead of as a filepath, it must be written to a file before it can be used. This configuration option determines where that file is created."
      },
      "default_replication_method": {
        "type": [
          "string",
          "null"
        ],
        "default": "FULL_TABLE",
        "description": "Replication method to use if there is not a catalog entry to override this choice. One of `FULL_TABLE`, `INCREMENTAL`, or `LOG_BASED`.",
        "enum": [
          "FULL_TABLE",
          "INCREMENTAL",
          "LOG_BASED"
        ]
      },
      "stream_maps": {
        "type": [
          "object",
          "null"
        ],
        "properties": {},
        "title": "Stream Maps",
        "description": "Config object for stream maps capability. For more information check out [Stream Maps](https://sdk.meltano.com/en/latest/stream_maps.html)."
      },
      "stream_map_config": {
        "type": [
          "object",
          "null"
        ],
        "properties": {},
        "title": "User Stream Map Configuration",
        "description": "User-defined config values to be used within map expressions."
      },
      "faker_config": {
        "type": [
          "object",
          "null"
        ],
        "properties": {
          "seed": {
            "oneOf": [
              {
                "type": [
                  "number"
                ]
              },
              {
                "type": [
                  "string"
                ]
              },
              {
                "type": [
                  "boolean"
                ]
              },
              {
                "type": "null"
              }
            ],
            "title": "Faker Seed",
            "description": "Value to seed the Faker generator for deterministic output: https://faker.readthedocs.io/en/master/#seeding-the-generator"
          },
          "locale": {
            "oneOf": [
              {
                "type": [
                  "string"
                ]
              },
              {
                "type": "array",
                "items": {
                  "type": [
                    "string"
                  ]
                }
              },
              {
                "type": "null"
              }
            ],
            "title": "Faker Locale",
            "description": "One or more LCID locale strings to produce localized output for: https://faker.readthedocs.io/en/master/#localization"
          }
        },
        "title": "Faker Configuration",
        "description": "Config for the [`Faker`](https://faker.readthedocs.io/en/master/) instance variable `fake` used within map expressions. Only applicable if the plugin specifies `faker` as an addtional dependency (through the `singer-sdk` `faker` extra or directly)."
      },
      "flattening_enabled": {
        "type": [
          "boolean",
          "null"
        ],
        "title": "Enable Schema Flattening",
        "description": "'True' to enable schema flattening and automatically expand nested properties."
      },
      "flattening_max_depth": {
        "type": [
          "integer",
          "null"
        ],
        "title": "Max Flattening Depth",
        "description": "The max depth to flatten schemas."
      },
      "batch_config": {
        "type": [
          "object",
          "null"
        ],
        "properties": {
          "encoding": {
            "type": [
              "object",
              "null"
            ],
            "properties": {
              "format": {
                "type": [
                  "string",
                  "null"
                ],
                "title": "Batch Encoding Format",
                "description": "Format to use for batch files.",
                "enum": [
                  "jsonl",
                  "parquet"
                ]
              },
              "compression": {
                "type": [
                  "string",
                  "null"
                ],
                "title": "Batch Compression Format",
                "description": "Compression format to use for batch files.",
                "enum": [
                  "gzip",
                  "none"
                ]
              }
            },
            "title": "Batch Encoding Configuration",
            "description": "Specifies the format and compression of the batch files."
          },
          "storage": {
            "type": [
              "object",
              "null"
            ],
            "properties": {
              "root": {
                "type": [
                  "string",
                  "null"
                ],
                "title": "Batch Storage Root",
                "description": "Root path to use when writing batch files."
              },
              "prefix": {
                "type": [
                  "string",
                  "null"
                ],
                "title": "Batch Storage Prefix",
                "description": "Prefix to use when writing batch files."
              }
            },
            "title": "Batch Storage Configuration",
            "description": "Defines the storage layer to use when writing batch files"
          }
        },
        "title": "Batch Configuration",
        "description": "Configuration for BATCH message capabilities."
      }
    }
  }
}

@edgarrmondragon edgarrmondragon added this pull request to the merge queue Dec 9, 2024
Merged via the queue into main with commit 01d4ce8 Dec 9, 2024
22 checks passed
@edgarrmondragon edgarrmondragon deleted the hub-bot-metadata-2024-12-09-12235274777-110-1 branch December 9, 2024 19:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants