Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature Request]: return filenames to result in pipeline #4697

Open
gaisa88 opened this issue Dec 11, 2024 · 8 comments
Open

[Feature Request]: return filenames to result in pipeline #4697

gaisa88 opened this issue Dec 11, 2024 · 8 comments

Comments

@gaisa88
Copy link

gaisa88 commented Dec 11, 2024

What would you like to happen?

in text file output have
@iNJECTiON(name = "ADD_TO_RESULT")
can you make returning filenames to next step in pipelines?
image

Issue Priority

Priority: 3

Issue Component

Component: Pipelines

@hansva
Copy link
Contributor

hansva commented Dec 11, 2024

We could add a tab like in the text file input to add the additional information to the stream.

Current workaround could be to create a field with the filename and use the "accept file name from field" option.
Then you will have the filename field in the stream after the text file output too.

@gaisa88
Copy link
Author

gaisa88 commented Dec 11, 2024

yes but if you can see im added date time format

@hansva
Copy link
Contributor

hansva commented Dec 11, 2024

FYI: the option is not turned on, but yes then you have to generate a filename including the date.
I'm giving you an option that works now. until this feature is implemented

@bamaer
Copy link
Contributor

bamaer commented Dec 11, 2024

what are you trying to achieve here? there's no way you would be able to use either the filename or the date format in the Vertica bulk loader that comes after your text file output.

@gaisa88
Copy link
Author

gaisa88 commented Dec 11, 2024

im loading by 1000 rows to minio and i need get url to insert to vertica table, how i can get filename in pipeline?

how i can blocked "Get files from result"?
image

@hansva
Copy link
Contributor

hansva commented Dec 11, 2024

let's not mix things in one location. Start a discussion or a mail to the user mailing list

@AlexLogov
Copy link

AlexLogov commented Dec 24, 2024

image
May I suggest a pull request to solve this problem?
Added a checkbox to select the option to include the "filename" field in the output fields. All static and dynamic file names are placed there.

@AlexLogov
Copy link

what are you trying to achieve here? there's no way you would be able to use either the filename or the date format in the Vertica bulk loader that comes after your text file output.

If add the file name to the resulting fields, it can be used for Vertica bulk loader as well

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants