censor database name, user and password in mysqldump invocation

[castilma]

Describe the bug

The output of wordmove pull -d shows database name, user and password

$ wordmove pull -dv

▬▬ Using Movefile: ./movefile.yml ▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬

▬▬ Pulling Database ▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬

   [...]
   remote | mysqldump --host=[secret] --user=[UNCENSORED DATABASE USER] --password=[UNCENSORED PASSWORD] --result-file="[secret]/wp-content/dump.sql" [UNCENSORED DTABASE NAME]
   [...]

Expected behavior

The [UNCENSORED *] parts above should be [secret].

movefile.yml
(Not posted since I think it's irrelevant to the problem.)

Environment (please complete the following information):

• OS: MacOS 11.6.2
• Ruby: ruby 2.7.5p203 (2021-11-24 revision f69aeb8314) [x86_64-darwin20]
• Wordmove: 5.2.2

Doctor

• running the wordmove doctor command returns all green

Irrelevant to this issue?: I use mysql_options to set a nonstandard socket, and doctor doesn't seem to respect those options.

[...]
▬▬ Validating movefile section: local ▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬

    ❌  error | [/database/mysql_options] key 'mysql_options:' is undefined.

▬▬ Validating movefile section: staging ▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬

    ✅  success | Formal validation passed

▬▬ Using Movefile: ./movefile.yml ▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬

▬▬ Checking local database commands and connection ▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬

    ✅  success | `mysql` command is in $PATH

    ✅  success | `mysqldump` command is in $PATH

    ❌  error |   We can't connect to the MySQL server using credentials
                specified in the Movefile. Double check them or try
                to debug your system configuration.

                The command used to test was:

                mysql --host=localhost --user=[...] --password=[...] -e'QUIT'


    ❌  error |   We can't connect to the database using credentials
                specified in the Movefile, or the database does not
                exists. Double check them or try to debug your
                system configuration.

                The command used to test was:

                mysql --host=localhost --user=[...] --password=[...] [...] -e'QUIT'

[...]

[alessandro-fazzi]

Hello,

wordmove consider secrets string the followings:

        secrets << options.dig(env, :database, :password)
        secrets << options.dig(env, :database, :host)
        secrets << options.dig(env, :vhost)
        secrets << options.dig(env, :ssh, :password)
        secrets << options.dig(env, :ssh, :host)
        secrets << options.dig(env, :ftp, :password)
        secrets << options.dig(env, :ftp, :host)
        secrets << options.dig(env, :wordpress_path)

where options are all the keys in movefile.yml and env is the local root key or any other root key representing a remote environment.

Thus only db password and host. Db name and db user are not intended as to be cencored.

Talking about the password, this is what I reproduce on same wordmove version as your

   remote | mysqldump --host=[secret] --user=sshwordmove --password=[secret] --result-file="[secret]/wp-content/dump.sql" sshwordmove

Each string is censored using a really basic regular expression costructed like, e.g.: having

database:
  host: 'myhost.com'
  password: 'badpassword'

the regex will be myhost\.com|badpassword. Each string's char is escaped to be used as literal into the resulting regex and strings are concatenated with an "or" |.

That's FYI and for my revision and memory.

Problem is that I cannot imagine what's breaking the toy here. I'd really need to have a string in order to reproduce the failure. I know that the string is your DB password, but if you'd be able to change it and tell me the old failing one I could write a test in order to check what's happening.

The described logger behaviour has also a bit of test coverage here

wordmove/spec/logger/logger_spec.rb

Line 1 in 96e2b41

describe Wordmove::Logger do

[castilma]

The passwort contains a #.

passwort: "R4ndom#+Str1nG"

The output shows a backslash:

   remote | mysqldump --host=[secret] --user=[] --password=R4ndom\#+Str1nG [...]

You might want to add a testcase with | as well as anything that would need to be escaped in a shell, like spaces, semicolons, quotes etc..

[alessandro-fazzi]

Thank you very much for the string :) I'll do some testing ASAP trying to make it fail.

You might want to add a testcase with | as well as anything that would need to be escaped in a shell, like spaces, semicolons, quotes etc..

A word of context: I do not need to escape those string for the shell, but for the regex engine. Shell escaping exists and is implemented in the context of command execution. Here we're in the context of the stdout logger.
Moreover what I'm testing is that wordmove is able to substitute string on stdout, not the escape algorith self: Regexp.escape is part of the ruby stdlib and doesn't need me to test it. But for sure adding some chars in the tested string could be useful.

I'll come back with news.

[alessandro-fazzi]

Hi @castilma ,

I've done a bit of testing; you can read what I tested at https://github.com/welaika/wordmove/pull/642/files and see results at https://github.com/welaika/wordmove/runs/5167816431?check_suite_focus=true#step:7:6

I've tested the behaviour on top of the branch bringing the 6.0.alpha version: I'm not able to spend effort on the previous versions. But I had bet that involved code shuould have remained the same. Even if I'm wrong, I'd consider the bug fixed in the next candidate version, thus I'm not prone to consolidate the bug.

We're on the path to move 6.0 from alpha to beta since we've done some internal testing and some commuity driven testing too. I don't know if it could be possible for you, but I'd ask if you could do some testing with the alpha version.

Preliminary info about the upcoming version can be found at #624 and a user testing experience at #632

[castilma]

How can I install 6.0 as ruby gem? Or do I need to clone the repo?

Here we're in the context of the stdout logger.

But doesn't the logger log the executed commandlines? And those contain the shellescaped secrets? So it should look for shellescaped secrets, too.
I left a review on #642.

[alessandro-fazzi]

Thank you very much

[alessandro-fazzi]

This is confirmed as a bug in the form that the script puts shell-escaped strings on STDOUT than search for secrets by using unescaped strings.

Stil not decied how to definitely approach the problem, but I've officialy to deal with it :)

Thanks again for the useful report and for the smart analysis

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.