refactor: clean up mongo script & update mongo client

[cfnelson]

This PR includes the changes made to the script to get our mongodump/mongorestore working and to clean up & simplify the process. 🔥

Changes Made: 💃

• switched to using 3.4 mongo tool/shell instead of 3.2 & using https instead of http. Meteor by default now uses 3.4, next Meteor release will be using mongo 3.6 and the script will likely be required to be updated again.
• Added comments so others can have a little idea of what is going on.
• Added --verbose option so we can see the whole process in the logs and verify the outcome or for debugging upon failure or odd behaviour.
• enabled/enforce --ssl as previous versions were not using an ssl connection to dump & restore the db 😱
• using --autheniticationDatabase option to be more clear how the two scripts work
• switched to long form param names to make it more clear what each option is
• Removed unnecessary collection drop script for --drop option on mongorestore if we wish to be even more restrictive on what we restore we could add the --nsExclude or --nsInclude options to the mongorestore command.
• changed --out to mongo_dump as mongodump is bad naming and can cause confusion as the outpath shared its name with the mongodump command

[pauldowman]

It looks like you're not explicitly dropping all collections now, which means if one doesn't appear in the data that we're restoring (because it has already been dropped on production) it won't be dropped.

[pauldowman]

Also it looks like it's losing the ability to set a custom port, which is sometimes needed.

[cfnelson]

@pauldowman the --PORT option isn't necessary as it can be attached in the --host string that you pass through. Unless you are talking about something else? See the below image for example syntax.

I was aware of the --drop behaviour but was unsure of how to handle for the first upload. The behaviour of mongorestore is if the --db is not passed it will overwrite the existing db and replace it completely. I attempted this but it seemed to fail I suspected the mongodump output we have isn't correct format.
See docs here.

If the staging DB has additional collections that were completely dropped from the production DB (something we don't normally do) and are no longer being used in the App if these still persist in the staging DB it wouldn't affect the App in anyway. I also suspect this script will further change once we update to Mongo 3.6.

[pauldowman]

OK, the port can just be on the host then if you prefer that.

If the staging DB has additional collections that were completely dropped from the production DB (something we don't normally do) and are no longer being used in the App if these still persist in the staging DB it wouldn't affect the App in anyway

I disagree that this is OK. It can definitely be a problem. Why not just drop them first as we did before?

[cfnelson]

The script was confusing to others (asked about it multiple times), I also found it to be buggy & the way you connect via the mongo shell would have required an additional environment variable.

I believe the recommended approach to achieve our goal is to not specify the db name in mongorestore which causes mongorestore to recreate the db from the mongodump. See here below:

I can improve the script further using the above recommended approach but I will need to test it or change the mongodump slightly as I think its missing the database name in the dump output.

[pauldowman]

What I'm saying is that mongorestore doesn't drop collections that no longer exist.

[pauldowman]

I'm pretty sure tahtmongorestore doesn't drop collections that no longer exist, so we need to make sure that happens. I don't think what we have here does that. Adding back the forEach would fix it, or if there's some nicer way that's fine with me.