Skip to content

Latest commit

 

History

History
435 lines (302 loc) · 16.2 KB

README.md

File metadata and controls

435 lines (302 loc) · 16.2 KB
Raw

Build Status

🔧 Setup

For easy setup read the running locally with docker-compose section.

Follow these steps to setup the App for creators.brave.com. This guide presumes you are using OSX and Homebrew.

  1. Install Ruby. For a Ruby version manager try rbenv. Follow the Installation section instructions and ensure your version is at least 1.1.2. Once installed run rbenv install. Be sure to restart your terminal before continuing.

  2. Install Node 6.12.3 or greater: brew install node

  3. Install Postgresql 9.5+: brew install postgresql

    If you get the error psql: FATAL: role “postgres” does not exist. You'll need to create the /usr/local/opt/postgres/bin/createuser -s postgres

  4. Install Redis: brew install redis

  5. Install Ruby gems: gem install bundler foreman mailcatcher.

  6. Install Yarn for Node dependency management: brew install yarn

  7. Install project dependencies

    Ruby dependencies: bundle install

    Possible errors:

    • Nokogiri, with libxml2. Try installing a system libxml2 with brew install libxml2 and then bundle config build.nokogiri --use-system-libraries then again bundle install
    • Run gem install nokogiri -v '1.10.3' and then bundle install

    Node dependencies: yarn --frozen-lockfile

    Your version of Node must be v11.15.0 or earlier. For a node version manager, try NVM.

  8. Install git-secrets with brew install git-secrets This prevents AWS keys from being committed.

  9. (Optional) Get a .env file from another developer which contains development-mode env vars. You can start developing without this, but some functionality may be limited.

  10. Install Rails: gem install rails

    Be sure to restart your terminal before continuing.

  11. Setup SSL as described below.

HTTPS Setup

Local development of brave-intl uses HTTPS. This allow us to use web APIs such as U2F in development.

If you already have a key and certificate for the localhost domain place them in the ssl/ directory:

ssl/server.key
ssl/server.crt

If you don't, you will need to generate certificates for this domain:

bundle exec rake ssl:generate

When you first visit the application in a browser you may need to add an exception to trust this self-signed certificate. Sometimes this is under an "advanced" or "proceed" link.

Run

  1. Start Postgres and Redis: brew services start redis postgresql

  2. Create and initialize the database:

    rails db:create RAILS_ENV=development
rails db:migrate RAILS_ENV=development

    Note: If you receive a fatal-role error, try running /usr/local/opt/postgres/bin/createuser -s postgres due to being installed from homebrew. Further documentation is here.

    If you receive an error about Readline, try running:

    ln -s /usr/local/opt/readline/lib/libreadline.dylib /usr/local/opt/readline/lib/libreadline.7.dylib

    Issue for further documentation.

  3. Run Rails server and async worker: bundle exec puma -C config/puma.rb -e ${RACK_ENV:-development} bundle exec sidekiq -C config/sidekiq.yml -e ${RACK_ENV:-development}

  4. Visit https://localhost:3000

  5. To test email, run a local mail server with: mailcatcher

  6. To view the emails sent to your inbox visit: http://localhost:1080

  7. Run webpack separately: ./bin/webpack-dev-server

  8. Compile landing page assets: cd public/landing-page; rake assets:clobber; rake assets:precompile; yarn install; yarn build

API Setups

Google API Setup

Setup a google API project:

  • Login to your google account (dev), or the Brave google account (staging, production)
  • Go to https://console.developers.google.com
  • Select "Create Project" then "Create" to setup a new API project
  • Give the project a name such as "creators-dev"
  • Select "+ Enable APIs and Services"
  • Enable "Google+ API" and "YouTube Data API v3"
  • Back at the console select Credentials, then select the "OAuth consent screen" sub tab
  • Fill in the details. For development you need the Product name, try "Creators Dev (localhost)"
  • Then Select "Create credentials", then "OAuth client ID"
    • Application type is "Web application"
    • Name is "Creators"
    • Authorized redirect URIs is http://localhost:3000/publishers/auth/google_oauth2/callback
    • select "Create"
  • Record the Client ID and Client secret and enter them in your GOOGLE_CLIENT_ID and GOOGLE_CLIENT_SECRET variables
  • Back at the console select "Create credentials" and select API key. This will be used for youtube channel stats via the data api.
  • Record the API and enter it in your YOUTUBE_API_KEY variable

You may need to wait up to 10 minutes for the changes to propagate.

These steps based on directions at the omniauth-google-oauth2 gem.

Twitch API Setup

Setup a twitch API project:

  • Login to your Twitch account (dev), or the Brave Twitch account (staging, production)
  • Go to https://dev.twitch.tv/dashboard
  • Make sure you first set up 2 factor auth for your twitch account
  • Select "Register Your Application" under "Applications"
  • Give the project a name such as "creators-dev"
  • Give the app a name and application category.
  • Use the redirect URI https://localhost:3000/publishers/auth/register_twitch_channel/callback in development.
  • Use the 'Website Integration' category
  • Save the app
  • Once the application is created, click on "Manage" where you can view the Client ID and create the Client Secret
  • Create a Client ID and secret, saving each of them.
    • Update your env to include TWITCH_CLIENT_ID="your-app-id"
    • Update your env to include TWITCH_CLIENT_SECRET="your-app-secret"

Twitter API Setup

  • Apply for a developer account at developer.twitter.com
  • Select "Create an App"
  • Give the app a name like "Brave Payments Dev"
  • Make sure "Enable Sign in with Twitter" is checked
  • Set the callback url to https://localhost:3000/publishers/auth/register_twitter_channel/callback. If it does not allow you to set localhost, use a place holder for now, and later add the correct callback url through apps.twitter.com instead.
  • Fill in the remaining information and hit "Create"
  • Navigate to your app settings -> permissions and ensure it is readonly and requests the user email
  • Regenerate your Consumer API keys
  • Update your env to include TWITCH_CLIENT_ID="your-api-key" and TWITTER_CLIENT_SECRET="your-api-secret-key"
  • Save

reCAPTCHA Setup

In order to test the rate limiting and captcha components you will need to setup an account with Google's reCAPTCHA. Instructions can be found at the reCAPTCHA gem repo. Add the api keys to your Env variables.

Local Eyeshade Setup

  1. Follow the setup instructions for bat-ledger
  2. Add export API_EYESHADE_BASE_URI="http://127.0.0.1:3002" to your secrets script
  3. Add export API_EYESHADE_KEY="00000000-0000-4000-0000-000000000000" to your secrets script

To stop using Eyeshade locally, set API_EYESHADE_BASE_URI="".

Local Vault-Promo-Services Setup

  1. Request access to Vault-Promo-Services and ip2tags
  2. Follow the setup instructions
  3. Create and run a vault-promo-services.sh start script like this
export DATABASE_URL="services"
export PGDATABASE="services"
export AUTH_TOKEN=1234
export S3_KEY="X"
export S3_SECRET="x"
export WINIA32_DOWNLOAD_KEY="/"
export WINX64_DOWNLOAD_KEY="/"
export OSX_DOWNLOAD_KEY="/"
export TEST=1

dropdb services
createdb services
for folder in ./migrations/*; do
  psql services < ${folder}/up.sql
done
npm start
  • If you run into an issue about a missing .mmdb file, run fetch.sh in node_modules/ip2tags
  1. Add the following into your Publishers start script
export API_PROMO_BASE_URI="http://127.0.0.1:8194"
export API_PROMO_KEY="1234"

Development

See CONTRIBUTING before submitting any PRs.

Config

Configuration is set in config/secrets.yml via environment variables.

We use the dotenv gem to load variables specified in .env into the rails app, only in the development and test environments. This makes sure they are only loaded for the context of the running rails app and that they don't pollute the shell environment.

Automagic addon vars

Some variables are set automagically with Heroku addons:

  • MAILGUN_* - For sending emails.
  • NEW_RELIC_APP_NAME, NEW_RELIC_LICENSE_KEY - New Relic app monitoring.
  • REDIS_URL - For Sidekiq and rack-attack

Generating Referral Charts

As part of a view we have a chart on the dashboard. There isn't an easy way to mock this out, so there is a rake task to allow developers to easily test this locally.

You must first have a channel added and the promo activated for this to work.

rails database_updates:mock_data:populate_promo_stats

A picture of the chart generated by the promo server

Other vars

A few variables are not configured in secrets.yml: currently none

Linting

For Ruby we use rubocop to standardize our project. Our linter config is based upon the AirBnb style guide.

To run simply open the project and run in the terminal

bundle exec rubocop

For Typescript/Stylesheets we use tslint and stylelint respectively.

To run simply open the project and run in the terminal

yarn lint

Testing

If you have Docker set up on your machine you can run all the tests by running:

make docker-test

Alternatively if you have the environment set up on your machine you can run the following steps.

Ruby

bin/rake test

We use capybara which runs selenium tests, which depends on chromium. If you don't installed, you'll get an error "can't find chrome binary". On debian you can install it like:

sudo apt-get install chromium

And on mac with:

brew cask install chromium

We also use ImageMagick to process user uploaded images. If you don't have it already, you might get an error "You must have ImageMagick or GraphicsMagick installed". You can install on mac with:

brew install imagemagick

Javascript

We use jest for our javascript testing framework. You can run the tests through the following command.

yarn test

Running locally with docker-compose

First, install docker and docker compose.

Check out publishers.

You can add any environment variables that need to be set by creating a .env file at the top of the repo. Docker compose will automatically load from this file when launching services.

Build the docker images

make docker-dev-build

and bring up the full stack

make docker-dev

If you wish to make modifications to the compose files you can place a file named docker-compose.override.yml at the top of the repo. For example you can expose ports on your system for the databases with this docker-compose.override.yml:

version: "2.1"

services:
  mongo:
    ports:
      - "27017:27017"
  redis:
    ports:
      - "6379:6379"
  postgres:
    ports:
      - "5432:5432"

to start with docker build the app and eyeshade images

docker-compose build

Create the databases

docker-compose run app yarn install; docker-compose run app rake db:setup; docker-compose run eyeshade-worker sh -c "cd eyeshade && ./bin/migrate-up.sh"

Adding balances to Eyeshade

By default when you create a channel it will not have a balance on Eyeshade, the accounting server. To test wallet code with non nil balances, you must add them first.

To add a contribution to a channel account:

rails "docker:add_contribution_balance_to_account[youtube#channel:UCOo92t8m-tWKgmw276q7mxw, 200]" # Adds 200 BAT to youtube#channel:UCOo92t8m-tWKgmw276q7mxw

To add add a referral balance to an owner account:

rails "docker:add_referral_balance_to_account[publishers#uuid:967a9919-34f4-4ce6-af36-e3f592a6eab7, 400]" # Adds 400 BAT to youtube#channel:UCOo92t8m-tWKgmw276q7mxw

The new balance should be reflected on the dashboard.

Adding a new type of channel

The easiest possible way to add a new channel is to find the Omniauth gem for the specified integration. A few examples include omniauth-soundcloud, omniauth-github, or omniauth-facebook

  1. Add the gem to the Gemfile

  2. Run bundle install

  3. Run rails generate property INTEGRATION_channel_details (Note: replace INTEGRATION with the name of the integration, e.g. github, soundcloud, vimeo, etc)

  4. Run rails db:migrate

  5. Register a new route in config/initializers/devise.rb

  6. Add a new controller method in app/controllers/publishers/omniauth_callbacks_controller.rb similar to register_github_channel or register_reddit_channel

  7. Add the link and icon to /app/views/application/_choose_channel_type.html.slim

  8. Add translations in en.yml for helpers.publisher.channel_type and helpers.publisher.channel_name

    channel_type:
  youtube: YouTube channel
  website: Website
  <INTEGRATION>: Your <INTEGRATION> Name
 channel_name:
   youtube: YouTube
   website: the website
   <INTEGRATION>: <INTEGRATION> Name

  9. Add assets for the new integration. Both a 32x32 png and a SVG of the logo.

Run Tests

Tests can be run on the container with

docker-compose run app rake test

Other one off commands can be run as above, but replacing rake test. Note this spawns a new container.

Debugging

Debugging with byebug and pry can be done by attaching to the running process. First get the container id with docker ps

docker ps
CONTAINER ID        IMAGE                    COMMAND                  CREATED                  STATUS              PORTS                                            NAMES
234f116cd942        publishers_app           "foreman start --pro…"   Less than a second ago   Up 2 seconds        0.0.0.0:3000->3000/tcp                           publishers_app_1
b592d489a8d3        redis                    "docker-entrypoint.s…"   15 minutes ago           Up 3 seconds        6379/tcp                                         publishers_redis_1
f1c86172def7        schickling/mailcatcher   "mailcatcher --no-qu…"   15 minutes ago           Up 2 seconds        0.0.0.0:1025->1025/tcp, 0.0.0.0:1080->1080/tcp   publishers_mailcatcher_1

Then attach to the container and you will hit your binding.pry breakpoints

docker attach 234f116cd942

To connect with a bash shell on a running container use:

docker exec -i -t 234f116cd942 /bin/bash
root@234f116cd942:/var/www#