Skip to content

Commit

Permalink
Transition audio test to mocha testing framework
Browse files Browse the repository at this point in the history
  • Loading branch information
Kunal Nain committed Jun 1, 2022
1 parent 1382a28 commit 8eb9faf
Show file tree
Hide file tree
Showing 17 changed files with 2,481 additions and 0 deletions.
131 changes: 131 additions & 0 deletions integration/mochaTests/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,131 @@
# Mocha Tests

Currently, integration tests are being transitioned over from KITE to Mocha. `mochaTests` contains the tests that are being transitioned from KITE to Mocha.

## Test Types

There are two types of Mocha tests: integration and browser compatibility.

### Integration tests
Integration tests are minimal tests that run on latest Chrome on MacOs. These tests are used to test the basic functionality across the more popular browsers.

### Browser Compatibility Tests
Browser compatibility tests are used to test a feature across several different browser and OS combinations. This test is used to make sure that JS SDK features are compatible with a wide variety of browser and OS. There is a default set of browser and OS that are used for this test. In a case where a feature is not compatible with one of the browser or OS then the user can pass in a custom config.

In the following section, we will go over the schema of the custom JSON config.
#### JSON Config Schema

```json
{
"clients": [
{
/*
Browser name
@required
*/
"browserName": "chrome",
/*
Browser Version
@required
*/
"browserVersion": "latest",
/*
Platform
@required
*/
"platform": "MAC"
}
]
}
```
## Running tests

### Running Integration Test Locally
You can run the integration tests locally. By default, integration tests will run on latest version of Chrome that is installed on your machine.
You will have to make sure that you have the required drivers installed on your machine that the selenium test uses. You can find more information about the different drivers available at the `selenium-webdriver` npm page: [https://www.npmjs.com/package/selenium-webdriver](https://www.npmjs.com/package/selenium-webdriver).

If you have an older version of driver installed then you will need to have an older version of browser on your machine as well. Generally, it is recommended to do local testing on latest version. If you need to test on older version, you can run a browser compatibility test with saucelabs.

Sample command to run an integration test locally:

```bash
npm run test -- --test-name AudioTest --host local --test-type integration-test
```

Browser compatiblity tests will run across a variety of browser and OS combinations so it is recommended to use SauceLabs for them.

### Running Browser Compatibility Tests on SauceLabs
To run the test on SauceLabs, the username and access key will be required for authentication. Update the following commands with the credentials and add the credentials to environment variables.
```bash
export SAUCE_USERNAME=<Sauce Labs account username>
```
```bash
export SAUCE_ACCESS_KEY=<Sauce Labs access key>
```

SauceLabs will open a browser and load a test url. You need to have a serverless deployment of Chime JS SDK demo. You can set the demo url as an environment variable with the following command:
```bash
export TEST_URL=<JS SDK Demo URL>
```

Following command can be used to run a browser compatibility tests with default settings on SauceLabs:

```bash
npm run test -- --test-name AudioTest --host saucelabs --test-type browser-compatibility
```

There are scenarios where a test might not be compatible with one of the browsers or OS. In that case, user can provide a custom config with updated clients array. `sample_test.config.json` is a sample test config already provided.
Following command can be used to run a browser compatibility test with custom config:

```bash
npm run test -- --test-name AudioTest --host saucelabs --test-type browser-compatibility --config browserCompatibilityTest/desktop/sample_test.config.json
```

### Running Browser Compatibility Tests on DeviceFarm
To run the tests on DeviceFarm, you will need an AWS account. Once you have an AWS account, you will need to setup two environment variables that will allow the AWS SDK to authenticate to your AWS account.
```bash
export AWS_ACCESS_KEY_ID=<YOUR_ACCESS_KEY_ID>
export AWS_SECRET_ACCESS_KEY=<YOUR_SECRET_ACCESS_KEY>
```

You will need to create a DeviceFarm project in the AWS account that you are planning to use. There are two types of devicefarm projects that can be created: `Mobile Device Testing` and `Desktop Browser Testing`. For this section, we will focus mainly on desktop browser testing.
After the project is created you will need to setup a `PROJECT_ARN` as an environment variable. Project arn is used by the device farm API to identify the project to create test sessions inside.
```bash
export PROJECT_ARN=<YOUR_PROJECT_ARN>
```

Following command can be used to run a browser compatibility test with default settings on DeviceFarm:
```bash
npm run test -- --test-name AudioTest --host devicefarm --test-type browser-compatibility
```

Just like with SauceLabs, DeviceFarm can run browser compatibility tests with a custom config:
```bash
npm run test -- --test-name AudioTest --host devicefarm --test-type browser-compatibility --config browserCompatibilityTest/desktop/sample_test.config.json
```

## Logging

In the integration tests, logging is implemented in a unique way. This section will go over the two distinct ways of logging:
1. Use `logger.log` for inline logging
2. Use `logger.pushLogs` for buffered logging

Developers can use `logger.log` as a normal console log with some enhanced functionality. The main difference between `logger.log` and `console.log` is that logger log will take `LogLevel` as a parameter. Different log levels print to the console in different colors and have a minor difference in the logs which makes it easier to read the logs. There are four different types of log levels.
```ts
const LogLevel = {
INFO: 'INFO',
WARN: 'WARN',
ERROR: 'ERROR',
SUCCESS: 'SUCCESS',
};
```

Integration tests use `mocha` as their test framework. Mocha prints the test results and some limited logs to the console automatically. Any console logs that are inside of `it` will print before the mocha logs.
This makes debugging hard as the logs seem out of sync and random. To get around this issue buffered logging was added. Inside of `it` blocks, `logger.pushLogs` can be used to add logs to the buffer and a call to `logger.printLogs` inside of the `afterEach` hook will print the logs in the desired order.
```ts
afterEach(async function () {
this.logger.printLogs();
});
```

Mocha provides hooks like `before`, `after`, `beforeEach`, `afterEach` to perform actions at specific time of the testing cycle. Any logs printed inside the hooks will print in line so inline logging can be used and there is no need for buffered logging.
40 changes: 40 additions & 0 deletions integration/mochaTests/configs/BaseConfig.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
const config = {
firefoxOptions: {
browserName: 'firefox',
'moz:firefoxOptions': {
args: ['-start-debugger-server', '9222'],
prefs: {
'media.navigator.streams.fake': true,
'media.navigator.permission.disabled': true,
'media.peerconnection.video.h264_enabled': true,
'media.webrtc.hw.h264.enabled': true,
'media.webrtc.platformencoder': true,
'devtools.chrome.enabled': true,
'devtools.debugger.prompt-connection': false,
'devtools.debugger.remote-enabled': true,
},
},
},
chromeOptions: {
browserName: 'chrome',
'goog:chromeOptions': {
args: ['--use-fake-device-for-media-stream', '--use-fake-ui-for-media-stream'],
},
},
safariOptions: {
browserName: 'safari',
},
sauceOptions: {
platformName: process.env.PLATFORM_NAME || 'macOS 10.14',
browserVersion: process.env.BROWSER_VERSION || 'latest',
'sauce:options': {
username: process.env.SAUCE_USERNAME,
accessKey: process.env.SAUCE_ACCESS_KEY,
noSSLBumpDomains: 'all',
extendedDebugging: true,
screenResolution: '1280x960',
},
},
};

module.exports = config;
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
{
"clients": [
{
"browserName": "chrome",
"browserVersion": "latest",
"platform": "MAC"
}
]
}
44 changes: 44 additions & 0 deletions integration/mochaTests/eslintrc.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
{
"root": true,
"parser": "@typescript-eslint/parser",
"parserOptions": {
"project": ["./tsconfig.json"]
},
"plugins": [
"@typescript-eslint",
"simple-import-sort"
],
"extends": [
"plugin:@typescript-eslint/eslint-recommended",
"prettier/@typescript-eslint",
"plugin:prettier/recommended"
],
"rules": {
"no-var": "error",
"@typescript-eslint/ban-ts-comment": "off",
"@typescript-eslint/no-empty-function": "off",
"@typescript-eslint/no-inferrable-types": "off",

"@typescript-eslint/no-explicit-any": "error",
"@typescript-eslint/explicit-member-accessibility": [
"error",
{
"accessibility": "no-public",
"overrides": {
"parameterProperties": "explicit"
}
}
],
"@typescript-eslint/explicit-function-return-type": [
"error",
{ "allowExpressions": true }
],
"@typescript-eslint/no-parameter-properties": "off",
"@typescript-eslint/no-unused-vars": [
"error",
{ "argsIgnorePattern": "^_", "varsIgnorePattern": "^_" }
],
"simple-import-sort/sort": "error",
"eqeqeq": ["error", "always"]
}
}
7 changes: 7 additions & 0 deletions integration/mochaTests/jsconfig.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
{
"compilerOptions": {
"module": "commonjs",
"target": "es6"
},
"exclude": ["node_modules", "**/node_modules/*"]
}
Loading

0 comments on commit 8eb9faf

Please sign in to comment.