-
Notifications
You must be signed in to change notification settings - Fork 472
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Transition audio test to mocha testing framework
- Loading branch information
Kunal Nain
committed
Jun 1, 2022
1 parent
1382a28
commit 8eb9faf
Showing
17 changed files
with
2,481 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,131 @@ | ||
# Mocha Tests | ||
|
||
Currently, integration tests are being transitioned over from KITE to Mocha. `mochaTests` contains the tests that are being transitioned from KITE to Mocha. | ||
|
||
## Test Types | ||
|
||
There are two types of Mocha tests: integration and browser compatibility. | ||
|
||
### Integration tests | ||
Integration tests are minimal tests that run on latest Chrome on MacOs. These tests are used to test the basic functionality across the more popular browsers. | ||
|
||
### Browser Compatibility Tests | ||
Browser compatibility tests are used to test a feature across several different browser and OS combinations. This test is used to make sure that JS SDK features are compatible with a wide variety of browser and OS. There is a default set of browser and OS that are used for this test. In a case where a feature is not compatible with one of the browser or OS then the user can pass in a custom config. | ||
|
||
In the following section, we will go over the schema of the custom JSON config. | ||
#### JSON Config Schema | ||
|
||
```json | ||
{ | ||
"clients": [ | ||
{ | ||
/* | ||
Browser name | ||
@required | ||
*/ | ||
"browserName": "chrome", | ||
/* | ||
Browser Version | ||
@required | ||
*/ | ||
"browserVersion": "latest", | ||
/* | ||
Platform | ||
@required | ||
*/ | ||
"platform": "MAC" | ||
} | ||
] | ||
} | ||
``` | ||
## Running tests | ||
|
||
### Running Integration Test Locally | ||
You can run the integration tests locally. By default, integration tests will run on latest version of Chrome that is installed on your machine. | ||
You will have to make sure that you have the required drivers installed on your machine that the selenium test uses. You can find more information about the different drivers available at the `selenium-webdriver` npm page: [https://www.npmjs.com/package/selenium-webdriver](https://www.npmjs.com/package/selenium-webdriver). | ||
|
||
If you have an older version of driver installed then you will need to have an older version of browser on your machine as well. Generally, it is recommended to do local testing on latest version. If you need to test on older version, you can run a browser compatibility test with saucelabs. | ||
|
||
Sample command to run an integration test locally: | ||
|
||
```bash | ||
npm run test -- --test-name AudioTest --host local --test-type integration-test | ||
``` | ||
|
||
Browser compatiblity tests will run across a variety of browser and OS combinations so it is recommended to use SauceLabs for them. | ||
|
||
### Running Browser Compatibility Tests on SauceLabs | ||
To run the test on SauceLabs, the username and access key will be required for authentication. Update the following commands with the credentials and add the credentials to environment variables. | ||
```bash | ||
export SAUCE_USERNAME=<Sauce Labs account username> | ||
``` | ||
```bash | ||
export SAUCE_ACCESS_KEY=<Sauce Labs access key> | ||
``` | ||
|
||
SauceLabs will open a browser and load a test url. You need to have a serverless deployment of Chime JS SDK demo. You can set the demo url as an environment variable with the following command: | ||
```bash | ||
export TEST_URL=<JS SDK Demo URL> | ||
``` | ||
|
||
Following command can be used to run a browser compatibility tests with default settings on SauceLabs: | ||
|
||
```bash | ||
npm run test -- --test-name AudioTest --host saucelabs --test-type browser-compatibility | ||
``` | ||
|
||
There are scenarios where a test might not be compatible with one of the browsers or OS. In that case, user can provide a custom config with updated clients array. `sample_test.config.json` is a sample test config already provided. | ||
Following command can be used to run a browser compatibility test with custom config: | ||
|
||
```bash | ||
npm run test -- --test-name AudioTest --host saucelabs --test-type browser-compatibility --config browserCompatibilityTest/desktop/sample_test.config.json | ||
``` | ||
|
||
### Running Browser Compatibility Tests on DeviceFarm | ||
To run the tests on DeviceFarm, you will need an AWS account. Once you have an AWS account, you will need to setup two environment variables that will allow the AWS SDK to authenticate to your AWS account. | ||
```bash | ||
export AWS_ACCESS_KEY_ID=<YOUR_ACCESS_KEY_ID> | ||
export AWS_SECRET_ACCESS_KEY=<YOUR_SECRET_ACCESS_KEY> | ||
``` | ||
|
||
You will need to create a DeviceFarm project in the AWS account that you are planning to use. There are two types of devicefarm projects that can be created: `Mobile Device Testing` and `Desktop Browser Testing`. For this section, we will focus mainly on desktop browser testing. | ||
After the project is created you will need to setup a `PROJECT_ARN` as an environment variable. Project arn is used by the device farm API to identify the project to create test sessions inside. | ||
```bash | ||
export PROJECT_ARN=<YOUR_PROJECT_ARN> | ||
``` | ||
|
||
Following command can be used to run a browser compatibility test with default settings on DeviceFarm: | ||
```bash | ||
npm run test -- --test-name AudioTest --host devicefarm --test-type browser-compatibility | ||
``` | ||
|
||
Just like with SauceLabs, DeviceFarm can run browser compatibility tests with a custom config: | ||
```bash | ||
npm run test -- --test-name AudioTest --host devicefarm --test-type browser-compatibility --config browserCompatibilityTest/desktop/sample_test.config.json | ||
``` | ||
|
||
## Logging | ||
|
||
In the integration tests, logging is implemented in a unique way. This section will go over the two distinct ways of logging: | ||
1. Use `logger.log` for inline logging | ||
2. Use `logger.pushLogs` for buffered logging | ||
|
||
Developers can use `logger.log` as a normal console log with some enhanced functionality. The main difference between `logger.log` and `console.log` is that logger log will take `LogLevel` as a parameter. Different log levels print to the console in different colors and have a minor difference in the logs which makes it easier to read the logs. There are four different types of log levels. | ||
```ts | ||
const LogLevel = { | ||
INFO: 'INFO', | ||
WARN: 'WARN', | ||
ERROR: 'ERROR', | ||
SUCCESS: 'SUCCESS', | ||
}; | ||
``` | ||
|
||
Integration tests use `mocha` as their test framework. Mocha prints the test results and some limited logs to the console automatically. Any console logs that are inside of `it` will print before the mocha logs. | ||
This makes debugging hard as the logs seem out of sync and random. To get around this issue buffered logging was added. Inside of `it` blocks, `logger.pushLogs` can be used to add logs to the buffer and a call to `logger.printLogs` inside of the `afterEach` hook will print the logs in the desired order. | ||
```ts | ||
afterEach(async function () { | ||
this.logger.printLogs(); | ||
}); | ||
``` | ||
|
||
Mocha provides hooks like `before`, `after`, `beforeEach`, `afterEach` to perform actions at specific time of the testing cycle. Any logs printed inside the hooks will print in line so inline logging can be used and there is no need for buffered logging. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,40 @@ | ||
const config = { | ||
firefoxOptions: { | ||
browserName: 'firefox', | ||
'moz:firefoxOptions': { | ||
args: ['-start-debugger-server', '9222'], | ||
prefs: { | ||
'media.navigator.streams.fake': true, | ||
'media.navigator.permission.disabled': true, | ||
'media.peerconnection.video.h264_enabled': true, | ||
'media.webrtc.hw.h264.enabled': true, | ||
'media.webrtc.platformencoder': true, | ||
'devtools.chrome.enabled': true, | ||
'devtools.debugger.prompt-connection': false, | ||
'devtools.debugger.remote-enabled': true, | ||
}, | ||
}, | ||
}, | ||
chromeOptions: { | ||
browserName: 'chrome', | ||
'goog:chromeOptions': { | ||
args: ['--use-fake-device-for-media-stream', '--use-fake-ui-for-media-stream'], | ||
}, | ||
}, | ||
safariOptions: { | ||
browserName: 'safari', | ||
}, | ||
sauceOptions: { | ||
platformName: process.env.PLATFORM_NAME || 'macOS 10.14', | ||
browserVersion: process.env.BROWSER_VERSION || 'latest', | ||
'sauce:options': { | ||
username: process.env.SAUCE_USERNAME, | ||
accessKey: process.env.SAUCE_ACCESS_KEY, | ||
noSSLBumpDomains: 'all', | ||
extendedDebugging: true, | ||
screenResolution: '1280x960', | ||
}, | ||
}, | ||
}; | ||
|
||
module.exports = config; |
9 changes: 9 additions & 0 deletions
9
integration/mochaTests/configs/browserCompatibilityTest/desktop/sample_test.config.json
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,9 @@ | ||
{ | ||
"clients": [ | ||
{ | ||
"browserName": "chrome", | ||
"browserVersion": "latest", | ||
"platform": "MAC" | ||
} | ||
] | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,44 @@ | ||
{ | ||
"root": true, | ||
"parser": "@typescript-eslint/parser", | ||
"parserOptions": { | ||
"project": ["./tsconfig.json"] | ||
}, | ||
"plugins": [ | ||
"@typescript-eslint", | ||
"simple-import-sort" | ||
], | ||
"extends": [ | ||
"plugin:@typescript-eslint/eslint-recommended", | ||
"prettier/@typescript-eslint", | ||
"plugin:prettier/recommended" | ||
], | ||
"rules": { | ||
"no-var": "error", | ||
"@typescript-eslint/ban-ts-comment": "off", | ||
"@typescript-eslint/no-empty-function": "off", | ||
"@typescript-eslint/no-inferrable-types": "off", | ||
|
||
"@typescript-eslint/no-explicit-any": "error", | ||
"@typescript-eslint/explicit-member-accessibility": [ | ||
"error", | ||
{ | ||
"accessibility": "no-public", | ||
"overrides": { | ||
"parameterProperties": "explicit" | ||
} | ||
} | ||
], | ||
"@typescript-eslint/explicit-function-return-type": [ | ||
"error", | ||
{ "allowExpressions": true } | ||
], | ||
"@typescript-eslint/no-parameter-properties": "off", | ||
"@typescript-eslint/no-unused-vars": [ | ||
"error", | ||
{ "argsIgnorePattern": "^_", "varsIgnorePattern": "^_" } | ||
], | ||
"simple-import-sort/sort": "error", | ||
"eqeqeq": ["error", "always"] | ||
} | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,7 @@ | ||
{ | ||
"compilerOptions": { | ||
"module": "commonjs", | ||
"target": "es6" | ||
}, | ||
"exclude": ["node_modules", "**/node_modules/*"] | ||
} |
Oops, something went wrong.