Testing of PIE Components

Overview

The PIE components are tested using a combination of component, accessibility, and visual tests to ensure they meet the required functionality, accessibility standards, and visual consistency across different screen sizes. This documentation provides an overview of the testing strategies and technologies used for the PIE Chip component as an example.

Testing Technologies

Playwright: A Node.js library to automate Chromium, Firefox, and WebKit with a single API. It is used for writing and running tests for the PIE components.

Percy: A visual testing tool that captures screenshots and compares them to baseline images to detect visual changes.

Axe Core: Used to automate the checking for accessibility issues, ensuring that components comply with Web Content Accessibility Guidelines (WCAG). In our testing setup, Axe Core is integrated with Playwright to perform accessibility checks on the PIE components.

Levels of Testing

Component Testing

Component tests are designed to verify the functionality and behavior of individual components in isolation. They ensure that components render correctly, handle user interactions, and emit events as expected.

Below is a basic test example from the component spec that demonstrates how to test the rendering of the PieChip component and perform a simple assertion to check its visibility.

import { test, expect } from '@sand4rt/experimental-ct-web';
import { PieChip } from '../../src/index.ts';

const componentSelector = '[data-test-id="pie-chip"]';

test.describe('PieChip - Component tests', () => {
    test('should render successfully', async ({ mount, page }) => {
        // Arrange
        await mount(PieChip, {
            slots: {
                default: 'Label',
            },
        });

        // Act
        const chip = page.locator(componentSelector);

        // Assert
        await expect(chip).toBeVisible();
    });
});

Note

We are planning to transition to using the native @playwright/test library. This change is part of our broader strategy to integrate Storybook for handling the mounting of our components, which will streamline our testing process and improve maintainability.

Accessibility (a11y) Testing

Accessibility tests ensure that components comply with Web Content Accessibility Guidelines (WCAG) to provide an inclusive experience for all users. These tests check for issues such as missing ARIA attributes, incorrect semantic markup, and other potential accessibility barriers.

Below is an example of an accessibility test for the Pie Chip component. This test uses the Axe Core accessibility testing tool to analyze the component for any WCAG compliance violations.

import { test, expect } from '@justeattakeaway/pie-webc-testing/src/playwright/webc-fixtures.ts';
import { PieChip, type ChipProps } from '../../src/index.ts';

test.describe('PieChip - Accessibility tests', () => {
    test('a11y - should test the PieChip component WCAG compliance', async ({ makeAxeBuilder, mount }) => {
        await mount(
            PieChip,
            {
                props: {} as ChipProps,
                slots: {
                    default: 'Label',
                },
            },
        );

        const results = await makeAxeBuilder().analyze();

        expect(results.violations).toEqual([]);
    });
});

Note

Our accessibility tests leverage a custom fixture built on top of the @playwright/test library. This fixture, defined in the playwright-fixtures.ts file, extends the base test functionality by providing a makeAxeBuilder function for use in accessibility test specs.

Visual Testing

Visual tests ensure PIE components maintain visual consistency across different screen sizes and variations. These tests help detect unintended visual changes that could affect the user interface's appearance and usability.

We use Percy in conjunction with Playwright to perform visual testing. Percy captures screenshots of the components and compares them to baseline images to identify any visual discrepancies. This process helps ensure that any changes to the codebase do not inadvertently alter the visual presentation of the components. Below is an example of a visual test for the PieChip component. This test captures screenshots of the component for different prop variations and compares them against baseline images:

import { test } from '@playwright/test';
import percySnapshot from '@percy/playwright';
import { percyWidths } from '@justeattakeaway/pie-webc-testing/src/percy/breakpoints.ts';
import { BasePage } from '@justeattakeaway/pie-webc-testing/src/helpers/page-object/base-page.ts';
import { variants } from '../../src/defs.ts';

variants.forEach((variant) => test(`should render all prop variations for Variant: ${variant}`, async ({ page }) => {
    const basePage = new BasePage(page, `chip--${variant}-prop-variations`);

    basePage.load();

    await page.waitForTimeout(5000);

    await percySnapshot(page, `PIE Chip - Variant: ${variant}`, percyWidths);
}));

Rendering Component Variations in Storybook

We use the createVariantStory function to generate stories that render all combinations of component properties. This function is particularly useful for visual testing as it systematically displays each variation of a component, ensuring comprehensive visual coverage.

For example, in the pie-chip.test.stories.ts file, the createVariantStory function is used to create stories for different variants of the Pie Chip component:

// Define the prop options for the matrix
const sharedPropOptions = {
    disabled: [true, false],
    isSelected: [true, false],
    isLoading: [true, false],
    isDismissible: [true, false],
    showIcon: [true, false],
    slot: ['Hello World'],
};

const defaultPropOptions = {
    ...sharedPropOptions,
    variant: ['default'],
};

const ghostPropOptions = {
    ...sharedPropOptions,
    variant: ['ghost'],
};

const outlinePropOptions = {
    ...sharedPropOptions,
    variant: ['outline'],
};

export const DefaultPropVariations = createVariantStory<Omit<ChipProps, 'aria'> >(Template, defaultPropOptions);
export const GhostPropVariations = createVariantStory<Omit<ChipProps, 'aria'>>(Template, ghostPropOptions);
export const OutlinePropVariations = createVariantStory<Omit<ChipProps, 'aria'>>(Template, outlinePropOptions);

Note

Any stories used for testing purposes should live in the pie-storybook/stories/testing directory so that they're not shown on the production Storybook deployment.