Summary
General principles of testing
- Testing should resemble the way it is going to be used
- When writing tests, you have to make sure that you write them from a perspective of someone that uses your code. If it is a backend REST APIs, then you want to make sure that your business cases are covered, and those should be reflected in your test. If you are testing frontend, it should cover the UX.
- Before you write a test, think about the instructions you would give to a tester to test your software.
- Avoid nesting tests, because they are not very readable, and also they do not help you with test isolation.
Test Isolation
All (or at least most) test should be written in isolation. That means that
add something more here...
Setup Functions
Setup function is a function that abstracts away the utilities for testing. This is useful for UI testing as well as for backend testing. This should be done when you have a lot of tests that do the same thing.
Example
import { render } from '@testing-library/react'
import userEvent from '@testing-library/user-event'
import * as React from 'react'
import Login from '../login'
async function setup() {
const handleSubmit = jest.fn()
const utils = await render(<Login onSubmit={handleSubmit} />)
const user = { username: 'michelle', password: 'smith' }
const changeUsernameInput = value =>
userEvent.type(utils.getByLabelText(/username/i), value)
const changePasswordInput = value =>
userEvent.type(utils.getByLabelText(/password/i), value)
const clickSubmit = () => userEvent.click(utils.getByText(/submit/i))
return {
...utils,
handleSubmit,
user,
changeUsernameInput,
changePasswordInput,
clickSubmit,
}
}
async function setupSuccessCase() {
const utils = await setup()
utils.changeUsernameInput(utils.user.username)
utils.changePasswordInput(utils.user.password)
utils.clickSubmit()
return utils
}
async function setupWithNoPassword() {
const utils = await setup()
utils.changeUsernameInput(utils.user.username)
utils.clickSubmit()
const errorMessage = utils.getByRole('alert')
return { ...utils, errorMessage }
}
async function setupWithNoUsername() {
const utils = await setup()
utils.changePasswordInput(utils.user.password)
utils.clickSubmit()
const errorMessage = utils.getByRole('alert')
return { ...utils, errorMessage }
}
test('calls onSubmit with the username and password', async () => {
const { handleSubmit, user } = await setupSuccessCase()
expect(handleSubmit).toHaveBeenCalledTimes(1)
expect(handleSubmit).toHaveBeenCalledWith(user)
})
test('shows an error message when submit is clicked and no username is provided', async () => {
const { handleSubmit, errorMessage } = await setupWithNoUsername()
expect(errorMessage).toHaveTextContent(/username is required/i)
expect(handleSubmit).not.toHaveBeenCalled()
})
test('shows an error message when password is not provided', async () => {
const { handleSubmit, errorMessage } = await setupWithNoPassword()
expect(errorMessage).toHaveTextContent(/password is required/i)
expect(handleSubmit).not.toHaveBeenCalled()
}) Testing Coverage
- It shows which lines of your code the test is run against and which ones still need to be tested
- It is composed of the following
- Statements: Statements that we reun at least once
- Branches: Percentage of branches that were run. Branch is one part of conditional statement like
if statementorternary - Functions: percentage of fns that were run at least once
- Lines: percentage of lines that were run at least once
You should never aim for 100%, because it give you diminishing returns (unless it is a library)
Mocking
- Mocking is a technique that allows you to override the functionality of a 3rd party service whose code you did not write.
- Mocking is usually used while testing your application but also when you want to develop in isolation (offline)
- You have to be careful with what you mock, because the more you mock, the less confident you can be that your software is going to work.
Mocking and testing
- When writing mocks for testing, you want to mock 3rd party APIs, especially those that can cost you money every time you run a test (email service, payment service) and store that in a
JSONfile.
E2E Testing
- It is a form of testing that resembles user experience all the way from the backend to the frontend and ensures that the application works as a whole. It can replace a human being doing manual tests and they can be run every time you make changes to your codebase and deploy the application
Tools
Playwright
It is an E2E framework for testing application.
Browser Context
It is an API that allows you to add context to a browser. This is useful for adding JWT tokens or cookies that are required for a test
- Pros
- Uses Browser Contexts for each test which helps with Test isolation
- Cons
Authenticated E2E Test
It is a test that omits authentication and allows you to test the authenticated part of it (which is most of the application). You want to avoid testing authentication for every test that requires user authentication, because that will slow your tests down. This part of your application should be tested separately and if you do have a test, you can just skip it in other parts.
Authenticated E2E Test with Playwright
To write an authenticated E2E test with Playwright, you can use Browser Context and add either your token to localStorage or to add cookies.
test('Users can add 2FA to their account and use it when logging in', async ({
page,
insertNewUser,
}) => {
const password = faker.internet.password()
const user = await insertNewUser({ password })
invariant(user.name, 'User name is not defined')
const session = await prisma.session.create({
data: {
expirationDate: getSessionExpirationDate(),
userId: user.id,
},
})
const cookieSession = await sessionStorage.getSession(session.id)
cookieSession.set(sessionKey, session.id)
const cookieConfig = cookieParser.parseString(
await sessionStorage.commitSession(cookieSession),
) as any
page.context().addCookies([{ ...cookieConfig, domain: 'localhost' }])
await page.goto('/settings/profile')
await page.getByRole('link', { name: /enable 2fa/i }).click()
await expect(page).toHaveURL(`/settings/profile/two-factor`)
const main = page.getByRole('main')
await main.getByRole('button', { name: /enable 2fa/i }).click()
const otpUrlString = await main
.getByLabel(/One-time Password URI/i)
.textContent()
invariant(otpUrlString, 'OTP URL is not defined')
const otpUrl = new URL(otpUrlString)
const options = Object.fromEntries(otpUrl.searchParams.entries())
const { otp } = generateTOTP(options)
await main.getByRole('textbox', { name: /code/i }).fill(otp)
await main.getByRole('button', { name: /submit/i }).click()
await expect(page).toHaveURL('/settings/profile/two-factor')
await page.goto('/settings/profile')
await page.getByRole('button', { name: /logout/i }).click()
await page.getByRole('link', { name: /login/i }).click()
await expect(page).toHaveURL('/login')
await page.getByRole('textbox', { name: /username/i }).fill(user.username)
await page.getByRole('textbox', { name: /password/i }).fill(password)
await page.getByRole('button', { name: /log in/i }).click()
await expect(page).toHaveURL(/verify*/i)
const { otp: otp2 } = generateTOTP(options)
await page.getByRole('textbox', { name: /code/i }).fill(otp2)
await page.getByRole('button', { name: /submit/i }).click()
await expect(page).toHaveURL('/')
}) Unit Testing
It is a test that tests a small unit of logic. Most of the time, it will be a function that you write. You should write unit tests for logic that have following characteristics:
- It is complex and there are a lot of edge cases
- It is reusable and if it breaks, it can have disastrous consequences for the business. It can be conversion from cents to gbp etc.
Tools
Vitest
It is a library that is used for writing unit tests but it also works with react testing library.
vitest.config.ts
/// <reference types="vitest" />
import react from '@vitejs/plugin-react'
import { defineConfig } from 'vite'
export default defineConfig({
test: {
include: ['./app/**/*.test.{ts,tsx}'],
setupFiles: ['./tests/setup/setup-test-env.ts'],
globalSetup: ['./tests/setup/global-setup.ts'],
restoreMocks: true,
coverage: {
include: ['app/**/*.{ts,tsx}'],
all: true,
},
},
}) Spy
Spy is a function that allows you to override how the function was called and change the implementation of it for testing purposes.
When using spy, you should check how many times a function has been called, because if you do not expect it to be called more than once, it is a bug.
const apples = 0
const cart = {
getApples: () => 13
}
test("", () => {
const spy = vi.spyOn(card, "getApples")
// The below overrides the returned value of `getApples` method
spy.mockImplementation(() => apples)
apples = 1
expect(cart.getApples()).toBe(1)
expect(cart.getApples()).toHaveBeencalledTimes(1)
// Here, it is restoring the original returned value of a function
spy.mockRestore()
expect(cart.getApples()).toBe(13)
}) Typing Spies
To satisfy typescript when assigning spies to a variable, you can use SpyInstance<Parameters<typeof console.error>>
Hooks
Hooks are functions that trigger during a particular moment within a test running cycle
beforeAll
beforeAll can be used when you want to run a function before all your tests. It is only called once.
import {beforeAll} from "vitest"
beforeAll(() => {
// .. Do something once, like environment setup.
}) beforeEach
beforeEach is used when you want to run a function every time a test runs.
import {beforeEach} from "vitest"
beforeEach(() => {
// Do something every time a new test runs
}) afterAll
afterAll is similar to beforeAll, the difference is that i runs after all test finish running. This is useful for closing your database connection or cleanup your mocks
import {afterAll} from "vitest"
afterAll(() => {
// logic to cleanup your test setup
}) afterEach
afterEach is like beforeEach, but it runs after each test. It is useful for cleaning up your mocks
afterEach(() => {
vi.mockRestore()
// logic to cleanup your mocks
}) Setup Environment
Setup Files
Setup files are files where you place your logic that you want to run before each test file. This is good for your global setup environment of your test. It can be reseting handlers, cleaning up react testing library or global mocks.
import 'dotenv/config'
import './db-setup.ts'
import '#app/utils/env.server.ts'
import '@testing-library/jest-dom/vitest'
import { installGlobals } from '@remix-run/node'
import { cleanup } from '@testing-library/react'
import { afterEach, beforeEach, vi, type SpyInstance } from 'vitest'
import { server } from '../mocks/index.ts'
import './custom-matchers.ts'
installGlobals()
afterEach(() => server.resetHandlers())
afterEach(() => cleanup())
export let consoleError: SpyInstance<Parameters<typeof console.error>>
beforeEach(() => {
const originalConsoleError = console.error
consoleError = vi.spyOn(console, 'error')
consoleError.mockImplementation(
(...args: Parameters<typeof console.error>) => {
originalConsoleError(...args)
throw new Error(
'Console error was called. Call consoleError.mockImplementation(() => {}) if this is expected.',
)
},
)
}) globalSetup
Global setup is a file where you can add all logic that should run before your tests run. It can be spinning up a database, connecting to external services and anything that absolutely must be running during the test.
import path from 'node:path'
import { execaCommand } from 'execa'
import fsExtra from 'fs-extra'
export const BASE_DATABASE_PATH = path.join(
process.cwd(),
'./tests/prisma/base.db',
)
export async function setup() {
const exists = await fsExtra.pathExists(BASE_DATABASE_PATH)
if (exists) return
await execaCommand(
'prisma migrate reset --force --skip-seed --skip-generate',
{
stdio: 'inherit',
env: { ...process.env, DATABASE_URL: `file:${BASE_DATABASE_PATH}` },
},
)
} UI Testing
General Principles of UI testing
- When testing the UI, you should always aim to grab DOM elements using roles, because by doing so, you also check that your application is accessible.
Component Testing
Component testing is a type of test that tests the UI component. Those tests focus on two users:
- The end user - we want to check that the user, by doing actions will be able to get the result they expect
- Developer - we want to make sure that when certain values passed to our component, component works as expected.
Tools
React Testing Library
React Testing Library is a plugin for vitest and jest that allows you test your UI. It simulates the DOM using jsdom. On top of that, it provides you with DOM assertions that allow you to assert DOM elements.
/**
* @vitest-environment jsdom
*/
import { renderHook, screen } from '@testing-library/react'
import { userEvent } from '@testing-library/user-event'
import { useState } from 'react'
import { expect, test } from 'vitest'
import { useCounter } from './use-counter'
function TestComponent() {
const { count, increment, decrement } = useCounter(0)
return (
<div>
<output>{count}</output>
<button onClick={increment}>+</button>
<button onClick={decrement}>-</button>
</div>
)
}
test('increments and decrements', async () => {
const user = userEvent.setup()
await render(<TestComponent />)
expect(screen.getByRole('status').textContent).toBe(0)
await user.click(screen.getByRole('button', { name: '+' }))
expect(screen.getByRole('status').textContent).toBe(1)
await user.click(screen.getByRole('button', { name: '-' }))
expect(screen.getByRole('status').textContent).toBe(0)
}) When to use queryBy/AllBy and findBy/getBy
When you want to assert elements that you know that exist, you should use findBy or getBy, because it returns the element. If you want to assert non-existence of an element, you should use queryBy/AllBy, because it returns an array of elements, and you can just check the length of an array.
Clean up after every test.
It is important to clean up after every UI test to maintain test isolation. RTL provides you with cleanup function that you can run in afterEach hook of every test, or you can add it to your global setup file.
Hooks Testing
Hooks testing is testing the logic of your React hook. Hooks should only be tested when the logic is business critical or logic within it is complicated. There are two ways of testing hooks
Using renderHook utility
To be able to test your hook, you need use renderHook provided by React Testing Library. This allows you to inject the hook within a fake component that it creates to manage the lifecycle of a component for you.
test('hook: prevents default on the first click, and does not on the second', async () => {
const { result } = await renderHook(() => useDoubleCheck())
expect(result.current.doubleCheck).toBe(false)
const mock = vi.fn()
const event = new MouseEvent('click', {
bubbles: true,
cancelable: true,
}) as unknown as React.MouseEvent<HTMLButtonElement>
act(() => result.current.getButtonProps({ onClick: mock }).onClick(event))
expect(result.current.doubleCheck).toBe(true)
expect(mock).toHaveBeenCalledOnce()
expect(event.defaultPrevented).toBe(true)
mock.mockClear()
const event2 = new MouseEvent('click', {
bubbles: true,
cancelable: true,
}) as unknown as React.MouseEvent<HTMLButtonElement>
act(() => result.current.getButtonProps({ onClick: mock }).onClick(event2))
expect(result.current.doubleCheck).toBe(true)
expect(mock).toHaveBeenCalledOnce()
expect(event2.defaultPrevented).toBe(false)
}) Gotchas
When using it, you have to remember to always deconstruct the object to get result property (it is because of how JS works).
Usage within a component
You can also test your hooks in a more traditional way by creating a test component that uses this hook, and html elements, that you can use to check whether it works or not.
test('TestComponent: prevents default on the first click, and does not on the second', async () => {
const user = userEvent.setup()
await render(<TestComponent />)
const button = screen.getByRole('button')
const output = screen.getByRole('status')
expect(button).toHaveTextContent(/submit/i)
await user.click(button)
expect(button).toHaveTextContent('Are you sure?')
expect(output).toHaveTextContent('yes')
await user.click(button)
expect(button).toHaveTextContent('Are you sure?')
expect(output).toHaveTextContent('no')
})
test('blurring the button starts things over', async () => {
const user = userEvent.setup()
await render(<TestComponent />)
const status = screen.getByRole('status')
const button = screen.getByRole('button')
await user.click(button)
expect(button).toHaveTextContent('You sure?')
expect(status).toHaveTextContent('Default Prevented: yes')
await user.click(document.body)
// button goes back to click me
expect(button).toHaveTextContent('Click me')
// our callback wasn't called, so the status doesn't change
expect(status).toHaveTextContent('Default Prevented: yes')
})
test('hitting "escape" on the input starts things over', async () => {
const user = userEvent.setup()
await render(<TestComponent />)
const status = screen.getByRole('status')
const button = screen.getByRole('button')
await user.click(button)
expect(button).toHaveTextContent('You sure?')
expect(status).toHaveTextContent('Default Prevented: yes')
await user.keyboard('{Escape}')
// button goes back to click me
expect(button).toHaveTextContent('Click me')
// our callback wasn't called, so the status doesn't change
expect(status).toHaveTextContent('Default Prevented: yes')
}) HTTP Mocking
HTTP Mocking when writing tests for your frontend is important when you want to test components that make requests, or you want to test a component that has 3rd party code in it that makes requests. Also, HTTP Mocking helps you to work offline when the system goes down, because you do not have to rely on the system.
Tools
MSW
MSW is a library that allows you to intercept your requests and change their implementation.
MSW Setup
import { rest, HttpResponse } from 'msw'
import { setupServer } from 'msw/node'
const { json } = HttpResponse
const server = setupServer(
http.get('https://example.com/book/:bookId', () => {
return json({ title: 'Lord of the Rings' })
}),
)
beforeAll(() => {
server.listen()
})
afterAll(() => {
server.close()
}) Overriding request handlers
You can override reset handlers with server.use. This is very useful when you want to write a test for an error.
const server = setupServer(
http.get('https://example.com/book/:bookId', () => {
return json({ title: 'Lord of the Rings' })
}),
)
afterEach(() => {
// Resets all runtime request handlers added via `server.use`
server.resetHandlers()
})
beforeAll(() => {
server.listen()
})
afterAll(() => {
server.close()
})
test('handles 500 errors from the API', () => {
server.use(
// Adds a runtime request handler for posting a new book review.
http.get('https://example.com/book/:bookId', () => {
return json({ error: 'Something went wrong' }, { status: 500 })
}),
)
// Performing a "GET /book/:bookId" request
// will get the mocked response from our runtime request handler.
})
test('asserts another scenario', () => {
// Since we call `server.resetHandlers()` after each test,
// this test suite will use the default GET /book/:bookId response.
}) Using database for HTTP Mocks
To make sure that your test is resembling how your system works and to avoid having to mock your database, you should have a copy of a database for each of your tests. This helps with test isolation.
When setting up your database, you have to make sure your environment is setup before you run the test. To do that, you can use globalSetup. You then, have to import it to your setupFile on top of the file, because it must run before anything else.
Database Setup for test
To setup database for your test, you have to do the following:
- Create a base of a database with required seeds etc. so you can copy it for every test instead of creating a new one and doing it every time you run a test
- Make sure you override the database URL variable (or any other variable that is needed for your ORM)
- Reset database before all tests start to avoid situations where during seeding process, test errors, because the entity in a database already exists
- After each test, you delete everything from the database to clean up after yourself
- Disconnect from the database once you finish running the test.
setupFiles
import path from 'node:path'
import { execaCommand } from 'execa'
import { afterAll, beforeAll, beforeEach } from 'vitest'
import {BASE_DATABASE_PATH} from "globalSetupFile"
const databaseFile = `./tests/prisma/data.${VITEST_POOOL_ID || 0}.db`
const databasePath = path.join(process.cwd(), databaseFile)
process.env.DATABASE_URL = databasePath
beforeAll(async () => {
await fsExtra.copyFile(BASE_DATABASE_PATH, databaseFile)
})
beforeEach(async () => {
const { prisma } = await import('#app/utils/db.server.ts')
await prisma.user.deleteMany()
})
afterAll(async () => {
const { prisma } = await import('#app/utils/db.server.ts')
await prisma.$disconnect()
}) globalSetup
import path from 'node:path'
import { execaCommand } from 'execa'
import fsExtra from 'fs-extra'
export const BASE_DATABASE_PATH = path.join(
process.cwd(),
'./tests/prisma/base.db',
)
export async function setup() {
const exists = await fsExtra.pathExists(BASE_DATABASE_PATH)
if (exists) return
await execaCommand(
'prisma migrate reset --force --skip-seed --skip-generate',
{
stdio: 'inherit',
env: { ...process.env, DATABASE_URL: `file:${BASE_DATABASE_PATH}` },
},
)
} VITEST
When running test with Vitest and creating files dynamically, you have to make sure you use VITEST_POOL_ID as a variable to not override a file when running a test.
Notes
- When writing tests you want to make sure that the human can use your application, not your computer.
- When trying to decide how to write a test, take a step back and think how you would do it if you were given a list of instruction to a human tester.
- Test should resemble the way softer is used to give you a confidence that it is working.
E2E testing
- testing an app from the user’s experience all the way to the backend components
- It is basically replacing a human person for testing purposes
- When testing use roles instead of ids etc. you are testing whether you application is accessible as well
Playwright
Playwright is an E2E framework for testing application
Fixtures
Fixtures is a functionality in Playwright that allows you to establish an environment for each test and create this in isolation. You can pass functions that will be reused in each test. They also help to group tests based on their meaning.
https://playwright.dev/docs/test-fixtures
Test example
import { expect, test } from '@playwright/test'
import { prisma } from '#app/utils/db.server.ts'
import { createUser } from '../db-utils.ts'
// 🐨 get a prisma client from #app/utils/db.server.ts
// 🐨 get the createUser util from ../db-utils.ts
test('Search from home page', async ({ page }) => {
// 🐨 create a new user in the database.
const user = await prisma.user.create({
select: { username: true, name: true },
data: createUser(),
})
await page.goto('/')
// 🐨 fill in the new user's username
await page.getByRole('searchbox', { name: /search/i }).fill(user.username)
await page.getByRole('button', { name: /search/i }).click()
// 🐨 swap out "kody" for the user's username
// 💯 handle encoding this properly using URLSearchParams
const searchParams = new URLSearchParams({ search: user.username }).toString()
await page.waitForURL(`/users?${searchParams}`)
await expect(page.getByText('Epic Notes Users')).toBeVisible()
const userList = page.getByRole('main').getByRole('list')
await expect(userList.getByRole('listitem')).toHaveCount(1)
// 🐨 update this to be user's name (with a fallback to their username)
await expect(page.getByAltText(user.name ?? user.username)).toBeVisible()
await page.getByRole('searchbox', { name: /search/i }).fill('__nonexistent__')
await page.getByRole('button', { name: /search/i }).click()
await page.waitForURL(`/users?search=__nonexistent__`)
await expect(userList.getByRole('listitem')).not.toBeVisible()
await expect(page.getByText(/no users found/i)).toBeVisible()
}) Fixtures
- Fixtures are a fixed state of the software under test used as a baseline for running test cases
- they are used to give the test everything it needs and nothing more
- They are isolated between tests. They can be grouped based on their meaning instead of their common setup
import { test as base } from '@playwright/test'
import { prisma } from '#app/utils/db.server.ts'
import { createUser } from '../db-utils.ts'
const test = base.extend<{
insertNewUser(): Promise<{
id: string
name: string | null
username: string
}>
}>({
insertNewUser: async ({}, use) => {
const userData = createUser()
let userId: string | undefined = undefined
await use(async () => {
const newUser = await prisma.user.create({
select: { id: true, name: true, username: true },
data: userData,
})
userId = newUser.id
return newUser
})
await prisma.user.deleteMany({ where: { id: userId } })
},
}) E2E Mocking
- You should be mocking 3rd party APIs, especially those that could possibly cost you money to run tests against.
- However, you have to be careful how many services you mock, because the more you mock, the less confidence you have
- remember, you tests should resemble the way your application will be used
Example
For example, you may want to mock an email API that sends an email with a verification. In this case, you may want to save that email as JSON file somewhere and read that file so that you can read it in your test.
test('onboarding with link', async ({ page, getOnboardingData }) => {
const onboardingData = getOnboardingData()
await page.goto('/')
await page.getByRole('link', { name: /log in/i }).click()
await expect(page).toHaveURL(`/login`)
const createAccountLink = page.getByRole('link', {
name: /create an account/i,
})
await createAccountLink.click()
await expect(page).toHaveURL(`/signup`)
await page.getByRole('textbox', { name: /email/i }).fill(onboardingData.email)
await page.getByRole('button', { name: /submit/i }).click()
await expect(
page.getByRole('button', { name: /submit/i, disabled: true }),
).toBeVisible()
await expect(page.getByText(/check your email/i)).toBeVisible()
const email = await waitFor(() => {
return readEmail(onboardingData.email.toLowerCase())
// 🐨 once you've implemented your requireEmail function in the resend mock
// use it here to get the email that was set to the onboardingData.email
})
expect(email.to).toBe(onboardingData.email.toLowerCase())
expect(email.from).toBe('hello@epicstack.dev')
expect(email.subject).toMatch(/welcome/i)
const onboardingUrl = extractUrl(email.text)
invariant(onboardingUrl, 'Onboarding URL not found')
await page.goto(onboardingUrl)
await expect(page).toHaveURL(`/onboarding`)
await page
.getByRole('textbox', { name: /^username/i })
.fill(onboardingData.username)
await page.getByRole('textbox', { name: /^name/i }).fill(onboardingData.name)
await page.getByLabel(/^password/i).fill(onboardingData.password)
await page.getByLabel(/^confirm password/i).fill(onboardingData.password)
await page.getByLabel(/terms/i).check()
await page.getByLabel(/remember me/i).check()
await page.getByRole('button', { name: /Create an account/i }).click()
await expect(page).toHaveURL(`/`)
await page.getByRole('link', { name: onboardingData.name }).click()
await expect(page).toHaveURL(`/users/${onboardingData.username}`)
await page.getByRole('button', { name: /logout/i }).click()
await expect(page).toHaveURL(`/`)
}) Authenticated E2E
- If you want to write a test where the user is authenticated (which is mostly 90% of the application), you have to mock this bit. With Playwright, you can just write to a bowser context https://playwright.dev/docs/api/class-browsercontext#browser-context-add-cookies. You can create a fixture that does it for you.
test('Users can add 2FA to their account and use it when logging in', async ({
page,
insertNewUser,
}) => {
const password = faker.internet.password()
const user = await insertNewUser({ password })
invariant(user.name, 'User name is not defined')
const session = await prisma.session.create({
data: {
expirationDate: getSessionExpirationDate(),
userId: user.id,
},
})
const cookieSession = await sessionStorage.getSession(session.id)
cookieSession.set(sessionKey, session.id)
const cookieConfig = cookieParser.parseString(
await sessionStorage.commitSession(cookieSession),
) as any
page.context().addCookies([{ ...cookieConfig, domain: 'localhost' }])
await page.goto('/settings/profile')
await page.getByRole('link', { name: /enable 2fa/i }).click()
await expect(page).toHaveURL(`/settings/profile/two-factor`)
const main = page.getByRole('main')
await main.getByRole('button', { name: /enable 2fa/i }).click()
const otpUrlString = await main
.getByLabel(/One-time Password URI/i)
.textContent()
invariant(otpUrlString, 'OTP URL is not defined')
const otpUrl = new URL(otpUrlString)
const options = Object.fromEntries(otpUrl.searchParams.entries())
const { otp } = generateTOTP(options)
await main.getByRole('textbox', { name: /code/i }).fill(otp)
await main.getByRole('button', { name: /submit/i }).click()
await expect(page).toHaveURL('/settings/profile/two-factor')
await page.goto('/settings/profile')
await page.getByRole('button', { name: /logout/i }).click()
await page.getByRole('link', { name: /login/i }).click()
await expect(page).toHaveURL('/login')
await page.getByRole('textbox', { name: /username/i }).fill(user.username)
await page.getByRole('textbox', { name: /password/i }).fill(password)
await page.getByRole('button', { name: /log in/i }).click()
await expect(page).toHaveURL(/verify*/i)
const { otp: otp2 } = generateTOTP(options)
await page.getByRole('textbox', { name: /code/i }).fill(otp2)
await page.getByRole('button', { name: /submit/i }).click()
await expect(page).toHaveURL('/')
}) Unit Testing
- Function/logic that necessitate their own test should have following characteristics:
- It is really complex and has many edge cases
- It is widely used and would be disastrous if it broke
Mocking
- Writing tests for functions that do not have side effects is usually straight forward
- That is why on the unit level, you have to mock implementation of functions that are used within your function. It may be browser APIs, modules, functions or fetch requests.
Vitest
Vitest is a library that is used primarly for unit testing.
Here is a setup
/// <reference types="vitest" />
import react from '@vitejs/plugin-react'
import { defineConfig } from 'vite'
export default defineConfig({
plugins: [react()],
css: { postcss: { plugins: [] } },
test: {
include: ['./app/**/*.test.{ts,tsx}'],
restoreMocks: true,
coverage: {
include: ['app/**/*.{ts,tsx}'],
all: true,
},
},
})
Spies
Spy is a function that records how it was called and allows you to change the implementation.
let apples = 0
const cart = {
getApples: () => 13,
}
const spy = vi.spyOn(cart, 'getApples')
spy.mockImplementation(() => apples)
apples = 1
expect(cart.getApples()).toBe(1)
expect(spy).toHaveBeenCalledTimes(1)
// then you can clean things up like so:
spy.mockRestore()
expect(cart.getApples()).toBe(13) In the example above, we are spying on cart.getApples, then we change the implementation so that it returns the value from apples variable. We then change the value of apples, and check whether it is correct or not.
Also, it is important to check how many times something has been called, because if we do not expect it to be called more than once, it means that there’s a bug.
test('undefined falls back to Unknown', () => {
const spy = vi.spyOn(console, 'error')
spy.mockImplementation(() => {})
expect(getErrorMessage(undefined)).toBe('Unknown Error')
expect(spy).toHaveBeenCalledOnce()
expect(spy).toHaveBeenCalledWith(
'Unable to get error message for error',
undefined,
)
spy.mockRestore()
}) To satisfy Typescript when assigning spies to a variable, you can use this
SpyInstance<Parameters<typeof console.error>>
Hooks
import { beforeEach, afterEach, beforeAll, afterAll } from 'vitest'
beforeAll(async () => {
// do something before all tests
})
afterAll(async () => {
// do something after all tests
})
beforeEach(async () => {
// do something before each test
})
afterEach(async () => {
// do something after each test
}) Below is an example of how to use hooks and spies together
import { faker } from '@faker-js/faker'
import { expect, test, vi, type SpyInstance, beforeEach } from 'vitest'
import { getErrorMessage } from './misc.tsx'
let consoleError: SpyInstance<Parameters<typeof console.error>>
beforeEach(() => {
const originalConsoleError = console.error
consoleError = vi.spyOn(console, 'error')
consoleError.mockImplementation(
(...args: Parameters<typeof console.error>) => {
originalConsoleError(...args)
throw new Error(
'console.error was called. If you expect this, call consoleError.mockImplementation(() => {}) in your test',
)
},
)
})
test('Error object returns message', () => {
const message = faker.lorem.words(2)
expect(getErrorMessage(new Error(message))).toBe(message)
})
test('String returns itself', () => {
const message = faker.lorem.words(2)
expect(getErrorMessage(message)).toBe(message)
})
test('undefined falls back to Unknown', () => {
consoleError.mockImplementation(() => {})
expect(getErrorMessage(undefined)).toBe('Unknown Error')
expect(consoleError).toHaveBeenCalledWith(
'Unable to get error message for error',
undefined,
)
expect(consoleError).toHaveBeenCalledTimes(1)
}) The more your tests resemble the way your software is used, the more confidence they can give you. - @kentcdodds
Setup Environment
- You want to create a global setup environment for your tests
- https://vitest.dev/config/#setupfiles
- Essentially, whatever is in this file, it will run while running tests
Component Testing
- When writing component tests, you have to focus on two users
- The end user
- Developer
- While using testing library/react, you simulate DOM using
jsdom. - Dom assertion with
testing-library/jest-domallow you to assert DOM element.
Example
/**
* @vitest-environment jsdom
*/
import { renderHook, screen } from '@testing-library/react'
import { userEvent } from '@testing-library/user-event'
import { useState } from 'react'
import { expect, test } from 'vitest'
import { useCounter } from './use-counter'
function TestComponent() {
const { count, increment, decrement } = useCounter(0)
return (
<div>
<output>{count}</output>
<button onClick={increment}>+</button>
<button onClick={decrement}>-</button>
</div>
)
}
test('increments and decrements', async () => {
const user = userEvent.setup()
await render(<TestComponent />)
expect(screen.getByRole('status').textContent).toBe(0)
await user.click(screen.getByRole('button', { name: '+' }))
expect(screen.getByRole('status').textContent).toBe(1)
await user.click(screen.getByRole('button', { name: '-' }))
expect(screen.getByRole('status').textContent).toBe(0)
}) In fact, the only situation where you ever use queryBy* or queryAllBy* queries is when you’re trying to assert on non-existence of elements. Otherwise, you should be using the other variants (get* or find*). For more details, refer to the docs.
- If you want to know how the dom looks like while testing, you can use
screen.debug()
Coverage
- Coverage shows you which lines of your code is run against or your test and which ones still need to be tested
- coverage report consists of four metrics
- Statements: of statements that we re run at least once
- Branches: Percentage of branches that we rere run. Branch is one part of a conditiopnal statement luike an if statement or a ternary
- Functions: percent of functions that were run at least once
- Lines: The percent of liunes that we re run at least once
- You have to remember to always clean up your tests with
cleanupfrom testing library andafterEachfrom jest or vitest. If you don’t use global variables, this will never work.
Hooks
- Testing hooks should happen when the hook is business critical or logic within is complicated
- Use use
renderHookthat it creates a fake component where the hook is being used and managed the lifecycle of a component for you - Always use
result.currentotherwise you might get wrong results userEventshould be used when you want to fire some events like clicking etc.
test('hook: prevents default on the first click, and does not on the second', async () => {
const { result } = await renderHook(() => useDoubleCheck())
expect(result.current.doubleCheck).toBe(false)
const mock = vi.fn()
const event = new MouseEvent('click', {
bubbles: true,
cancelable: true,
}) as unknown as React.MouseEvent<HTMLButtonElement>
act(() => result.current.getButtonProps({ onClick: mock }).onClick(event))
expect(result.current.doubleCheck).toBe(true)
expect(mock).toHaveBeenCalledOnce()
expect(event.defaultPrevented).toBe(true)
mock.mockClear()
const event2 = new MouseEvent('click', {
bubbles: true,
cancelable: true,
}) as unknown as React.MouseEvent<HTMLButtonElement>
act(() => result.current.getButtonProps({ onClick: mock }).onClick(event2))
expect(result.current.doubleCheck).toBe(true)
expect(mock).toHaveBeenCalledOnce()
expect(event2.defaultPrevented).toBe(false)
}) You can also test your hooks using components like so
test('TestComponent: prevents default on the first click, and does not on the second', async () => {
const user = userEvent.setup()
await render(<TestComponent />)
const button = screen.getByRole('button')
const output = screen.getByRole('status')
expect(button).toHaveTextContent(/submit/i)
await user.click(button)
expect(button).toHaveTextContent('Are you sure?')
expect(output).toHaveTextContent('yes')
await user.click(button)
expect(button).toHaveTextContent('Are you sure?')
expect(output).toHaveTextContent('no')
})
test('blurring the button starts things over', async () => {
const user = userEvent.setup()
await render(<TestComponent />)
const status = screen.getByRole('status')
const button = screen.getByRole('button')
await user.click(button)
expect(button).toHaveTextContent('You sure?')
expect(status).toHaveTextContent('Default Prevented: yes')
await user.click(document.body)
// button goes back to click me
expect(button).toHaveTextContent('Click me')
// our callback wasn't called, so the status doesn't change
expect(status).toHaveTextContent('Default Prevented: yes')
})
test('hitting "escape" on the input starts things over', async () => {
const user = userEvent.setup()
await render(<TestComponent />)
const status = screen.getByRole('status')
const button = screen.getByRole('button')
await user.click(button)
expect(button).toHaveTextContent('You sure?')
expect(status).toHaveTextContent('Default Prevented: yes')
await user.keyboard('{Escape}')
// button goes back to click me
expect(button).toHaveTextContent('Click me')
// our callback wasn't called, so the status doesn't change
expect(status).toHaveTextContent('Default Prevented: yes')
}) Code Snippet
/**
* Use this hook with a button and it will make it so the first click sets a
* `doubleCheck` state to true, and the second click will actually trigger the
* `onClick` handler. This allows you to have a button that can be like a
* "are you sure?" experience for the user before doing destructive operations.
*/
export function useDoubleCheck() {
const [doubleCheck, setDoubleCheck] = useState(false)
function getButtonProps(
props?: React.ButtonHTMLAttributes<HTMLButtonElement>,
) {
const onBlur: React.ButtonHTMLAttributes<HTMLButtonElement>['onBlur'] =
() => setDoubleCheck(false)
const onClick: React.ButtonHTMLAttributes<HTMLButtonElement>['onClick'] =
doubleCheck
? undefined
: e => {
e.preventDefault()
setDoubleCheck(true)
}
const onKeyUp: React.ButtonHTMLAttributes<HTMLButtonElement>['onKeyUp'] =
e => {
if (e.key === 'Escape') {
setDoubleCheck(false)
}
}
return {
...props,
onBlur: callAll(onBlur, props?.onBlur),
onClick: callAll(onClick, props?.onClick),
onKeyUp: callAll(onKeyUp, props?.onKeyUp),
}
}
return { doubleCheck, getButtonProps }
}
/**
* Simple debounce implementation
*/
function debounce<Callback extends (...args: Parameters<Callback>) => void>(
fn: Callback,
delay: number,
) {
let timer: ReturnType<typeof setTimeout> | null = null
return (...args: Parameters<Callback>) => {
if (timer) clearTimeout(timer)
timer = setTimeout(() => {
fn(...args)
}, delay)
}
}
/**
* Debounce a callback function
*/
export function useDebounce<
Callback extends (...args: Parameters<Callback>) => ReturnType<Callback>,
>(callback: Callback, delay: number) {
const callbackRef = useRef(callback)
useEffect(() => {
callbackRef.current = callback
})
return useMemo(
() =>
debounce(
(...args: Parameters<Callback>) => callbackRef.current(...args),
delay,
),
[delay],
)
}
export async function downloadFile(url: string, retries: number = 0) {
const MAX_RETRIES = 3
try {
const response = await fetch(url)
if (!response.ok) {
throw new Error(`Failed to fetch image with status ${response.status}`)
}
const contentType = response.headers.get('content-type') ?? 'image/jpg'
const blob = Buffer.from(await response.arrayBuffer())
return { contentType, blob }
} catch (e) {
if (retries > MAX_RETRIES) throw e
return downloadFile(url, retries + 1)
}
}
function callAll<Args extends Array<unknown>>(
...fns: Array<((...args: Args) => unknown) | undefined>
) {
return (...args: Args) => fns.forEach(fn => fn?.(...args))
} HTTP Mocking
-
You usually don’t want to mock your own code but in case of 3rd parties, you do want to in case system goes down. Otherwise, your tests will be broken too
-
Without mocking, you cannot work offline and you relay on their system
-
For testing and mocks, you can use
server.useAPI
import { rest, HttpResponse } from 'msw'
import { setupServer } from 'msw/node'
const { json } = HttpResponse
const server = setupServer(
http.get('https://example.com/book/:bookId', () => {
return json({ title: 'Lord of the Rings' })
}),
)
beforeAll(() => {
server.listen()
})
afterEach(() => {
// Resets all runtime request handlers added via `server.use`
server.resetHandlers()
})
afterAll(() => {
server.close()
})
test('handles 500 errors from the API', () => {
server.use(
// Adds a runtime request handler for posting a new book review.
http.get('https://example.com/book/:bookId', () => {
return json({ error: 'Something went wrong' }, { status: 500 })
}),
)
// Performing a "GET /book/:bookId" request
// will get the mocked response from our runtime request handler.
})
test('asserts another scenario', () => {
// Since we call `server.resetHandlers()` after each test,
// this test suite will use the default GET /book/:bookId response.
}) Setup Functions
- Avoid nesting tests, because they are not readable. Do the following instead
import { render } from '@testing-library/react'
import userEvent from '@testing-library/user-event'
import * as React from 'react'
import Login from '../login'
// here we have a bunch of setup functions that compose together for our test cases
// I only recommend doing this when you have a lot of tests that do the same thing.
// I'm including it here only as an example. These tests don't necessitate this
// much abstraction. Read more: https://kcd.im/aha-testing
async function setup() {
const handleSubmit = jest.fn()
const utils = await render(<Login onSubmit={handleSubmit} />)
const user = { username: 'michelle', password: 'smith' }
const changeUsernameInput = value =>
userEvent.type(utils.getByLabelText(/username/i), value)
const changePasswordInput = value =>
userEvent.type(utils.getByLabelText(/password/i), value)
const clickSubmit = () => userEvent.click(utils.getByText(/submit/i))
return {
...utils,
handleSubmit,
user,
changeUsernameInput,
changePasswordInput,
clickSubmit,
}
}
async function setupSuccessCase() {
const utils = await setup()
utils.changeUsernameInput(utils.user.username)
utils.changePasswordInput(utils.user.password)
utils.clickSubmit()
return utils
}
async function setupWithNoPassword() {
const utils = await setup()
utils.changeUsernameInput(utils.user.username)
utils.clickSubmit()
const errorMessage = utils.getByRole('alert')
return { ...utils, errorMessage }
}
async function setupWithNoUsername() {
const utils = await setup()
utils.changePasswordInput(utils.user.password)
utils.clickSubmit()
const errorMessage = utils.getByRole('alert')
return { ...utils, errorMessage }
}
test('calls onSubmit with the username and password', async () => {
const { handleSubmit, user } = await setupSuccessCase()
expect(handleSubmit).toHaveBeenCalledTimes(1)
expect(handleSubmit).toHaveBeenCalledWith(user)
})
test('shows an error message when submit is clicked and no username is provided', async () => {
const { handleSubmit, errorMessage } = await setupWithNoUsername()
expect(errorMessage).toHaveTextContent(/username is required/i)
expect(handleSubmit).not.toHaveBeenCalled()
})
test('shows an error message when password is not provided', async () => {
const { handleSubmit, errorMessage } = await setupWithNoPassword()
expect(errorMessage).toHaveTextContent(/password is required/i)
expect(handleSubmit).not.toHaveBeenCalled()
}) Test Database
Creating Isolated databases for our tests
When running tests, it is important to create a copy/to have an isolated database for each of our tests. It helps with isolation for each test.
Isolation
It is important to isolate your dev environment from test but also tests from other tests to avoid situations where your test fails, because there are leftovers from your tests.
Setup
When creating a setup of a database, you have to make sure your env is setup before you run tests. Vitest has globalSetup where you can add all of it there, and it will run before tests start to run.
Module imports
When using imports, they execute from top to the bottom and even if you try to run something before imports, it will still import modules and run whatever is there first
console.log("three") // This will run after "one" and "two" module imports
import {one} from "one"
import {two} from "two" If you want to evaluate something before a module is evaluted, there are two solutions
import {one} from "one"
console.log("three")
const {two} = await import("two")
// Or
import {one} from "one"
import "./three"
import {two} from "two"
Database setup for test\
- You need to override DatabaseURL variable (or whatever variables you need for your ORM)
- Reset database before all tests start
- After each test you need to delete everything from the database to clean up after yourself
- Once all tests are done, you need to disconnect from a DB and delete the file
Make sure you create a databasefile for each test with vitest_pool_id variable to make sure databases are isolated
import path from 'node:path'
import { execaCommand } from 'execa'
import { afterAll, beforeAll, beforeEach } from 'vitest'
const databaseFile = `./tests/prisma/data.${VITEST_POOOL_ID || 0}.db`
const databasePath = path.join(process.cwd(), databaseFile)
process.env.DATABASE_URL = databasePath
beforeAll(async () => {
await fsExtra.copyFile(BASE_DATABASE_PATH, databaseFile)
})
beforeEach(async () => {
const { prisma } = await import('#app/utils/db.server.ts')
await prisma.user.deleteMany()
})
afterAll(async () => {
const { prisma } = await import('#app/utils/db.server.ts')
await prisma.$disconnect()
}) import 'dotenv/config'
import './db-setup.ts'
import '#app/utils/env.server.ts'
import '@testing-library/jest-dom/vitest'
import { installGlobals } from '@remix-run/node'
import { cleanup } from '@testing-library/react'
import { afterEach, beforeEach, vi, type SpyInstance } from 'vitest'
import { server } from '../mocks/index.ts'
import './custom-matchers.ts'
installGlobals()
afterEach(() => server.resetHandlers())
afterEach(() => cleanup()) You also want to copy the database file that you create.
global-setup.ts
import path from 'node:path'
import { execaCommand } from 'execa'
import fsExtra from 'fs-extra'
export const BASE_DATABASE_PATH = path.join(
process.cwd(),
'./tests/prisma/base.db',
)
export async function setup() {
const exists = await fsExtra.pathExists(BASE_DATABASE_PATH)
if (exists) return
await execaCommand(
'prisma migrate reset --force --skip-seed --skip-generate',
{
stdio: 'inherit',
env: { ...process.env, DATABASE_URL: `file:${BASE_DATABASE_PATH}` },
},
)
}