CLI Usage

Installation

The CLI is included with the runners package:

npm install runners
# or
pnpm add runners

Basic Usage

Run Specific Runners

npx runners run --url https://example.com runner1 runner2

Run All Discovered Runners

npx runners run --url https://example.com

Use a Config File

npx runners run --config runners.config.ts

Commands

run

Execute runners locally.

Syntax:

npx runners run [options] [runner-names...]

Options:

Option	Description	Default
`--url <url>`	URL to pass to all runners	None
`--config <path>`	Path to config file	`runners.config.ts`
`--region <region>`	Region identifier	None
`--timeout <ms>`	Timeout in milliseconds	30000
`--help`	Show help message	-
`--version`	Show version	-

Examples:

# Run specific runners with URL
npx runners run --url https://example.com cookieBannerTest exampleTitleTest

# Run with config file
npx runners run --config runners.config.ts

# Run all discovered runners
npx runners run --url https://example.com

# Run with custom timeout
npx runners run --url https://example.com --timeout 60000 runner1

# Run with region
npx runners run --url https://example.com --region us-east-1 runner1

Configuration File

Create runners.config.ts in your project root:

import { defineConfig } from "runners/config";

export default defineConfig({
  url: "https://example.com",
  region: "us-east-1",
  runners: ["cookieBannerTest", "exampleTitleTest"],
  timeout: 30000,
});

Config Options:

Option	Type	Description
`url`	`string`	URL to pass to runners
`region`	`string`	Region identifier
`runners`	`string[]`	List of runner names to execute
`timeout`	`number`	Timeout in milliseconds

Example:

import { defineConfig } from "runners/config";

export default defineConfig({
  url: "https://example.com",
  runners: ["cookieBannerTest"],
});

Runner Discovery

The CLI automatically discovers runners from:

src/**/*.ts
runners/**/*.ts

Runners must have the "use runner" directive to be discovered.

Module-Level Directive

"use runner";

export const runner1: Runner = async (ctx) => { /* ... */ };
export const runner2: Runner = async (ctx) => { /* ... */ };

Function-Level Directive

export const runner: Runner = async (ctx) => {
  "use runner";
  // ... implementation
};

Output Format

Success

✓ Running runners...

✓ cookieBannerTest
  Status: pass
  Duration: 1234ms

✓ exampleTitleTest
  Status: pass
  Duration: 567ms

✓ All runners passed (2/2)

Failure

✓ Running runners...

✓ cookieBannerTest
  Status: pass
  Duration: 1234ms

✗ exampleTitleTest
  Status: fail
  Duration: 567ms
  Error: Title not found

✗ Some runners failed (1/2)

Error

✓ Running runners...

✓ cookieBannerTest
  Status: pass
  Duration: 1234ms

✗ exampleTitleTest
  Status: error
  Duration: 567ms
  Error: Failed to load page: timeout

✗ Some runners errored (1/2)

Exit Codes

0 - All runners passed
1 - One or more runners failed or errored
2 - Invalid command or configuration

Advanced Usage

Debug Mode

Enable debug logging:

DEBUG=true npx runners run --url https://example.com runner1

Or:

RUNNERS_DEBUG=true npx runners run --url https://example.com runner1

Custom Input

Pass custom input to runners via config:

import { defineConfig } from "runners/config";

export default defineConfig({
  url: "https://example.com",
  runners: [
    {
      name: "customRunner",
      input: {
        url: "https://example.com",
        customField: "value",
      },
    },
  ],
});

Multiple URLs

Run the same runners against multiple URLs:

for url in https://example.com https://test.com; do
  npx runners run --url "$url" runner1 runner2
done

CI Integration

Example GitHub Actions workflow:

name: Run Runners

on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: pnpm/action-setup@v2
        with:
          version: 8
      - uses: actions/setup-node@v3
        with:
          node-version: 18
          cache: 'pnpm'
      - run: pnpm install
      - run: pnpm exec playwright install
      - run: pnpm exec runners run --url https://example.com

Troubleshooting

Runners Not Found

If runners aren't being discovered:

Check that files are in src/ or runners/ directories
Verify "use runner" directive is present
Ensure runners are exported
Check file extensions are .ts (not .js)

Playwright Errors

If you see Playwright executable errors:

pnpm exec playwright install

Timeout Issues

Increase timeout for slow runners:

npx runners run --url https://example.com --timeout 60000 slowRunner

Type Errors

Ensure TypeScript is properly configured:

pnpm exec tsc --noEmit

Tips

Use Config Files: Avoid long command lines with config files
Debug Mode: Use DEBUG=true to see detailed logs
Exit Codes: Use exit codes in CI/CD pipelines
Parallel Execution: Runners execute sequentially; use orchestrator for parallel execution
Error Handling: Check exit codes and error messages for debugging

Installation

The CLI is included with the runners package:

npm install runners
# or
pnpm add runners

Basic Usage

Run Specific Runners

npx runners run --url https://example.com runner1 runner2

Run All Discovered Runners

npx runners run --url https://example.com

Use a Config File

npx runners run --config runners.config.ts

Commands

run

Execute runners locally.

Syntax:

npx runners run [options] [runner-names...]

Options:

Option	Description	Default
`--url <url>`	URL to pass to all runners	None
`--config <path>`	Path to config file	`runners.config.ts`
`--region <region>`	Region identifier	None
`--timeout <ms>`	Timeout in milliseconds	30000
`--help`	Show help message	-
`--version`	Show version	-

Examples:

# Run specific runners with URL
npx runners run --url https://example.com cookieBannerTest exampleTitleTest

# Run with config file
npx runners run --config runners.config.ts

# Run all discovered runners
npx runners run --url https://example.com

# Run with custom timeout
npx runners run --url https://example.com --timeout 60000 runner1

# Run with region
npx runners run --url https://example.com --region us-east-1 runner1

Configuration File

Create runners.config.ts in your project root:

import { defineConfig } from "runners/config";

export default defineConfig({
  url: "https://example.com",
  region: "us-east-1",
  runners: ["cookieBannerTest", "exampleTitleTest"],
  timeout: 30000,
});

Config Options:

Option	Type	Description
`url`	`string`	URL to pass to runners
`region`	`string`	Region identifier
`runners`	`string[]`	List of runner names to execute
`timeout`	`number`	Timeout in milliseconds

Example:

import { defineConfig } from "runners/config";

export default defineConfig({
  url: "https://example.com",
  runners: ["cookieBannerTest"],
});

Runner Discovery

The CLI automatically discovers runners from:

src/**/*.ts
runners/**/*.ts

Runners must have the "use runner" directive to be discovered.

Module-Level Directive

"use runner";

export const runner1: Runner = async (ctx) => { /* ... */ };
export const runner2: Runner = async (ctx) => { /* ... */ };

Function-Level Directive

export const runner: Runner = async (ctx) => {
  "use runner";
  // ... implementation
};

Output Format

Success

✓ Running runners...

✓ cookieBannerTest
  Status: pass
  Duration: 1234ms

✓ exampleTitleTest
  Status: pass
  Duration: 567ms

✓ All runners passed (2/2)

Failure

✓ Running runners...

✓ cookieBannerTest
  Status: pass
  Duration: 1234ms

✗ exampleTitleTest
  Status: fail
  Duration: 567ms
  Error: Title not found

✗ Some runners failed (1/2)

Error

✓ Running runners...

✓ cookieBannerTest
  Status: pass
  Duration: 1234ms

✗ exampleTitleTest
  Status: error
  Duration: 567ms
  Error: Failed to load page: timeout

✗ Some runners errored (1/2)

Exit Codes

0 - All runners passed
1 - One or more runners failed or errored
2 - Invalid command or configuration

Advanced Usage

Debug Mode

Enable debug logging:

DEBUG=true npx runners run --url https://example.com runner1

Or:

RUNNERS_DEBUG=true npx runners run --url https://example.com runner1

Custom Input

Pass custom input to runners via config:

import { defineConfig } from "runners/config";

export default defineConfig({
  url: "https://example.com",
  runners: [
    {
      name: "customRunner",
      input: {
        url: "https://example.com",
        customField: "value",
      },
    },
  ],
});

Multiple URLs

Run the same runners against multiple URLs:

for url in https://example.com https://test.com; do
  npx runners run --url "$url" runner1 runner2
done

CI Integration

Example GitHub Actions workflow:

name: Run Runners

on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: pnpm/action-setup@v2
        with:
          version: 8
      - uses: actions/setup-node@v3
        with:
          node-version: 18
          cache: 'pnpm'
      - run: pnpm install
      - run: pnpm exec playwright install
      - run: pnpm exec runners run --url https://example.com

Troubleshooting

Runners Not Found

If runners aren't being discovered:

Check that files are in src/ or runners/ directories
Verify "use runner" directive is present
Ensure runners are exported
Check file extensions are .ts (not .js)

Playwright Errors

If you see Playwright executable errors:

pnpm exec playwright install

Timeout Issues

Increase timeout for slow runners:

npx runners run --url https://example.com --timeout 60000 slowRunner

Type Errors

Ensure TypeScript is properly configured:

pnpm exec tsc --noEmit

Tips

Use Config Files: Avoid long command lines with config files
Debug Mode: Use DEBUG=true to see detailed logs
Exit Codes: Use exit codes in CI/CD pipelines
Parallel Execution: Runners execute sequentially; use orchestrator for parallel execution
Error Handling: Check exit codes and error messages for debugging

CLI Usage

Installation

Basic Usage

Run Specific Runners

Run All Discovered Runners

Use a Config File

Commands

run

Configuration File

Runner Discovery

Module-Level Directive

Function-Level Directive

Output Format

Success

Failure

Error

Exit Codes

Advanced Usage

Debug Mode

Custom Input

Multiple URLs

CI Integration

Troubleshooting

Runners Not Found

Playwright Errors

Timeout Issues

Type Errors

Tips

See Also

CLI Usage

Installation

Basic Usage

Run Specific Runners

Run All Discovered Runners

Use a Config File

Commands

run

Configuration File

Runner Discovery

Module-Level Directive

Function-Level Directive

Output Format

Success

Failure

Error

Exit Codes

Advanced Usage

Debug Mode

Custom Input

Multiple URLs

CI Integration

Troubleshooting

Runners Not Found

Playwright Errors

Timeout Issues

Type Errors

Tips

See Also