Guide

How to write and run UI and API tests with NATL. Start with install, get a green run, then work through the language. Jump around later with the list below.

Install

You need Node.js 18 or newer.

npm install -g @natl/cli @natl/adapter-playwright
npx playwright install chromium
natl --version

If that prints a version, you are set. If not, see Troubleshooting.

To start a project from scratch (example hits the live sandbox):

mkdir my-tests && cd my-tests
natl init
natl run tests/

Credentials for the sandbox: demo@natl.dev / secret. Offline twin: examples/fixtures/sandbox.html in the repo (set base_url in natl.config.yaml).

First run

After natl init, natl run tests/ opens the docs sandbox, logs in, and pings a static JSON API. You need network for the Pages URL (or point base_url at the local fixture).

There is also a login example in the repo that uses a local HTML file:

git clone https://github.com/arslan-ahmetjanov/natl.git
cd natl/examples
natl run login.yaml

You should see PASS. Change the assert text to something wrong, run again, read the error, then put it back. That is the fastest way to see how failures look.

Scenario shape

name: Sandbox login smoke
engine: playwright
tags: [smoke]
timeout: 15000
retries: 0
vars:
  base_url: ./fixtures/sandbox.html
  user: demo@natl.dev
  pass: secret
steps:
  - goto: $base_url
  - fill: "#email"
    with: $user
  - fill: "#password"
    with: $pass
  - click: "#login-btn"
  - assert: ".welcome"
    text: "Welcome, demo"

Fields at the top of a file:

A single step can also set timeout, save, or debug. To change engine in the middle of a run, wrap steps in with:. A bare engine: on one step is not what switches context.

Project config and env

Defaults live in natl.config.yaml next to your tests (NATL walks up from the file until it finds one):

engine: playwright
browser: chromium
timeout: 15000
base_url: https://staging.example.com
headless: true
viewport: { width: 1280, height: 720 }
locator_strategy: css
artifacts_dir: artifacts
retries: 0
trace: on-fail
video: off
soft_assert_screenshot: false

Different stands without editing every scenario:

natl run tests/ --env staging
natl run tests/ --config config/prod.yaml

--env staging loads config/staging.yaml. Do not pass both --env and --config.

What wins, from strongest to weakest: CLI flags, then fields in the test file, then the env profile, then natl.config.yaml, then built-in defaults.

UI steps

- goto: $base_url
- fill: "#email"
  with: $user
- select: "#country"
  with: "KZ"
- check: "#agree"
- click: "#submit"
- wait: 500
- wait: ".modal hidden"

Clicks, fills, and asserts already wait for the target up to timeout. You rarely need wait: ".ok visible" right before an assert. Keep wait: for delays or states like hidden.

Assert

- assert: ".welcome"
  text: "Hello"              # is: works the same
- assert: ".welcome"
  contains: "Hell"
- assert: $welcome
  visible: true
- assert: ".banner"
  hidden: true
- assert: ".price"
  attr: class contains "sale"
- assert: current_url == "https://example.com/app"
- assert: current_url contains "/app"
- assert: $ping.status == 200
- assert: ".price"
  text: "$10"
  soft: true
- soft_assert: ".stock"      # same as soft: true
  visible: true

For attr: you can use ==, !=, contains, or matches. A hard assert stops the run. Soft ones keep going and fail the run at the end if any of them failed.

Read values, set, log, screenshot

- get_text: ".title"
  save: title
- get_attr: "a"
  attr: href
  save: href
- set: $total = $price * $qty
- log: "total=$total"
- log: { message: "x", level: warn }   # info, debug, warn, error
- debug: true
- screenshot:
    file: artifacts/shot.png
    full_page: true

Scroll, swipe, long press

These work on desktop and mobile web the same way. You do not branch on device type.

- scroll: "#footer"
  into_view: true            # or delta_x / delta_y
- swipe: "#carousel"
  direction: left            # left, right, up, down
  distance: 200
- long_press: "#card"
  duration_ms: 700

Variables, secrets, expressions

secrets:
  env:
    file: .env
    encoding: utf-8

vars:
  user: $env.TEST_USER
  pass: $secret.TEST_PASS

Comparisons: == != > < >= <= contains matches and or not.

Helpers: now, today, random_int, random_string, random_uuid, random_email, len, contains, trim, upper, lower, replace, match, join, map, filter, range.

current_url is available during URL checks. There is a working example in examples/env_login.yaml.

Page objects

# pages/login.yaml
locator_strategy: css
elements:
  email: "#email"
  title:
    strategy: xpath
    value: "//h1"
actions:
  login:
    - fill: $email
      with: $user
    - fill: "#password"
      with: $pass
    - click: "#login-btn"

# scenario
imports:
  - pages/login.yaml
steps:
  - goto: $base_url
  - do: login.login
    user: $user
    pass: $pass
  - include: login/login
    vars: { user: $user }

Use do: page.action in normal scenarios. include: is the lower-level form when you want a file path or to pull in a whole file. Extra keys next to do: become variables for that action.

A page file can be only elements and actions, with no steps. When you run a folder, NATL skips pages/ and config/.

Locator strategy is chosen in this order: the element object, then the page file, then project config, then css.

Cases and data

cases:
  - { name: user1, user: "a@test.com", pass: secret, expect: "Welcome, A" }
  - { label: user2, user: "b@test.com", pass: secret, expect: "Welcome, B" }
steps:
  - fill: "#email"
    with: $user
  - assert: ".welcome"
    text: $expect

data:
  - { name: one }
  - { name: two }
steps:
  - for: $item in $data
    steps:
      - log: $item.name

For a table of inputs, use cases:. For loops over lists or range, use data: with for:. The report adds the row name or label, or [case N] if you omit both.

HTTP

engine: http
vars:
  base_url: http://127.0.0.1:8765
steps:
  - get: $base_url/get
    save: ping
  - post: $base_url/post
    headers: { X-Test: "1" }
    body: { hello: world }
    save: echo
  - put: …
  - patch: …
  - delete: …
  - assert: $echo.status == 200
  - assert: $echo.ok == true

What gets saved: { status, headers, body, ok }. Relative URLs use vars.base_url when that value is an http(s) URL. The old api: form still parses; new files should use the verbs above.

UI and HTTP in one scenario:

engine: playwright
steps:
  - goto: $base_url
  - with: http
    steps:
      - get: $api_base/get
        save: ping
      - assert: $ping.status == 200
  - click: "#refresh"

The HTTP examples in the repo expect a local stub:

node stubs/echo-server.mjs
natl run http_only.yaml
natl run ui_http_block.yaml

if / for / repeat / parallel

- if: $role == "admin"
  then:
    - click: "#admin"
  else:
    - click: "#user"

- for: $i in range(1, 5)
  steps:
    - log: $i

- repeat: 3                 # or "3 times", or times: 3
  steps:
    - click: "#next"        # $i is set each time

- repeat:
    until: $done == true
  steps:
    - click: "#poll"        # stops after 1000 iterations

- parallel:                 # wait: all by default, or any
  - get: $api/a
    save: a
  - get: $api/b
    save: b

before_each and after_each run around steps on every attempt, including retries.

Tags, retries, CLI

name: Checkout
tags: [smoke, checkout]
retries: 1

natl run tests/ --tags smoke,auth    # match any listed tag
natl run tests/ --grep "Login"       # regex on name or path
natl run tests/ --retries 2
natl run tests/ --env staging
natl run tests/ --headed
natl run tests/ --engine selenium
natl run tests/ --trace on-fail
natl run tests/ --video off

If you pass both --tags and --grep, a file must match both. No matches means exit code 1. Retries start the whole scenario again with a new browser context. The last attempt is the result that counts.

How to write steps

Prefer normal YAML with sibling keys. That is what editors and the schema expect:

- fill: "#email"
  with: $user
- wait: ".dashboard visible"
- assert: ".welcome"
  text: "Hello"

There is also a one-line form the runner accepts. Many editors will mark it as invalid YAML, so I would not commit it:

- fill: "#email" with: $user
- assert: ".welcome" text: "Hello"
- scroll: $footer into_view

For autocomplete, point your YAML extension at the schemas in @natl/core. The CLI README shows the yaml.schemas setting.

Where to go next

• examples/ — POM, cases, HTTP, gestures, soft assert
• Troubleshooting
• Adapters
• Canon — fixed design choices