🐐 Goat

goat (Go API Tester), a CLI tool to simplify and automate integration testing of HTTP APIs by using script files.

Example Goatfile

use util/login

### Setup

LOGIN {{.instance}}/api/v1/auth

[Header]
Content-Type: application/json

[Body]
{ 
    "username": "{{.username}}",
    "password": "{{.password}}"
}

[Script]
assert(response.StatusCode == 200, `Status code was ${response.StatusCode}`);

---

### Tests

GET {{.instance}}/api/v1/list

[Script]
assert(response.StatusCode == 200, `Status code was ${response.StatusCode}`);
print(response.Body);

📖 Hey and welcome to the Goat documentation!

Here you will learn how to use the Goat CLI tool and how to write Goatfiles.

For better understanding and conciseness, we highly suggest to take a look at the Glossary before continuing into the documentation.

Glossary

To make this documentation more concise and understandable, here you can find a list of all special terms and their definitions used across the documentation.

Goatfile - A file with the extension .goat which is interpreted and executed by the Goat CLI.
action - Can be any executable entity defined in a Goatfile like a request, log entry or execution.
batch - Defines the execution of multiple requests and actions across one or multiple Goatfiles.

Getting Started

First of all, you need to install the goat CLI tool. It is distributed as a single binary without any dependencies. So simply download the latest release binaries from GitHub fitting your system configuration.

You can also use the provided installation script, if you want an easy one-line setup.

$ curl -Ls https://raw.githubusercontent.com/studio-b12/goat/main/scripts/download.sh | bash -

Installation with `go install`

Alternatively, if you have the Go toolchain installed on your system, you can also simply use go install to build and install the tool for your system. This is also very useful if no pre-built binary is available for your specific system configuration.

Details on how to install the Go toolchain are available here.

Simply use the following command to install the program.

go install github.com/studio-b12/goat/cmd/goat@latest

This will install the latest tagged version. You can also specify a specific version or branch after the @.

go install github.com/studio-b12/goat/cmd/goat@dev

First Goatfile

After that, simply use the following command to initalize a new Goatfile.

goat --new

This will generate a new Goatfile with the name tests.goat in your current working directory with some simple examples and documentation. You can use this file as your starting point to play around with Goat or to create your project's integration test structure.

Command Line Tool

The goat command is used to create, validate and execute Goatfiles.

You can pass one or multiple Goatfiles or directories as positional arguments. For example:

goat issue_32.goat tests/integrationtests

When passing in a directory, Goat will look for any *.goat files recursively. Files and directories prefixed with an underscore (_) are ignored. This is especially useful for Goatfiles which are only supposed to be imported or executed in other Goatfiles. If you want to read more about this, take a look into the Project Structure section.

Flags

In the following, further information is provided about the various flags which can be passed to the goat CLI.

-a ARGS, --args ARGS
Pass params into the execution as key-value pairs. If you want to pass multiple args, specify each pair with its own parameter.
Example: -a hello=world -a user.name=foo -a "user.password=bar bazz"
--delay DELAY, -d DELAY
Delay all requests by the given duration. The duration is formatted according to the format of Go's time.ParseDuration function.
Example: -d 1m30s
--dry
Only parse the Goatfile(s) without executing any requests.
--gradual, -g
Advance the execution of each request manually via key-presses.
--json
Use JSON format instead of pretty console format for logging.
--loglevel LOGLEVEL, -l LOGLEVEL
Logging level. Here you can see which values you can use for log levels.
Example: -l trace
--new
Create a new base Goatfile. When a file directory name is passed as positional parameter, the new Goatfile will be created under that directory name.
--no-abort
Do not abort the batch execution on error.
--no-color
Suppress colored log output.
--params PARAMS, -p PARAMS
Pass parameters defined in parameter files. These can be either TOML, YAML or JSON files. If you want to pass multiple parameter files, specify each one with its own parameter.
Example: -p ./local.toml -p ~/credentials.yaml
--profile PROFILE, -P PROFILE
Use parameters from profiles defined in a profile config in your home's configuration directory. Here you can read more about how profiles work.
Example: -P foo -P bar
--reduced-errors, -R
Hide template errors in teardown steps. This can be useful when running tests to hide some noise from failing teardown steps due to missing variables.
--silent, -s
Disable all logging output. Only print and println statements will be printed. This is especially useful if you want to use Goatfiles within other scripts.
--skip SKIP
Section(s) to be skipped during execution.
Example: --skip teardown
--secure
Enable TLS certificate validation.
--help, -h
Display the help message.
--version
Display the installed version.

Profiles

Profiles are like parameter files but located in a central file in your home config directory and usable from everywhere. This should solve the problem when you have similar parameters across multiple projects, so you don't need to duplicate them everywhere or figure out the path to your centrally stored parameter files.

The location of this file depends on your system type.

Operating System	Location
Linux	`$HOME/.config/goat/profiles` or `$XDG_CONFIG_HOME/goat/profiles.` if `$XDG_CONFIG_HOME` is set
Darwin (OSX)	`$HOME/Library/Application Support/goat/profiles.*`
Windows	`%AppData%\goat\profiles.*`

Goat looks for any file in the config directory called profiles.*. Supported file types and extensions are as follows.

.toml
.yaml or .yml
.json

The structure of a profile file consists of a top level map containing the names of the profiles as keys and the parameter maps as values.

Here is a quick example.

default:
  instance: https://example.com
  credentials:
    token: foobar

staging:
  instance: https://staging.example.com
  credentials:
    token: barbaz

lowprevileges:
  credentials:
    token: lowtoken

local:
  instance: https://localhost:8080

Profiles can be addressed when calling goat with the --profile (or -P) parameter. You can define multiple profiles by chaining multiple of these parameters. For example:

goat -P live -P lowprevileges test.goat

The values of the profiles are then merged in order of given parameters.

The default profile is always loaded into the parameters if present. There you can define parameters that you always want to use when executing goat. Other passed profiles and parameters will overwrite values in the default profile as usual.

Parameters passed via environment variables or via the --parameter flag will also overwrite profile values.

How does it work?

In the following sections you will learn more about how the internals of Goat work. This will help you to understand how to create Goatfiles fitting your testing situation and work with them.

State Management

Goat initializes a state for every Goatfile execution. A state initially consists of the parameters which were passed via a configuration file, environment variables, or the --args CLI parameter.

This state is then passed to every request in the executed Goatfile. The request can read and alter the state. When one request has finished, the state is passed on to the next request and so on until the execution has finished.

Below, you can see a very simple example of what a state lifecycle could look like.

State with `use`

Because use effectively merges the imported Goatfile together with the root Goatfile to a single batch execution, the state is shared between them. So if you define a variable in a Goatfile imported via use, the variable will be accessible in subsequent imported Goatfiles as well as in the root Goatfile.

State with `execute`

Invoking another Goatfile via the execute command will make Goat handle the executed Goatfile like a separate batch with its own, disconnected state.

Lifecycle

In this section, you will learn the basics on how exactly batches and requests are executed and when which section of a batch or test is evaluated.

Sections

First of all, a batch consists of four logical sections.

Default: Contains default request parameters which are passed on to each executed request in the batch. This is the only section which does not actually execute actions and supports partial request fields.
Setup: Contains actions which should be executed before the actual test steps are ran.
Tests: The actual test actions.
Teardown: Steps that are run after the test executions, e.g., to clean up stuff from the Setup and Test sections.

Batch Lifecycle

Below, you can find a simplified schematic of how a single test batch is executed.

%%{init: {'theme':'dark'}}%%
flowchart TD
  collect_args[Collect params\nand arguments]
  entry[Parse entry Goatfile]
  resolve_imports[Resolve imports]
  parse_imports[Parse imports]
  merge_files[Merge files]
  create_state[Create state]
  setup_actions[Execute setup actions]
  setup_failed{Failed?}
  teardown_actions[Execute Teardown actions]
  test_actions[Execute Test actions]
  test_failed{Failed?}

  entry
    --> resolve_imports
    --> parse_imports -.-> entry
  
  parse_imports
    --> merge_files
    --> setup_actions

  collect_args 
    ----> create_state
    --> setup_actions
    --> setup_failed -- yes --> teardown_actions
        setup_failed -- no --> test_actions
          --> test_failed -- yes --> teardown_actions
              test_failed -- no --> teardown_actions

Every batch begins with a single entrypoint Goatfile (if you execute Goat on a folder of Goatfiles, each Goatfile in that folder will be seen as an entrypoint Goatfile and the batch execution will be executed indiviually for each file).

First, all imported (see use) Goatfiles are resolved, parsed and merged together with the entrypoint Goatfile to one single batch execution. So all Default, Setup, Test and Teardown entries are merged together for each file.

"Simultaneously", all parameters are collected from passed parameter files, environment variables and arguments (see Command Line Tool). These form the initial state.

After that, all Setup actions are executed. If any setup action has failed, the rest of the setup and the entire Test section is skipped. Finally, all teardown steps are executed and the batch exits in a failed state summarizing all errors that occurred.

If the Setup has completed successfully, the Test section is executed. Just as in the Setup section, if any of the Test actions fails, the entire section is skipped, the teardown actions are executed and the batch exits in a failed state.

If the Test section has completed successfully, the Teardown section is executed. Here, if any action fails, the execution continues instead of skipping the rest of the actions to ensure a complete cleanup as intended. If any of the Teardown actions fail, the batch execution will result in a failed state as well.

Action Lifecycle

Below, you can see a simplified lifecycle diagram of the three actions Request, Execute and Log Section.

%%{init: {'theme':'dark'}}%%
flowchart TD
  type{Type of Action}

  exit_with_failure(End with Failure)
  exit_with_success(End with Success)

  %% --- Request Type -----------------------------------------------

  apply_defaults[Apply defaults]
  pre_substitute[Substitute parameters\nfrom state]
  run_prescript[Run PreScript]
  prescript_success{Successful?}
  substitute[Re-substitute parameters\nfrom state]
  apply_options[Apply request options]
  condition_option{Condition option\nmatches?}
  run_request[Run Request]
  run_script[Run Script]
  script_success{Successful?}

  type 
    -- Request --> apply_defaults
    --> pre_substitute
    --> run_prescript
    --> prescript_success -- no --> exit_with_failure
  
  prescript_success 
    -- yes --> substitute
    --> apply_options
  
  condition_option
    -- yes --> run_request
    --> run_script
    --> script_success -- no --> exit_with_failure

  script_success -- yes --> exit_with_success

  apply_options --> condition_option -- no --> exit_with_success

  %% --- Execute Type -----------------------------------------------

  create_state_from_params[Create state from\npassed parameters]
  parse_goatfile[Parse Goatfile]
  execute_goatfile[Execute Goatfile]
  execute_success{Successful?}
  apply_captured[Apply captured\nreturn Values]

  type
    -- Execute --> create_state_from_params
    --> parse_goatfile
    --> execute_goatfile
    --> execute_success -- no --> exit_with_failure

  execute_success
    -- yes --> apply_captured
    -------> exit_with_success
    
  %% --- Log Section Type -----------------------------------------------

  print_logsection[Print Section to Log]

  type
    -- Log Section --> print_logsection 
    -----------> exit_with_success

Request

A Request action begins with the application of all default parameters from the Default section of the batch. After that, the parameters from the current state are substituted for the template parameters in the request definition. With the resulting state, the PreScript section is executed. If the execution failed, the request ends with a failure state. Otherwise, the new state is extracted and all templates are re-substituted using the new state. Following this, the request options are evaluated and applied. If the option condition evaluates to false, the request is skipped which ends the request in a success state. Otherwise, the actual request is now executed. Finally, the Script section is executed using the current state. Depending on the result, the request will end with a failure or success state.

Execute

An Execute action "calls" another Goatfile with specified parameters and capture values. The defined parameters are used to initialize a new state. After that, the referenced Goatfile is parsed and executed with this new state. This execution is a whole new Batch execution in itself.

If this batch execution fails, the action results in a failed state as well. If it is successful, the defined values to be captured in the return statement are merged with the current state of the executing batch.

Log Section

A Log Section is simply an action that prints a visual separator as INFO entry into the log to visually separate between test sections. This should never result in a failed action state.

Goatfile

A Goatfile contains the request definitions for a Goat test suite. Goatfiles are written in plain UTF-8 encoded text using the Goatfile syntax.

Goatfiles have the file extension .goat.

Below, you can see a really simple example Goatfile.

### Tests

GET https://api.github.com/repos/{{.repo}}

[Script]
assert(response.StatusCode === 200, `Invalid response code: ${response.StatusCode}`);
assert(response.Body.language === "Go");

---

GET https://api.github.com/repos/{{.repo}}/languages

[Script]
info('Languages:\n' + JSON.stringify(response.Body, null, 2));
assert(response.StatusCode === 200, `Invalid response code: ${response.StatusCode}`);

More conclusive examples can be found here.

Specification

Below, you can see a simple synopsis of the different building blocks of a Goatfile.

         Import  |  use ./setup.goat
                 |
        Comment  |  // This is a comment!
                 |
Section Heading  |  ### Setup
                 |
Context Section  |  ##### Upload Tests
                 |
   Method & URL  |  POST https://example.com
                 |  
        Headers  |  [Header]
                 |  X-Requested-With: XMLHttpRequest
                 |  Content-Type: application/json
                 |  Hash: {{ sha256 .data }}
                 |
           Body  |  [Body]
                 |  ```
                 |  {
                 |    "hello": "world",
                 |    "hash": "{{ sha256 .data }}",
                 |    "dontescapethis": "\{\{ \}\}"
                 |  }
                 |  ```
                 |
  Option Blocks  |  [QueryParams]
                 |  page = 2
                 |  items = 30
                 |  filters = ["time", "name"]
                 |
                 |  [MultipartFormdata]
                 |  image = @myimage.jpeg
                 |
                 |  [Options]
                 |  cookiejar = "foo"
                 |  storecookies = false
                 |
         Script  |  [Script]
                 |  assert(response.status >= 200 && response.status < 400);
                 |  assert(response.body["name"] == "somename");
                 |
                 |  // capture a variable to be used in subsequent responses
                 |  var id = response.body["id"];
                 |
        Request  |  ---
      Separator  |
                 |
        Execute  |  execute ./testFileUpload (
                 |    file="file_a.txt"
                 |    token="{{.auth.token}}"
                 |  ) return (
                 |    fileId as fileId_a
                 |  )

In the following sections, you will find a detailed rundown for each component of a Goatfile.

Comment
Import Statement
Execute Statement
Section
Defaults Section
Log Section
Request
- Method and URL
- Options
- Headers
- Query Parameters
- Body
- PreScript
- Script

Comment

Syntax

CommentDefinition :
LineComment | BlockComment

LineComment :
// .* NL

BlockComment :
/* .* */

Example

// This is a line comment!

/// This is a line comment as well!

/*
  This is a
    multiline
  block comment!
*/

Explanation

Contents of comments are ignored by the parser.

Import Statement

Syntax

UseExpression :
use StringLiteral

Example

use ../path/to/goatfile

Explanation

External Goatfiles can be imported using the use statement.

Imported Goatfiles behave like they are merged with the root Goatfile. So when you import a Goatfile B into a Goatfile A, all actions in all sections of B will be inserted in front of the actions in the sections of A. Meanwhile, the order of the sections Setup, Tests and Teardown stays intact.

Cyclical or repeated imports are not allowed.

Schematic Example:

A.goat

use B

### Setup

GET A1

### Tests

GET A2

### Teardown

GET A3

B.goat

### Setup

GET B1

### Tests

GET B2

### Teardown

GET B3

Result (virtual):

### Setup

GET B1
GET A1

### Tests

GET B2
GET A2

### Teardown

GET B3
GET A3

Execute Statement

Syntax

ExecuteExpression :
execute StringLiteral

Parameters :
( (KeyValuePair (WS|NL)+ )* ) ReturnStatement?

ReturnStatement :
( (ReturnPair (WS|NL)+ )* )

ReturnPair :
StringLiteral as StringLiteral

KeyValuePair :
StringLiteral WS* = WS* StringLiteral

Example

execute "../utils/login" (
  username="{{.credentials.username}}"
  password="{{.credentials.password}}"
) return (
  userId as userId
)

Explanation

The execute statement allows to run an external Goatfile inside another Goatfile in its own state context. Parameters for the executed Goatfile can be passed and values in the resulting state can be taken over into the calling Goatfile's state.

In contrast to the use directive, the executed Goatfile A is run like a completely separate Goatfile execution with its own isolated state which does not share any values with the state of the executing file B. All parameters which shall be available in A must be passed as a list of key-value pairs. Resulting state values of A can then be captured by the state of B by listing them in the return statement with the name of the parameter in the state of A and the name the value shall be accessible under in B.

Executed Goatfiles are parsed in place, so they are only statically checked once they are executed within the executing Goatfile.

Section

Syntax

SectionDefinition :
SectionHeader NL+ SectionContent

SectionHeader :
### StringLiteral

Example

### Tests

// ...

Explanation

A section defines a discrete segment in a Goatfile. This can either be a specialized section like the Defaults section or a request section like Setup, Tests and Teardown. More information on these section types can be found under Lifecycle.

Defaults Section

Syntax

DefaultsBlock :
### defaults NL+ PartialRequestProperties?

Example

### Defaults

[Options]
storecookies = false

[Header]
Authorization: basic {{.token}}

[Script]
assert(response.StatusCode === 200);

Explanation

The Defaults section body is structured like a request but without the method and URL section. This includes the blocks [Options], [Header], [QueryParams], [Body], [PreScript] and [Script].

Values specified in [Header], [Options] and [QueryParams] will be merged with the values set in each request. Values set in a request will overwrite values set in the defaults, if specified in both.

On the other hand, values specified in [Body], [PreScript] or [Script] will be used if not specified in the requests. If these blocks are specified in the request, they will overwrite the default values as well (even if they are empty).

Default values will also be applied if imported via a use directive. Multiple Defaults sections will be merged together in order of specification.

Merge Example

Here is a quick example on how the merges will result.

Let's assume the following defaults are set.

### Defaults

[Header]
Content-Type: application/json
Accept-Language: de;q=0.8, en

[Options]
storecookies = false

[Script]
assert(response.Status === 200);

Our first request definition looks like this:

GET {{.instance}}/api/v1/users/me

[Header]
Authorization: bearer sDrYbXdm2tCYnb8p

The resulting merged request fields will look as following.

GET {{.instance}}/api/v1/users/me

[Header]
Content-Type: application/json
Accept-Language: de;q=0.8, en
Authorization: bearer sDrYbXdm2tCYnb8p

[Options]
storecookies = false

[Script]
assert(response.Status === 200);

The second request looks like this.

POST {{.instance}}/api/v1/login

[Header]
Content-Type: multipart/form-data; boundary=bbecc22460604a4d97e62895cc6b254a

[Options]
storecookies = true

[Body]
```
--bbecc22460604a4d97e62895cc6b254a
Content-Disposition: form-data; name="username"

foo
--bbecc22460604a4d97e62895cc6b254a
Content-Disposition: form-data; name="password"

bar
--bbecc22460604a4d97e62895cc6b254a--
```

[Script]
assert(response.Status === 200);
assert(response.Body === expectedUserId);

And the resulting merged parameters will look as following.

POST {{.instance}}/api/v1/login

[Header]
Accept-Language: de;q=0.8, en
Content-Type: multipart/form-data; boundary=bbecc22460604a4d97e62895cc6b254a

[Options]
storecookies = true

[Body]
```
--bbecc22460604a4d97e62895cc6b254a
Content-Disposition: form-data; name="username"

foo
--bbecc22460604a4d97e62895cc6b254a
Content-Disposition: form-data; name="password"

bar
--bbecc22460604a4d97e62895cc6b254a--
```

[Script]
assert(response.Status === 200);
assert(response.Body === expectedUserId);

Log Section

Syntax

LogSectionDefinition :
##### Text NL

Example

##### User Authentication Tests

// ...

Explanation

A log section will produce a log output containing the given text to provide a visual separation between requests.

These sections can be used everywhere where requests can be specified as well.

Request

Syntax

RequestDefinition :
RequestHeader NL (RequestBlock NL)* RequestDelimiter NL

RequestHeader :
StringLiteral WS+ StringLiteral

RequestBlock :
RequestBlockHeader NL RequestBlockContent

RequestDelimiter :
--- -*

Example

GET https://example.com/api/users

[Header]
Accept: application/json
Authentication: basic {{ .credentials.token }}

[Script]
assert(response.StatusCode === 200);

---

Explanation

Define a request to be executed. The request definition starts with the request header consisting of the method followed by the URI (separated by one or more spaces). The request header is the only mandatory field for a valid request definition.

After that, you can specify more details about the request in different blocks. In the following documentation sections, all available blocks are listed and explained.

Method and URL

Syntax

RequestHeader :
StringLiteral WS+ StringLiteral

Example

GET https://example.com/api/users

Explanation

The request header defines the method and URL for a request and is the only mandatory element to define a request.

The method can be any uppercase string.

The URL can either be defined as an unquoted string literal or as a quoted string if spaces are required in the URL. Template substitution is supported.

✅ Valid

GET https://example.com/api/users

✅ Valid

GET https://example.com/api/users/{{.userId}}

✅ Valid

GET "https://example.com/api/users/some user"

✅ Valid

GET "https://example.com/api/users/{{ .userId }}"

❌ Invalid

GET https://example.com/api/users/{{ .userId }}

❌ Invalid

GET https://example.com/api/users/some user

Options

RequestOptions :
[Options] NL+ RequestOptionsContent

RequestOptionsContent :
TomlKeyValues

Example

[Options]
cookiejar = "admin"
condition = {{ isset . "userId" }}
delay = "5s"

Explanation

Define additional options to the requests. The format of the contents of this block is TOML.

The option values support template substitution.

Below, all available options are explained.

`cookiejar`

Type: string | number
Default: "default"

Defines the cookie jar to be used for saving and storing cookies. A cookie jar can be specified by either a number or a string. Every cookie jar contains a separate set of cookies collected from requests performed with that cookie jar specified.

`storecookies`

Type: boolean
Default: true

Defines if a cookie set by a request's response shall be stored in the cookie jar.

`sendcookies`

Type: boolean
Default: true

Defines if cookies stored in the specified cookie jar shall be sent to the server on request.

`noabort`

Type: boolean
Default: false

When enabled, a batch execution will not be canceled when the request execution or assertion failed.

`alwaysabort`

Type: boolean
Default: false

Forces a batch request to abort if the request execution or assertion failed, even if the --no-abort CLI flag has been passed.

`condition`

Type: boolean
Default: true

Defines if a request shall be executed or not. This is useful in combination with template substitution.

For example, the following request will only be executed when localAddress is set in the current state.
[Options]
condition = {{ isset . "localAddress" }}

`delay`

Type: string
Default: 0

A duration formatted as a Go time.ParseDuration compatible string. Execution will pause for this duration before the request is executed.

`responsetype`

Type: string
Default: ""

Explicit type declaration for body parsing. Implicit body parsing (json/xml) can be prevented by setting this option to raw.

`followredirects`

Type: bool
Default: true

Define whether or not to follow redirect responses on GET requests.

RequestOptions :
[Header] NL+ RequestHeaderContent

RequestHeaderContent :
HeaderKV*

HeaderKV :
/[A-Za-z\-]+/ : WS* /.*/ NL

Example

[Header]
Content-Type: application/json
Accept: application/json
X-Some-Custom-Header: foo bar bazz

Explanation

Define HTTP headers sent with the request in HTTP conform header representation.

Template parameters in the headers' value fields will be substituted.

QueryParams

QueryParams :
[QueryParams] NL+ QueryParamsContent

QueryParamsContent :
TomlKeyValues

Example

[QueryParams]
page = {{.page}}
count = 100
field = ["username", "age", "id"]
token = "{{.apitoken}}"

Explanation

Define additional query parameters which will be appended to the request URL. The format of the contents of this block is TOML.

An array of values will be represented as a repetition of the same query parameter with the different contained values assigned.

The example from above results in the following query parameters.
page=5&count=100&field=username&field=age&field=id

Template parameters in parameter values will be substituted.

Auth

RequestOptions :
[Auth] NL+ RequestAuthContent

RequestAuthContent :
TomlKeyValues

Example

[Auth]
username = "foo"
password = "{{.credentials.password}}"

Explanation

Auth is a utility block for easily defining basic or token authorization. When defined, the Authorization header will be set accordingly to the request.

Basic Auth

If you want to add basic auth to your request, simply define a username and password. If both values are set, the username and password are joined by a : and encoded using end-padded base64. The encoded value is then set to the Authorization header using the basic auth type (as defined in RFC 7671).

Example
[Auth]
username = "foo"
password = "bar"
his input will result in the following header.
Authorization: basic Zm9vOmJhcg==

Token Auth

You can also specify a token set as Authorization header. If defined, the token will be prefixed with a token type.

Example
[Auth]
type = "bearer"
token = "foobarbaz"
This input will result in the following header.
Authorization: bearer foobarbaz
[Auth]
token = "foobarbaz"
This input will result in the following header.
Authorization: foobarbaz

Body

RequestOptions :
[Body] NL+ RequestBodyContent

RequestBodyContent :
BlockDelimitedContent | UndelimitedContent

BlockDelimitedContent :
BlockDelimiter NL /.*/ NL BlockDelimiter

UndelimitedContent :
(/.*/ NL)* NL

BlockDelimiter :
```

Example

[Body]
```
{
  "user": {
    "name": "{{.userName}}",
    "favorite_programming_langs": [
      "Go",
      "Rust"
    ],
  }
}
```

Explanation

Define the data to be transmitted with the request.

If you want to use the template parameter braces ({{, }}) without substitution, you can escape them using a backslash.

Example:

[Body]
```
{
  "user": {
    "name": "\{\{This will not be substituted\}\}",
    "favorite_programming_langs": [
      "Go",
      "Rust"
    ],
  }
}
```

FormData

FormData :
[FormData] NL+ FormDataContent

FormDataContent :
TomlKeyValues

Example

[FormData]
someString = "some string"
someInt = 42
someFile = @files/goat.png:image/png

Explanation

Defines entries in a key-value pair format which will be sent in the request body as multipart/form-data request. The format of the contents of this block is TOML.

The file's content type can be specified after the file descriptor, separated by a colon (:). Otherwise, the content type will default to application/octet-stream.

The example from above results in the following body content.

--e8b9253313450dbcf0d09df1a0f3ff36dd00e888339415a239ce167f279c
Content-Disposition: form-data; name="someInt"

42
--e8b9253313450dbcf0d09df1a0f3ff36dd00e888339415a239ce167f279c
Content-Disposition: form-data; name="someFile"; filename="goat.png"
Content-Type: image/png

<binary file content>
--e8b9253313450dbcf0d09df1a0f3ff36dd00e888339415a239ce167f279c
Content-Disposition: form-data; name="someString"

some string
--e8b9253313450dbcf0d09df1a0f3ff36dd00e888339415a239ce167f279c--

Template parameters in parameter values will be substituted.

PreScript

RequestOptions :
[PreScript] NL+ RequestPreScriptContent

RequestPreScriptContent :
BlockDelimitedContent | UndelimitedContent

BlockDelimitedContent :
BlockDelimiter NL /.*/ NL BlockDelimiter

UndelimitedContent :
(/.*/ NL)* NL

BlockDelimiter :
```

Example

[PreScript]
```
var fileName = requestedFile.Metadata.Name;
```

Explanation

A script section which is evaluated before the template parameters are substituted and the actual request is executed. This can be used to put values from previous responses into variables which can then be used in the request parameters.

Example:

POST http://example.com/api/user

[Body]
// ...

[Script]
assert(response.StatusCode === 201);
var user = response.Body;

---

GET http://example.com/api/user/{{.userid}}

[PreScript]
var userid = user.id;

PreScript will always be executed before template parameters in the request definition are substituted. This makes it possible to use the results in various fields like [Options], [Body], [Header] or [Script].

Scripts are written in ES5.1 conform JavaScript. More on that can be found in the Script section documentation.

Script

RequestOptions :
[Script] NL+ RequestScriptContent

RequestScriptContent :
BlockDelimitedContent | UndelimitedContent

BlockDelimitedContent :
BlockDelimiter NL /.*/ NL BlockDelimiter

UndelimitedContent :
(/.*/ NL)* NL

BlockDelimiter :
```

Example

[Script]
```
assert(response.StatusCode === 200, `Response status code was ${response.StatusCode}`);
assert(response.Body.UserName === "Foo Bar");
```

Explanation

A script section which is executed after a request has been performed and a response has been received. This is generally used to assert response values like status codes, header values or body content.

Scripts are written in ES5.1 conform JavaScript.

The context of the script always contains the current values in the batch state as global variables.

A special variable set in each [Script] section is the response variable, which contains all information about the request response. The Response object contains the following fields.

type Response struct {
	StatusCode    int
	Status        string
	Proto         string
	ProtoMajor    int
	ProtoMinor    int
	Header        map[string][]string
	ContentLength int64
	BodyRaw       []byte
	Body          any
}

Body is a special field containing the response body content as a JavaScript object which will be populated if the response body can be parsed. Parsers are currently implemented for json and xml and are chosen depending on the responsetype option or the Content-Type header. If neither are set, the raw response string gets set as Body. By setting the responsetype to raw, implicit body parsing can be prevented.

In any script section, a number of built-in functions like assert can be used, which are documented here.

If a script section throws an uncaught exception, the test will be evaluated as failed.

Templating

One of the most powerful features of Goat is the ability to use variables from the state in almost every value field in a request definition.

These values are substituted using the powerful templating engine of Go. We will only go over the most basic syntax. Please review the Go template documentation for full details.

Syntax

Template parameters in Goatfiles are defined inside double-braces ({{ }}). Variables from the state are referenced using a dot notation. Let's take a look at the following example.

We have the following state.

instance: "http://example.com"

credentials:
  username: "foobar"
  password: "password"

Now, we can reference the values in our state as follows:

POST {{.instance}}/api/v1/auth/login

[Body]
{
  "username": "{{.credentials.username}}"
  "password": "{{.credentials.password}}"
}

Functions can be called using the name of the function followed by the parameters separated by spaces. Lets take a look at an example.

We assume the following state:

name: "Max"

{{ printf "Hello, %s!" .name }}

This will result in the string value "Hello, Max!".

As you can see, you can use the built-in templating functions provided by Go.
There are also a lot of built-in functions provided by Goat which you can use in your Goatfiles.

Built-ins

The following built-in functions are available in templates used in Goatfiles.

`base64`

base64 <value: string> -> string

Returns the input value as base64 encoded string with padding.

Example:

{{ base64 "hello world" }}

`base64Url`

base64Url <value: string> -> string

Returns the input value as base64url encoded string with padding.

Example:

{{ base64Url "hello world" }}

`base64Unpadded`

base64Unpadded <value: string> -> string

Returns the input value as base64 encoded string without padding.

Example:

{{ base64Unpadded "hello world" }}

`base64UrlUnpadded`

base64UrlUnpadded <value: string> -> string

Returns the input value as base64url encoded string without padding.

Example:

{{ base64UrlUnpadded "hello world" }}

`formatTimestamp`

formatTimestamp <value: Date> <format?: string> -> string
formatTimestamp <value: string> <valueFormat: string> <format?: string> -> string

Takes either a Date object and an optional output format or a timestamp string, the format for the input timestamp and an optional output format and returns the formatted timestamp.

The format is according to Go's time package definition. You can also specify the names of the predefined formats like rfc3339 or DateOnly. If no format is passed, the time will be represented as Unix seconds.

`md5`

md5 <value: string> -> string

Returns the HEX encoded MD5 hash of the given input value.

Example:

{{ md5 "hello world" }}

`sha1`

sha1 <value: string> -> string

Returns the HEX encoded SHA1 hash of the given input value.

Example:

{{ sha1 "hello world" }}

`sha256`

sha256 <value: string> -> string

Returns the HEX encoded SHA256 hash of the given input value.

Example:

{{ sha256 "hello world" }}

`sha512`

sha512 <value: string> -> string

Returns the HEX encoded SHA512 hash of the given input value.

Example:

{{ sha512 "hello world" }}

`randomString`

randomString <length?: integer> -> string

Returns a random string with the given length. If no length is passed, the default length is 8 characters.

Example:

{{ randomString 16 }}

`randomInt`

randomInt <n?: integer> -> int

Returns a random integer in the range [0, n) where n is given as parameter. if no parameter is passed, n defaults to the max int value.

Example:

{{ randomInt 256 }}

`timestamp`

timestamp <format?: string> -> string

Returns the current timestamp in the given format.

Example:

{{ timestamp "Mon, 02 Jan 2006 15:04:05 MST" }}

`isset`

isset <map: map[string]any> <key: string> -> bool

Returns true when the given key is present and its corresponding value is not nil in the given map. Otherwise, false is returned.

Example:

{{ isset . "username" }}

`json`

json <value: any> <ident?: string | int> -> string

Serializes a given value into a JSON string. You can pass a string value used as ident or a number of spaces used as indent.

Example:

{{ json .someObject 2 }}

Scripting

Scripting sections like [Script] and [PreScript] use a dedicated scripting micro-engine for maximum flexibility in your test setup and procedures.

Goat uses ES5.1 conform JavaScript interpreted by the goja micro-engine.

In each script instance, you have access to the current state values via the global environment variables. Also, you can define global variables using the var statement to define values which will be saved in the state after successful script execution.

Example:

Initial State
foo = 1

Script Execution
var bar = foo * 2;

Resulting State
foo = 1
bar = 2

Also, some built-in functions are available in each script instance.

Builtins

The following built-in functions are available in each script instance.

`assert`

function assert(expression: bool, fail_message?: string): void;

Takes an expression which, when evaluated to false, will throw an assert exception. You can pass an additional fail_message which will be shown in the exception. This can be used to assert values in responses and fail test execution if they are invalid.

Example

assert(response.StatusCode >= 400, `Status code was ${response.StatusCode}`);

`assert_eq`

function assert_eq(value: any, expected: any, fail_message?: string): void;

Takes a value and an expected value and deep-equals them. That means, when comparing objects and lists, their structure as well as primitive contenst are compared as well. If the comparison fails, it will throw an exception which will display both compared values. You can also pass an additional fail_message to further specify the error output.

Example

assert_eq(response.StatusCode, 200, "invalid status code");

`print`

function print(...message: string[]): void;

Prints the given message to the terminal without a leading new line.

Example

print("Hello world!");

`println`

function println(...message: string[]): void;

Prints the given message to the terminal with a leading new line.

Example

println("Hello world!");

`info`

function info(...message: string[]): void;

Logs an info log entry to the output logger(s) with the given message.

Example

info("Hello world!");

`warn`

function warn(...message: string[]): void;

Logs a warn log entry to the output logger(s) with the given message.

Example

warn("Hello world!");

`error`

function error(...message: string[]): void;

Logs an error log entry to the output logger(s) with the given message.

Example

error("Hello world!");

`fatal`

function fatal(...message: string[]): void;

Logs a fatal log entry to the output logger(s) with the given message. This will also abort the batch execution.

Example

fatal("Hello world!");

`debug`

function debug(...message: string[]): void;

Logs a debug log entry to the output logger(s) with the given message.

Example

debug("Hello world!");

`infof`

function infof(format: string, ...values: any[]): void;

Logs an info log entry to the output logger(s) with the given format formatted with the given values. Formatting is handled according to Go's formatting implementation.

Example

infof("Hello %s!", "World");

`warnf`

function warnf(format: string, ...values: any[]): void;

Logs a warn log entry to the output logger(s) with the given format formatted with the given values. Formatting is handled according to Go's formatting implementation.

Example

warnf("Hello %s!", "World");

`errorf`

function errorf(format: string, ...values: any[]): void;

Logs an error log entry to the output logger(s) with the given format formatted with the given values. Formatting is handled according to Go's formatting implementation.

Example

errorf("Hello %s!", "World");

`fatalf`

function fatalf(format: string, ...values: any[]): void;

Logs a fatal log entry to the output logger(s) with the given format formatted with the given values. Formatting is handled according to Go's formatting implementation. This will also abort the batch execution.

Example

fatalf("Hello %s!", "World");

`debugf`

function debugf(format: string, ...values: any[]): void;

Logs a debug log entry to the output logger(s) with the given format formatted with the given values. Formatting is handled according to Go's formatting implementation.

Example

debugf("Hello %s!", "World");

`jq`

function jq(object: any, src: string): any[];

For more complex analysis on JSON responses, jq can be used to perform JQ commands on any passed object. Goat uses itchyny/gojq as implementation of JQ. Here you can find more information about the JQ syntax.

The results are always returned as a list. If there are no results, an empty list is returned. When the command compilation fails or the JQ execution fails, the function will throw an exception.

Example

Explanation: Lists all instances recursively where the value of href is an object and is empty.

jq(response.Body, `..
    | if type == "object" then .href else empty end
    | if type == "object" then . else empty end
    | select( . | length == 0 )`);

Project Structure

When you write extensive integration tests for your products, you can use different strategies to re-use often utilized snippets. In this document, you will find some inspiration for how large sets of Goatfiles can be structured.

Centralizing commonly used snippets

As you might know, you can extract often used snippets into their own Goatfiles which can then be re-used in other files either via the use or the execute statement.

If you have multiple projects which share a similar backend logic, the best approach is to collect shared routines in a separate dependency. For example, you could create a separate Git repository with some utility Goatfiles, which is then added as a Git sub-module to your main projects. Make sure to prefix the name of the target directory in your projects with an underscore (_), so that the Goat CLI will not try to execute them by accident.

Example use-cases for that could be

logging in with one or more user accounts
setting default headers for all requests
encapsulating common procedures like creating and cleaning up entities

File Structure

In our projects, we employ the following file structure for API integration tests in all our projects. Maybe you can use this for inspiration for your own structure.

integrationtests/
├── _shed/
├── issues/
│   └── 123/
│       └── main.goat
├── tests/
│   └── users/
│       ├── _util.goat
│       ├── create.goat
│       ├── list.goat
│       └── delete.goat
├── params.toml.template
├── local.toml
├── staging.toml
└── ci.toml

As you can see, we have a directory called integrationtests/ in all of our projects which contains all tests, test utilities, as well as parameter files.

_shed/ is the name of our dependency containing some utility Goatfiles used, e.g., to create new users, log in as users with different permissions, set request defaults, and much more. As you can see, this directory is prefixed with _, so it will not be executed when calling Goat on the integrationtests/ directory.

issues/ contains sub-directories with the names of tickets on our issue board. These are there to demonstrate misbehaviour cases of our API, so these tests should fail on the latest dev state. These can also be used to test against fixes of these issues. When the issue is resolved, these tests should be moved into the respective tests/ sub-directory.

tests/ contains the actual integration tests grouped by features. These tests should always pass against the latest dev state, otherwise something might be broken.

We use some user-specific parameters passed into the tests which are stored in different *.toml files in the integrationtests/ directory. These should be specified in the project's .gitignore, because every developer might have their own parameters like API keys or user credentials. You could also put a parameter file for automatic tests in there (like ci.toml in our example) which is commited into the repository. The params.toml.template is a template file to base custom parameter files on. This is handy because the integration tests expect specific parameters to work with. An example params.toml.template could look as follows:

# The server instance to connect to.
instance = "http://localhost:10001"

# Credentials for a user with non-admin privileges.
[credentials.low]
username = "test@test"
password = "password"
apikey = "some api key"

# Credentiasl for a user with admin privileges.
[credentials.admin]
username = "root@root"
password = "password"
apikey = "some api key"

Documentation

To simplify the usage of our Goatfiles, we employ a system of documentation to record what a Goatfile does, which parameters it expects and – if it is a Goatfile meant to be executed or used in other Goatfiles – which state variables it creates that can be further used or captured in an execute's return statement.

The documentation is prefixed with three forward slashes and sits at the very top of the Goatfile. First, the purpose of the file is stated, followed by required parameters. After that, you can specify the variables created in the execution in the same manner.

Let's take the following as an example for a Goatfile which can be imported in your test via a use directive. Its purpose is to log in two users with two different privilege groups.

/// Logs in with a `low` and an `admin` user to the system and stores
/// the session in the `default` cookiejar for the `low` user and
/// in the `admin` cookiejar for the `admin` user.
/// 
/// 
/// Required Parameters
/// -----------------------------------------------------------------------
/// instance:                       The root address of the API instance.
/// credentials.low.username:       Username of the `low` user
/// credentials.low.password:       Password of the `low` user
/// credentials.admin.username:     Username of the `admin` user
/// credentials.admin.password:     Password of the `admin` user
/// 
/// 

### Setup

POST {{.instance}}/api/auth/login

[Header]
Content-Type: application/json

[Body]
{
  "username": "{{.credentials.low.username}}",
  "password": "{{.credentials.low.password}}"
}

[Script]
assert(response.StatusCode == 200, `Status code was ${response.StatusCode}`);

---

POST {{.instance}}/api/auth/login

[Options]
cookiejar = "admin"

[Header]
Content-Type: application/json


[Body]
{
  "username": "{{.credentials.admin.username}}",
  "password": "{{.credentials.admin.password}}"
}

[Script]
assert(response.StatusCode == 200, `Status code was ${response.StatusCode}`);

---

After this Goatfile is executed with use, the cookie jars for the two users persist and can be used with further requests in the executing Goatfile.

In contrast, this second example shows a Goatfile which is created to be used with an execute statement in another Goatfile:

/// Creates a new user.
/// 
/// 
/// Parameters
/// -----------------------------------------------------------------------
/// instance:   The root address of the API instance.
/// username:   User name of the new user.
/// password:   Password of the new user.
/// 
///
/// Returns
/// -----------------------------------------------------------------------
/// userid:     The ID of the new user.


// ...

The ID of the created user is returned so that the executing Goatfile can use it further but no other state information is retained.