mountebank - over the wire test doubles

the apothecary

Proxies

Proxies are one of the most powerful features of mountebank, rivaled only by the mighty injection. Proxies support record/replay behavior to easily capture a rich set of test data for your test scenarios. Each proxy definition allows you to define the fields which should be included in newly created predicates.

proxy response types take the following parameters:

Parameter	Default	Type	Description
`to`	required	A URL without the path (e.g. http://someserver:3000 or tcp://someserver:3000)	Defines the origin server that the request should proxy to.
`predicateGenerators`	`[]`	array	An array of objects that defines how the predicates for new stubs are created. Each object in the array defines the fields to generate predicates from. See below for examples.
`mode`	`proxyOnce`	string, one of `proxyOnce`, `proxyAlways` or `proxyTransparent`.	Defines the replay behavior of the proxy. The default `proxyOnce` mode doesn't require you to explicitly do anything to replay the proxied responses. The `proxyAlways` mode requires you to run the `mb replay` command (or equivalent) to switch from record mode to replay mode, but allows a richer set of data to be recorded. The `proxyTransparent` mode proxies the request but does not record any data. See below for details.
`addWaitBehavior`	`false`	boolean	If true, mountebank will add a `wait` behavior to the response with the same latency that the proxied call took. This is useful in performance testing scenarios where you want to simulate the actual latency of downstream services that you're virtualizing.
`addDecorateBehavior`	`null`	string, JavaScript	If defined, mountebank will add a `decorate` behavior to the saved response.

http and https proxies add additional optional parameters:

Parameter	Default	Type	Description
`cert`	`null`	A PEM-formatted string	The SSL client certificate
`key`	`null`	A PEM-formatted string	The SSL client private key
`ciphers`	Here	A valid cipher (see this page for formats)	For older (and insecure) https servers, this field allows you to override the cipher used to communicate
`secureProtocol`	`TLS_method`	A valid OpenSSL protocol method name	The SSL method to use
`passphrase`	`null`	string	Shared passphrase used for a single private key
`injectHeaders`	`{}`	object	Key-value pairs of headers to inject into the proxied request.

tcp proxies support the following optional configuration:

Parameter	Default	Type	Description
`keepalive`	`false`	boolean	If true, mountebank will not close the connection on each proxied request

It is occasionally useful to capture how long the original proxied request takes. mountebank stores the number of milliseconds for the request in the _proxyResponseTime field in the response. Setting the addWaitBehavior flag will add that latency to the saved response.

Note, if you use a corporate proxy, then the standard shell http_proxy or https_proxy environment variables will be honored.

Select the behavior of the proxy below for a relevant example:

Understanding predicateGenerators

Recording a rich set of test data through proxying requires also capturing the appropriate predicates from the request, so that saved responses are only replayed when the requests are similar. The predicateGenerators field defines the template for the generated predicates. Each object in the predicateGenerators array takes the following fields:

Parameter	Default	Type	Description
`matches`	`{}`	object	The fields that need to be equal in subsequent requests to replay the saved response. Set the field value `true` to generate a predicate based on it. Nested fields, as in JSON fields or HTTP headers, are supported as well, as long as the leaf keys have a `true` value. If you set the parent object key (e.g. `query`) to `true`, the generated predicate will use `deepEquals`, requiring the entire object graph to match.
`caseSensitive`	`false`	boolean	Determines if the match is case sensitive or not. This includes keys for objects such as query parameters.
`except`	`""`	string	Defines a regular expression that is stripped out of the request field before matching.
`xpath`	`null`	object	Defines an object containing a `selector` string and, optionally, an `ns` object field that defines a namespace map. The predicate's scope is limited to the selected value in the request field.
`jsonpath`	`null`	object	Defines an object containing a `selector` string. The predicate's scope is limited to the selected value in the request field.
`predicateOperator`	`deepEquals` or `equals`	string	Allows you to override the predicate operator used in the generated predicate. This is most often used to substitute an `exists` operator, e.g., for whether the given xpath expression exists in the incoming request or not. At times, it may be useful to use a `contains` operator if future requests can add more information to the field.
`inject`	`null`	string	Defines a JavaScript function that allows programmatic creation of the predicates.
`ignore`	`{}`	object	Use this option to ignore specific key of field from request based on `match` field.

With the exception of matches and inject, the fields correspond to the standard predicate parameters. Each object in the predicateGenerators array generates an object in the newly created stub's predicates array. You can decide how strictly you want the generated predicates to match object fields. The following example matches the root query field in its entirety:

POST /imposters HTTP/1.1
Host: localhost:38419
Accept: application/json
Content-Type: application/json

{
  "port": 3000,
  "protocol": "http",
"stubs": [{
  "responses": [{
    "proxy": {
      "to": "http://origin-server.com",
      "predicateGenerators": [{
        "matches": { "query": true },
        "caseSensitive": true
      }]
    }
  }]
}]}

This will generate a deepEquals predicate at the same level, which requires that all keys and values in the querystring match (although the order can be different). We added the caseSensitive parameter, which will also require the cases of the query keys and values to match. If the incoming request is to /test?q=mountebank&page=1, the following stub will be generating (the saved response is elided for clarity):

{
"stubs": [
  {
    "predicates": [{
      "caseSensitive": true,
      "deepEquals": {
        "query": {
          "q": "mountebank",
          "page": "1"
        }
      }
    }],
    "responses": [{
      "is": { ... }
        "_proxyResponseTime": 5
      }
    }]
  },
  {
    "responses": [{
      "proxy": {
        "to": "http://origin-server.com",
        "predicateGenerators": [{
          "matches": { "query": true },
          "caseSensitive": true
        }]
      }
    }]
  }
]}

Just as with predicates, we could have also specified only the query key we cared about:

POST /imposters HTTP/1.1
Host: localhost:38419
Accept: application/json
Content-Type: application/json

{
  "port": 3000,
  "protocol": "http",
"stubs": [{
  "responses": [{
    "proxy": {
      "to": "http://origin-server.com",
      "predicateGenerators": [{
        "matches": {
          "query": { "q": "mountebank" }
        }
      }]
    }
  }]
}]}

The same request to /test?q=mountebank&page=1 now generates a more limited predicate:

{
"stubs": [
  {
    "predicates": [{
      "equals": {
        "query": { "q": "mountebank" }
      }
    }],
    "responses": [{
      "is": { ... }
        "_proxyResponseTime": 5
      }
    }]
  },
  {
    "responses": [{
      "proxy": {
        "to": "http://origin-server.com",
        "predicateGenerators": [{
          "matches": {
            "query": { "q": "mountebank" }
          }
        }]
      }
    }]
  }
]}

xpath

The xpath and jsonpath predicate parameters work by limiting the scope of the generated predicate to the matching value in the proxied request. If the selector matches multiple values in the proxied request, they will all be part of the generated predicate, using standard predicate array matching rules. For example, following stub will look for number tags in the XML body:

POST /imposters HTTP/1.1
Host: localhost:38419
Accept: application/json
Content-Type: application/json

{
  "port": 3000,
  "protocol": "http",
"stubs": [{
  "responses": [{
    "proxy": {
      "to": "http://origin-server.com",
      "predicateGenerators": [{
        "matches": { "body": true },
        "xpath": { "selector": "//number" }
      }]
    }
  }]
}]}

We'll send it the following XML body:

POST / HTTP/1.1
Host: localhost:3000

<doc>
  <number>1</number>
  <number>2</number>
  <number>3</number>
</doc>

Rather than matching the entire body, the generated predicate will require all three values matching the xpath selector in the original proxied request to be present (in any order):

{
"stubs": [
  {
    "predicates": [{
      "xpath": { "selector": "//number" },
      "deepEquals": { "body": ["1", "2", "3"] }
    }],
    "responses": [{
      "is": { ... }
    }]
  },
  {
    "responses": [{
      "proxy": {
        "to": "http://origin-server.com",
        "predicateGenerators": [{
          "matches": { "body": true },
          "xpath": { "selector": "//number" }
        }]
      }
    }]
  }
]}

jsonpath

Similarly, the following stub looks for number elements in JSON:

POST /imposters HTTP/1.1
Host: localhost:38419
Accept: application/json
Content-Type: application/json

{
  "port": 3000,
  "protocol": "http",
"stubs": [{
  "responses": [{
    "proxy": {
      "to": "http://origin-server.com",
      "predicateGenerators": [{
        "matches": { "body": true },
        "jsonpath": { "selector": "$..number" }
      }]
    }
  }]
}]}

We'll send it the following JSON body:

POST / HTTP/1.1
Host: localhost:3000

[
  { "number": 1 },
  { "number": 2 },
  { "number": 3 }
]

Again, the generated predicate gets scoped to the jsonpath matches.

{
"stubs": [
  {
    "predicates": [{
      "jsonpath": { "selector": "$..number" },
      "deepEquals": { "body": [1, 2, 3] }
    }],
    "responses": [{
      "is": { ... }
    }]
  },
  {
    "responses": [{
      "proxy": {
        "to": "http://origin-server.com",
        "predicateGenerators": [{
          "matches": { "body": true },
          "jsonpath": { "selector": "$..number" }
        }]
      }
    }]
  }
]}

inject

In advanced scenarios, you need more fine-grained control over the creation of the predicates. In such scenarios, you can use the inject option.

The inject option requires the --allowInjection command line option.

The inject field takes a string representing a JavaScript function that is expected to return an array of predicate objects. Since it provides full control over the predicate generation, the inject option will ignore any other parameters, like xpath. The function accepts a single object parameter, which contains the following fields:

Field	Description
`request`	The entire request object, containing all request fields
`logger`	A logger object with `debug`, `info`, `warn`, and `error` functions to write to the mountebank logs.

In this example, we'll add a predicate if a specific header exists. Here's the injection function fully expanded:


function (config) {
  const predicate = { exists: { headers: { 'X-Transaction-Id': false } } };
  if (config.request.headers['X-Transaction-Id']) {
    config.logger.debug('Requiring X-Transaction-Id header to exist in predicate');
    predicate.exists.headers['X-Transaction-Id'] = true;
  }
  return [predicate];
}

First let's add the imposter. The function must be passed as a single string, which makes it largely unreadable inline.

POST /imposters HTTP/1.1
Host: localhost:38419
Accept: application/json
Content-Type: application/json

{
  "port": 3000,
  "protocol": "http",
"stubs": [{
  "responses": [{
    "proxy": {
      "to": "http://origin-server.com",
      "predicateGenerators": [{
        "inject": "function (config) {\n  const predicate = { exists: { headers: { 'X-Transaction-Id': false } } };\n  if (config.request.headers['X-Transaction-Id']) {\n    config.logger.debug('Requiring X-Transaction-Id header to exist in predicate');\n    predicate.exists.headers['X-Transaction-Id'] = true;\n  }\n  return [predicate];\n}"
      }]
    }
  }]
}]}

We'll send it the following HTTP request:

POST / HTTP/1.1
Host: localhost:3000
X-Transaction-Id: 100

SUCCESS

Since the X-Transaction-Id header was passed, the generated predicate requires it to exist to replay the response later.

{
"stubs": [
  {
    "predicates": [{
      "exists": { "headers": { "X-Transaction-Id": true } }
    }],
    "responses": [{
      "is": { ... }
    }]
  },
  {
    "responses": [{
      "proxy": {
        "to": "http://origin-server.com",
        "predicateGenerators": [{
          "inject": "function (config) {\n  const predicate = { exists: { headers: { 'X-Transaction-Id': false } } };\n  if (config.request.headers['X-Transaction-Id']) {\n    config.logger.debug('Requiring X-Transaction-Id header to exist in predicate');\n    predicate.exists.headers['X-Transaction-Id'] = true;\n  }\n  return [predicate];\n}"
        }]
      }
    }]
  }
]}

ignore

Support ignoring certain keys in predicateGenerators

POST /imposters HTTP/1.1
Host: localhost:38419
Accept: application/json
Content-Type: application/json

{
  "port": 3000,
  "protocol": "http",
"stubs": [
  {
    "responses": [{
      "proxy": {
        "to": "http://origin-server.com",
        "predicateGenerators": [{
          "matches": { "query": true },
          "ignore": { "query": "startDate" }
        }]
      }
    }]
  }
]}

Then we get a request /path?limit=100&enhanced=true&startDate=2017-09-07&endDate=2017-10-11

The saved predicate should not have the startDate key.

{
"stubs": [
  {
    "predicates": [{
      "deepEquals": { "query": { "limit": "100", "enhanced": "true", "endDate": "2017-10-11" } }
    }],
    "responses": [{
      "is": { ... }
    }]
  },
  {
    "responses": [{
      "proxy": {
        "to": "http://origin-server.com",
        "predicateGenerators": [{
            "matches": { "query": true },
            "ignore": { "query": "startDate" }
        }]
      }
    }]
  }
]}

Understanding proxy modes

The mode defines the behavior of the proxy.

proxyOnce - ensures that the same request (defined by the predicates) is never proxied twice. mountebank only records one response for each request, and automatically replays that response the next time the request predicates match.
proxyAlways - All calls will be proxied, allowing multiple responses to be saved for the same logical request. You have to explicitly tell mountebank to replay those responses.
proxyTransparent - All calls will be proxied, but will not be recorded.

proxyOnce

The default proxyOnce mode is simpler; it always creates a new stub in front of the stub with the proxy response, relying on mountebank's first-match policy to automatically replay the saved response in the new stub the next time a request matches the predicates. Imagine the following stubs array, set by us when we create the imposter:

POST /imposters HTTP/1.1
Host: localhost:38419
Content-Type: application/json

{
  "port": 2000,
  "protocol": "http",
"stubs": [{
  "responses": [{
    "proxy": {
      "to": "http://origin-server.com",
      "mode": "proxyOnce",
      "predicateGenerators": [{ "matches": { "path": true } }]
    }
  }]
}]}

When we issue an HTTP call to /test, the stub will proxy all of the request details to http://origin-server.com/test, and save off the response in a new stub in front of the stub with the proxy response:

{
"stubs": [
  {
    "predicates": [{ "deepEquals": { "path": "/test" } } ],
    "responses": [{
      "is": {
        "body": "Downstream service response",
        ...
      }
    }]
  },
  {
    "responses": [{
      "proxy": {
        "to": "http://localhost:2001",
        "mode": "proxyOnce",
        "predicateGenerators": [{ "matches": { "path": true } }]
      }
    }]
  }
]}

Because of mountebank's first-match policy on stubs, the next time the imposter receives a request to /test, the saved predicates on the newly created stub will match, and the recorded response will be replayed. If the imposter receives a call to /different-path, then it will proxy again, creating a new stub, because the path is different.

proxyAlways

The proxyAlways mode saves stubs behind the proxy stub. This allows you to record a richer set of interactions with the origin server because it can record multiple responses for the same logical request (as defined by the predicates). The consequence is that it requires you to save off the imposter representation and remove or reorder the proxy stubs to replay those interactions. The easiest way to do that is with the mb replay command.

Let's say you had the following stubs array:

POST /imposters HTTP/1.1
Host: localhost:38419
Content-Type: application/json

{
  "port": 4000,
  "protocol": "http",
"stubs": [{
  "responses": [{
    "proxy": {
      "to": "http://origin-server.com",
      "mode": "proxyAlways",
      "predicateGenerators": [{ "matches": { "path": true } }]
    }
  }]
}]}

Every time we send a request to /test, it will be proxied to http://origin-server.com/test. The first time we do that, just like the proxyOnce example, it will add a new stub. The difference is that the new stub will be created at the end of the array:

{
"stubs": [
  {
    "responses": [
      {
        "proxy": {
          "to": "http://origin-server.com",
          "mode": "proxyAlways",
          "predicateGenerators": [{ "matches": { "path": true } }]
        }
      }
    ]
  },
  {
    "predicates": [{ "deepEquals": { "path": "/test" } }],
    "responses": [{
      "is": {
        "body": "Request number 1",
        ...
      }
    }]
  }
]}

Because the proxy occurs before the new stub, the next request will continue to use the proxy response. This allow us to capture multiple responses for the same logical request. If we once again send a request to the /test path, since a stub with the predicate already exists, the response will be added to the existing stub:

{
"stubs": [
  {
    "responses": [
      {
        "proxy": {
          "to": "http://origin-server.com",
          "mode": "proxyAlways",
          "predicateGenerators": [{ "matches": { "path": true } }]
        }
      }
    ]
  },
  {
    "predicates": [{ "deepEquals": { "path": "/test" } }],
    "responses": [
      {
        "is": {
          "body": "Request number 1",
          ...
        }
      },
      {
        "is": {
          "body": "Request number 2",
          ...
        }
      }
    ]
  }
]}

This configuration allows you to capture as rich a set of data as your downstream system provides and play it back with the appropriate predicates. The only additional complexity with proxyAlways is that you have to explicitly switch to replay mode. Conceptually, replaying is as simple as removing the proxies. The easiest way to do that is by using the mb replay command, passing in the port if you're using a nonstandard port for running mb. If, for example, we had previously started mountebank on port 3535, we could switch to replay mode with the following call:

mb replay --port 3535

As always, there's more than one way to do it. You can instead use the API, retrieving all imposters with a call to GET /imposters?replayabe=true&removeProxies=true and sending the payload in a call PUT /imposters.

Now if we look at the imposter configuration, you'll notice that the proxy has been removed. Only the saved responses will be used.

{
"stubs": [
  {
    "predicates": [{ "deepEquals": { "path": "/test" } }],
    "responses": [
      {
        "is": {
          "body": "Request number 1",
          ...
        }
      },
      {
        "is": {
          "body": "Request number 2",
          ...
        }
      }
    ]
  }
]}

proxyTransparent

The proxyTransparent mode does not save any responses and simply proxies the requests transparently.

Let's say you had the following stubs array:

POST /imposters HTTP/1.1
Host: localhost:38419
Content-Type: application/json

{
  "port": 5050,
  "protocol": "http",
"stubs": [{
  "responses": [{
    "proxy": {
      "to": "http://origin-server.com",
      "mode": "proxyTransparent"
    }
  }]
}]}

Every time we send a request to /test, it will be proxied to http://origin-server.com/test. There will be no stub created off the back of the request.

Adding latency to the saved responses with addWaitBehavior

Every response saves the time it took to call the downstream service in the _proxyResponseTime field. You can tell mountebank to automatically translate that into a wait behavior, which will add the same latency to the saved response. This is a useful technique during performance testing, where you want to virtualize downstream services but still want realistic latencies.

Let's create the following proxy configuration:

POST /imposters HTTP/1.1
Host: localhost:38419
Content-Type: application/json

{
  "port": 6000,
  "protocol": "http",
"stubs": [{
  "responses": [{
    "proxy": {
      "to": "http://origin-server.com",
      "addWaitBehavior": true
    }
  }]
}]}

Next we'll send a request to our proxy imposter to create a new saved response:

GET / HTTP/1.1
Host: origin-server.com

If you look at the imposter configuration, you'll notice the new wait behavior. The value will be the same as the _proxyResponseTime field, which means that the saved response will add the same latency as the real service.

{
"stubs": [
  {
    "predicates": [],
    "responses": [{
      "is": {
        "statusCode": 200,
        ...
        "_proxyResponseTime": 219
      },
      "behaviors": [
        {
          "wait": 219
        }
      ]
    }]
  },
  {
    "responses": [{
      "proxy": {
        "to": "http://origin-server.com",
        "addWaitBehavior": true
      }
    }]
  }
]}

Decorating saved responses with addDecorateBehavior

A decorate behavior on a proxy response affects the proxy response itself, not the saved response that the proxy creates. If you want to add a decorate behavior on the saved response, you do so with the addDecorateBehavior field.

We'll decorate our saved responses with the following JavaScript function:

config => {
  config.response.body = config.response.body + ' DECORATED!';
}

Add that function in the addDecorateBehavior field of the proxy configuration:

POST /imposters HTTP/1.1
Host: localhost:38419
Content-Type: application/json

{
  "port": 5000,
  "protocol": "http",
"stubs": [{
  "responses": [{
    "proxy": {
      "to": "http://origin-server.com",
      "addDecorateBehavior": "config => { config.response.body = config.response.body + ' DECORATED!'; }"
    }
  }]
}]}

Next we'll send a request to our proxy imposter to create a new saved response. Notice that the decorator doesn't affect the proxied response (to do that, we'd have to add the same function as a decorate behavior on the proxy response too):

GET / HTTP/1.1
Host: origin-server.com

HTTP/1.1 200 OK
Connection: close
Date: Fri, 19 May 2017 19:39:02 GMT
Transfer-Encoding: chunked

downstream service response

If you look at the imposter configuration, you'll notice the new decorate behavior.

{
"stubs": [
  {
    "predicates": [],
    "responses": [{
      "is": {
        "body": "downstream service response",
        ...
      },
      "behaviors": [
        {
          "decorate": "config => { config.response.body = config.response.body + ' DECORATED!'; }"
        }
      ]
    }]
  },
  {
    "responses": [{
      "proxy": {
        "to": "http://origin-server.com",
        "addDecorateBehavior": "config => { config.response.body = config.response.body + ' DECORATED!'; }"
      }
    }]
  }
]}

Changing the proxied request headers with injectHeaders

The injectHeaders field allows you to modify the request headers before passing them on to the downstream service. To demonstrate, let's create a mirror imposter so we can see what headers are sent. This imposter just takes the request headers it receives and sends them back in the response:

injectHeaders only work for http/s proxies

POST /imposters HTTP/1.1
Host: localhost:38419
Content-Type: application/json

{
  "port": 7002,
  "protocol": "http",
  "name": "Mirror",
  "stubs": [{
    "responses": [{
      "is": { "body": "The body." },
      "behaviors": [
        { "decorate": "function (req, res) { res.headers = req.headers; }" }
      ]
    }]
  }]
}

Now let's set up another imposter that will proxy requests to the mirror imposter. We'll also set up the inject headers field to insert some custom headers in the outgoing request:

POST /imposters HTTP/1.1
Host: localhost:38419
Content-Type: application/json

{
  "port": 7001,
  "protocol": "http",
  "name": "Inject Headers",
  "stubs": [{
    "responses": [{
      "proxy": {
        "to": "http://localhost:7002",
        "injectHeaders": {
           "X-My-Custom-Header-One": "my first value",
           "X-My-Custom-Header-Two": "my second value"
        }
      }
    }]
  }]
}

Then we send a request to our proxy imposter:

GET / HTTP/1.1
Host: localhost:7001

HTTP/1.1 200 OK
Host: localhost:7002
Accept: application/json
Connection: keep-alive
X-My-Custom-Header-One: my first value
X-My-Custom-Header-Two: my second value
Date: Thu, 09 Jan 2014 02:30:31 GMT
Transfer-Encoding: chunked

The body.

Now we can see that the X-My-Custom-Header-One and Two headers were returned back to us, reflected back from the mirror imposter after being injected into the outgoing request.