HTTP request
The HTTP request component allows you to execute HTTP requests.
It supports methods GET, POST, PUT, DELETE, and PATCH and has built-in support for basic and bearer token authentication. It also features so-called hooks, which allow you to execute JavaScript code before and after the request is made.
See how it works:
HTTP Request Explainer
Configuration
General
General can be overridden using the incoming object.If the configuration
body is overridden, make sure to also use bodyAs to parse the body correctly.The request method used when sending the request. (default: GET)
Override key: method (string)
The URL to send the request to.
Override key: url (string)
security.trustedDomains configuration property. To block all domains, set the property to an empty string.Enables you to configure headers.
headers is a JSON object with the header name as the key and the header value as the value.
Override key: headers (JSON object)
setHeader hook in the beforeHook, it will only override headers set in the Configure headers config value.Enables you to configure the request body.
Override key: body (any)
bodyAs or handle the body within a beforeHook with the hooks: setJSONBody, setFormBody or setMultipartBody which will result in same behavior.The type of the body. This will set the Content-Type header for the request. (default: text). options: json, binary, form-data, text and x-www-form-urlencoded
Override key: bodyAs (string)
When using json or x-www-form-urlencoded an object with key-value pairs is expected in the body.
When using the binary option, the localReference or externalReference is required in the body. If a binary is used from the content store, the localReference is required. The _id of the binary object is expected as value. When using a binary stored on the filesystem, the externalReference is required. The file path to the binary is expected as value.
When using the form-data option, at least one of the multipartText or multipartBinary is required in the body.
There is an example for both scenarios at the end of this section.
Response options
Response typeDefines how the HTTP response body is returned. (default: json)
Options:json – Parses the response as JSON.xml – Parses the response as XML.
Allows attributes without value. By default boolean attributes are ignored. When set to true:
<hello checked>
world
</hello>
will be parsed to:
{
hello: {
"@_checked": true,
"#text": "world"
}
}
Force rendering a tag with text-node. Otherwise, it creates a property with the tag name and assigns the value directly.
Example:
<hello>
world
</hello>
With alwaysCreateTextNode set to false:
{
hello: "world"
}
With alwaysCreateTextNode set to true:
{
hello: {
"#text": "world"
}
}
Comments are parsed with prop name @_comments.
Example:
<hello>
<!--world-->
</hello>
Will parse to:
{
hello: {
"@_comments": "world"
}
}
Attributes are ignored by the parser. By default set to true. This also means that any configuration regarding attributes will not apply.
Parse Attribute ValueForce parsing the attribute value. This is relevant when attribute values have another type than string.
Example:
<hello attr="true">
world
</hello>
With parseAttributeValue set to false:
{
hello: {
"@_attr": "true"
}
}
With parseAttributeValue set to true:
{
hello: {
"@_attr": true
}
}
Force parsing the tag value. Same logic applies as with parseAttributeValue.
Used to keep the order of tags in the result object.
Remove NS PrefixRemove namespace string from tag and attribute names.
Example:
<ns:hello ns:attr="true">
world
</ns:hello>
Will parse to:
{
"hello": {
"@_attr": "true"
}
}
Automatically trim whitespace characters from the start and end of text nodes when parsing XML.
Default: trimValues = true
If the checkbox is unchecked, trimValues becomes false and whitespace is preserved.
Example:
<user>
<id> 123 </id>
<username>
John Doe
</username>
</user>
Will parse to (when trimValues is true):
{
"user": {
"id": "123",
"username": "John Doe"
}
}
Will parse to (when trimValues is false):
{
"user": {
"id": "\t123\t",
"username": "\n \n John Doe \n \n\t\t",
"#text": "\n\t\t\n\t\t\n\t"
}
}
Unpaired Tags are the tags which don't have matching closing tag. Eg <br> in HTML. You can parse unpaired tags by setting the unpaired tags separated by a comma (,).
Example:
<hello>
world
<br>
</hello>
With unpairedTags set to br will parse to:
{
hello: {
"#text": "world",
unpaired: ""
}
}
text – Returns the response as plain text.buffer – Returns raw binary data as a buffer.content store – Stores the response stream in the content store.file store – Stores the response stream on the file system.The directory where the downloaded file will be stored when using file-store.
Can be set in the component settings or dynamically via the incoming message using outputPath.
The name of the file to be saved when using file-store.
Can be set in the component settings or dynamically via the incoming message using filename. If not provided, a name is automatically generated.
If enabled it will only resolve the body of the response. This works for the output of the component. The response body won't be resolved within the after-hook yet. When using the body there, make sure to explicitly look up ctx.result.body.
Hooks
A hook lets you execute plain JavaScript code within a sandbox making it secure and safe.
It has the same functions as the code component with a few additions:
ctx gives you access to the context of the message. This allows you to access the incoming message and the response using ctx.value and ctx.result respectively.
setLocal function to add data.setMethod(method: string); is used to set the method of the request.
setURL(url: string, query: string); is used to set the URL of the request.
setHeader(key: string, value: string); is used to set a header of the request.
setResponseType(responseType: string); is used to set the response type of the request. options: json, xml, text, buffer, content-store and file-store. When using json, xml, text or buffer, the response will get parsed into the expected responseType. When content-store is used, the binary in the response will be stored automatically to the content store and will output the fileName and localReference referring to the _id of the binary object. When file-store is used, the binary is stored on the file system to the path used in Output Path.
content-store option, the _id of the binary is generated based on the request URL. If multiple binaries share the same request URL, they may overwrite each other. To avoid this, consider appending distinguishing metadata (e.g., query parameters or headers) to the URL. This will ensure the binary gets a unique identifier.setBody(body: string); is used to set the body of the request.
setJSONBody(body: object); is used to set the body of the request as JSON.
setFormBody(body: object); is used to set the body of the request as form data.
await setMultipartBody(body: object); is used to set the body of the request as multipart form data.
Any of the above functions will overwrite the configured options within the config of the component. This can be handy when using values dynamically from the incoming message.
await request(url: string, options: object); executes a request. The options object can contain nearly all configuration options.
Example options object:
{
"method": "GET",
"url": "https://example.com",
"headers": {
"Content-Type": "application/json"
},
"body": {
"hello": "world"
},
"bodyAs": "json", // options: json, binary, form-data, text and x-www-form-urlencoded
"responseType": "json", // options: json, xml, text, buffer, content-store and file-store
"requestTimeout": 1000,
"responseTimeout": 1000,
"followRedirects": true,
"maxRedirects": 10,
"authType": "none", // options: none, basic, bearer, oauth and ntlm
}
authType is set in the options, the component will use the configured authentication settings.setLocal(key: string, value: any); use to set a local variable in the context of message. This can be used to share data between the two hooks.
getLocal(key: string); use to get a local variable in the context of message.
Before hook
The before hook is triggered before the request is sent. It allows you to modify the request before it is sent. The following functions are available in the before hook to help you do dynamic requests:
After hook
The after hook is triggered after the request is made. It allows you to post-process the response before it is sent.
Calling done
Because JavaScript allows you to write code asynchronously, you need to call the done function when your code is done. When you are not using any asynchronous syntax, the done call should be at the end of your script.
// your code
done();
done before the return statement.Authentication
TypeThe type of authentication used. (default: No Authentication).
For Basic Authentication, the username and password are required.
For Bearer Authentication, the token is required.
For NTLM Authentication, the username and password are required. The domain and workstation are optional. When creating or updating entities in SharePoint, make sure to set the NTLM Token Type to digest.
For OAuth2 - Client Authentication, the token URL, client ID, and client secret are required. The scope field is optional. When set, it will be used to retrieve the access token.
token URL from the OAuth2 - Client Authentication must be included in the list of trusted domains.TLS Options
Allow invalid certificatesWhen set to true, the server certificate is not verified. Verification happens at the connection level, before the HTTP request is sent. This setting can be useful when your server is using self-signed certificates. (default: false)
One or more certificate authorities to trust when making a TLS connection.
Example: ./ca.pem
When enabled, the client certificate and key are used to authenticate the client to the server. (default: false)
The client certificate file.
Example: ./client.pem
The client key file.
Example: ./client.key
Rate limit
Request limitThe max amount of requests during the interval.
IntervalThe interval in milliseconds in which the requests happen.
Max concurrentThe maximum concurrent executions per message.
In the HTTP request component it is possible to schedule multiple messages in parallel depending on the maxConcurrent value, where every single message could make multiple requests. This message will be used as a context for all requests that originate from this message. Each context will be scheduled with priority over new messages, only limited by the maxConcurrent value. The scheduler has a request limit and an interval, limiting the amount of requests per interval (in ms). Below is an image displaying the scheduling with the following configuration:
{
"interval": 500,
"requestLimit": 6,
"maxConcurrent": 3
}
Timeout
Request timeoutThe request timeout in milliseconds. The time in which the headers need to be received.
Response timeoutThe response timeout in milliseconds. The total time for the request.
Redirects
Follow redirectsIf enabled it will follow redirects.
Maximum number of redirects to followThe maximum number of redirects to follow.
Content Store
Enable Content Store / Enable lookup functionIf enabled, the component creates a connection to the Content Store. This allows the component to read and write files from and to the Content Store and enables the use of the lookup function. For more information about the lookup function, see the Code component.
Connection stringA MongoDB connection string.
Example: mongodb://<username>:<password>@localhost:27017/<databaseName>
Here <databaseName> is the database to use.
Whether or not to use TLS in case your mongoDB requires TLS.
Allow Invalid CertificatesChecking this will disable certificate validation. Warning: specifying this option in a production environment makes your application insecure and potentially vulnerable to expired certificates and to foreign processes posing as valid client instances.
Certificate Authority FileOne or more certificate authorities to trust when making a TLS connection.
Example: ./ca.pem
Disables in-memory caching for lookups (when using the lookup or lookupByKey fn in the hooks). When enabled, each lookup is fetched directly from the Content Store instead of using cached results. For more information about the lookup function, see the Code component.
Download Files
When the response body has a stream and it needs to be downloaded to the content store, set the responseType to content-store and make sure content-store is enabled in the settings of the component. It will result in a response object with localReference. This refers to the id of the saved binary in the content store. When the file needs to be stored on the file system instead of the content store, set the responseType to file-store in the component settings. This results in a response object containing externalReference, which refers to the filename, and outputPath, which refers to the full path of the file on the file system. When using file-store as the responseType, the filename and output path (directory) can be set dynamically by providing the filename and outputPath key in the incoming message or using handlebars in the settings if other keys are provided. But keep in mind that, when filename or outputPath are present, these values take precedence over the component settings including setting custom keys with handlebars. When filename is not set in the settings or incoming message, it will automatically create an id for the filename, similar to the content-store.
Upload Files
To upload a file, you can set bodyAs to binary or form-data. When using form-data, you can also add other form fields.
Example:When uploading a file using binary and the file source is a file path:
{
"externalReference" : "C:/Users/xill/Desktop/test.txt",
}
When uploading a file using binary and the file source is the Content Store:
{
"localReference" : "f02a73168ae19095d70f8f0ebc84cd135add4ccf8e008a9723a53c4865609f13ceed0079b8f56f92f1906843c8891e3165d198645c3bd33df5a224665a1060d1",
}
When uploading a file using form-data and the file source is a file path:
{
"multipartText" : {
"field1" : "value1",
"field2" : "value2"
},
"multipartBinary":{
"field3" : "C:/Users/xill/Desktop/test.txt",
}
}
{
"multipartText" : {
"field1" : "value1",
"field2" : "value2"
},
"multipartBinary":{
"field3" : {
"externalReference" : "C:/Users/xill/Desktop/test.txt",
"fileName" : "test.txt"
}
}
}
form-data a key is not unique and can be used multiple times. To do this you can wrap the key value pairs in an array.{
"multipartText" : [
{"field1" : "value1"},
{"field1" : "value2"}
],
"multipartBinary":[
{"field3" : "C:/Users/xill/Desktop/test.txt"}
{"field3" : "C:/Users/xill/Desktop/test2.txt"}
]
}
{
"multipartText" : {
"field1" : "value1",
"field2" : "value2"
},
"multipartBinary":{
"field3" : {
"localReference" : "f02a73168ae19095d70f8f0ebc84cd135add4ccf8e008a9723a53c4865609f13ceed0079b8f56f92f1906843c8891e3165d198645c3bd33df5a224665a1060d1",
"fileName" : "test.txt"
}
}
}
field3 now is an object with a localReference property and a mandatory fileName property. This is because the filename can't be derived from the binary when retrieving it from a localReference. When using an externalReference the fileName is optional.{
"multipartText" : [
{"field1" : "value1"},
{"field1" : "value2"}
],
"multipartBinary": [
{
"field3" : {
"localReference" : "f02a73168ae19095d70f8f0ebc84cd135add4ccf8e008a9723a53c4865609f13ceed0079b8f56f92f1906843c8891e3165d198645c3bd33df5a224665a1060d1",
"fileName" : "test.txt"
}
},
{
"field3" : {
"localReference" : "f02a73168ae19095d70f8f0ebc84cd135add4ccf8e008a9723a53c4865609f13ceed0079b8f56f92f1906843c8891e3165d198645c3bd33df5a224665a1060d1",
"fileName" : "test2.txt"
}
}
]
}
Multi-part form data
form-data as the bodyAs value, at least one of multipartText or multipartBinary is required. This property should be an object with the field names as keys and the field values as values.