HttpClient

Overview

HttpClient is a global table of methods that allows communication to other devices and web services via a custom HTTP protocol. The HttpClient API only functions over the control network.

The HttpClient API is compatible with Radius NX, Edge and Prism devices, but will not run on Solus NX DSPs.

HttpClient.

The following methods and properties are available for a HttpClient object.

HttpClient.Download

The Download method allows remote text content hosted on a website to be acquired. Only content which can be expressed as a single string can be accessed. Stringified JSON objects can be downloaded and converted back to a table using the JSON API's decode method.

The Download method returns a Boolean indicating if the call was accepted. If more than 8 Download or Upload calls are open without a callback, a false will be returned, otherwise a true will be returned, and the callback will eventually be called. The EventHandler callback function must be specified to receive a response or error.

Up to 8 Download and Upload calls may be open awaiting a response or timeout for all modules combined. They can all come from one module or be spread out. Once a response or timeout occurs, the slot is free for another access. The Download or Upload call will immediately fail if this limit is exceeded and no callback for that command will occur.

The Download method requires a Parameters argument which is itself a table of arguments. If a table is not provided, the function will return nil. The Parameters table can contain URL, Headers, User, Password, Timeout, or EventHandler, each indexed by the parameter name.

• Url: A proper Url parameter consists of a formatted web domain name such as “http://www.symetrix.co”. Consider using CreateUrl to construct the URL to be accessed. The maximum URL length is 1023 characters, more than this number of characters will be truncated without warning. Note that the output of the CreateUrl method is 4095characters. The Download method requires a Url parameter. However, the command will not be rejected if the Url parameter is blank. Instead, any errors caused by non-existent or malformed URLs will be returned to the callback as an error.

• Headers: The Headers parameter is itself a table of all the required headers. Each header has a suitable formatted array element such as ["content-type"] = "text/*". Headers are not required. Each header is a key and value, each key and value can be a maximum of 3823 characters, longer will be truncated without warning. More information on common headers can be found on MDN.

• User: The User parameter is the username for authentication if required. The default is no User value. Without a User, no Password entered will be used. The User is limited to 31 characters, longer will be truncated without warning.

• Password: The Password parameter is the password for authentication if required. The default is no password. Without a User, no password entered will be used. The password is limited to 31 characters, longer will be truncated without warning.

• Timeout: The Timeout parameter is the number of seconds to wait for a response before returning a timeout error. It can be any number but floating point will be rounded up to the next whole second. If not specified, the default timeout is 30 seconds.

• EventHandler: The EventHandler parameter is the function that should be called when a response is received, an error is encountered or there is a timeout.

The EventHandler callback function should be of the form: EventHandler(table Parameters, number ReturnCode, string Data, string Error, table Headers).

• Parameters: The Parameters argument will receive the Parameters table sent to the HttpClient.Download method. It is preserved in until returning from the EventHandler.

• Return Code: The ReturnCode is a number received from destination URL. Normally, 200 will be received for a good response but there are many possible HTTP Status codes that could be returned. Normally, the EventHandler will perform different actions depending on the code.

• Data: The Data argument is a string containing the returned data. If the string is a stringified JSON object, the json library can be used to convert back. The maximum length for a string returned will be 64000 characters. If the content is larger than this, the Data will be nil and error text will be returned, but the ReturnCode will still be whatever return code was returned by the site. Note: Large strings can be inefficient to process in Lua and should be used with caution as they may cause execution errors in the Intelligent Module for consuming too much processing time.

• Error: The Error argument is a string describing the error condition encountered or nil for no error. The errors are generated by computer (when running the script offline) or the Symetrix device firmware (when running the script online). The maximum error string length is 255 characters. Some common errors you may see include:

• "Download returned too much data to handle!"

• "Download returned too much headers data to handle!"

• "Upload returned too much data to handle!"

• "Download returned too much headers data to handle!"

• "Timeout waiting for response from Http request service!"

• Headers: The Headers argument is a table of the headers returned from the destination URL if successful. More information on common headers can be found on MDN.

Typical use:

-- callback function

function Response(Table, ReturnCode, Data, Error, Headers)
    print(string.format("URL requested = '%s'.", Table.Url))

    if (200 == ReturnCode) then
        print("Success!")
        print(string.format("Data returned = '%s'", Data))
    else
        print(string.format("Failed, error code #%d, '%s'", ReturnCode, Error))
    end
end

HttpClient.Download { Url = "https://www.symetrix.co", Headers = { ["Content-type"] = "text/*" }, Timeout = 3, EventHandler = Response }

This code will generate a response similar to the following:

URL requested = 'https://www.symetrix.co'.
Success!
Data returned = 'nil'

HttpClient.Upload

The Upload method allows remote content hosted on a website to be modified with text content. Only content which can be expressed as a single string can be sent. JSON objects can be uploaded if first stringified using the JSON API’s encode method.

The Upload method returns a Boolean indicating if the call was accepted. If more than 8 Download or Upload calls are open without a callback, a false will be returned, otherwise a true will be returned, and the callback will eventually be called. The EventHandler callback function must be specified to receive a response or error.

Up to 8 Download and Upload calls may be open awaiting a response or timeout for all modules combined. They can all come from one module or be spread out. Once a response or timeout occurs, the slot is free for another access. The Download or Upload call will immediately fail if this limit is exceeded and no callback for that command will occur.

The Upload method requires a Parameters argument which is itself a table of arguments. If a table is not provided, the function will return nil. The Parameters table can contain URL, Headers, User, Password, Data, Method, Timeout, or EventHandler.

• Url: The Url parameter consists of a formatted web domain name such as “http://www.symetrix.co”. Consider using CreateUrl to construct the URL to be accessed. The maximum URL parameter length is 1023 characters, more than this number of characters will be truncated without warning. The Upload method requires a Url and Data parameter. However, the command will not be rejected if the Url parameter is blank. Instead, any errors caused by non-existent or malformed URLs will be returned to the callback as an error.

• Headers: The Headers parameter is itself a table of all the required headers. Each header has a suitable formatted array element such as ["content-type"] = "text/*". Headers are not required. Each header is a key and value, each key and value can be a maximum of 3823 characters, longer will be truncated without warning. More information on common headers can be found on MDN.

• User: The User parameter is the username for authentication if required. The default is no User value. Without a User, no Password entered will be used. The User is limited to 31 characters, longer will be truncated without warning.

• Password: The Password parameter is the password for authentication if required. The default is no Password. Without a User, no Password entered will be used. The Password is limited to 31 characters, longer will be truncated without warning.

• Data: The Data parameter is the text data to be sent to the website in the form of a string. The Upload method requires a Data argument, but it can be an empty string or nil. The maximum length for a string sent will be 64000 characters. If the content is larger than this, it will be truncated without warning before sending. Note: Large strings can be inefficient to process in Lua and should be used with caution as they may cause execution errors in the Intelligent Module for consuming too much processing time.

• Method: The Method parameter specifies the method to be used for uploading. Typically, it is “POST” but can be “PUT” if the server supports it. If nil or an empty string is specified, POST is used. Any other method will be passed to the HTTP engine directly. The Method is limited to 31 characters; longer will be truncated without warning.

• Timeout: The Timeout parameter is the number of seconds to wait for a response before returning a timeout error. It can be any number, but floating point will be rounded up to the next whole second. If not specified, the default timeout is 30 seconds.

• EventHandler: The EventHandler parameter is the function that should be called when a response is received, an error is encountered or there is a timeout.

The callback function should be of the form: EventHandler (table Parameters, number ReturnCode, string Data, string Error, table Headers).

• Parameters: The Parameters argument will receive the Parameters table sent to the HttpClient.Upload method. It is preserved in memory until returning from the EventHandler.

• ReturnCode: The ReturnCode is a number received from the destination URL. Normally, 200 will be received for a good response but there are many possible HTTP Status codes that could be returned. Normally, the EventHandler will perform different actions depending on the code.

• Data: The Data argument is a string containing the returned data. If the string is a stringified JSON object, the json API can be used to convert back. The maximum length for a string returned will be 64000 characters. If the content is larger than this, the Data will be nil and error text will be returned, but the ReturnCode will still be whatever return code was returned by the site. Note: Large strings can be inefficient to process in Lua and should be used with caution as they may cause execution errors in the Intelligent Module for consuming too much processing time.

• Error: The Error argument is a string describing the error condition encountered or nil for no error. The maximum error string length is 255 characters.

• Headers: The Headers argument is a table of the headers returned from the website if successful. More information on common headers can be found on MDN.

Typical use:

--Use Post Test Server to test upload
--https://ptsv2.com/s/howitworks.html
ptsUrl = "Insert your Post Test URL Here"

-- Upload callback function
function UploadResponse(Table, ReturnCode, Data, Error, Headers)
    print("")
    print("Upload Callback:")
    print(string.format("URL requested = '%s'.", Table.Url))
    print(string.format("Data uploaded = '%s'.", Table.Data))
    print("Return Code: "..ReturnCode)
    print("Headers:")

    if (Headers ~= nil) then
        for headerName, headerValue in pairs(Headers) do
            print(string.format( "\t"..headerName..": "..headerValue ) )
        end
    else
        print("\tNone")
    end

    if (200 == ReturnCode) then
        print("Upload Success!")
        print(string.format("Data returned = '%s'", Data))
    else
        print(string.format("Failed, error code#%d, '%s'", ReturnCode, Error))
    end
end

--Do Upload
HttpClient.Upload { Url = ptsUrl, Method = "POST", Data = "test message", Timeout = 3, EventHandler = UploadResponse }

This code will generate a response similar to the following when run with a valid Post Test Server URL:

Upload Callback:
URL requested = 'http://ptsv2.com/t/rx49p-1611678942'.
Data uploaded = 'test message'.
Return Code: 200
Headers:
      X-Cloud-Trace-Context:  0365d7484760d3389ff39ce47de2d4ff    
      Content-Type:  text/html; charset=utf-8    
      Content-Length:  5229    
      Date:  Tue, 26 Jan 2021 16:36:12 GMT    
      Vary:  Accept-Encoding    
      Server:  Google Frontend
Upload Success!
Data returned = '<!DOCTYPE html>
…

HttpClient.CreateUrl

Combines a table of options to create a URL out of a hostname with any combination of port number, path and queries into a single formatted URL for use by the Download and Upload methods. The script may make the string itself and does not need to use this method if not helpful.

The CreateUrl method returns exactly one value, the constructed Lua string. If an error occurs, nil is returned. The maximum returned string length is 4095 characters.

The CreateUrl method requires a Parameters argument which is itself a table of arguments. If a table is not provided, the function will return nil. The Parameters table can contain Host, Port, Path, and Query options, each indexed by the parameter name.

• Host: The Host option should typically contain the “http://” or “https://” prefix. However, the presence of the prefix will not be verified. It could be prepended after the method is called if needed. A DNS resolution will not be performed on the Host option. The Host option should not contain spaces or other special characters as no automatic URL encoding will occur. The maximum length of the Host is 1023 characters. Longer strings will be truncated.

• Port: The Port option allows inclusion of an optional port number in the URL. The port number will be ignored if not an integer or if it is outside the range of 1 to 65535.

• Path: The Path option allows inclusion of an optional refining path to the URL within the host name space. The Path option does not need to have a leading slash. If a slash is not present at the start of a non-empty path and it is not present at the end of a non-empty URL, it will be added as needed in the URL construction. Spaces and some other special characters will be URL encoded, replacing those characters with valid URL character equivalents such as a space being replaced with “%20”. There are exceptions likes the ":" character that will not be URL encoded; if you need them encoded, you must do it manually. The maximum length of the Path is 255 characters.

• Query: The Query option is an optional table which contains one or more indexed parameters. Each parameter is an index and value. The required “?” charter will be prepended by the method to the first query and the “&” character will added by the method between items as needed. An empty Query table is the same as no table. The maximum length for each key is 255 characters. Longer keys will be truncated. The maximum length for each value is 255 characters. Longer values will be truncated. Additional Key/Value pairs will not be added if adding them will cause the maximum URL length to be exceeded. The Keys and Values are URL encoded by default, but this may be overridden by the Encode option described below.

• Encode: The Encode option is an optional bool that controls whether the "/" character in the Query index and Query value fields will be URL encoded, replacing those characters with a valid URL character equivalent. If not provided, a value of true will be used. This is useful if the queries have already been encoded or if you need to allow "/" through without encoding.

Any trailing fragments desired in the URL need to be added manually after calling this method.

Typical use:

print (HttpClient.CreateUrl( { Host = "http:/www.symetrix.co"} ))
print (HttpClient.CreateUrl( { Host = "http:/www.symetrix.co", Port = 49474} ))
print (HttpClient.CreateUrl( { Host = "http:/www.symetrix.co", Path = "Path/with : and &"} ))
print (HttpClient.CreateUrl( { Host = "http:/www.symetrix.co", Port = 49474, Path = "Path/with : and &"} ))
print (HttpClient.CreateUrl( { Host = "http:/www.symetrix.co", Path = "Path/with : and &", Query = { key = "headerValue", ["key with space"] = "header value with space"} } ))
print (HttpClient.CreateUrl( { Host = "http:/www.symetrix.co", Port = 49474, Path = "Path with : and &", Query = { key = "headerValue", ["key with space"] = "header value with space"} } ))
print (HttpClient.CreateUrl( { Host = "http:/www.symetrix.co", Path = "Path/with : and &", Query = { key = "headerValue", ["key+space"] = "header+value+with+space"}, Encode = true } ))
print (HttpClient.CreateUrl( { Host = "http:/www.symetrix.co", Path = "Path/with : and &", Query = { key = "headerValue", ["key+space"] = "header+value+with+space"}, Encode = false } ))

This code will produce the following output:

http:/www.symetrix.co
http:/www.symetrix.co:49474
http:/www.symetrix.co/Path/with%20%3a%20and%20%26
http:/www.symetrix.co:49474/Path/with%20%3a%20and%20%26
http:/www.symetrix.co/Path/with%20%3a%20and%20%26?key=headerValue&key%20with%20space=header%20value%20with%20space
http:/www.symetrix.co:49474/Path%20with%20%3a%20and%20%26?key=headerValue&key%20with%20space=header%20value%20with%20space
http:/www.symetrix.co/Path/with%20%3a%20and%20%26?key=headerValue&key%2bspace=header%2bvalue%2bwith%2bspace
http:/www.symetrix.co/Path/with%20%3a%20and%20%26?key=headerValue&key+space=header+value+with+space

HttpClient.EncodeParams

The EncodeParams method does parameter formatting and URL encoding on a passed in table of parameters. This is useful when generating a URL to be passed to the Download or Upload methods.

The EncodeParams method returns exactly one value, the encoded Lua string. If anything but a table is passed in, a nil is returned. The returned value will not exceed 4095 characters. The string will be truncated if the input or encoded strings exceed this.

The Parameters argument is a table which contains one or more indexed parameters. Each parameter is an index and value. The index and value will be used exactly as written. The required “?” character will be prepended by the method to the first query and the “&” character will added by the method between items as needed. An empty Query table is the same as no table. Spaces and other special characters in either index or value fields will be URL encoded, replacing those characters with valid URL character equivalents such as a space being replaces with “%20”. The maximum length for each key is 3823 characters. Longer keys will be truncated. The maximum length for each value is 3823 characters. Longer values will be truncated. Additional Key/Value pairs will not be added if adding them will cause the maximum URL length to be exceeded.

Typical use:

output = HttpClient.EncodeParams( { key = "value", ["valid key"] = "valid value"} )
print(output)

This code will produce the following output:

key=value&valid%20key=valid%20value

HttpClient.EncodeString

The EncodeString does URL encoding on a passed in string. Spaces and other special characters are replaced with their valid URL character equivalents. This is a tool provided to allow the programmer to create URL compatible strings for use by the Download and Upload methods.

The EncodeString method returns exactly one value, the encoded Lua string. If an error occurs, nil is returned.

The EncodeString method uses the following arguments.

• Input: The Input string argument and the returned value will not exceed 4095 characters, if the Input argument or the string produced would be longer than this, it will be truncated.

• EncodeSlash: The EncodeSlash argument determines if forward slashes (‘/’) should be encoded or left intact. The argument is optional and if not present, defaults to true so forward slashes are encoded.

Typical Use:

originalString = "Path/with : and &"
print(HttpClient.EncodeString(originalString))
print(HttpClient.EncodeString(originalString, false))

This code will produce the following output:

Path%2fwith%20%3a%20and%20%26
Path/with%20%3a%20and%20%26

HttpClient.DecodeString

The DecodeString does URL decoding on a passed in string. Encoded spaces and other special characters are replaced with their normal readable characters. This is a tool provided to allow the programmer to deconstruct a URL or returned content from the Download method.

The DecodeString method returns exactly one value, the decoded Lua string. If anything but a string is passed in, nil is returned.

The DecodeString method uses the following argument.

• Input: The Input string argument and the returned value will not exceed 4095 characters, it the Input argument or the string produced would be longer than this, it will be truncated.

Typical Use:

print(HttpClient.DecodeString("path%20with%20%3A%20and%20%26"))

This code will produce the following output:

Path with : and &

Usage Examples

The following examples illustrate how these can be used:

Example 1 – Putting Upload and Download together

This example shows how HttpClient.Upload and HttpClient.Download can be used together.

json = require("json")

--Post Test Server
--https://ptsv2.com/s/howitworks.html
ptsUrl = "https://ptsv2.com/t/a7nrl-1610735518/post"
--Note the Post Test Server in this example returns HTTP URL to the JSON version of the upload so that it can be subsequently downloaded. This was done by editing the "Body" configuration option to "{{URL-JSON}}".

-- Upload callback function
function UploadResponse(Table, ReturnCode, Data, Error, Headers)
      print("")
      print("Upload Callback:")
      print(string.format("URL requested = '%s'.", Table.Url))
      print(string.format("Data uploaded = '%s'.", Table.Data))
      print("Return Code: "..ReturnCode)
      print("Headers:")

      if (Headers ~= nil) then
            for headerName, headerValue in pairs(Headers) do
                  print(string.format( "\t"..headerName..": "..headerValue ) )
            end
      else
            print("\tNone")
      end

      if (200 == ReturnCode) then
            print("Upload Success!")
            print(string.format("Data returned = '%s'", Data))

            --download the data just uploaded from the address provided by the return data
            HttpClient.Download { Url = Data, Headers = { ["Content-type"] = "text/*" }, Timeout = 3, EventHandler = DownloadResponse }
      else
            print(string.format("Failed, error code#%d, '%s'", ReturnCode, Error))
      end
end

-- Download callback function

function DownloadResponse(Table, ReturnCode, Data, Error, Headers)
      print("")
      print("Download Callback:")
      print(string.format("URL requested: '%s'.", Table.Url))
      print("Return Code: "..ReturnCode)
      print("Headers:")

      if (Headers ~= nil) then
            for headerName,headerValue in pairs(Headers) do
                  print(string.format( "\t"..headerName..": "..headerValue ) )
            end
      else
            print("\tNone")
      end

      if (200 == ReturnCode) then
            print("Download Success!")
            print(string.format("Data returned = '%s'", Data))
            decodedData = json.decode(Data)
            print("Received: "..decodedData.Body)
      else
            print(string.format("Failed, error code#%d, '%s'", ReturnCode, Error))
      end
end

--Run the Example

HttpClient.Upload { Url = ptsUrl, Method = "POST", Data = "test message", Timeout = 3, EventHandler = UploadResponse }

This code will produce the following output:

Upload Callback:
URL requested = 'https://ptsv2.com/t/a7nrl-1610735518/post'.
Data uploaded = 'test message'.
Return Code: 200
Headers:
      Server:  Google Frontend
      Access-Control-Allow-Origin:  *
      Content-Length:  59
      Date:  Tue, 26 Jan 2021 16:51:41 GMT
      Content-Type:  text/plain; charset=utf-8
      Vary:  Accept-Encoding
      X-Cloud-Trace-Context:  aba8faf879dc03fdea1e77fd5237d104
Upload Success!
Data returned = 'http://ptsv2.com/t/a7nrl-1610735518/d/6483465671278592/json'

Download Callback:
URL requested: 'http://ptsv2.com/t/a7nrl-1610735518/d/6483465671278592/json'.
Return Code: 200
Headers:
      Server:  Google Frontend
      Vary:  Accept-Encoding
      Content-Length:  661    
      Content-Type:  application/json
      Date:  Tue, 26 Jan 2021 16:51:41 GMT
      X-Cloud-Trace-Context:  f282d7f7d4e548093d215da67afb8f4c
Download Success!
Data returned = '{"Timestamp":"2021-01-26T16:51:41.287636Z","Method":"POST","RemoteAddr":"127.0.0.1:27812","ID":6483465671278592,"Headers":{"Accept":["*/*"],"Content-Length":["12"],"Forwarded":["for=\"2601:602:9600:2f8:212f:c94c:85a5:3738\";proto=https"],"Traceparent":["00-aba8faf879dc03fdea1e77fd5237d104-643f005180f011e6-00"],"User-Agent":["curl/7.55.1"],"X-Cloud-Trace-Context":["aba8faf879dc03fdea1e77fd5237d104/7223492677381132774"],"X-Forwarded-For":["2601:602:9600:2f8:212f:c94c:85a5:3738, 169.254.1.1"],"X-Forwarded-Proto":["https"],"X-Google-Apps-Metadata":["domain=gmail.com,host=ptsv2.com"]},"FormValues":{},"Body":"test message","Files":null,"MultipartValues":null}'
Received: test message

Example 2 - Who’s it from?

It is often needed to perform different actions depending on the source URL of the HttpClient.Download or HttpClient.Upload. This can be done with different callback functions or conditionals within a single callback function as demonstrated below.

url1 = "https://www.symetrix.co"
url2 = "https://duckduckgo.com"

-- callback function
function Response(Table, ReturnCode, Data, Error, Headers)
    print(string.format("URL requested = '%s'.", Table.Url))
    if (200 == ReturnCode) then
        print("Success!")
        if Table.Url == url1 then
            --do one thing
            print("Doing url1 thing")
        elseif Table.Url == url2 then
            --do another thing
            print("Doing url2 other thing")
        end
    else
        print(string.format("Failed, url %s, error code #%d, '%s'", Table.url, ReturnCode, Error))
    end
end

HttpClient.Download { Url = url1, Headers = { ["Content-type"] = "text/*" }, Timeout = 3, EventHandler = Response }

HttpClient.Download { Url = url2, Headers = { ["Content-Type"] = "text/*" } , Timeout = 3, EventHandler = Response }

This code will produce the following output:

URL requested = 'https://www.symetrix.co'.
Success!
Doing url1 thing
URL requested = 'https://duckduckgo.com'.
Success!
Doing url2 other thing

This example can easily be applied to many different URL paths.

Example 3 – Helper Functions

When working with HTTP REST applications, it may be required to convert tables of strings into a single path string with forward slash separators between each table element. This would allow the creation of a complex path from a list of specifiers.

Likewise, it may be required to separate that concatenated string on the slashes into a table of strings. Two Lua helper functions are provided for this.

Concatenation:

function createPathFromTable(inTable)
    if (type(inTable) == "table") then
        return(table.concat(inTable, "/"))
    else
        return("")
    end
end

Splitting:

function createTableFromPathString(inString)
    local outTable = {n = 0}
    if (type(inString) == "string") then
        local pattern = "(.-)" .. "/"
        local last = 1
        local head, tail, cap = inString:find(pattern, 1)

        while head do
            if head ~= 1 or cap ~= "" then
                table.insert(outTable,cap)
            end
            last = tail + 1
            head, tail, cap = inString:find(pattern, last)
        end

        if last <= #inString then
            cap = inString:sub(last)
            table.insert(outTable, cap)
        end
    end
    return(outTable)
end

The following code snippet shows how these functions may be used:

originalPathTable = {"dsp","channels",1,"output","fader"}
newPathString = createPathFromTable(originalPathTable)
print(newPathString)
newPathTable = createTableFromPathString(newPathString)
print(table.unpack(newPathTable))

This code will produce the following output:

dsp/channels/1/output/fader
dsp   channels    1     output      fader