REST

Sender

This service implements a REST web service. Preparing the payload (or intercepting) before sending it to the hive can be done via the TSInterceptor service.

registry.goingrid.io/services/rest:v0.1.0
Using

The REST service awaits a http request from any REST client and forwards it to the hive. Optionally you can use the TSInterceptor to transform the incoming http request to a readable ingrid protocol message format.

Table of contents

Service properties

This service implements the ingrid protocol message. Following properties are being used:

Property Usage
Class See below
Operation See below
Control See below
Data Accepts any given data, can be manipulated via the TSInterceptor

Class and Operation

The class and operation can be manipulated via the TSInterceptor.

In strict mode the class and operation will be retrieved via the URL (e.g. https://rest/{class}/{operation}).

Control

The controls can be manipulated via the TSInterceptor.

The control Ingrid-Break=true can be used to get an instant preview of data that will be sent to the hive (take a look at the example below).

Accepts any given Control (list of allowed can be set via the property ControlAllowList) via HTTP header fields or via Data fields with a specified prefix (property ControlPrefix). The latter overrides any HTTP header fields.

Service capabilities

Basically, if you want to use the service, you distinguish between using strict mode or not using it. The latter assumes that the TSInterceptor is configured and being used, otherwise you will only have problems because the rest forwards the data to the hive and the hive accepts only data in the format of the ingrid protocol message.

Any http method and URL path will be accepted.

Using it with TSInterceptor

The only case where you do not want to use the TSInterceptor, is when you are using the rest in strict mode and only want to pass data that is already in the format of the ingrid protocol message.

Plain text 

POST https://rest.app.goingrid.io HTTP/1.1
Content-Type: text/plain; charset=utf-8
Authorization: Basic admin nutz

Hello there

// ./assets/tsinterceptor/funcs.ts
import { Response, IngridResponse, IngridMsg, IngridResult, IngridData } from "https://raw.githubusercontent.com/itdistrict/ingrid-deno/master/lib/ingrid.ts";
/*
* Funcs
*/
export function restBefore(type: string, value: any, options: Record<string, string>): IngridResponse {
    let resp = new IngridResponse();

    if (value == "") {
      resp.status_code = 400;
      resp.status = "Empty payload";
      return resp;
    }

    let ingMsg : IngridMsg = new IngridMsg();
    if (type == "text") {
        let ingData: IngridData = {};
        ingData["message"] = [value];
        ingMsg.Data = ingData;
    }

    ingMsg.Class = "echo";
    ingMsg.Operation = "run";

    resp.data = ingMsg;

    return resp;
}


export function restAfter(type: string, value: any, options: Record<string, string>): Response {
  let resp = new Response();

  resp.data = value.Data.Example;

  let http_response_headers = JSON.parse(options.http_response_headers);
  http_response_headers["Content-Type"] = "text/plain";
  options.http_response_headers = JSON.stringify(http_response_headers);

  resp.options = options;
  return resp;
}

rest:
  image: ${ING_REGISTRY}/trigger/rest:latest
  deploy:
    restart_policy:
      condition: on-failure
    placement:
      constraints:
        - node.role == manager
    labels:
      - "traefik.enable=true"
      - "traefik.docker.network=ing-entry"
      - "traefik.http.routers.rest.rule=Host(`rest.${ING_FQDN}`)"
      - "traefik.http.routers.rest.entrypoints=https"
      - "traefik.http.routers.rest.tls=true"
      - "traefik.http.routers.rest.tls.certresolver=le"
      - "traefik.http.routers.rest.middlewares=auth"
      - "traefik.http.services.rest.loadbalancer.server.port=80"
      - "traefik.http.services.rest.loadbalancer.server.scheme=http"
  environment:
    NAME: "rest"
    SERVERHOST: "hive"
    SERVERAUTH: "file:///run/secrets/ing-hive-key"
    OUTPUTCHANNEL: "main" # default anyway
    INTERCEPTBEFORE: "http://tsinterceptor:8099/restBefore"
    INTERCEPTAFTER: "http://tsinterceptor:8099/restAfter"
  depends_on:
    - hive
  secrets:
    - ing-hive-key
  networks:
    - ing-entry
    - ing-middle

tsinterceptor:
  image: ${ING_REGISTRY}/worker/tsinterceptor:latest
  environment:
    PORT: 8099
  volumes:
    - ./assets/tsinterceptor:/app/static
  depends_on:
    - hive
  networks:
    - ing-entry
    - ing-middle

JSON 

POST https://rest.app.goingrid.io HTTP/1.1
Content-Type: application/json
Authorization: Basic admin nutz

{
  "msg": "Hello there"
}

// ./assets/tsinterceptor/funcs.ts
import { Response, IngridResponse, IngridMsg, IngridResult, IngridData } from "https://raw.githubusercontent.com/itdistrict/ingrid-deno/master/lib/ingrid.ts";
/*
* Funcs
*/
export function restBefore(type: string, value: any, options: Record<string, string>): IngridResponse {
    let resp = new IngridResponse();

    if ("msg" in value == false && value.msg == "") {
      resp.status_code = 400;
      resp.status = "Empty payload";
      return resp;
    }

    let ingMsg : IngridMsg = new IngridMsg();
    if (type == "json") {
        let ingData: IngridData = {};
        ingData["message"] = [value.msg];
        ingMsg.Data = ingData;
    }

    ingMsg.Class = "echo";
    ingMsg.Operation = "run";

    resp.data = ingMsg;

    return resp;
}


export function restAfter(type: string, value: any, options: Record<string, string>): Response {
  let resp = new Response();

  resp.data = {
    "echo_back": value.Data.Bla
  };

  let http_response_headers = JSON.parse(options.http_response_headers);
  http_response_headers["Content-Type"] = "application/json";
  options.http_response_headers = JSON.stringify(http_response_headers);

  resp.options = options;
  return resp;
}

rest:
  image: ${ING_REGISTRY}/trigger/rest:latest
  deploy:
    restart_policy:
      condition: on-failure
    placement:
      constraints:
        - node.role == manager
    labels:
      - "traefik.enable=true"
      - "traefik.docker.network=ing-entry"
      - "traefik.http.routers.rest.rule=Host(`rest.${ING_FQDN}`)"
      - "traefik.http.routers.rest.entrypoints=https"
      - "traefik.http.routers.rest.tls=true"
      - "traefik.http.routers.rest.tls.certresolver=le"
      - "traefik.http.routers.rest.middlewares=auth"
      - "traefik.http.services.rest.loadbalancer.server.port=80"
      - "traefik.http.services.rest.loadbalancer.server.scheme=http"
  environment:
    NAME: "rest"
    SERVERHOST: "hive"
    SERVERAUTH: "file:///run/secrets/ing-hive-key"
    OUTPUTCHANNEL: "main" # default anyway
    INTERCEPTBEFORE: "http://tsinterceptor:8099/restBefore"
    INTERCEPTAFTER: "http://tsinterceptor:8099/restAfter"
  depends_on:
    - hive
  secrets:
    - ing-hive-key
  networks:
    - ing-entry
    - ing-middle

tsinterceptor:
  image: ${ING_REGISTRY}/worker/tsinterceptor:latest
  environment:
    PORT: 8099
  volumes:
    - ./assets/tsinterceptor:/app/static
  depends_on:
    - hive
  networks:
    - ing-entry
    - ing-middle

XML 

POST https://rest.app.goingrid.io HTTP/1.1
Content-Type: text/xml
Authorization: Basic admin nutz

<user>
    <firstname>Clark</firstname>
    <lastname>Kent</lastname>
</user

import { Response, IngridResponse, IngridMsg, IngridResult, IngridData } from "https://raw.githubusercontent.com/itdistrict/ingrid-deno/master/lib/ingrid.ts";
import { xmlJS } from "../lib/xml-js.ts";
/*
* Funcs
*/
export function restBefore(type: string, value: any, options: Record<string, string>): IngridResponse {
  let resp = new IngridResponse();
  if (type == "text") {
      let ingMsg : IngridMsg = new IngridMsg();
      let ingData: IngridData = {};
      let respData = JSON.parse(xmlJS.xml2json(value, { compact: true, spaces: 4 }));
      ingData["firstname"] = [respData.user.firstname._text];
      ingData["lastname"] = [respData.user.lastname._text];
      ingMsg.Data = ingData;
      resp.data = ingMsg;
  } else {
      // not supported type, want xml
      resp.status = "Not supported type"
      resp.status_code = 400
  }
  return resp;
}

export function restAfter(type: string, value: any, options: Record<string, string>): Response {
  let resp = new Response();
  // Before interceptor is always from type json, because the data is coming from the hive
  if (type == "json") {
      resp.data = xmlJS.js2xml(value, { compact: true, spaces: 0 });
  }

  let http_response_headers = JSON.parse(options.http_response_headers);
  http_response_headers["Content-Type"] = "application/xml";
  options.http_response_headers = JSON.stringify(http_response_headers);

  resp.options = options;
  return resp;
}

rest:
  image: ${ING_REGISTRY}/trigger/rest:latest
  deploy:
    restart_policy:
      condition: on-failure
    placement:
      constraints:
        - node.role == manager
    labels:
      - "traefik.enable=true"
      - "traefik.docker.network=ing-entry"
      - "traefik.http.routers.rest.rule=Host(`rest.${ING_FQDN}`)"
      - "traefik.http.routers.rest.entrypoints=https"
      - "traefik.http.routers.rest.tls=true"
      - "traefik.http.routers.rest.tls.certresolver=le"
      - "traefik.http.routers.rest.middlewares=auth"
      - "traefik.http.services.rest.loadbalancer.server.port=80"
      - "traefik.http.services.rest.loadbalancer.server.scheme=http"
  environment:
    NAME: "rest"
    SERVERHOST: "hive"
    SERVERAUTH: "file:///run/secrets/ing-hive-key"
    OUTPUTCHANNEL: "main" # default anyway
    INTERCEPTBEFORE: "http://tsinterceptor:8099/restBefore"
    INTERCEPTAFTER: "http://tsinterceptor:8099/restAfter"
  depends_on:
    - hive
  secrets:
    - ing-hive-key
  networks:
    - ing-entry
    - ing-middle

tsinterceptor:
  image: ${ING_REGISTRY}/worker/tsinterceptor:latest
  environment:
    PORT: 8099
  volumes:
    - ./assets/tsinterceptor:/app/static
  depends_on:
    - hive
  networks:
    - ing-entry
    - ing-middle

Options 

import { Response, IngridResponse, IngridMsg, IngridResult, IngridData } from "https://raw.githubusercontent.com/itdistrict/ingrid-deno/master/lib/ingrid.ts";

export function restBefore(type: string, value: any, options: Record<string, string>): IngridResponse {
  // The http_request is used to determine what http request was issued.
  // You can restrict http methods, check for explicit url paths, etc.
  let http_request = JSON.parse(options.http_request);
  console.log(http_request);

  return new IngridResponse();
}


export function restAfter(type: string, value: any, options: Record<string, string>): Response {
  // The same http_request that was issued in the restBefore.
  let http_request = JSON.parse(options.http_request);
  console.log(http_request);
  // The http_response_headers can be used to control the response headers.  
  let http_response_headers = JSON.parse(options.http_response_headers);
  console.log(http_response_headers);
  http_response_headers["Content-Type"] = ["application/json"];
  options.http_response_headers = JSON.stringify(http_response_headers);

  let resp = new Response();
  resp.options = options;
  return resp;
}

Validation

You can validate the data and stop the rest from sending it to the hive. Every status NOT between 200 and 299 will abort further processing and send the response back to the client.

The same applies to the after interception, the only difference is that the actual http response code is set.

import { Response, IngridResponse, IngridMsg, IngridResult, IngridData } from "https://raw.githubusercontent.com/itdistrict/ingrid-deno/master/lib/ingrid.ts";

export function restBefore(type: string, value: any, options: Record<string, string>): IngridResponse {
  let resp = new IngridResponse();
  if (type == "json" && "username" in value && value.username != "tony") {
    resp.status = "Useranme not provided";
    resp.status_code = 400;
    return resp;
  }
  // do something
  return resp;
}

Strict mode

If you want to send and retrieve data only within the ingrid protocol, you can use the rest service in strict mode by setting the property Strict=true.

POST https://rest.app.goingrid.io/echo/run HTTP/1.1
Content-Type: application/json
Authorization: Basic admin nutz

{
    "message": ["Hello"]
}

rest:
  image: ${ING_REGISTRY}/trigger/rest:latest
  deploy:
    restart_policy:
      condition: on-failure
    placement:
      constraints:
        - node.role == manager
    labels:
      - "traefik.enable=true"
      - "traefik.docker.network=ing-entry"
      - "traefik.http.routers.rest.rule=Host(`rest.${ING_FQDN}`)"
      - "traefik.http.routers.rest.entrypoints=https"
      - "traefik.http.routers.rest.tls=true"
      - "traefik.http.routers.rest.tls.certresolver=le"
      - "traefik.http.routers.rest.middlewares=auth"
      - "traefik.http.services.rest.loadbalancer.server.port=80"
      - "traefik.http.services.rest.loadbalancer.server.scheme=http"
  environment:
    NAME: "rest"
    SERVERHOST: "hive"
    SERVERAUTH: "file:///run/secrets/ing-hive-key"
    OUTPUTCHANNEL: "main" # default anyway
    STRICT: "true"
  depends_on:
    - hive
  secrets:
    - ing-hive-key
  networks:
    - ing-entry
    - ing-middle

Break control

If you want to get an instant preview of the data that will be sent to the hive, use the control Ingrid-Break=true.

POST https://rest.app.goingrid.io/echo/run HTTP/1.1
Content-Type: application/json
Authorization: Basic admin nutz
Ingrid-Break: true

{
    "message": ["Hello"]
}

rest:
  image: ${ING_REGISTRY}/trigger/rest:latest
  deploy:
    restart_policy:
      condition: on-failure
    placement:
      constraints:
        - node.role == manager
    labels:
      - "traefik.enable=true"
      - "traefik.docker.network=ing-entry"
      - "traefik.http.routers.rest.rule=Host(`rest.${ING_FQDN}`)"
      - "traefik.http.routers.rest.entrypoints=https"
      - "traefik.http.routers.rest.tls=true"
      - "traefik.http.routers.rest.tls.certresolver=le"
      - "traefik.http.routers.rest.middlewares=auth"
      - "traefik.http.services.rest.loadbalancer.server.port=80"
      - "traefik.http.services.rest.loadbalancer.server.scheme=http"
  environment:
    NAME: "rest"
    SERVERHOST: "hive"
    SERVERAUTH: "file:///run/secrets/ing-hive-key"
    OUTPUTCHANNEL: "main" # default anyway
    CONTROLALLOWLIST: "ingrid-break" # lowercase because of traefik, otherwise 'Ingrid-Break'
    STRICT: "true"
  depends_on:
    - hive
  secrets:
    - ing-hive-key
  networks:
    - ing-entry
    - ing-middle

Dev mode

By enabling dev mode DEVMODE=true all http headers will be passed as controls (equivalent to CONTROLALLOWLIST=*) and following http headers will be added to the http response:

Access-Control-Allow-Origin: *
Access-Control-Allow-Credentials: true
Access-Control-Allow-Methods: *
Access-Control-Allow-Headers: *

Healt check

The health check determines if the service is running probably and connected to the hive.

GET https://rest.app.goingrid.io/healthcheck HTTP/1.1
Authorization: Basic admin nutz

Service configuration

Parameter Default Description
Port 80 Port the http server will listen on
ResponseHeaders "" Set response headers (\r\n separated)
DevMode false DevMode enabled will open up the CORS headers and Controls
RequestTimeout "60s" Timeout of the request to the hive
BasePath "/" Base path used to prepend to the url (e.g. /rest)
HealthcheckPath "healthcheck" Path to the health check (only accessable via GET method)
Strict false Strict enabled will only accept the ingrid protocol message
ControlPrefix "" Prefix used to identify controls in data objects
ControlAllowList* "" List of allowed controls that will be passed to the request (comma or semicolon separated)
InterceptBefore "" URL to a mapper function before the actual request, mandatory if strict = false
InterceptAfter "" URL to a mapper function after the actual request , mandatory if strict = false
DefaultClass "rest" Default class used if not set in the request
DefaultOperation "input" Default operation used if not set in the request

* Another valid value is * which allows every Control to be passed. Should be only used in production.

Additionally the REST service includes all properties of the service configuration and the output configuration.

Configuration parameter Intercept

The configuration parameters InterceptBefore and InterceptAfter are being used to configure the endpoints of the used TSInterceptor service (e.g https://tsinterceptor/hello). So in order to make the http service work, one or more TSInterceptor services need to be configured as well.