Protocol

The wire protocol used to communicate between the clients (services, tools, etc.) and within the hive is a simple, text-based publish/subscribe style protocol. Clients connect to and communicate with the hive through a regular TCP/IP socket.

Protocol modes

There are two modes ingrid supports.

Calling

Simply a request, response principe. A request is sent and a service either waits on the response with a certain timeout or receives a response asynchronously.

Execution

Executing a request is like publishing it without waiting for any response. The advantage is that the request can be sent to several services. Ingrid logging, auditing and application metrics are based on this principle, by just executing or publishing the request without waiting for a response and therefore not overloading the traffic.

Compare Call vs Execution

Exec Call
Parallel:
e.g. 5 subscriber listen to a channel: if channel got input, all 5 subscribers start working with this input.		 Sequential:
e.g. 5 subscriber listen to a channel. If channel got input, only 1 subscribers of the 5 subscribers takes this input, processes it and give feedback to the workflow
Wait:
Workflow does not wait for feedback from subscriber –> Shoot and forget		 Wait:
Workflow waits for response from function/channel –> Shoot and wait for answer
Example:
- 1 subscriber for stdout
- 1 subscriber for elasticsearch
- 1 subscriber for splunk		 Example:
5 subscriber for AD-Processing
Only 1 subscriber of the 5 will process the input. The other 4 are waiting for further inputs.

Protocol conventions

Channel names: Channel names are case-sensitive and must be non-empty alphanumeric strings with no embedded whitespace. All ascii alphanumeric characters except spaces/tabs and separators which are “.” and “>” are allowed. Channel names can be optionally token-delimited using the dot character (e.g: “my.channel”).

Protocol message

The ingrid protocol message will be parsed to JSON and sent via a text-based to the hive and back. The following table describes the ingrid protocol message. Names are case sensitive.

Click the name to see more detailed information:

Name Type Description
ID string Unique id to identify request
Class string Used to classify a request
Operation string Used to add more clarification to the request
Control map[string][]string Operating parameters for the called service
Data map[string][]string Dynamic payload (Data to process)
List []map[string][]string List of Dataobjects (Multidata requests)
Result Result Issued by the response of a services

Not every service implements the ingrid protocol message. Services like worker/storage-filesystem will deliver the payload in an unstructed byte array. Services will be labeled if they do not implement the default ingrid protocol.

ID

Services that send data to the hive add an unique UUID to every request.

Class

Class is used to classify a request.

Operation

In addition to the Class, Operation is used to add more clarification to the request.

Control

Services accept Controls. Those Controls influence the behaviour of the service. Take a look at the following example:

// csv service can handle this control property
...
"Control": {
    "filename": ["example.csv"]
}
...

Data

Wheres Control is a representation of operating parameters, Data is used to submit payload for processing to a service. This is the main “Thing”. All the Data goes in here. This is a map with the key as Attribute and a String Array for the Values.

Take a look at the following example:

// httpserver can handle any data submit to the hive
...
"Data": {
    "firstname": ["Tony"],
    "lastname": ["Stark"]
}
...

List

Same as Data, with List you can submit an array full of Data.

In some rare cases, u may want or need to have multiple “Data” objects at once, to implement some more advanced logic.

Take a look at the following example:

// httpserver can handle any data submit to the hive
...
"List": [
    {
        "firstname": ["Tony"],
        "lastname": ["Stark"]
    },
    {
        "firstname": ["Clark"],
        "lastname": ["Kent"]
    }
]
...

Result

Services from type of worker usually return a response with data. Result contains the response. The following table describes the Result.

Name Type Description
Code int A status code issued by the response of the service. List of available status codes.
Message string Status message belonging to the status code.
Data map[string][]string Same as the Data property from the ingrid protocol message.
List []map[string][]string Same as the List property from the ingrid protocol message.