ingrid docs > Reference > Services > Queue

Queue

Receiver Sender

This module implements a queue for ingrid

docker pull registry-dev.goingrid.io/services/queue:latest:v0.1.1

Table of Content

Properties
Valid Controls from Controls Section
Service configuration
- CLI / ENV
From Workflow

Properties

The queue service accepts following properties:

Class = Objectclass to work with
Operation = Operation to perform with the given Data

Valid Controls from Controls Section

Control	Value	Description
command	queue (default)	The Data provided will be added to the Queue
command	status	Returns the Status of the Queue
command	clear / delete / wipe	Clears the Queue
command	delete / clear / wipe	With a Control named “key” removes only the Entry with the given Key
command	pause / stop	Pauses the Queue
command	unpause / resume / start	Resumes the Queue
command	list / show	List all Queue Entries
command	get / fetch	Fetch a Queue Entry
command	step / next / one	Process the first Entry in the Queue or the one providedb< the Control “key”
channel	main	Overrides the Target Channel for the Entry
mode	call	call or execute the Entry from the Queue
template_define	default	Override the Named Template to run this Requests to (Multi Queue Object Creation )
requeue_timer	10	Wait time beweeen retries, if a Queue Entry call failed
requeue_count	3	Retries before drop the entry and send to deadletter_slap, if configured
deadletter_slap	dead_letter	Dead Entries will be forwarded to this slap
groupby		List command will be grouped by the given attribute (Count per same value)
list		If the command is list and the groupby attribute is set, you can filter for one specific value

Add Single Queue Entry

To add a single Queue Entry, the minimum u need to send to the Queue worker is an object with data.

Rest Call:

POST http://yourserver HTTP/1.1
content-type: application/json
Authorization: Basic admin secret
channel: debug_echo

{
    "Class": "queue",
    "Operation": "action",
    "Data": {
        "my": ["data"],
        "name": ["yourname"]
    }
}

Docker:

  resolver:
    image: ${ING_REGISTRY}/services/queue:latest
    deploy:
      restart_policy:
        condition: on-failure
    environment:
      NAME: "queue"
      SERVERHOST: "hive"
      SERVERAUTH: "file:///run/secrets/ing-hive-key"
      INPUTCHANNEL: "queue_action"
      LOGLEVEL: "DEBUG"
      DEBUG_TEMPLATE: "TRUE"
      TEMPLATE: "hive://queue_template.tmpl"
    depends_on:
      - hive
    secrets:
      - ing-hive-key
    networks:
      - ing-worker

Response:

{
  "Code": 200,
  "Message": "Success",
  "Data": {
    "queuesize": [
      "1"
    ]
  },
  "List": []
}

This will add a Queue Entry on the Queue worker, that will be executed against the channel debug_echo If it was successful, it will be removed from the Queue

Add Multiple Queue Entry

To add a single Queue Entry, the minimum u need to send to the Queue worker is an object with data. Hence: It calls the “default” template define in the loaded Template and cerated 5 Queue Entries. You can have multiple define templates in your loaded Template. just create a new define template with [[ define “yournamehere” ]]. Close it with [[ end ]]

Rest Call:

Make sure you enabled the the Header Control template_define in the Rest Server

POST http://yourserver HTTP/1.1
content-type: application/json
Authorization: Basic admin secret
template_define: default

{
    "Class": "queue",
    "Operation": "action",
    "Data": {
        "keys": ["user1", "user2", "user3", "user4", "dario"]
    }
}

Internal:

{
    "Class": "queue",
    "Operation": "action",
    "Control": {
        "template_define": ["default"]
    },
    "Data": {
        "keys": ["user1", "user2", "user3", "user4", "dario"]
    }
}

Docker:

  resolver:
    image: ${ING_REGISTRY}/services/queue:latest
    deploy:
      restart_policy:
        condition: on-failure
    environment:
      NAME: "queue"
      SERVERHOST: "hive"
      SERVERAUTH: "file:///run/secrets/ing-hive-key"
      INPUTCHANNEL: "queue_action"
      LOGLEVEL: "DEBUG"
      DEBUG_TEMPLATE: "TRUE"
      TEMPLATE: "hive://queue_template.tmpl"
    depends_on:
      - hive
    secrets:
      - ing-hive-key
    networks:
      - ing-worker

Template:

[[ define "default" ]]
    [[ $list :=  getValues (getRequestData (getRequest)) "keys" ]]
    [[ print $list ]]
    [[ range $v := $list ]]
        [[ print $v ]]
        [[ $newReq := setRequestOperation (setRequestClass newRequest "debug") "echo" ]]
        [[ $data := newData ]]
        [[ $data = addValue $data "mykey" $v ]]
        [[ $data = addValue $data "name" $v ]]
        [[ $newReq = setRequestData $newReq $data ]]
        [[ $controls := newControl ]]
        [[ $controls = addValue $controls "mode" "call" ]]
        [[ $controls = addValue $controls "channel" "debug_echo" ]]
        [[ $newReq = setRequestControl $newReq $controls ]]
        [[ addEntry $newReq ]]
    [[ end ]]
[[ end ]]

Response

{
  "Code": 200,
  "Message": "Success",
  "Data": {
    "queuesize": [
      "5"
    ]
  },
  "List": []
}

This will add multiple Queue Entries on the Queue worker, that will be executed against the channel defined in the template The Template Function addEntry will add the given Object $newReq to the Queue with the Parameters / Data specified.

Other Commands / Operations

The other commands are:

status
get / fetch
list / show
pause / stop
unpause / resume / start
clear / wipe / delete
step / next / one

Status Command

Rest Call:

POST http://yourserver HTTP/1.1
content-type: application/json
Authorization: Basic admin secret
command: status

{
    "Class": "queue",
    "Operation": "action",
    "Data": { }
}

Response:

{
  "Code": 200,
  "Message": "Success",
  "Data": {
    "queuesize": [
      "0"
    ],
    "status": [
      "running"
    ]
  },
  "List": []
}

List Commands

Rest Call:

This example utilizes the Control Prefix from the Rest Server. The Rest Server will will remove the _ and add a Control command

POST http://yourserver HTTP/1.1
content-type: application/json
Authorization: Basic admin secret

{
    "Class": "queue",
    "Operation": "action",
    "Data": {
        "_command": ["list"]
    }
}

Response:

{
  "Code": 200,
  "Message": "Success",
  "Data": {
    "objects": [
      "416262d5-115c-4d3f-8dbe-e06a70605918",
      "203b792f-543a-4b45-883e-beae6b6b48d5",
....
      "80469d1e-8253-4269-8bba-304cfd3d057d",
      "5a8617c3-dd07-4ded-8ecb-e0bfbe99746d"
    ],
    "queuesize": [
      "28"
    ],
    "status": [
      "paused"
    ]
  },
  "List": []
}

List GroupBy Commands

Rest Call:

POST http://yourserver HTTP/1.1
content-type: application/json
Authorization: Basic admin secret

{
    "Class": "queue",
    "Operation": "action",
    "Data": {
        "_command": ["list"],
        "_groupby" : ["password"]
    }
}

Response:

{
  "Code": 200,
  "Message": "Success",
  "Data": {
    "queuesize": [
      "28"
    ],
    "secret11": [
      "12"
    ],
    "secret55": [
      "16"
    ],
    "status": [
      "paused"
    ]
  },
  "List": []
}

List GroupBy Filtered Commands

Rest Call:

POST http://yourserver HTTP/1.1
content-type: application/json
Authorization: Basic admin secret

{
    "Class": "queue",
    "Operation": "action",
    "Data": {
        "_command": ["list"],
        "_groupby" : ["password"],
        "_list" : ["secret11"]
    }
}

Response:


{
  "Code": 200,
  "Message": "Success",
  "Data": {
    "objects": [
      "cf70fd74-ceaa-47b9-8e31-69c5d547b87a",
      "d9f0e697-ad13-4afa-bff8-9ff86e3ffdff",
...
      "80469d1e-8253-4269-8bba-304cfd3d057d",
      "5a8617c3-dd07-4ded-8ecb-e0bfbe99746d"
    ],
    "queuesize": [
      "28"
    ],
    "status": [
      "paused"
    ]
  },
  "List": []
}

Step Command

Rest Call:

POST http://yourserver HTTP/1.1
content-type: application/json
Authorization: Basic admin secret
command: step
key: cf70fd74-ceaa-47b9-8e31-69c5d547b87a

{
    "Class": "queue",
    "Operation": "action",
    "Data": { }
}

Response:

{
  "Code": 200,
  "Message": "Success",
  "Data": {
    "queuesize": [
      "0"
    ],
    "status": [
      "running"
    ]
  },
  "List": []
}

Service configuration

Like every other service, the log service includes all properties of the service configuration and since it listens on input the input configuration.

CLI / ENV

The following parameters are specific to this worker. They can be either used environment variables (all uppercase) or within the CLI (- as prefix).

Parameter	Default	Description
TemplateChannel	DefaultStorageChannel	Hive Channel to fetch the Templates from
DefaultChannel	main	Default Hive Channel to send queued items to
DefaultCommand	queue	Default Command for fetched messages, when no _command is given
DefaultMode	call	Default Mode for Queue Objects, when no _mode is given
DefaultTemplateDefine	””	Default template define name if no define name is specified in Controls
DefaultDeadLetterSlap	””	slap to send dead objects to. Default is drop em.
PersistPath	Os TempDir + ingrid_queue	Local Storage, where we persist the Queue. Set empty for no Persistance
DefaultRequeueTime	30	Seconds to wait before a ‘Failed’ objects is retried
DefaultRequeueCount	3	After what amount of retries, the object will be a dead letter
Template	””	Queue Template to use
StartPaused	false	Start the Queue paused
AutoPause	false	Switch into paused mode, after a completet Queue run

From Workflow

If you want to add some objects from a Workflow you can use a snippet like this

TODO EXAMPLE

Attachments

wf-queue-test.bpmn (3 ko)