Parallelize

"part": {
    "part_id": "number",
    "total_parts": "number",
    "part_queue": "string"
}

The part hook identifies if there are multiple tasks running in parrallel that must be completed before continuing. Having a part hook decorator will block the oncomplete hook until all the parts are complete. The part_queue should be unique as to not conflict with other routes in the exchange. Example: part.<svc>-<method>-<id>

Note: The maximum length of a queue in AMQP is usually 255 bytes.

The part hook is to obsecure from the client how the program is internally solving a payload. This allows the solution to be program dependent. The workflow of when a part hook is identified should be as follows,

flowchart LR
  task>Task Part i]
  process[/Process/]

  subgraph HookEval
    direction TB
    publish(("Publish to part queue"))
    check{"Is len(part_queue) == total_parts"}
    yes[/execute oncomplete hook/]
    nothing

    publish --> check
    check -- yes --> yes
    check -- no --> nothing
  end

  finishflow(((return OK)))

  task --> process
  process --> HookEval
  HookEval --> finishflow

Creating the Decorator

The decorator can be manually added or also generated by the service itself

Generation

If the method being triggered has a method with the suffix _part, the worker should follow the following flow to generate part decorators. When using the generator, the other hooks are ignored.

flowchart LR
  task>Task]
  check{"does method '[method]_part' exist AND no existing part decorator?"}

  method[/"run [method]"/]
  yes[/execute oncomplete hook/]

  partmethod[/"run [method]_part"/]
  subgraph PartMethod
    direction TB

    gen[/"make part queue"/]
    publish(("publish with part decorator"))
    task_i>Task Part i]
    task_k>Task Part k]

    gen --> publish
    publish --> task_i
    publish --> task_k
  end

  finishflow(((return OK)))

  task --> check
  check -- yes --> partmethod --> PartMethod
  check -- no --> method --> yes

  PartMethod --> finishflow
  yes --> finishflow

Use Case

A task can be split into smaller tasks in order to streamline workloads or deal with larger workloads in a horizontal approach. The system needs to keep track of all the splits in order to know when the oncomplete hook should be executed if needed.