Event handling on -> one

[alvin]

I have two test workflows, designed for me to test + understand Synapse's internal event handling:

• The first, when run, creates an event (with static data)
• The second is meant to POST an external URL when the event from the first workflow occurs

document:
  dsl: 1.0.0-alpha2
  namespace: default
  name: generate-order-event
  version: 0.1.0
do:
- emitOrderEvent:
    emit:
      event:
        with:
          source: https://petstore.com
          type: com.petstore.order.placed.v1
          data: Big Dog

My issue is that I the 2nd one (below) doesn't trigger POST to webhook.site -- EXCEPT if I manually create a post-order-from-event instance (which isn't how this is supposed to work, right?). Is my syntax wrong (I tried as JSON too so I know it is valid in a "mechanical" sense)... Or do I have the wrong idea about what is supposed to happen with "schedule -> on" ? I read all the docs I could find and that's what I think it's supposed to do...

document:
  dsl: 1.0.0-alpha2
  namespace: default
  name: post-order-from-event
  version: 0.1.0
schedule:
  "on":
    one:
      with:
        type: com.petstore.order.placed.v1
do:
  - postTheOrderRemotely:
      call: http
      with:
        method: post
        endpoint: https://webhook.site/1f523c5c-7e89-40db-aef4-12b056235311
        body:
          data: something happened

[alvin]

can confirm that even the verbatim example for "Emit" also does not pass the front-end JSON validation that was just committed, so perhaps there is presently some issue with my syntax vs. what the server needs to see?

document:
  dsl: '1.0.0-alpha2'
  namespace: test
  name: emit-example
  version: '0.1.0'
do:
  - emitEvent:
      emit:
        event:
          with:
            source: https://petstore.com
            type: com.petstore.order.placed.v1
            data:
              client:
                firstName: Cruella
                lastName: de Vil
              items:
                - breed: dalmatian
                  quantity: 101

Validation:

Invalid definition

The workflow definition is not valid.
:

/document:

/do:

/document/dsl:

/document/namespace:

/document/name:

/document/version:

/do/0:

/do/0/emitEvent:

    oneOf: Expected 1 matching subschema but found 0
    oneOf: Expected 1 matching subschema but found 0
    required: Required properties ["do"] are not present
    required: Required properties ["fork"] are not present
    required: Required properties ["for","do"] are not present
    required: Required properties ["listen"] are not present
    required: Required properties ["raise"] are not present
    required: Required properties ["run"] are not present
    required: Required properties ["set"] are not present
    required: Required properties ["switch"] are not present
    required: Required properties ["try","catch"] are not present
    required: Required properties ["wait"] are not present
    required: Required properties ["call","with"] are not present
    required: Required properties ["call","with"] are not present
    required: Required properties ["call","with"] are not present
    required: Required properties ["call","with"] are not present
    required: Required properties ["call"] are not present

/do/0/emitEvent/emit:

    : All values fail against the false schema
    : All values fail against the false schema
    : All values fail against the false schema
    : All values fail against the false schema
    : All values fail against the false schema
    : All values fail against the false schema
    : All values fail against the false schema
    : All values fail against the false schema
    : All values fail against the false schema
    : All values fail against the false schema
    : All values fail against the false schema
    : All values fail against the false schema
    : All values fail against the false schema
    : All values fail against the false schema
    : All values fail against the false schema
    : All values fail against the false schema

/do/0/emitEvent/emit/event:

    required: Required properties ["source","type"] are not present

/do/0/emitEvent/emit/event/with:

[cdavernas]

Or do I have the wrong idea about what is supposed to happen with "schedule -> on" ?

Looks you got it right! The second flow should start upon consuming the order placed event.

You should see a correlation created, with a start/create outcome.

[cdavernas]

Does it work if you skip the with?
Like:

emit:
  event:
    type: ....
    source: ....

It looks like the example you are using is invalid 😞
Do you mind reporting it (at the airport, so a bit challenging for me at the time being 🤣)?

When in doubt, check the dsl-reference.md, which should be correct.

[alvin]

Yeah I tried earlier without the "with" (and various other permutations) it would accept it before, but now that the schema is validating on the client it won't accept the version without "with" - but before it would accept the version above, and "appear to" run...

And yes, I have been consulting the DSL + DSL Reference files extensively throughout (I don't want to waste others' bandwidth when I can debug things on my own - but I do have my limits being brand new to this spec + framework!)

Enjoy your time away! (I am travelling as well, but working a little while I go)

[cdavernas]

And yes, I have been consulting the DSL + DSL Reference files extensively throughout

Oh Im sure you did, my intent was not to throw you off, rather to point out that those docs are the source of truth 😜

now that the schema is validating on the client it won't accept the version without "with" - but before it would accept the version above, and "appear to" run...

Hmmm. I'll check what's happening when I've got an hour or so. Aside from client side validation, which you can skip using the CLI, it should work as expected (at least it did last week) but I'll confirm/infirm asap, and fix the software if need be 😉

Enjoy your time away!

Thanks a lot, you too!!!

[alvin]

...and another limitation is that I've never really worked in C# or dotnet :)

However I do work very extensively in fullstack JS/TS + JSON schema, so hopefully I can be at least a little bit useful :)

Based on your suggestion, this morning now I've validated the example against the spec JSON Schema

I checked both with and without the "with" (with an external validator tool that I use regularly) and both appear to be valid.

However the front-end of synapse there are (now) various errors with the different permutations (one example below)

Additionally it shows this validation error in red at bottom (among many other things
required: Required properties ["source","type"] are not present

Yet if I play with moving source/type around, no combination really seems to solve it for the parsing that synapse wants to see?...

But again, prior to the front-end validation commit, different versions were "accepted" via the UI.

And "runs" were completed without errors... however the other workflow with the "on" handler (see original pair above) did not automatically create and run a new instance, so the external POST side-effect didn't occur unless I manually triggered that 2nd workflow.

I have no idea if there is a system issue or if i'm feeding configs in in wrong.

And ... along the lines of your CLI suggestion, what would be ideal for me is if I could create a script and populate a folder of workflows an ingest them programmatically at startup -- because IF synapse works for me it will involve tight coupling with a very configuration-oriented "host" system that's built entirely in JS/TS - and synapse (or eventually a TS implementation of the spec) would be the "router" that makes different things happen at the right times.

I know there is SOME way to inject these config files this if I code it all myself, but just thought I'd ask if there's already something like a "startup workflows folder" kind of config option?

Thanks again - and no rush on replies here!

[cdavernas]

Yet if I play with moving source/type around, no combination really seems to solve it for the parsing that synapse wants to see?...

That's wierd indeed, I'll check what's happening. It might be a bug introduced by the latest front end validation PR.

And "runs" were completed without errors... however the other workflow with the "on" handler (see original pair above) did not automatically create and run a new instance

Oh, I think I get it now! So far, events published by emit tasks are sent directly to the configured event sink, which I guess is not the same than your correlator's event endpoint. I need to fix Synapse to also publish emitted events onto itself, thus enabling the use case you are trying to test. In the time being, you can use either CLI or Swagger (/api/doc) to create your flow, and configure the event sink to be the same than the correlator's (/api/events/pub) intake endpoint.

And ... along the lines of your CLI suggestion, what would be ideal for me is if I could create a script and populate a folder of workflows an ingest them programmatically at startup

That can be easily achieved, I'd be happy to add that feature!

[alvin]

Amazing thank you so much Charles

And yes that makes sense about the event bus stuff - part of why I asked here in the first place was that I wasn't sure if what I was trying/testing was even "supposed to" work but then when I tried things and saw the "correlator" stuff jump to life I got excited and was like 🤔 maybe it does have an "internal" event bus. I'll give that sink config stuff a go!

And also thanks for the tip on the swagger docs/endpoint - that might be more practical for now (and will also make it more obvious to me what kinds of integrations I'd be able to layer in myself)

[cdavernas]

I wasn't sure if what I was trying/testing was even "supposed to" work

It is supposed to work indeed, I forgot to implement the event forwarding in the API. Many thanks for uncovering it!

[cdavernas]

The validation problems you faced occurred because the changes made to emit had not yet been commitrd to SW schema, but were already implemented in the SDK.

Creating a flow using dsl 1.0.0-alpha3 should work as expected.

As for the internal bus, the PR has been merged today, I'll release Synapse alpha3 tomorrow 😉

[alvin]

Welcome back!

And regarding the schema that's great - I had noticed some small (but quite time-consuming) discrepancies with the schema in my implementation work over the past couple of weeks so that is good to know. (My schema is maybe 65% as complex as the full DSL as I'm starting out with a subset of the DSL in an "embedded" scenario for now)

One particularly thorny area was some oneOf match bugs with "for" vs. "do" task signatures (because a "for" task has a "do" property, the AJV validator needed more specificity which at long last after realizing what was happening, I added to DoTask via

      "properties": {
        "do": {
          "$ref": "#/definitions/taskList",
          "title": "DoTaskConfiguration",
          "description": "The configuration of the tasks to perform sequentially."
        }
      },
      "not": {
        "required": ["for"]
      }

(a hack, perhaps, but it got me unblocked).

I'm a bit fuzzy on recall here but another thing that may have helped was ensuring consistent application of "$ref": "#/definitions/taskBase"in all of the task types my sub-schema is supporting, and also some vetting of unevaluatedProperties and considering some cases where the meaning of 'anyOf' was perhaps more valid than 'oneOf'.

In any case, if the team working on the spec (or Synapse) finds themselves experiencing any validation issues that "feel" like any of the above, feel free to reach out and I'm happy to provide more specific input or feedback.

And yes, I will check out the differences in the alpha3 schema, thanks so much!