Support configuring event subscription expressions to limit total subscriptions

[DavidBoike]

Describe the feature.

Is your feature related to a problem? Please describe.

Let's say you have an event MyNamespace.MyEvent and need to version it so you create MyNamespace.V2.MyEvent.

On the subscription bundle, 2 rules will be created:

• [NServiceBus.EnclosedMessageTypes] LIKE '%MyNamespace.MyEvent%'
• [NServiceBus.EnclosedMessageTypes] LIKE '%MyNamespace.V2.MyEvent%'

The way filters are created is not currently configurable. Although you can configure the Azure Service Bus entities however you like and the messages will flow correctly, you can't make NServiceBus installers do anything but create the 2 rules above.

In a system with many event types where versioning like this is required (for whatever reason, that's unimportant) this can quickly get out of hand since subscriptions on a bundle are a limited resource.

Describe the requested feature

Instead, an advanced option could allow customization of the filter. Something like a Func<Type, string> for creating the filter:

var asb = new AzureServiceBusTransport(…);

asb.CreateFilterExpression = type => $"%{type.Name}%";

This results of expression would be deduped, so the previous example would only create one rule:

• [NServiceBus.EnclosedMessageTypes] LIKE '%MyEvent%'

Describe alternatives you've considered

There doesn't seem to be any other alternative, as the feature is to make something configurable that is currently hard-coded.

Additional Context

When implementing, the following should be considered:

• New users have a tendency to "fiddle with all the knobs" even if they should leave things at their defaults. The existing hard-coded version should be the default, but the implementation should look at ways to design the API to make clear that this is an advanced API that shouldn't be used unless you know what you're doing, as the transport would have to take the filter expression creator at its word, and a misconfiguration could result in events not being properly delivered where they need to go.
• Need to look at the scripting tool and see how the asb-transport endpoint subscribe command can be made to support the same configuration, since a command-line utility can't take a Func.

Original idea from @SeanFeldman

[danielmarbach]

I think, conceptually, we should take a step back and look at the broader picture a bit (*). Currently, the transport rule creation policy is centered around a 1:1 relationship between the event type and the rule. This means for every event type subscribed, a rule will be created that turns the event type into the SQL filter described here in the issue. While exposing a function delegate as suggested here might allow tweaking the rule filter, such a solution might suffer from the shortcoming of "not knowing the full picture".

With the v8 transport seam redesign, auto-subscribe passed all subscribed event types to the subscription manager. This means the subscription topology is known during the auto-subscribe time. Once that full picture is known, it becomes possible to ask the question of "given all these subscription intents what is the minimal set of rules required to represent the subscription rules".

The rules expressed need to be stable not just for the auto-subscription phase but also for the manual subscribe/unsubscribe phase to make sure the rule(s) outcome is deterministic. A Func<Type, string> would not be able to take that into account.

In Amazon SQS we had a similar challenge to solve. The number of policies on a queue is a finite and small number. The number of event types subscribed on a single endpoint could easily be larger in the context of 1:1 (event type to policy) and then the policy propagation would fail. We solved this in SQS by having policy rules that either match a namespace with wildcard or an account level. This allows to "condense" multiple event types subscribed to only a handful of policies.

I think what we are discussing here falls into a similar territory because, after all we intend to condense potentially multiple event types subscribed to a smaller number of rules. As far as I understand, rule evaluation comes with a runtime cost associated with it. Given that the topology design requires SQL filters (which are more expensive than correlation filters) I would assume having less rules would benefit the rule evaluation and thus improve the throughput. Furthermore, it would enable scenarios in this issue too.

(*) I think the title of this issue does that pretty well already, while the description is centered around a possible approach/solution that takes a single event type into account and my comments are referring to that part by trying to think "outside the box of the solution proposal".

[SeanFeldman]

The primary concern is not the cost but the number of entities a namespace can have. If it would be solvable by money, it wouldn't be an issue.