IB adapter InstrumentId format to '[email protected]'

[rsmb7z]

Hi @limx0, @cjdsellers,
In regards to IB I see that the InstrumentId format is localSymbol.exchange where exchange serve as Venue. On the other hand if I see the Backtest Engine which require one account for one Venue each. So from different struggle and understanding what I understood is that Nautilus Trader starting with other adapters which are mostly I believe single Venue based (some sort of different symbol mechanism and Venue being the place where account is). So is there any particular reason for using 'localSymbol.exchange'? Why not similar to [email protected]. Using the later one I noticed few benefits though;

• Virtually single Venue samilar to any other adapter [a place where you are doing the trade not where the security reside]
• If you want to Backtest different securities i.e. EUR/USD.IDEALPRO, AAPL.NASDAQ, ORCL.NYSE, because the Venue is different so Backtest will need to have venue seperate leading to different running Account balances. Whereas in real life scenario there is single account and you can trade anything.
• Also for Live Execution I see the Risk Engine doesn't give any warning about Venue not present. Although Order can still move because of adding default routing to True.

Let me know if you need more information.

[limx0]

Hey @rsmb7z, thanks for your thoughts!

@cjdsellers and I discussed briefly your PR and these questions, I'm going to reply to both your PR (#864) and this discussion.

We currently have another mechanism in nautilus that attempts to handle this situation when running live, the RoutingConfig, which allows registering an execution client (like InteractiveBrokers) for routing to another exchange (ie NASDAQ).

Configuring this in a live node looks something like:

config_node = TradingNodeConfig(
    exec_clients={
        "IB": InteractiveBrokersExecClientConfig(
            routing=RoutingConfig(
                default=True,
                venues={Venue("NASDAQ"), Venue("NYSE")}
            )
        ),
    },
)

This would mean that any orders send for AAPL.NASDAQ will be routed to the InteractiveBrokers exec client by default.

Regarding your PR, we are hesitant to make any changes to the InstrumentId, as this could have wide-reaching consequences within nautilus, and the current symbol.venue is the correct way to represent these instruments (we think!) for both live and historic trading (one of the main selling points for nautilus is that you should not have to change your strategy code between live and backtesting). We also need instrument definition, market data and execution fills we receive to all share the correct instrument id, otherwise many trading strategies won't work.

That being said, you correctly raise some questions about routing and execution that are not being handled as well as they could be right now. The most pressing issue I see currently is that we are unable to correctly subscribe to data / instruments or place orders via a broker in the correct way. The IB adapter should be able to place orders via SMART routing, or DMA to an exchange, but when those fills come through we need to correctly identify where they were executed (for commission handling etc).

I think this ultimately boils down to improvements for the IB adapter, not changing the way we handle instruments in nautilus (and possibly some additions to how we can tag orders sent from the strategy as SMART or DMA) , but I'd like to hear any further thoughts you have.

[rsmb7z]

Hi @limx0
Yes I have been trying to use RoutingConfig with the exception that the list is passed as string. I tested passing through Venue list but that raises error because RoutingConfig is expecting string. So with the RoutingConfig I only see the difference when default value is set to True or False. Venues are not taking any affect either because its not a Venue list (as in your response) or some other issue and will keep getting message that no account is registered for Venue,

routing=RoutingConfig(
    default=True,
    venues=["IDEALPRO", "GLOBEX", "NYMEX", "NASDAQ", "NYSE"],
)

With above error and looking and some config parameters, Core system is expecting something like Account at Venue sort of relation (although allowing default routing) and adapter is setting InstrumentId relationship Symbol at Venue. So I believe it will be the core side changes to make it work as expectation so I was of thinking to adjust the adapter side according to core.
Also curious to know what would happen to default routing where there is another adapter for similar broker with multiple exchange added i.e tdameritrade (TDA). And a scenario that I want to route AAPL.NASDAQ to IB and MSFT.NASDAQ to TDA.

As I was talking specifically for IB adapter so mentioned [email protected] but its more appropriate to write as [email protected]. This change doesn't affect the existing use of live and historic trading with same strategy code. Only difference will be providing the input InstrumentId [email protected] or [email protected] instead of AAPL.NASDAQ and it will be enough rest works like before. Furthermore it will still allow to use data collecting data from one source and execution on other which is the perfect architect of nautilus.
You can review further to check for the impact.

[cjdsellers]

Hi @rsmb7z

So here is the definition for the RoutingConfig, and you're correct its a list of strings (these configs need to be serializable over the wire, hence the primitive type):

class RoutingConfig(NautilusConfig):
    """
    Configuration for live client message routing.

    Parameters
    ----------
    default : bool
        If the client should be registered as the default routing client
        (when a specific venue routing cannot be found).
    venues : list[str], optional
        The venues to register for routing.
    """

    default: bool = False
    venues: Optional[frozenset[str]] = None

You've also correctly identified the account <-> venue one-to-one mapping, this is intentional, partly for simplicity.

So as per Brads thoughts, we don't want to change the semantics of the InstrumentId, the reason for this is we want to avoid the symbol or venue components of the ID becoming specific to live trading, or implying any routing instructions - the ID should be generic for backtesting, live and the catalog.

Having said that, you've given us some good ideas for how we could add some more flexibility when submitting orders for smart routing e.g. SMART or DMA to a certain exchange.

Where this can be specified is likely to be the client_id (this param is used to specify custom routing), and the @ convention you suggest could potentially be utilized here e.g. SMART@IB for the client_id.

The other part to this is how we indicate where you were filled... this is trickier because sending an order with an instrument ID of AAPL.IB and client ID of SMART@IB could have you filled across different exchanges, so we may introduce another field into event messages to carry that information back into the system.

This is still being determined and discussed. In the mean time we'll close your PR for now pending the final solution.

Many thanks for your thoughts and efforts on this! 🙏