Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor Resolver to support custom per-TLD resolvers #26

Merged
merged 7 commits into from
Apr 9, 2021

Conversation

vyzo
Copy link
Contributor

@vyzo vyzo commented Apr 6, 2021

See protocol/web3-dev-team#42

We will have to implement our own DoH basic resolver, as the search for a viable library proved fruitless. But that should live in its own repo, it doesn't belong to madns.

@vyzo vyzo requested review from Stebalien, raulk and aschmahmann April 6, 2021 21:13
Copy link
Member

@lidel lidel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Quick feedback on "TLD matching" in real life being a bit more nuanced.

resolve.go Outdated
Comment on lines 66 to 67
parts := strings.Split(domain, ".")
tld := parts[len(parts)-1]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Matching only the last DNS label may be not enough: TLD in the contexts we care about can have more than one label, for example .co.uk. In practice, anything from https://publicsuffix.org/list/public_suffix_list.dat can act as de facto TLD in the browser context and our solution should be flexible enough to support a custom resolver for each.

For foo.bar.example.com we should check com first, but if not found, retry with example.com, bar.example.com and foo.bar.example.com

This will not only make our solution flexible enough to support PublicSuffixList for browsers, but also enable users to set up a custom resolver per domain name (when the full name is a match).

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For foo.bar.example.com we should check com first, but if not found, retry with example.com, bar.example.com and foo.bar.example.com

@lidel shouldn't this go the other way (i.e. check if we have an entry for foo.bar.example.com, then (if not found) bar.example.com then example.com then com)?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

implemented left to right resolution; it will first try the full domain and then all the subdomains, left to right and fall back to the default.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ack, left to right works better and enables multiple levels of rules within a single TLD.

@vyzo vyzo marked this pull request as ready for review April 7, 2021 18:24
@vyzo vyzo removed the request for review from raulk April 8, 2021 10:22
resolve.go Outdated Show resolved Hide resolved
Copy link

@aschmahmann aschmahmann left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left a few comments, but overall LGTM. Thanks 😄

resolve_test.go Show resolved Hide resolved
Comment on lines 26 to -22
type Resolver struct {
Backend Backend

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@vyzo IIRC this is making a choice to break backwards compatibility here a bit for the sake of future extensibility.

Two breakages:

  • Rename Backend to BasicResolver, both not perfect names but Backend is probably worse.
    • We're not really gaining much out of the name change, but it also seems low cost for people to update. If we really wanted to we could just alias type Backend BasicResolver and deprecate it, but idk if it's worth bothering
  • Constructing Resolver is now different and we can't do Resolver.Backend anymore to get out the BasicResolver
    • Neither of these seems like a big deal and moving to a construction with Options gives us more flexibility to add more options in the future

cc @Stebalien in case you have any issues with the breakages.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, conscious choice -- having the fields public was just a bad idea.
I think the original emerged in a hacky way when the ability to change the backend was added.

resolve.go Outdated Show resolved Hide resolved
vyzo added 2 commits April 8, 2021 18:43

const dnsaddrTXTPrefix = "dnsaddr="

type Backend interface {
// BasicResolver is a low level interface for DNS resolution
type BasicResolver interface {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if I have enough context here, but doesn't DNS allow us to lookup A, AAAA and TXT records in the same query? This interface suggests that looking up both would cause 2 queries (and potentially 2 roundtrips).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In theory it does, but in practice it doesn't work as most DNS servers only respond to the first one.
There is a caveat about that in the dns library.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, they have different usage, we don't do concurrent A/AAAA and TXT resolution in our usage patterns.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To be fair, at some point may have some overlap in our usage patterns.
For example, a request for DNSLink might be accompanied by a lookup for a content routing hint (if we implement some form of ipfs/kubo#6516).

That being said, I don't think we should do any premature optimizations.
When time comes, adding cache for raw DNS records (that respects their TTL) will be enough.

@vyzo vyzo force-pushed the feat/rework-resolver branch from 7f98306 to 45cdfcf Compare April 8, 2021 15:45
@vyzo
Copy link
Contributor Author

vyzo commented Apr 8, 2021

Let's wait for @Stebalien to chime in wrt the interface (breaking) changes before merging.

@vyzo
Copy link
Contributor Author

vyzo commented Apr 9, 2021

@Stebalien GoForIt'ed in slack, so merging.

@vyzo vyzo merged commit 963a26a into master Apr 9, 2021
@vyzo vyzo deleted the feat/rework-resolver branch April 9, 2021 09:46
@Stebalien Stebalien mentioned this pull request May 11, 2021
27 tasks
@aschmahmann aschmahmann mentioned this pull request May 14, 2021
71 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants