You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: concepts/discovery.mdx
+36-60Lines changed: 36 additions & 60 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -3,25 +3,25 @@ title: "Discovery"
3
3
---
4
4
5
5
6
-
Discovery is the glue that connects an [Endpoint](/concepts/endpoints#endpoint-identifiers) to something we can dial. Discovery services resolve EndpointIds to either a home Relay URL or direct-dialing information.
6
+
Discovery is the glue that connects an [Endpoint](/concepts/endpoints#endpoint-identifiers) to something we can dial. Discovery services resolve EndpointIDs to either a home Relay URL or direct-dialing information.
7
7
8
8
<Note>
9
9
More details on discovery in the discovery module [documentation](https://docs.rs/iroh/latest/iroh/discovery/index.html)
10
10
</Note>
11
11
12
-
Endpoint discovery is an automated system for an [Endpoint](/concepts/endpoints) to retrieve addressing information. Each iroh endpoint will automatically publish their own addressing information with configured discovery services. Usually this means publishing which [Home Relay](/concepts/relay#home-relay) an endpoint is findable at, but they could also publish their direct addresses.
12
+
Endpoint discovery is an automated system for an [Endpoint](/concepts/endpoints) to retrieve addressing information. Each iroh endpoint will automatically publish their own addressing information with configured discovery services. Usually this means publishing which [Home Relay](/concepts/relays) an endpoint is findable at, but they could also publish their direct addresses.
13
13
14
14
15
15
## Discovery Services
16
16
17
-
There are four different implementations of the discovery service in iroh. **By default, iroh uses DNS discovery** to resolve EndpointIds to addresses. Local and DHT discovery are **disabled by default** and must be explicitly configured if you want to use them.
17
+
There are four different implementations of the discovery service in iroh. **By default, iroh uses DNS discovery** to resolve EndpointIDs to addresses. Local and DHT discovery are **disabled by default** and must be explicitly configured if you want to use them.
18
18
19
19
20
20
| Name | Description | Default |
21
21
| --- | --- | --- |
22
-
|[DNS](#node-discovery-via-dns)| uses a custom Domain Name System server | ✅ Enabled |
22
+
|[DNS](#endpoint-discovery-via-dns)| uses a custom Domain Name System server | ✅ Enabled |
23
23
|[Local](#local-discovery)| uses an mDNS-like system to find endpoints on the local network | ❌ Disabled |
|[DHT](#dht-discovery)| uses the BitTorrent Mainline DHT | ❌ Disabled |
26
26
27
27
### The motivation
@@ -32,27 +32,27 @@ iroh. Now, devs no longer need to worry about opening up ports on their servers
32
32
/ firewalls or be resigned to only creating peer-to-peer connections to
33
33
computers inside their local NAT.
34
34
35
-
But even with holepunching, you need to know *where* to dial. Dialing a node in
35
+
But even with holepunching, you need to know *where* to dial. Dialing an endpoint in
36
36
iroh needs either an IP address to talk to, or the URL of a relay to which the
37
-
remote node is connected. To make things easier, you can use
37
+
remote endpoint is connected. To make things easier, you can use
38
38
[tickets](/concepts/tickets) early-on. Tickets are easily encodable
39
-
bytestrings that contain a node id, socket addresses and a relay URL. Tickets
39
+
bytestrings that contain an endpoint ID, socket addresses and a relay URL. Tickets
40
40
work well, but they are long - and they expire: People change networks or
41
41
relays, and then old tickets cannot find you anymore. So while tickets work well
42
42
in some scenarios, they are not frictionless or universally usable.
43
43
44
-
But what if you didn’t need to know the relay URL or socket address in order to
45
-
dial a node? Could we enable iroh users to dial peers using _the least amount of
46
-
information possible_, ie can we enable dialing _only by Node ID_, and no other
44
+
But what if you didn't need to know the relay URL or socket address in order to
45
+
dial an endpoint? Could we enable iroh users to dial peers using _the least amount of
46
+
information possible_, ie can we enable dialing _only by EndpointID_, and no other
47
47
address data?
48
48
49
49
### The solution
50
50
51
-
We had two “guiding lights” while doing research on global node discovery:
51
+
We had two "guiding lights" while doing research on global endpoint discovery:
52
52
53
53
First, we needed to see a path forward that would allow for a fully distributed
54
54
topology, even if our first solution had a federated structure. If we (or some
55
-
contributor) wanted to create or opt into a fully p2p version of global node
55
+
contributor) wanted to create or opt into a fully p2p version of global endpoint
56
56
discovery in the future, that needed to work in tandem with whatever federated
57
57
solution we came up with.
58
58
@@ -61,7 +61,7 @@ protocols and specs unnecessarily. Can we instead leverage standards that have
61
61
stood the test of time (and scrutiny) in novel ways to solve our problem? It
62
62
turns out, we can! And we can do it using one of the oldest and most dependable
63
63
technologies we have on the internet: DNS. Using the DNS standard along side
64
-
Pkarr (public-key addressable resource records), **we now have global node
64
+
Pkarr (public-key addressable resource records), **we now have global endpoint
65
65
discovery in iroh**!
66
66
67
67
## The approach
@@ -74,53 +74,52 @@ names_ to [_resource
74
74
records_](https://datatracker.ietf.org/doc/html/rfc1035#section-3.2), of which
75
75
IP addresses are one type. Pkarr allows us to publish DNS records that resolve
76
76
**elliptic curve keys** (not domain names) to resource records instead. Our
77
-
**Node IDs** in iroh are elliptic curve public keys. This means we can publish
78
-
an association between iroh Node IDs and some resource records. Importantly,
77
+
**EndpointIDs** in iroh are elliptic curve public keys. This means we can publish
78
+
an association between iroh EndpointIDs and some resource records. Importantly,
79
79
these records are signed, so that you can verify that the record was actually
80
-
published by the node with the given Node ID.
81
-
82
-
As long as iroh has a Node ID and its associated relay URL (the address of the
83
-
relay server that node uses to hole-punch and proxy relay packets), we can dial
84
-
that node. So the Pkarr packet currently only needs to contain the Node ID and
85
-
the relay URL of its preferred relay server (which we call its “home relay”).
86
-
When Pkarr publishing is enabled on your iroh node, your node will create a
87
-
Pkarr packet with its Node ID and relay URL, sign it, and defaults to publishing
80
+
published by the endpoint with the given EndpointID.
81
+
82
+
As long as iroh has an EndpointID and its associated relay URL (the address of the
83
+
relay server that endpoint uses to hole-punch and proxy relay packets), we can dial
84
+
that endpoint. So the Pkarr packet currently only needs to contain the EndpointID and
85
+
the relay URL of its preferred relay server (which we call its "home relay").
86
+
When Pkarr publishing is enabled on your iroh endpoint, your endpoint will create a
87
+
Pkarr packet with its EndpointID and relay URL, sign it, and defaults to publishing
88
88
on an `iroh-dns` server that is run by [the n0 team](https://n0.computer).
89
89
90
-
From there, others can discover your dialing information by resolving your Node
91
-
ID using regular DNS. It’s worth noting that others must still learn your node
92
-
ID for this to work,
90
+
From there, others can discover your dialing information by resolving your EndpointID using regular DNS. It's worth noting that others must still learn your endpoint
91
+
ID for this to work.
93
92
94
-
The following sections describe the format of the Pkarr publishing records and Node discovery via DNS queries in greater detail.
93
+
The following sections describe the format of the Pkarr publishing records and endpoint discovery via DNS queries in greater detail.
95
94
96
-
## Node discovery via DNS
95
+
## Endpoint discovery via DNS
97
96
98
-
When connecting to an unknown `EndpointId`, the DNS discovery mechanism in iroh will perform a DNS query to discover relays or addresses for the node. The DNS discovery is configured with a *origin domain* (which defaults to *dns.iroh.link*, a server run by n0). iroh will then perform a DNS query through the configured DNS resolver (which defaults to using the host system's nameservers):
97
+
When connecting to an unknown `EndpointId`, the DNS discovery mechanism in iroh will perform a DNS query to discover relays or addresses for the endpoint. The DNS discovery is configured with a *origin domain* (which defaults to *dns.iroh.link*, a server run by n0). iroh will then perform a DNS query through the configured DNS resolver (which defaults to using the host system's nameservers):
99
98
100
-
`_iroh.<z32-node-id>.<origin-domain> TXT`
99
+
`_iroh.<z32-endpoint-id>.<origin-domain> TXT`
101
100
102
101
-`_iroh` is the record name defined in this spec
103
-
-`<z32-node-id>` is the [z32](https://crates.io/crates/z32) encoding of the 32-byte long `EndpointId` (which is a string of 52 characters)
102
+
-`<z32-endpoint-id>` is the [z32](https://crates.io/crates/z32) encoding of the 32-byte long `EndpointId` (which is a string of 52 characters)
104
103
-`<origin-domain>` is the configured origin domain
105
104
-`TXT` is the queried record type
106
105
107
106
The returned TXT records must contain a string value of the form `key=value`, as defined in [RFC1464](https://www.rfc-editor.org/rfc/rfc1464).
108
107
109
108
This spec defines the following attributes:
110
109
111
-
-`relay=<url>`: The home relay for this node, e.g. `https://euw1-1.derp.iroh.network`.
112
-
-`addr=<addr> <addr> ..` A space-separated list of socket addresses for this iroh node. Each address is an IPv4 or IPv6 address with a port (e.g. `1.2.3.4:7367` or `[::1]:3521`)
110
+
-`relay=<url>`: The home relay for this endpoint, e.g. `https://euw1-1.derp.iroh.network`.
111
+
-`addr=<addr> <addr> ..` A space-separated list of socket addresses for this iroh endpoint. Each address is an IPv4 or IPv6 address with a port (e.g. `1.2.3.4:7367` or `[::1]:3521`)
113
112
114
113
<Note>
115
114
**Ready to use DNS discovery?** Learn how to configure DNS discovery in your application in the [DNS Discovery guide](/connecting/dns-discovery).
116
115
</Note>
117
116
118
-
## Node announces via `pkarr`
117
+
## Endpoint announces via `pkarr`
119
118
120
-
Nodes announce their address information in
119
+
Endpoints announce their address information in
121
120
[Pkarr](https://github.com/Nuhvi/pkarr/) signed packets. The TXT records, as
122
121
described below, are added to the `answers` section of a DNS server. Their name
123
-
must be `_iroh.<node-id>.` The encoded node ID must be the root name, no other
122
+
must be `_iroh.<endpoint-id>.` The encoded endpoint ID must be the root name, no other
124
123
origin but `.` (the single dot) is permitted.
125
124
126
125
Those packets are published to a Pkarr relay server, which is a HTTP service
@@ -137,26 +136,3 @@ with the DNS server's configured *origin domain* appended to all record names.
137
136
<Note>
138
137
**Want to publish via Pkarr?** This is handled automatically when using DNS discovery. See the [DNS Discovery guide](/connecting/dns-discovery) for configuration details.
139
138
</Note>
140
-
141
-
## Local Discovery
142
-
143
-
**Note: Local discovery is disabled by default and must be explicitly enabled.**
144
-
145
-
Global servers aren't the only way to find other iroh endpoints. Iroh also supports local
146
-
discovery, where endpoints on the same local network can find each other &
147
-
exchange dialing information without a relay using mDNS. This is useful for
148
-
offline environments, or for bootstrapping a network before a relay is available.
149
-
150
-
<Note>
151
-
**Want to enable local discovery?** Learn how to configure and enable mDNS-based local discovery in the [Local Discovery guide](/connecting/local-discovery).
152
-
</Note>
153
-
154
-
## DHT Discovery
155
-
156
-
**Note: DHT discovery is disabled by default and must be explicitly enabled.**
157
-
158
-
The Distributed Hash Table (DHT) discovery uses the BitTorrent Mainline DHT to publish and discover node addresses in a fully decentralized manner. This allows nodes to find each other without relying on centralized DNS servers.
159
-
160
-
<Note>
161
-
**Want to enable DHT discovery?** Learn how to configure and enable DHT-based discovery in the [DHT Discovery guide](/connecting/dht-discovery).
Copy file name to clipboardExpand all lines: concepts/endpoints.mdx
+28-6Lines changed: 28 additions & 6 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -44,7 +44,7 @@ See the [EndpointID](https://docs.rs/iroh/latest/iroh/type.EndpointId.html) docu
44
44
Endpoint Addresses or [`EndpointAddrs`](https://docs.rs/iroh/latest/iroh/struct.EndpointAddr.html) are a common struct you'll interact when working with iroh to tell iroh what & where to dial. In rust they look like this:
45
45
46
46
```rust
47
-
pubstructAddr {
47
+
pubstructEndpointAddr {
48
48
pubid:PublicKey,
49
49
pubaddrs:BTreeSet<TransportAddr>,
50
50
}
@@ -81,12 +81,34 @@ roundtrip required to either do initial address discovery, or update cached
81
81
addresses. So if you have a source of up to date home relay & dialing info,
82
82
provide it!
83
83
84
-
### Don't store relay_url & direct_addresses values
84
+
### What to persist in your application
85
85
86
-
If you're persisting the contents of `EndpointAddrs` in your app, it's probably
87
-
not worth keeping the `relay_url` and `direct_address` fields, unless you _know_
88
-
these details are unlikely to change. Providing stale details to the endpoint
89
-
can slow down connection construction.
86
+
When storing endpoint information in your application database, what you should
87
+
persist depends on whether you're using discovery:
88
+
89
+
**If you're using discovery (recommended):**
90
+
Store just the `EndpointID`. When you need to connect, construct an
91
+
`EndpointAddr` from the ID and let discovery resolve the current dialing
92
+
details. This is the most robust approach since relay URLs and direct addresses
93
+
can change frequently in P2P networks.
94
+
95
+
```rust
96
+
// Store in your database
97
+
letendpoint_id:EndpointId=...;
98
+
99
+
// Later, when connecting
100
+
letaddr=EndpointAddr::from(endpoint_id);
101
+
endpoint.connect(addr, ALPN).await?;
102
+
```
103
+
104
+
**If you're not using discovery:**
105
+
You'll need to store full `EndpointAddr` information (including the `addrs`
106
+
field with relay and direct address information). Without discovery, iroh has no
107
+
way to resolve an `EndpointID` to dialing details.
108
+
109
+
Keep in mind that stored dialing details can become stale quickly. Providing
110
+
outdated information may slow down connection establishment as iroh tries
111
+
addresses that no longer work before falling back to other methods.
Copy file name to clipboardExpand all lines: concepts/holepunching.mdx
+9-4Lines changed: 9 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -6,7 +6,12 @@ When two devices want to connect directly over the internet, they face a common
6
6
7
7
## The Problem: NAT Blocks Direct Connections
8
8
9
-
Imagine you're trying to video call a friend. Both of you are behind routers at home. When you try to connect directly to each other, your routers block the incoming connection because they don't recognize it as a response to something you requested. This is NAT protecting your network.
9
+
Imagine you're trying to video call a friend. Both of you are behind routers at
10
+
home. When you try to connect directly to each other, your routers' firewalls
11
+
block the incoming connection because they don't recognize it as a response to
12
+
something you requested. The combination of NAT (which translates addresses) and
13
+
firewall rules (which filter traffic) makes direct peer-to-peer connections
14
+
challenging.
10
15
11
16
Traditionally, this problem was solved by:
12
17
-**Port forwarding**: Manually configuring your router to allow specific connections (tedious and requires technical knowledge)
@@ -16,7 +21,7 @@ Neither solution is ideal for peer-to-peer applications.
16
21
17
22
## The Solution: Holepunching
18
23
19
-
**Holepunching** is a clever technique that tricks NAT routers into allowing direct connections between peers without manual configuration or relying entirely on relay servers.
24
+
**Holepunching** is a clever technique that works around NAT and firewall restrictions to allow direct connections between peers without manual configuration or relying entirely on relay servers.
20
25
21
26
Here's how it works:
22
27
@@ -33,9 +38,9 @@ Through the relay, peers exchange their:
33
38
34
39
### 3. Simultaneous Outbound Connection
35
40
36
-
Here's the magic: both peers try to connect to each other **at the same time**. When peer A sends a packet to peer B's public address, A's router creates a temporary rule allowing responses from B. When peer B sends a packet to peer A at the same moment, B's router does the same.
41
+
Here's the magic: both peers try to connect to each other **at the same time**. When peer A sends a packet to peer B's public address, A's NAT creates a mapping and the firewall creates a temporary rule allowing responses from B. When peer B sends a packet to peer A at the same moment, B's NAT and firewall do the same.
37
42
38
-
Because both routers now have rules expecting traffic from each other, the packets get through, and a direct connection is established!
43
+
Because both NAT mappings are established and both firewalls now have rules expecting traffic from each other, the packets get through, and a direct connection is established!
0 commit comments