Markdown fixes, reports content and metrics cross-links

content/docs/kademlia-dht.md

@@ -12,9 +12,7 @@ category = "peer discovery/routing"
 ## Overview
 
 The Kademlia Distributed Hash Table (DHT), or Kad-DHT, is a distributed hash table
-that is designed for P2P networks.
-
-Kad-DHT in libp2p is a subsystem based on the
+that is designed for P2P networks. Itis a subsystem based on the
 [Kademlia whitepaper](https://pdos.csail.mit.edu/~petar/papers/maymounkov-kademlia-lncs.pdf).
 
 Kad-DHT offers a way to find nodes and data on the network by using a
@@ -49,6 +47,8 @@ This design allows for efficient and effective lookups in the routing table when
 trying to find nodes or data that share similar prefixes.
 {% end %}
 
+If you are considering adopting a DHT for your application, the [ProbeLab](https://probelab.io) team has written a [blog post](https://probelab.io/blog/libp2p-dht-configuration-guide/) how to properly configure it.
+
 ## Peer routing
 
 The Kad-DHT uses a process called "peer routing" to discover nodes in the

content/reports/guides/_index.md

@@ -0,0 +1,9 @@
++++
+title = "Guides"
+description = "Configuration guides and network measurement reports from the ProbeLab team."
+weight = 4
+template = "reports/section.html"
++++
+
+Practical configuration guides and network measurement reports covering libp2p and IPFS.
+Each page is a short summary with a link to the full source.

content/reports/guides/bitswap-discovery-effectiveness.md

@@ -0,0 +1,29 @@
++++
+title = "Bitswap Discovery Effectiveness"
+description = "Measuring how effectively Bitswap discovers IPFS content without DHT lookups, and what that implies for ProviderSearchDelay."
+weight = 2
+
+[extra]
+author = "Guillaume Michel"
++++
+
+Before falling back to a DHT lookup, IPFS nodes try to find content via **Bitswap** by asking
+already-connected peers. This study measures how effective that broadcast-based discovery
+actually is on its own. Researchers modified kubo and go-bitswap to request **71,769 CIDs**
+(from Bitswap sniffs and IPFS Gateway logs) while blocking DHT lookups, running for 50 hours
+from a European data center.
+
+**Key findings:**
+
+- **98% of requested content was discovered by Bitswap alone**, without DHT assistance — far above the ~5% that was expected.
+- **Fast**: 75.98% of successful requests completed within 200 ms, and 95.20% within 1 second.
+- **Highly concentrated**: just 723 distinct providers served all successful requests, with the top 10 delivering ~60% of content.
+- **Expensive**: each request averaged 1,714 messages to 856 peers — heavy flooding compared to the DHT's ~15 messages.
+
+**Recommendations:** remove or shorten the 1-second `ProviderSearchDelay` (to 200–500 ms),
+start DHT lookups concurrently with Bitswap (only ~0.4% overhead), and explore selective
+broadcasting toward high-performing providers rather than network-wide flooding.
+
+{% alert(type="note") %}
+Read the full report in the [network-measurements repository](https://github.com/probe-lab/network-measurements/blob/main/results/rfm16-bitswap-discovery-effectiveness.md) (Guillaume Michel, December 2022).
+{% end %}

content/reports/guides/dht-routing-table-health.md

@@ -0,0 +1,29 @@
++++
+title = "DHT Routing Table Health"
+description = "Measuring how complete and current IPFS peers' Kademlia routing tables are despite high network churn."
+weight = 4
+
+[extra]
+author = "Guillaume Michel"
++++
+
+This report assesses the health of Kademlia DHT routing tables across the IPFS network: how
+many entries are stale or unreachable, how peers are distributed across k-buckets, and
+whether nodes actually maintain their 20 closest peers. The data comes from the
+[Nebula crawler](https://github.com/dennis-tra/nebula) over 28 crawls across 7 days
+(19–26 April 2022), reconstructing expected k-bucket contents with a binary trie to compare
+actual versus theoretical peer distributions.
+
+**Key findings:**
+
+- **95.21% of peers** keep at least 18 of their 20 closest peers in their routing table.
+- **Full k-buckets (0–8)** average only 0.12 missing peers each; non-full buckets (9+) show ~20% missing, as expected from the network design.
+- **Unreachable peers** account for just 3.78% in heavily populated buckets.
+- Peer distribution closely matches theoretical expectations for the network size.
+
+The overall takeaway: DHT routing tables are surprisingly resilient despite high churn,
+keeping peers discoverable and routing information current.
+
+{% alert(type="note") %}
+Read the full report in the [network-measurements repository](https://github.com/probe-lab/network-measurements/blob/main/results/rfm19-dht-routing-table-health.md) (Guillaume Michel, August 2022).
+{% end %}

content/reports/guides/libp2p-dht-configuration-guide.md

@@ -0,0 +1,30 @@
++++
+title = "libp2p DHT Configuration Guide"
+description = "How to configure go-libp2p-kad-dht and rust-libp2p/kad for your deployment instead of relying on the public IPFS Amino DHT defaults."
+weight = 1
+
+[extra]
+author = "Yiannis Psarras, Dennis Trautwein"
++++
+
+The defaults shipped with libp2p's Kademlia DHT implementations are calibrated for the
+public IPFS Amino DHT. If you're running a different kind of network, those defaults are
+rarely the right choice. This guide walks operators through tuning
+[go-libp2p-kad-dht](https://github.com/libp2p/go-libp2p-kad-dht) and
+[rust-libp2p/kad](https://github.com/libp2p/rust-libp2p) for their specific deployment.
+
+**Key takeaways:**
+
+- Configuration is organized into four priority tiers, from highest impact to marginal:
+  1. **Correctness gates** — operating mode, protocol identifier, and IP diversity filter. Getting these wrong produces silently broken nodes.
+  2. **Provide optimizations** — the Reprovide Sweep is effectively mandatory above ~10³ CIDs.
+  3. **Continuous tuning** — bucket size (`k`) and query parallelism (`α`) drive the replication-versus-latency trade-off.
+  4. **Marginal adjustments** — secondary parameters with limited isolated impact.
+- Five concrete deployment profiles are provided with recommended parameters: public server, resource-constrained client, peer-discovery overlay, private cluster, and high-throughput provider.
+
+For anyone deploying or maintaining a libp2p-based network and looking for production-ready
+configuration guidance beyond the defaults.
+
+{% alert(type="note") %}
+Read the full guide on the [ProbeLab blog](https://probelab.io/blog/libp2p-dht-configuration-guide/) (Yiannis Psarras & Dennis Trautwein, May 2026).
+{% end %}

content/reports/guides/provider-record-liveness.md

@@ -0,0 +1,26 @@
++++
+title = "Provider Record Liveness"
+description = "How long IPFS provider records remain retrievable in the DHT, and what that means for replication and republish intervals."
+weight = 3
+
+[extra]
+author = "Mikel Cortes-Goicoechea, Leonardo Bautista-Gomez (Barcelona Supercomputing Center)"
++++
+
+Provider records (PRs) map content IDs to the peers that host them. This report measures how
+long those records stay accessible in the IPFS DHT over time. Using the purpose-built **CID
+Hoarder** tool, the authors published 10,000 random CIDs over 36-hour windows, tracked PR
+holders roughly every 30 minutes, and tested replication values of K = 15, 20, 25, and 40,
+with and without Hydra-Booster nodes.
+
+**Key findings:**
+
+- **75% of initial PR holders remain online 48+ hours**, and 70% serve records for the full 24-hour specification period.
+- **K = 20 remains a suitable replication value**, balancing overhead, performance, and reliability.
+- **The network functions adequately without Hydra nodes**, though they add stability — the authors suggest halving their count rather than removing them.
+- **Raising the republish interval from 12 to 24 hours** could cut network overhead by ~25% without compromising availability.
+- **70% of the initial closest peers** stay among the K closest for 48+ hours.
+
+{% alert(type="note") %}
+Read the full report in the [network-measurements repository](https://github.com/probe-lab/network-measurements/blob/main/results/rfm17-provider-record-liveness.md) (Cortes-Goicoechea & Bautista-Gomez, August 2022).
+{% end %}

content/reports/research-initiatives/_index.md

@@ -1,5 +1,9 @@
 +++
 title = "Research Initiatives"
+description = "Published, peer-reviewed research and research initiatives from the ProbeLab team."
 weight = 3
 template = "reports/section.html"
 +++
+
+Peer-reviewed papers and research initiatives on libp2p and IPFS. Each page is a short
+summary with a link to the full publication.

content/reports/research-initiatives/design-and-evaluation-of-ipfs.md

@@ -0,0 +1,27 @@
++++
+title = "Design and Evaluation of IPFS"
+description = "The ACM SIGCOMM 2022 paper presenting the first large-scale measurement and evaluation of IPFS as a storage layer for the decentralized web."
+weight = 3
+
+[extra]
+author = "Dennis Trautwein et al."
++++
+
+Published at **ACM SIGCOMM 2022** (Amsterdam, August 2022), this peer-reviewed paper presents
+a comprehensive design overview and large-scale evaluation of the InterPlanetary File System
+(IPFS) — a content-addressable, peer-to-peer network for distributed data storage and
+delivery — as a decentralized alternative to centralized web infrastructure.
+
+**Key contributions and findings:**
+
+- IPFS spans **more than 2,700 Autonomous Systems across 152 countries**, predominantly outside the major cloud providers.
+- The network handles **millions of daily retrievals** supporting dozens of applications.
+- Both publication and retrieval performance prove suitable for a wide range of use cases.
+- The authors introduce measurement methodologies to characterize peer behavior, and release shared datasets and lessons learned.
+
+Authors: Dennis Trautwein, Aravindh Raman, Gareth Tyson, Ignacio Castro, Will Scott,
+Moritz Schubotz, Bela Gipp, and Yiannis Psaras.
+
+{% alert(type="note") %}
+Read the full publication on the [ProbeLab site](https://probelab.io/publications/design-and-evaluation-of-ipfs-a-storage-layer-for-the-decentralized-web/) (ACM SIGCOMM 2022).
+{% end %}

content/reports/research-initiatives/gossipsub.md

@@ -0,0 +1,27 @@
++++
+title = "GossipSub"
+description = "The research paper behind libp2p's attack-resilient publish-subscribe protocol, now powering messaging in Filecoin and Ethereum."
+weight = 4
+
+[extra]
+author = "Dimitris Vyzovitis, Yusef Napora, Dirk McCormick, David Dias, Yiannis Psaras"
++++
+
+GossipSub is a gossip-based publish-subscribe protocol built for permissionless blockchain
+networks, where fast message propagation must coexist with resilience against adversarial
+peers. This paper introduces its design and evaluates it against a range of attacks.
+
+**Key contributions:**
+
+- **Mesh construction** — an eager-push delivery model that keeps fan-out low while balancing bandwidth against propagation speed.
+- **Gossip dissemination** — a lazy-pull model that reaches nodes outside the mesh.
+- **Peer scoring** — reputation profiles for connected peers, so well-behaved nodes are preferred for mesh inclusion.
+- **Attack mitigation** — defenses tailored to Sybil-based attacks against each of the protocol's three core components.
+
+The authors validated GossipSub across **5,000+ virtual nodes on AWS**, demonstrating
+resilience against the considered attacks. The protocol was subsequently adopted as the
+messaging layer for **Filecoin and Ethereum 2.0**.
+
+{% alert(type="note") %}
+Read the full paper on [arXiv (2007.02754)](https://arxiv.org/abs/2007.02754) (Vyzovitis et al., 2020).
+{% end %}

content/reports/research-initiatives/nat-hole-punching.md

@@ -0,0 +1,30 @@
++++
+title = "NAT Hole Punching"
+description = "A measurement campaign evaluating the DCUtR hole-punching protocol across ~6.25M attempts in 372 networks, with an accompanying research paper."
+weight = 2
+
+[extra]
+author = "ProbeLab"
++++
+
+This report evaluates the success rate and performance of libp2p's DCUtR (Direct Connection
+Upgrade through Relay) protocol for NAT hole punching. The campaign ran from December 2022
+through January 2023, tracking roughly **6.25 million hole-punch attempts** across 372
+identified networks in 39 countries, using a honeypot to discover NAT'd peers, a central
+coordination server, and distributed Go and Rust clients.
+
+**Key findings:**
+
+- **~70% success rate** for hole punching across networks, independent of round-trip time to the relay.
+- **TCP and QUIC perform similarly**, but QUIC wins ~81% of races when both are available.
+- **First attempts succeed most often**; retries rarely help.
+- **IPv6 showed surprisingly low success rates**, flagged for further investigation.
+- **VPNs reduce effectiveness** by adding extra NAT layers.
+- **90% of successful direct connections** achieve lower latency than the relayed path.
+
+The findings produced three protocol-improvement proposals and surfaced address-reporting
+and IPv6 issues in libp2p implementations.
+
+{% alert(type="note") %}
+Read the full report in the [network-measurements repository](https://github.com/probe-lab/network-measurements/blob/main/results/rfm15-nat-hole-punching.md), and the accompanying research paper on [arXiv (2510.27500)](https://arxiv.org/abs/2510.27500).
+{% end %}

content/reports/research-initiatives/optimistic-provide.md

@@ -0,0 +1,31 @@
++++
+title = "Optimistic Provide"
+description = "How statistical network-size estimation and predictive termination cut IPFS DHT publication latency from ~15s to ~0.7s."
+weight = 1
+
+[extra]
+author = "Yiannis Psarras, Dennis Trautwein"
++++
+
+Publishing content to IPFS via the DHT was historically slow — often 13–20 seconds, sometimes
+minutes. The bottleneck was the rigid DHT-walk termination condition, which forced nodes to
+wait for specific peers that were frequently unreachable. **Optimistic Provide** rethinks when
+a provide operation can safely stop.
+
+**How it works:**
+
+1. **Network size estimation** — nodes estimate the global network size from peer distances observed during routine routing-table refreshes, adding zero network overhead.
+2. **Predictive termination** — instead of waiting for rigid confirmation, the algorithm uses statistical heuristics to stop early once it has reached the necessary peers with ~90% confidence.
+3. **Early return** — control returns after 15 of 20 peers confirm storage, while remaining requests complete asynchronously.
+
+**Results:**
+
+- Upload latency drops from ~15s to **~0.7s — a 10×+ improvement**, with content discoverable within ~1 second of publication.
+- Record availability stays comparable to the original approach (only modest, negligible-impact replication reductions).
+- Network overhead is reduced by ~40%.
+
+Shipped as the default in **Kubo v0.39.0** (February 2026).
+
+{% alert(type="note") %}
+Read the full write-up on the [ProbeLab blog](https://probelab.io/blog/optimistic-provide/) (Yiannis Psarras & Dennis Trautwein, March 2026).
+{% end %}

templates/shortcodes/alert.html

@@ -41,6 +41,6 @@
         {% if display_title %}
         <strong class="alert__title">{{ display_title }}</strong>
         {% endif %}
-        {{ body | safe }}
+        {{ body | markdown | safe }}
     </div>
 </div>

templates/shortcodes/details.html

@@ -22,6 +22,6 @@
         </svg>
     </summary>
     <div class="details__content">
-        {{ body | safe }}
+        {{ body | markdown | safe }}
     </div>
 </details>

templates/status/section.html

@@ -92,6 +92,16 @@ <h2 class="status-section__title">Hole Punching</h2>
                     repository. View detailed results and contribute to the test suite there.
                 </p>
             </section>
+
+            {# Network Metrics #}
+            <section class="status-section">
+                <header class="status-section__header">
+                    <h2 class="status-section__title">Network Metrics</h2>
+                    <p class="status-section__description">
+                        While these pages show information about how different libp2p implementation perform and interoperate, they do not show how the implementations perform in the wild at a larger scale. The <a href="https://probelab.io" target="_blank" rel="noopener noreferrer">ProbeLab</a> team is producing network-wide metrics for libp2p-based projects on their website which you can find <a href="https://probelab.io/networks/" target="_blank" rel="noopener noreferrer">here</a>.
+                    </p>
+                </header>
+            </section>
         </article>
     </div>
 </div>