RFC 3490 IDNA (Punycode, Nameprep) integration

[lo48576]

• RFC 3490: Internationalizing Domain Names in Applications (IDNA)

No need to implement the algorithm because there will already be some battle-tested crates for IDNA, but need to provide API from string types of iri-string to use them conveniently.

[lo48576]

Basic support can be introduced without breaking change, but extensive support will need major version bump because it requires dropping Copy trait from some types.

[lo48576]

IDNA conversion (both direction) via idna crate can fail for valid hostname. The current normalization implementation through fmt::Display trait is not highly compatible with it because the formatting itself is usually never expected to fail.

Note that fmt::Error usually represents the failure in the backend sink (such as String or impl io::Write), even though it can be used for failure of formatting itself.

Error in std::fmt - Rust:

This type does not support transmission of an error other than that an error occurred. This is because, despite the existence of this error, string formatting is considered an infallible operation. fmt() implementors should not return this Error unless they received it from their Formatter. The only time your code should create a new instance of this error is when implementing fmt::Write, in order to cancel the formatting operation when writing to the underlying stream fails.

[lo48576]

Firefox (147.0.4) accepts percent-encoded domain, but curl does not. In fact curl looks like rejecting any strings as a domain when it contains non-ASCII characters.

$ curl -sSD- https://xn--wgv71a119e.jp -o /dev/null
HTTP/1.1 200 OK
Date: Thu, 19 Feb 2026 23:03:10 GMT
Content-Type: text/html
Strict-Transport-Security: max-age=2592000
Transfer-Encoding: chunked

$ curl -sSD- curl -sS 'https://日本語.jp'
curl: (6) Could not resolve host: 日本語.jp (Misformatted domain name)
$ curl -sSD- 'https://%E6%97%A5%E6%9C%AC%E8%AA%9E+%E3%83%89%E3%83%A1%E3%82%A4%E3%83%B3.jp'
curl: (3) URL rejected: Bad hostname
$

[lo48576]

In fact curl looks like rejecting any strings as a domain when it contains non-ASCII characters.

This was simply because my environment has curl with IDN support disabled.

$ eix -e curl
[I] net-misc/curl
     Available versions:  8.15.0^t 8.16.0-r1^t (~)8.17.0^t 8.17.0-r1^t (~)8.18.0^t **9999*l^t {+adns +alt-svc brotli debug ech +ftp gnutls gopher +hsts +http2 +http3 +httpsrr idn +imap kerberos ldap mbedtls +openssl +pop3 +psl +quic rtmp rustls samba sasl-scram +smtp ssh ssl static-libs telnet test +tftp verify-sig +websockets zstd ABI_MIPS="n32 n64 o32" ABI_S390="32 64" ABI_X86="32 64 x32" CURL_QUIC="ngtcp2 +openssl" CURL_SSL="gnutls mbedtls +openssl rustls"}
     Installed versions:  8.18.0^t(2026-01-22 00:05:48)(adns alt-svc ftp hsts http2 http3 httpsrr imap openssl pop3 psl quic smtp ssl telnet tftp websockets zstd -brotli -debug -ech -gnutls -gopher -idn -kerberos -ldap -mbedtls -rtmp -rustls -samba -sasl-scram -ssh -static-libs -test -verify-sig ABI_MIPS="-n32 -n64 -o32" ABI_S390="-32 -64" ABI_X86="64 -32 -x32" CURL_QUIC="openssl -ngtcp2" CURL_SSL="openssl -gnutls -mbedtls -rustls")
     Homepage:            https://curl.se/
     Description:         A Client that groks URLs

$ curl --version
curl 8.18.0 (x86_64-pc-linux-gnu) libcurl/8.18.0 OpenSSL/3.5.5 zlib/1.3.2 zstd/1.5.7 c-ares/1.34.6 libpsl/0.21.5 nghttp2/1.68.0 nghttp3/1.15.0
Release-Date: 2026-01-07
Protocols: dict file ftp ftps http https imap imaps ipfs ipns mqtt pop3 pop3s rtsp smtp smtps telnet tftp ws wss
Features: alt-svc AsynchDNS HSTS HTTP2 HTTP3 HTTPS-proxy HTTPSRR IPv6 Largefile libz NTLM PSL SSL threadsafe TLS-SRP UnixSockets zstd

[lo48576]

My current thought is, using raw Unicode representation as IRI (not URI) is most natural, and IDNA should be used mainly to make a network request to DNS.
Percent-encoding for a domain does not help to make DNS request, but IRI is not solely for network access, so using percent-encoding for a domain should be perfectly valid.

I still have no idea how the default "normalization" should behave for non-ASCII domain.
IDNA (via idna crate) will introduce additional dependencies and may fail, however, iri-string might have to deal with Unicode normalization without IDNA support.

[lo48576]

RFC 3987 section 5.3.3 (Scheme-Based Normalization) mentions IDN:

Some IRI schemes may allow the usage of Internationalized Domain Names (IDN) [RFC3490] either in their ireg-name part or elsewhere. When in use in IRIs, those names SHOULD be validated by using the ToASCII operation defined in [RFC3490], with the flags "UseSTD3ASCIIRules" and "AllowUnassigned". An IRI containing an invalid IDN cannot successfully be resolved. Validated IDN components of IRIs SHOULD be character normalized by using the Nameprep process [RFC3491]; however, for legibility purposes, they SHOULD NOT be converted into ASCII Compatible Encoding (ACE).

Note that this is scheme-based normalization, which is protocol-specific to some extent.
This is not a convention for a "generic URI/IRI", so I think supporting this use case is not mandatory for iri-string.
However, this is very common use case with HTTP, so it is better to support IDNA as opt-in.
Only supporting ToAscii algorithm would be enough for normalization and making network request.

[lo48576]

Uts46 in idna::uts46 - Rust

Even ToASCII conversion has multiple parameters, and they should be provided by the user based on the usage of the URI/IRI (for example, whether the result would be used as part of a network request).

Maybe iri-string should accept (potentially-failing) conversion function as a parameter instead of using idna crate internally, to make the conversion sufficiently flexible?

[lo48576]

RFC 3986: Uniform Resource Identifier (URI): Generic Syntax (page 21)

The reg-name syntax allows percent-encoded octets in order to represent non-ASCII registered names in a uniform way that is independent of the underlying name resolution technology. Non-ASCII characters must first be encoded according to UTF-8 [STD63], and then each octet of the corresponding UTF-8 sequence must be percent-encoded to be represented as URI characters.
(snip)
When a non-ASCII registered name represents an internationalized domain name intended for resolution via the DNS, the name must be transformed to the IDNA encoding [RFC3490] prior to name lookup.

It looks reasonable to make percent-encoding default, and support IDNA encoding as opt-in.

[lo48576]

Added support (kind of) by recent changes and documented by 23f64cc.