rfq: add tls support for price oracles #1775

jtobin · 2025-08-30T10:29:30Z

(Draft, for now, as the test suite still needs a little work.)

Adds TLS support for communication with price oracles, mostly following the suggestions proposed in #1278. Adds configuration options for skipping certificate verification, distrusting the operating system's root CA list, and using a custom certificate.

Resolves #1278.

coveralls · 2025-08-30T10:53:42Z

Pull Request Test Coverage Report for Build 17361316751

Details

57 of 69 (82.61%) changed or added relevant lines in 5 files are covered.
35 unchanged lines in 9 files lost coverage.
Overall coverage increased (+8.1%) to 56.705%

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
rfq/tls.go	23	25	92.0%
tapcfg/server.go	3	6	50.0%
tapcfg/config.go	26	33	78.79%

Files with Coverage Reduction	New Missed Lines	%
tapdb/addrs.go	2	78.23%
tapdb/mssmt.go	2	91.36%
tapgarden/custodian.go	2	77.02%
itest/assertions.go	3	89.08%
itest/multisig.go	3	97.91%
proof/verifier.go	3	86.82%
mssmt/compacted_tree.go	4	80.0%
tapdb/assets_store.go	7	79.61%
universe/archive.go	9	80.05%

Totals
Change from base Build 17301129198:	8.1%
Covered Lines:	61073
Relevant Lines:	107703

💛 - Coveralls

jtobin · 2025-08-31T20:00:53Z

(Changed this from draft; I think the litd tests are failing for an unrelated reason.)

jtobin · 2025-09-03T12:21:18Z

(As pointed out by @ZZiigguurraatt, to be more precise: TLS support already existed for price oracles, but certificate verification was skipped entirely.)

jtobin · 2025-09-03T12:22:06Z

rfq/cli.go

+
+	PriceOracleTLSInsecure bool `long:"priceoracletlsinsecure" description:"Disable price oracle certificate verification."`
+
+	PriceOracleTLSNoSystemCAs bool `long:"priceoracletlsnosystemcas" description:"Disable use of the operating system's list of root CA's when verifiying price oracle certificates."`


typo: s/verifiying/verifying

ffranr

There are other cases where we need more precise control over TLS behavior. For example:

taproot-assets/proof/courier.go

Lines 309 to 320 in a17a67a

    
           // serverDialOpts returns the set of server options needed to connect to the 
        
           // server using a TLS connection. 
        
           func serverDialOpts() ([]grpc.DialOption, error) { 
        
           	var opts []grpc.DialOption 
        
           	// Skip TLS certificate verification. 
        
           	tlsConfig := tls.Config{InsecureSkipVerify: true} 
        
           	transportCredentials := credentials.NewTLS(&tlsConfig) 
        
           	opts = append(opts, grpc.WithTransportCredentials(transportCredentials)) 
        
           	return opts, nil 
        
           }

With that in mind, I wonder if we could define a more general, reusable solution in something like the new rfq/tls.go file, especially given the need for configuration and the importance of which package owns this logic.

ffranr · 2025-09-16T15:58:44Z

rfq/tls_test.go

+			name:           "invalid custom certificate",
+			expectInsecure: false,
+			tlsConfig: &TLSConfig{
+				Enabled:            true,
+				InsecureSkipVerify: false,
+				TrustSystemRootCAs: false,
+				CustomCertificates: []byte(invalidCertificate),
+			},
+		},


I'm not sure I see the purpose of invalidCertificate here. It doesn't look like the test actually exercises its invalidity.

More broadly, do we need certificate examples in our unit tests at all? It seems like we're testing the behavior of the underlying TLS/certificate library rather than the logic we're adding on top of it.

Refactor the function by introducing an internal helper to improve readability.

Refactor MacaroonWhitelist to handle public universe proof courier permissions independently. This clarifies the logic and avoids coupling courier access with other universe server permissions.

Remove the defaultMacaroonWhitelist map and inline its entries directly into the conditional logic within MacaroonWhitelist. This ensures that access to previously always-available endpoints is now governed by explicit user configuration (read/write/courier), improving permission control and aligning with expected access restrictions.

Separate the mint anchor transaction fee rate calculation from fundGenesisPsbt into anchorTxFeeRate. This refactor is part of a broader effort to simplify calling fundGenesisPsbt from unit tests.

Extract the wallet funding call into a closure that is passed as an argument. This prepares fundGenesisPsbt to become a standalone function, making it easier to call in unit tests.

Pass the pending batch and chain params into fundGenesisPsbt and convert it into a standalone function rather than a method on ChainPlanter. This change makes it easier to call fundGenesisPsbt from unit tests.

The batch key was only used for logging. This commit moves the log messages outside fundGenesisPsbt, simplifying the function for better code health.

The mock helper FundGenesisTx now returns the index of the change output. It also dynamically computes the index of the change output it adds. These enhancements will be useful when handling packets with supply pre-commitment outputs.

The funding routine now uses the refactored fundGenesisPsbt function, introduced in a previous commit. Which adds test coverage for the batch funding logic. An optional argument is also added to allow skipping funding.

RandSeedlingMintingBatch retires.

Refactored GetBlockTimestamp to call GetBlockHeaderByHeight and return an optional error type. Removed the timestamp-to-block-height cache, as it did not handle re-orgs correctly. This prepares the codebase for a more comprehensive caching mechanism to be added in a follow-up commit.

Introduce a reusable cache that stores full headers keyed by height and hash. Tracks confirmation depth and treats shallow entries as unsettled (return miss). Detects conflicts at a height and invalidates shallower headers on reorg. Size and random purge fraction are configurable (default 100k entries, 10 percent). Not yet used by LndRpcChainBridge.

…-rpc-macaroonwhitelist Improve `MacaroonWhitelist` Structure and Permission Granularity

Adds a block header cache to LndRpcChainBridge, which indirectly improves performance of methods like GetBlockTimestamp by avoiding repeated block header fetches.

…int-pre-commit Refactor `fundGenesisPsbt` and enhance test helpers for coverage

…k-header-cache lndservices: add reorg aware block header cache; use in ChainBridge

Introduces rfq/tls.go, which contains a basic TLSConfig type and default value of such. The default value, which for now only indicates that certificate verification should be skipped, is used in place of the 'dialInsecure' bool when setting up the price oracle RPC.

Adds both 'TrustSystemRootCAs' and 'CustomCertificates' to the rfq TLSConfig. The former indicates whether or not to trust the operating system's root CA list; the latter allows additional certificates (CA or self-signed) to be trusted. Also adds a basic unit test skeleton.

We don't skip certificate verification by default, and also default to trusting the operating system's root CA list.

Adds some basic test cases for configuring transport credentials.

Ensures that certificate verification is skipped when constructing a communication channel with the itest oracle harness.

Ensures the price oracle TLS toggle fits the existing pattern of flags defaulting to false.

lightninglabs-deploy · 2025-11-11T20:25:48Z

@GeorgeTsagk: review reminder
@jtobin, remember to re-request review from reviewers when ready

jtobin added security mainnet preparedness Goal issues to be completed before mainnet release RFQ Work relating to TAP channel Request For Quote (RFQ). oracle labels Aug 30, 2025

jtobin force-pushed the oracle-tls branch 2 times, most recently from 77cad42 to 9910220 Compare August 31, 2025 13:27

jtobin requested review from GeorgeTsagk and ffranr August 31, 2025 19:59

jtobin marked this pull request as ready for review August 31, 2025 20:00

jtobin commented Sep 3, 2025

View reviewed changes

levmi assigned jtobin Sep 4, 2025

levmi added this to Taproot-Assets Project Board Sep 4, 2025

github-project-automation bot moved this to 🆕 New in Taproot-Assets Project Board Sep 4, 2025

levmi moved this from 🆕 New to 🏗 In progress in Taproot-Assets Project Board Sep 4, 2025

levmi moved this from 🏗 In progress to 👀 In review in Taproot-Assets Project Board Sep 4, 2025

ffranr reviewed Sep 16, 2025

View reviewed changes

jtobin added this to the v0.8 milestone Sep 29, 2025

ffranr added 10 commits October 8, 2025 11:13

taprpc: simplify MacaroonWhitelist func code

4094e8f

Refactor the function by introducing an internal helper to improve readability.

taprpc: separate universe proof courier permissions in MacaroonWhitelist

cea2725

Refactor MacaroonWhitelist to handle public universe proof courier permissions independently. This clarifies the logic and avoids coupling courier access with other universe server permissions.

proof: update stale universe proof courier comment

fe70d4f

taprpc: add default permissions for MailboxInfo and Universe/Info

99c30a5

tapgarden: refactor fee rate calculation from fundGenesisPsbt

9f2fe58

Separate the mint anchor transaction fee rate calculation from fundGenesisPsbt into anchorTxFeeRate. This refactor is part of a broader effort to simplify calling fundGenesisPsbt from unit tests.

tapgarden: refactor fundGenesisPsbt to pass wallet funding as closure

687b851

Extract the wallet funding call into a closure that is passed as an argument. This prepares fundGenesisPsbt to become a standalone function, making it easier to call in unit tests.

tapgarden: refactor fundGenesisPsbt to pass in pending batch

affedad

Pass the pending batch and chain params into fundGenesisPsbt and convert it into a standalone function rather than a method on ChainPlanter. This change makes it easier to call fundGenesisPsbt from unit tests.

tapgarden: remove batch key argument from fundGenesisPsbt

f61140d

The batch key was only used for logging. This commit moves the log messages outside fundGenesisPsbt, simplifying the function for better code health.

ffranr and others added 16 commits October 10, 2025 13:28

tapgarden: add batch funding support to RandMintingBatch

2b04c29

The funding routine now uses the refactored fundGenesisPsbt function, introduced in a previous commit. Which adds test coverage for the batch funding logic. An optional argument is also added to allow skipping funding.

tapdb: use new funded mint batch in TestUpsertMintSupplyPreCommit

1fc3ea5

tapdb+tapgarden: replace RandSeedlingMintingBatch with RandMintingBatch

10b3fb4

RandSeedlingMintingBatch retires.

docs: add release note

9b16515

tapdb: add pagination and ordering to QueryAddrEvents

c0845a7

taprpc: add pagination and sorting to AddrReceivesRequest

9a786ad

rpc: add pagination and sorting to AddrReceives endpoint

1e085a3

cmd: add pagination and sorting to tapcli addr receives

aece748

docs: release notes for addrs receives pagination

12dccdd

itest: add test for address receives pagination

60fbb90

Merge pull request lightninglabs#1841 from lightninglabs/wip/refactor…

8f6b510

…-rpc-macaroonwhitelist Improve `MacaroonWhitelist` Structure and Permission Granularity

lndservices+tapcfg: use block header cache in LndRpcChainBridge

9853145

Adds a block header cache to LndRpcChainBridge, which indirectly improves performance of methods like GetBlockTimestamp by avoiding repeated block header fetches.

Merge pull request lightninglabs#1418 from lightninglabs/unit-tests-m…

57984a4

…int-pre-commit Refactor `fundGenesisPsbt` and enhance test helpers for coverage

Merge pull request lightninglabs#1849 from lightninglabs/wip/add-bloc…

83a4116

…k-header-cache lndservices: add reorg aware block header cache; use in ChainBridge

jtobin force-pushed the oracle-tls branch from f03cb78 to f360fa7 Compare October 21, 2025 11:23

jtobin changed the base branch from main to 0-8-0-staging October 21, 2025 11:24

jtobin force-pushed the oracle-tls branch from f360fa7 to 0e69a19 Compare October 21, 2025 11:25

jtobin added 7 commits October 21, 2025 15:26

tapcfg: add price oracle TLS config

f554957

We don't skip certificate verification by default, and also default to trusting the operating system's root CA list.

rfq: add tls test cases

ad205e2

Adds some basic test cases for configuring transport credentials.

itest: use insecure connection with oracle harness

2984546

Ensures that certificate verification is skipped when constructing a communication channel with the itest oracle harness.

docs: add release notes

fe806e8

rfq+tapcfg: disable tls when flag present

0e69a19

Ensures the price oracle TLS toggle fits the existing pattern of flags defaulting to false.

darioAnongba force-pushed the 0-8-0-staging branch from 2b3ac4f to 035a840 Compare November 14, 2025 09:38

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

rfq: add tls support for price oracles #1775

rfq: add tls support for price oracles #1775

Uh oh!

jtobin commented Aug 30, 2025 •

edited

Loading

Uh oh!

coveralls commented Aug 30, 2025 •

edited

Loading

Uh oh!

jtobin commented Aug 31, 2025

Uh oh!

jtobin commented Sep 3, 2025

Uh oh!

jtobin Sep 3, 2025

Uh oh!

ffranr left a comment

Uh oh!

ffranr Sep 16, 2025

Uh oh!

lightninglabs-deploy commented Nov 11, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants


		PriceOracleTLSInsecure bool `long:"priceoracletlsinsecure" description:"Disable price oracle certificate verification."`

		PriceOracleTLSNoSystemCAs bool `long:"priceoracletlsnosystemcas" description:"Disable use of the operating system's list of root CA's when verifiying price oracle certificates."`

	// serverDialOpts returns the set of server options needed to connect to the
	// server using a TLS connection.
	func serverDialOpts() ([]grpc.DialOption, error) {
	var opts []grpc.DialOption

	// Skip TLS certificate verification.
	tlsConfig := tls.Config{InsecureSkipVerify: true}
	transportCredentials := credentials.NewTLS(&tlsConfig)
	opts = append(opts, grpc.WithTransportCredentials(transportCredentials))

	return opts, nil
	}

rfq: add tls support for price oracles #1775

Are you sure you want to change the base?

rfq: add tls support for price oracles #1775

Uh oh!

Conversation

jtobin commented Aug 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

coveralls commented Aug 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Pull Request Test Coverage Report for Build 17361316751

Details

💛 - Coveralls

Uh oh!

jtobin commented Aug 31, 2025

Uh oh!

jtobin commented Sep 3, 2025

Uh oh!

jtobin Sep 3, 2025

Choose a reason for hiding this comment

Uh oh!

ffranr left a comment

Choose a reason for hiding this comment

Uh oh!

ffranr Sep 16, 2025

Choose a reason for hiding this comment

Uh oh!

lightninglabs-deploy commented Nov 11, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

jtobin commented Aug 30, 2025 •

edited

Loading

coveralls commented Aug 30, 2025 •

edited

Loading