CryptoScraper is a package for generalizing the retrieval of information from crypto block chains.
CryptoScraper supports the Swift Package Manager. To include CryptoScraper in your project add the following to your Package.swift file:
.package(url: "git@github.com:foscomputerservices/CryptoScraper.git", branch: "main"),
To begin, api keys need to be set in order to access various services. These keys are stored in environment variables as follows:
| Service | Environment Variable | Required |
|---|---|---|
| Etherscan API V2, every EVM chain's scanner | ETHER_SCAN_KEY | No |
| CoinGecko, a pro key (the pro endpoint) | COIN_GECKO_KEY | No |
| CoinGecko, a demo key (the free endpoint, 30 calls a minute) | COIN_GECKO_DEMO_KEY | No |
| CoinMarketCap | COIN_MARKETCAP_KEY | No |
Etherscan's API V2 serves every EVM chain's scanner with the one key, ETHER_SCAN_KEY, each scanner naming its chain by its chain id. A free key reads Ethereum, Polygon and Celo; BNB Smart Chain, Optimism, Avalanche C-Chain and Base need a paid plan, and token information (a token's decimals) needs a Standard plan. V2 does not serve Fantom, Ethereum Classic, Theta or COTI: Fantom's scanner is refused in V2's words, and the other three chains have no scanner (NilScanner).
To begin using the framework call the initialization method to initialize the block chains with their contract data.
try await CryptoScraper.initialize()In order for testing to succeed, a test contract (a wallet address) is needed for each chain. These contracts can be provided in the environment as follows:
| Service | Environment Variable |
|---|---|
| Bitcoin | BTC_TEST_CONTRACT_ADDRESS |
| Ethereum | ETH_TEST_CONTRACT_ADDRESS |
| Fantom | FTM_TEST_CONTRACT_ADDRESS |
| BNB | BSC_TEST_CONTRACT_ADDRESS |
| Matic | POLYGON_TEST_CONTRACT_ADDRESS |
| Optimism | OPTIMISTIC_TEST_CONTRACT_ADDRESS |
| Tron | TRON_TEST_CONTRACT_ADDRESS |
The package's platform floor is 26 on macOS, iOS, Mac Catalyst, tvOS and watchOS; Linux builds with a Swift 6 toolchain.
CryptoAsset builds on macOS, iOS, tvOS, watchOS and Linux. It is the one library an app holds, so CI builds it for generic iOS, tvOS and watchOS devices beside its macOS and Linux tests.
CryptoAssetLocalization renders CryptoAsset's amounts, prices and fractions for a reader through FOSMVVM's localizing encoder: LocalizableAmount, LocalizablePrice and LocalizableFraction, each drawn from the exact integer in the reader's locale, never through a Double. It links CryptoAsset, FOSFoundation and FOSMVVM; CryptoAsset itself never imports FOSMVVM. It is built and tested on macOS and Linux.
CryptoExchange, CryptoHyperliquid, CryptoKraken, CryptoCoinbase, CryptoOHLCV and CryptoReference are built and tested on macOS and Linux only. No app speaks to an exchange, only services read bars, and no app shows reference facts yet, so CI builds none of them for a device.
CryptoFearGreed, Alternative.me's Fear and Greed index, one value a day, links FOSFoundation alone and is built and tested on macOS and Linux only.
CryptoScraper, the original target, now depends on Foundation, FOSFoundation and CryptoAsset only; Web3 is gone. CI builds and tests it on macOS and Linux.
Three test steps gate every build on macOS and Linux, and none of them reaches the network: the libraries' contract tests, the behavioral suites (the libraries', and the CryptoScraper target's, whose suites are named D<n>_…Tests), and the CryptoScraper target's no-network suites (every suite whose name ends NoNetworkTests).
A behavioral test classified red is disabled in place, with its classification as the reason, so what runs is exactly what is expected to pass.
The CryptoScraper target's network suites do not gate yet. They reach the chains' scanners and the aggregators with the keys above, and they stay non-gating until the aggregators' token counts are done and the Bitcoin explorer's rate limit is met. A chain Etherscan's V2 refuses to the key's plan, or does not serve, is skipped in V2's words.
A new push to a branch or pull request cancels the CI run still in progress for it.
The contracts, the asset classes and Kraken's and Binance's exchange tables are generated by Scripts/import-assets.swift from CoinGecko's top coins and the exchanges' public listings, run with swift-sh from the repository's root: swift sh Scripts/import-assets.swift --top 1000 (with COIN_GECKO_KEY set it asks CoinGecko's pro endpoint). It writes every +Imported.swift file and prints what it left out and why. A generated file is never edited by hand; what the sources cannot state goes in the hand-written files beside it, such as KrakenExchangeChain+Overrides.swift.
Which holdings stand for a currency is declared by hand, in each exchange chain's own file beside its holdings (currencyDeclarations, a CurrencyDeclaration per currency), and added to the client's registry with the chain's holdings at the client's init. A holding that stands for a currency is accepted where the currency is named, one for one, with no conversion and no rate: Binance's USDT and USDC, Hyperliquid's USDC, Kraken's and Coinbase's dollars stand for iso4217:USD. AssetRegistry.holding(standingFor:on:) names the one holding on a chain that stands for a currency, and refuses where none or more than one does; AssetRegistry.currency(of:) names the currency a holding stands for.
For complete documentation, please refer to the DocC documentation in Xcode or here for an online version (see the documentation tab in the top-right).
All contributions are welcome! Please see CONTRIBUTING.md for more details.
This project is maintained by David Hunt owner of FOS Computer services, LLC.
CryptoScraper is under the MIT License. See the LICENSE file for more information.