feat: follow proxies and use Sourcify as the only source of reference ABIs

.github/workflows/ci.yml

@@ -28,7 +28,13 @@ jobs:
         uses: actions/checkout@v7
         with:
           fetch-depth: 0
-          submodules: "recursive"
+
+      - name: Checkout test registries
+        timeout-minutes: 10
+        run: >-
+          git submodule update --init --checkout
+          tests/registries/clear-signing-erc7730-registry
+          tests/registries/ledger-asset-dapps
 
       - name: Setup mise
         timeout-minutes: 10

.gitmodules

@@ -1,6 +1,8 @@
 [submodule "clear-signing-erc7730-registry"]
 	path = tests/registries/clear-signing-erc7730-registry
 	url = https://github.com/LedgerHQ/clear-signing-erc7730-registry
+	update = none
 [submodule "ledger-asset-dapps"]
 	path = tests/registries/ledger-asset-dapps
 	url = https://github.com/LedgerHQ/ledger-asset-dapps
+	update = none

docs/pages/developer.md

@@ -19,10 +19,14 @@ brew install mise
 1. **Clone the repository:**
 
 ```bash
-git clone --recursive git@github.com:LedgerHQ/python-erc7730.git
+git clone git@github.com:LedgerHQ/python-erc7730.git
 cd python-erc7730
+git submodule update --init --checkout tests/registries/clear-signing-erc7730-registry tests/registries/ledger-asset-dapps
 ```
 
+The test registries are git submodules that are skipped by a recursive submodule update (so that `pip` can install the
+library from git without an SSH key for a nested submodule), hence the explicit `--checkout`.
+
 2. **Install tools via mise:**
 
 ```bash

docs/pages/lint.md

@@ -1,14 +1,31 @@
 # Linter checks list
 ## ABI checks
+The four "not verified" / "not supported" / "could not fetch" checks below are reported as errors instead when `--require-verified` is passed, so that a strict run never passes without having checked every deployment.
+
+### Contract not verified
+- **Level**: ⚠️ Warning 
+- **Message**: `contract <address> on chain <chain_id> is not verified on Sourcify, descriptor ABIs will not be validated`
+- **Description**: The contract is not verified on Sourcify, so there is no reference ABI. Subsequent checks are skipped for the current deployment.
+
+### Proxy implementation not verified
+- **Level**: ⚠️ Warning 
+- **Message**: `contract <address> on chain <chain_id> is a proxy, and its implementation <address> is not verified on Sourcify, descriptor ABIs will not be validated`
+- **Description**: Sourcify resolved the contract as a proxy, but one of its implementations is not verified, so the reference ABI would be incomplete. Subsequent checks are skipped for the current deployment.
+
+### Chain not supported
+- **Level**: ℹ️ Info 
+- **Message**: `chain <chain_id> is not supported by Sourcify, descriptor ABIs will not be validated`
+- **Description**: Sourcify does not support the chain of the deployment, so no reference ABI can be fetched. Subsequent checks are skipped for the current deployment.
+
 ### Could not fetch ABI
 - **Level**: ⚠️ Warning 
 - **Message**: `Fetching reference ABI for chain id <chain_id> failed, descriptor ABIs will not be validated: <error>`
-- **Description**: ABI fetch from external sources (Sourcify, then Etherscan as a fallback) has failed. Subsequents checks are skipped for the current deployment.
+- **Description**: ABI fetch from Sourcify has failed for another reason, such as a rate limit, a network error or a proxy resolution error. Subsequent checks are skipped for the current deployment.
 
-### Proxy Contract
+### Deployment ABIs differ
 - **Level**: ⚠️ Warning 
-- **Message**: `Contract <url> is likely to be a proxy, validation of descriptor ABIs skipped`
-- **Description**: Contract detected as a potential proxy contract based on a simple heuristic. Subsequents checks are skipped.
+- **Message**: `The reference ABIs of the deployments do not all expose the same functions, display fields are validated against each distinct reference ABI: <chain id>:<address>, ...; <chain id>:<address>, ...`
+- **Description**: The deployments of the descriptor do not have the same reference ABI. Display fields are validated once per distinct ABI, so findings may apply to some chains only (the contract URL in each finding tells which).
 
 ### Extra function
 - **Level**: ⚠️ Warning 

docs/pages/usage_cli.md

@@ -67,9 +67,9 @@ excluded, please add it to `excluded` list to avoid this warning.
 
 ➡️ checking tether/calldata-usdt.json…
 🟠 warning: Function mismatch: Function approve(address,uint256) (selector: 0x095ea7b3) defined in descriptor ABIs does not match
-reference ABI (see https://etherscan.io/address/0xdac17f958d2ee523a2206206994597c13d831ec7#code)
+reference ABI (see https://repo.sourcify.dev/1/0xdac17f958d2ee523a2206206994597c13d831ec7)
 🟠 warning: Function mismatch: Function transfer(address,uint256) (selector: 0xa9059cbb) defined in descriptor ABIs does not match
-reference ABI (see https://etherscan.io/address/0xdac17f958d2ee523a2206206994597c13d831ec7#code)
+reference ABI (see https://repo.sourcify.dev/1/0xdac17f958d2ee523a2206206994597c13d831ec7)
 🔴 error: Invalid data path: "0xdAC17F958D2ee523a2206206994597C13D831ec7" is invalid, it must contain a data path to the address in the
 transaction data. It seems you are trying to use a constant address value instead, please note this feature is not supported (yet).
 
@@ -78,13 +78,15 @@ checked 61 descriptor files, some errors found ❌
 
 It can be called with single files or directories, in which case all descriptors will be checked.
 
-Use `--skip-abi-validation` to disable external ABI comparisons against Sourcify/Etherscan (useful for offline runs or faster local checks).
+Use `--skip-abi-validation` to disable external ABI comparisons against Sourcify (useful for offline runs or faster local checks).
+
+Use `--require-verified` to report contracts (or proxy implementations) that are not verified on Sourcify as errors instead of warnings. With the flag, a reference ABI that could not be fetched (rate limit, network error, proxy resolution failure) is an error too.
 
 ### `erc7730 generate`
 
 The `generate` command bootstraps a new descriptor file from ABIs or message schemas:
 ```shell
-# fetch ABIs from sourcify/etherscan and generate a new calldata descriptor
+# fetch ABIs from sourcify and generate a new calldata descriptor
 erc7730 generate --chain-id=1 --address=0x68b3465833fb72A70ecDF485E0e4C7bD8665Fc45
 
 # generate a new calldata descriptor using given ABI file
@@ -94,12 +96,11 @@ erc7730 generate --chain-id=1 --address=0x00000000000000000000000000000000000000
 erc7730 generate --chain-id=1 --address=0x0000000000000000000000000000000000000000 --schema schemas.json
 ```
 
-ABIs are fetched from [Sourcify](https://sourcify.dev) first, which requires no API key. If the contract is not
-verified on Sourcify, Etherscan is used as a fallback, which requires
-[setting up an Etherscan API key](https://docs.etherscan.io/getting-started/viewing-api-usage-statistics):
-```shell
-export ETHERSCAN_API_KEY=XXXXXX
-```
+ABIs are fetched from [Sourcify](https://sourcify.dev), which requires no API key. The contract (and its
+implementations, if it is a proxy) must be verified on Sourcify. Sourcify responses are cached for one hour in
+`~/.cache/erc7730` (or `$XDG_CACHE_HOME/erc7730`), set `ERC7730_NO_CACHE=1` to disable the cache.
+
+If you have a Sourcify API token, set `SOURCIFY_TOKEN` and it is sent with every Sourcify request.
 
 Please note that while the generator does its best to guess the right format based on fields name/type, the generated
 descriptor should be considered a starting point to refine.

src/erc7730/common/abi.py

@@ -128,7 +128,6 @@ def function_to_selector(abi: Function) -> str:
 @dataclass(kw_only=True)
 class Functions:
     functions: dict[str, Function]
-    proxy: bool
 
 
 _READ_ONLY_MUTABILITIES = frozenset({StateMutability.pure, StateMutability.view})
@@ -141,14 +140,12 @@ def get_functions(abis: list[ABI], *, include_read_only: bool = False) -> Functi
     :param include_read_only: if False (default), filter out pure/view functions that cannot produce transactions
     :return: Functions dataclass with selector->Function mapping
     """
-    functions = Functions(functions={}, proxy=False)
+    functions = Functions(functions={})
     for abi in abis:
         if abi.type == "function":
             if not include_read_only and abi.stateMutability in _READ_ONLY_MUTABILITIES:
                 continue
             functions.functions[function_to_selector(abi)] = abi
-            if abi.name in ("proxyType", "getImplementation", "implementation", "proxy__getImplementation"):
-                functions.proxy = True
     return functions