IntersectMBO / IntersectMBO/cardano-ledger
Look further at: MissingScriptWitnessesUTXOW (fromList [])
- Dominant language
- Haskell
- Stars
- 295
- Forks
- 179
- Avg merge
- 4d 7h
- Merged PRs (30d)
- 29
Description
In https://github.com/input-output-hk/cardano-ledger/pull/3039 we fixed the seemingly random
`MissingScriptWitnessesUTXOW (fromList [])` failure, by discarding traces where there was an unnecessary script witness.
It would be better to not generate transactions with these extra script witnesses. But this seems quite difficult to do, as generating
witnesses is intricately tied up with the fix-point calculation. Here are some notes that might be useful.
There is some evidence that this arises because the script hash appears as the PolicyId in a Value. At least that is the case in
one failing trace. In that trace here is the transaction
```haskell
ValidatedTx {
body = TxBodyConstr TxBodyRaw {
_inputs = fromList
[TxIn (TxId {_unTxId = SafeHash "7887389959cd0b37c21116c1658b195112605df6e1e7165921192b416eccaf32"}) (TxIx 1),
TxIn (TxId {_unTxId = SafeHash "87a0cb55e115848fc0486f17a3d7c68d3fe2f3b24fc0c3cd32dae24459b8526e"}) (TxIx 0)],
_collateral = fromList
[TxIn (TxId {_unTxId = SafeHash "87a0cb55e115848fc0486f17a3d7c68d3fe2f3b24fc0c3cd32dae24459b8526e"}) (TxIx 0)],
_outputs = StrictSeq {fromStrict = fromList
[(Addr Testnet (KeyHashObj (KeyHash "963f3c80fd6cb6104d6cd5e114661fc0d7e9dcb9385acb996986c71c"))
(StakeRefBase (KeyHashObj (KeyHash "bd455eb0e7847df7c1e5552cad84621355b94f69cb3999832b54af10"))),
Value 4342 (fromList
[(PolicyID {policyID = ScriptHash "42c7a014a4cd5537f64e5ae8ec7349db3d8603e16765dc37f8fb6e67"}, ####### Note PolicyID is the extra script
fromList [(79656c6c6f7730,560729),(79656c6c6f7733,477818),(79656c6c6f7735,276901)])]),SNothing),
(Addr Testnet (KeyHashObj (KeyHash "094d78230eb61aaf1ff09f43ff9d3ce63b7932d499d5131757ee62dd"))
(StakeRefBase (KeyHashObj (KeyHash "3bd2ccfdb4fec76fc4f35a6da648a5ffcc4082c6642c64f38de2ea22"))),
Value 4537741 (fromList []),SNothing)]},
_certs = StrictSeq {fromStrict = fromList
[DCertDeleg (RegKey (KeyHashObj (KeyHash "e459931b2628c08350543b7e5202acd8cee0941d51ee1b08b42e6d70"))),
DCertDeleg (Delegate (Delegation {_delegator = KeyHashObj (KeyHash "d840b13dcfcb99c5ad3fb54f0d007c335ed050c17f07b995458b2827"),
_delegatee = KeyHash "15b7af45e7293a42b8eabb19bb0fd568116581a8b99fceaffb3e4bab"}))]},
_wdrls = Wdrl {unWdrl = fromList
[(RewardAcnt {getRwdNetwork = Testnet, getRwdCred = ScriptHashObj
(ScriptHash "395e0b6c308dbdfd6e41354b68f833b96990ecd93721699ed90a2113")},Coin 0),
(RewardAcnt {getRwdNetwork = Testnet, getRwdCred = ScriptHashObj
(ScriptHash "504fca9befba2683bcf5797db908b891ff8e9aacfb3ce633fe495b45")},Coin 0),
(RewardAcnt {getRwdNetwork = Testnet, getRwdCred = ScriptHashObj
(ScriptHash "5c0a02a40d0b72ea023a6628116c3fd7ca00e853dde5f7c90606aeff")},Coin 0),
(RewardAcnt {getRwdNetwork = Testnet, getRwdCred = ScriptHashObj
(ScriptHash "64abdacc0e1cc6b7dcbcf601c28f3e882f5453dca4eb713591635f90")},Coin 0),
(RewardAcnt {getRwdNetwork = Testnet, getRwdCred = ScriptHashObj
(ScriptHash "7c06fcc873546a676c9b83947385ccf5101d2320456ec3f2dd8179f9")},Coin 0),
(RewardAcnt {getRwdNetwork = Testnet, getRwdCred = ScriptHashObj
(ScriptHash "9573fdfbcf7e2fdec139e2fcf7215883855933c9628928d5e5c0f3b4")},Coin 0)]},
_txfee = Coin 1561575,
_vldt = ValidityInterval {invalidBefore = SJust (SlotNo 536), invalidHereafter = SJust (SlotNo 537)},
_update = SNothing,
_reqSignerHashes = fromList [],
_mint = Value 0 (fromList []),
_scriptIntegrityHash = SNothing,
_adHash = SNothing,
_txnetworkid = SNothing},
wits = TxWitnessRaw {
_txwitsVKey = fromList [...]
_txwitsBoot = fromList [],
_txscripts = fromList
[(ScriptHash "395e0b6c308dbdfd6e41354b68f833b96990ecd93721699ed90a2113",
TimelockScript TimelockConstr Signature (KeyHash "380f0d3b9106f8e7e0ee50572044da97257a9d1c55f858f0ae1c9e33")),
EXTRANEOUS Comes from PolicyID of a Value with (Coin 4342) ...
(ScriptHash "42c7a014a4cd5537f64e5ae8ec7349db3d8603e16765dc37f8fb6e67",
TimelockScript TimelockConstr AllOf (StrictSeq {fromStrict = fromList
[TimelockConstr AllOf (StrictSeq {fromStrict = fromList [TimelockConstr AllOf (StrictSeq {fromStrict = fromList []})]})]})),
(ScriptHash "504fca9befba2683bcf5797db908b891ff8e9aacfb3ce633fe495b45",
TimelockScript TimelockConstr MOfN 3 (StrictSeq {fromStrict = fromList
[TimelockConstr AnyOf (StrictSeq {fromStrict = fromList
[TimelockConstr TimeStart (SlotNo 26),
TimelockConstr TimeExpire (SlotNo 26)]}),
TimelockConstr Signature (KeyHash "3a3c9207e9d6a4f55c9be45e2279d3d9826845e01573d8248fcde754"),
TimelockConstr Signature (KeyHash "2508e7e5b012b1af2b9942dc6837d68d38ac6f7fcd0261c08b444d4d")]})),
(ScriptHash "5c0a02a40d0b72ea023a6628116c3fd7ca00e853dde5f7c90606aeff",
TimelockScript TimelockConstr AnyOf (StrictSeq {fromStrict = fromList
[TimelockConstr TimeStart (SlotNo 26),
TimelockConstr TimeExpire (SlotNo 26)]})),
(ScriptHash "64abdacc0e1cc6b7dcbcf601c28f3e882f5453dca4eb713591635f90",
TimelockScript TimelockConstr AnyOf (StrictSeq {fromStrict = fromList
[TimelockConstr Signature (KeyHash "2508e7e5b012b1af2b9942dc6837d68d38ac6f7fcd0261c08b444d4d"),
TimelockConstr Signature (KeyHash "3a3c9207e9d6a4f55c9be45e2279d3d9826845e01573d8248fcde754"),
TimelockConstr AnyOf (StrictSeq {fromStrict = fromList
[TimelockConstr TimeStart (SlotNo 26),
TimelockConstr TimeExpire (SlotNo 26)]})]})),
(ScriptHash "7c06fcc873546a676c9b83947385ccf5101d2320456ec3f2dd8179f9",
TimelockScript TimelockConstr Signature (KeyHash "d3b51656ce61098c59bef02b99529e3b16d9165951cf0c030f915602")),
(ScriptHash "9573fdfbcf7e2fdec139e2fcf7215883855933c9628928d5e5c0f3b4",
TimelockScript TimelockConstr Signature (KeyHash "7006429f96d5a5775d9bca2e69b6f69caeeb8fc34ef2df63763ce35c"))],
_txdats = TxDatsRaw (fromList []),
_txrdmrs = RedeemersRaw (fromList [])},
isValid = IsValid True,
auxiliaryData = SNothing}
```
Note the output, where the Value has a policy ID with script hash "42c7a014a4cd5537f64e5ae8ec7349db3d8603e16765dc37f8fb6e67"
Note that, that is the script hash of the unneeded witness.
How does an extra witness lead to the MissingScriptWitnessesUTXOW (fromList []) failure? This arises because of this code in the Utxow rule in the alonzo era.
```haskell
validateMissingScripts pp sNeeded sReceived =
if HardForks.missingScriptsSymmetricDifference pp
then
sequenceA_
[ failureUnless (sNeeded `Set.isSubsetOf` sReceived) $
MissingScriptWitnessesUTXOW (sNeeded `Set.difference` sReceived),
failureUnless (sReceived `Set.isSubsetOf` sNeeded) $
ExtraneousScriptWitnessesUTXOW (sReceived `Set.difference` sNeeded)
]
else
failureUnless (sNeeded == sReceived) $
MissingScriptWitnessesUTXOW (sNeeded `Set.difference` sReceived) ### TRACE SHOWS WE ARE HERE
```
So he hardforks test is false, and we compute `(sNeeded `Set.difference` sReceived)` which turns out to be the empty set since
sReceived is a superset of sNeeded. Here are the values of those variables.
```
sneeded =fromList
[ScriptHash "395e0b6c308dbdfd6e41354b68f833b96990ecd93721699ed90a2113",
ScriptHash "504fca9befba2683bcf5797db908b891ff8e9aacfb3ce633fe495b45",
ScriptHash "5c0a02a40d0b72ea023a6628116c3fd7ca00e853dde5f7c90606aeff",
ScriptHash "64abdacc0e1cc6b7dcbcf601c28f3e882f5453dca4eb713591635f90",
ScriptHash "7c06fcc873546a676c9b83947385ccf5101d2320456ec3f2dd8179f9",
ScriptHash "9573fdfbcf7e2fdec139e2fcf7215883855933c9628928d5e5c0f3b4"]
received=fromList
[ScriptHash "395e0b6c308dbdfd6e41354b68f833b96990ecd93721699ed90a2113",
ScriptHash "42c7a014a4cd5537f64e5ae8ec7349db3d8603e16765dc37f8fb6e67", -- Extra script hash
ScriptHash "504fca9befba2683bcf5797db908b891ff8e9aacfb3ce633fe495b45",
ScriptHash "5c0a02a40d0b72ea023a6628116c3fd7ca00e853dde5f7c90606aeff",
ScriptHash "64abdacc0e1cc6b7dcbcf601c28f3e882f5453dca4eb713591635f90",
ScriptHash "7c06fcc873546a676c9b83947385ccf5101d2320456ec3f2dd8179f9",
ScriptHash "9573fdfbcf7e2fdec139e2fcf7215883855933c9628928d5e5c0f3b4"]
```
This explains why we get the error. But not why the extra witness is generated. But perhaps it is a usefull clue, that at least in this failure the hash comes from a PolicyID.
Once can reproduce this on commit 4fc32a6117db697bc928d1488d38e9e871e82dbe
using this command
`TASTY_PATTERN='/total amount of Ada is preserved (Chain)/' cabal test --test-options="--quickcheck-replay=76070 "`
Contributor guide
Research direction
Start at validateMissingScripts in the Alonzo Utxow rule and reproduce the failure with TASTY_PATTERN='/total amount of Ada is preserved (Chain)/' cabal test --test-options="--quickcheck-replay=76070" at commit 4fc32a6117db697bc928d1488d38e9e871e82dbe. Trace how the PolicyID script hash becomes an extra witness during fix-point calculation. Done means explaining and preventing the unnecessary witness without relying on discarded traces, with the reproduction passing.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- haskell
- Domain
- blockchain, testing
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100