WoWyrd · Rule review
Rule review: technical report
Client: Demo client with its own rule file (own-rules, synthetic). Data: 2026-08 to 2026-10 (monthly periods). Cut-off: 2026-10-01 (given with --cutoff). Generated: 2026-11-20 10:02 UTC.
Internal report. Do not share it outside your firm.
Method and definitions
The review measures what the client's own engine logged. Only the section "Your imported rules" replays the rule definitions the client gave, and that replay never replaces a logged number. Label delay: 14 days. Hits newer than 2026-10-01 minus the label delay are not yet mature. One alert takes 6 analyst minutes at 0.80 USD per minute. A declined payment that was not fraud costs 4.00 USD in customer friction. The client entered these costs. Every cost in this report is an estimate from them.
- A hit is a transaction on which the rule fired, from the client's rule-hit log. An alert is a transaction with at least one hit.
- A hit is mature when the transaction is older than the client's stated label delay at the cut-off, so a fraud label had time to arrive. Immature hits are reported, never counted as false alarms or as friction.
- The label of a transaction is the label row with the latest date known at or before the cut-off. A later label that arrives after the cut-off is not used.
- A declined payment has no outcome (selective labels): its hits are counted and reported, and left out of precision and false-alarm counts.
- Evaluable = mature and not declined. Caught = evaluable with a fraud label. False alarm = evaluable with no fraud and no disputed label. Disputed = evaluable with a disputed label: it is neither caught nor a false alarm, it is shown on its own, and it is left out of precision.
- Precision = caught / (caught + false alarms). Recall share = caught / all evaluable fraud in the period.
- Alert cost: one alert per transaction with any hit, whatever the number of rules; a false alarm costs analyst minutes x cost per minute, shared equally among the rules that fired on it. Friction: a mature declined payment with no fraud and no disputed label costs the client's friction cost, shared the same way.
- Decay: precision of the last three mature periods is at most half of the three before (needs six periods).
Totals
| Measure | Value |
|---|---|
| Transactions | 1,950 |
| Alerts (transactions with at least one hit) | 723 |
| Alerts not yet mature (newer than the label delay) | 140 |
| Alerts on declined payments (no outcome; left out of precision) | 80 |
| Confirmed fraud (label known by the cut-off) | 103 |
| Confirmed fraud with at least one hit | 82 |
| Evaluable fraud (mature, not declined) | 66 |
| False alarms | 450 |
| Declined payments with no fraud or disputed label (friction) | 42 |
| False-alarm cost (estimate) | 2,328.00 USD |
| Rule-hit rows that matched a transaction | 1,950 of 1,950 (100.0%) |
| Label rows that matched a transaction | 639 of 639 (100.0%) |
Source: transactions, rule-hit log and labels files, 2026-08 to 2026-10.
Per-rule metrics
Rules are sorted by false-alarm cost, highest first. The trend line shows precision per period.
| Rule | Hits | Not yet mature | Declined | Evaluable | Caught | False alarms | Precision | Recall share | Cost (USD) | Decayed | Precision trend |
|---|---|---|---|---|---|---|---|---|---|---|---|
| tp_u21_transfer_over_500 No description supplied by the client | 701 | 138 | 71 | 492 | 41 | 443 | 8.5% | 62.1% | 2,092.07 USD | No | |
| tp_u21_large_txn_9k No description supplied by the client | 80 | 15 | 9 | 56 | 19 | 36 | 34.5% | 28.8% | 90.93 USD | No | |
| tp_u21_wire_total_gt4999_7d No description supplied by the client | 95 | 15 | 80 | 0 | 0 | 0 | no data | 0.0% | 83.67 USD | No | |
| tp_ffiec_fan_in_5_senders_5d No description supplied by the client | 24 | 7 | 0 | 17 | 11 | 6 | 64.7% | 16.7% | 23.20 USD | No | |
| tp_u21_fan_out_5_benef_5d No description supplied by the client | 25 | 4 | 1 | 20 | 13 | 7 | 65.0% | 19.7% | 20.20 USD | No | |
| tp_u21_wire_count_gt2_7d No description supplied by the client | 22 | 0 | 18 | 4 | 2 | 2 | 50.0% | 3.0% | 17.93 USD | No |
Hits and precision per period
Each cell: hits / precision in that period.
| Rule | 2026-08 | 2026-09 | 2026-10 |
|---|---|---|---|
| tp_ffiec_fan_in_5_senders_5d No description supplied by the client | 15 / 73.3% | 9 / 0.0% | 0 / no data |
| tp_u21_fan_out_5_benef_5d No description supplied by the client | 16 / 60.0% | 9 / 80.0% | 0 / no data |
| tp_u21_large_txn_9k No description supplied by the client | 41 / 34.3% | 39 / 35.0% | 0 / no data |
| tp_u21_transfer_over_500 No description supplied by the client | 373 / 9.6% | 318 / 6.2% | 10 / no data |
| tp_u21_wire_count_gt2_7d No description supplied by the client | 20 / 66.7% | 2 / 0.0% | 0 / no data |
| tp_u21_wire_total_gt4999_7d No description supplied by the client | 51 / no data | 44 / no data | 0 / no data |
What changed since the previous report
Compared with the previous report, which covers data up to 2026-08-27 (35 days earlier). Its numbers are as measured at its own cut-off: labels that matured later are not added to it.
Per rule
Sorted by the size of the change in alerts, largest first. Precision is shown only with at least 30 hits with a known outcome in each run. Hit rate = alerts / transactions.
| Rule | Alerts then | Alerts now | Change | Precision then | Precision now | Change | Hit rate then | Hit rate now |
|---|---|---|---|---|---|---|---|---|
| tp_u21_transfer_over_500 | 324 | 701 | +116% | 3.3% | 8.5% | +5.1 points | 37.6% | 35.9% |
| tp_u21_wire_total_gt4999_7d | 32 | 95 | +197% | no data | no data | not measured | 3.7% | 4.9% |
| tp_u21_large_txn_9k | 36 | 80 | +122% | no data | 34.5% | not measured | 4.2% | 4.1% |
| tp_u21_wire_count_gt2_7d | 5 | 22 | +340% | no data | no data | not measured | 0.6% | 1.1% |
| tp_u21_fan_out_5_benef_5d | 15 | 25 | +67% | no data | no data | not measured | 1.7% | 1.3% |
| tp_ffiec_fan_in_5_senders_5d | 15 | 24 | +60% | no data | no data | not measured | 1.7% | 1.2% |
| R-509 | 0 | 0 | no change | no data | no data | not measured | 0.0% | 0.0% |
| R-610 | 0 | 0 | no change | no data | no data | not measured | 0.0% | 0.0% |
| R-900 | 0 | 0 | no change | no data | no data | not measured | 0.0% | 0.0% |
Rules added and removed
Added: none.
Removed: none.
Threshold changes
No threshold changed.
Requirements
No requirement file was given for this run, so requirement changes are not compared.
Recommendations from the previous report
Of the previous report's 7 recommendations: 0 changed as proposed, 0 partly changed, 2 not changed, 5 where we cannot tell.
Too few dated changes for a median (0; at least 3 needed).
Open recommendations by age: 7 up to 30 days, 0 of 31–90 days, 0 over 90 days.
Days count from the first report that made each recommendation. A change dated before that report, a change with no exact date and an upper bound ("at most") are shown but not counted. This is a measurement on your own records, not a judgement of your change process.
| Reference | No. in the previous report | Recommendation | Status | Source | Change date | First recommended | Days to change | Dates | Before → after | Predicted | Against the prediction |
|---|---|---|---|---|---|---|---|---|---|---|---|
| REC-73309477 | 1 | Turn off rule tp_u21_transfer_over_500 | Not changed. | your rule file | not known | 2026-11-20, in 1 report | open 0 days, in 1 report | — | Not measured. | Predicted: 9.5 fewer alerts per day (estimated on your data). | — |
| REC-16a49a98 | 2 | Add a rule for: Gambling winnings arrive with no deposits, then card transfers follow | Cannot tell: no change record or rule file covers it, and your data cannot show this kind of change. | none | not known | 2026-11-20, in 1 report | open 0 days, in 1 report | — | Not measured. | No effect was estimated for this recommendation. | — |
| REC-fc5701c9 | 3 | Add a rule for: Large loans are taken and repaid again and again | Cannot tell: no change record or rule file covers it, and your data cannot show this kind of change. | none | not known | 2026-11-20, in 1 report | open 0 days, in 1 report | — | Not measured. | No effect was estimated for this recommendation. | — |
| REC-5ff2bbff | 4 | Add a rule for: One type of operation makes up an unusual share of a user's activity | Cannot tell: no change record or rule file covers it, and your data cannot show this kind of change. | none | not known | 2026-11-20, in 1 report | open 0 days, in 1 report | — | Not measured. | No effect was estimated for this recommendation. | — |
| REC-8078ec06 | 5 | Add a rule for: Payment purposes do not fit the merchant's category code | Cannot tell: no change record or rule file covers it, and your data cannot show this kind of change. | none | not known | 2026-11-20, in 1 report | open 0 days, in 1 report | — | Not measured. | No effect was estimated for this recommendation. | — |
| REC-05f946b2 | 6 | Add a rule for: Regular transfers to a personal account look like business activity | Cannot tell: no change record or rule file covers it, and your data cannot show this kind of change. | none | not known | 2026-11-20, in 1 report | open 0 days, in 1 report | — | Not measured. | No effect was estimated for this recommendation. | — |
| REC-d74333d3 | 7 | Rule tp_u21_transfer_over_500, check 1: threshold 500 → 1000 | Not changed. | your rule file | not known | 2026-11-20, in 1 report | open 0 days, in 1 report | — | Not measured. | Predicted: 0.9 fewer alerts per day (backtest on past data). | — |
Observed after the change, not proven to be caused by it. Traffic, fraud patterns and other changes in the same weeks also move these numbers. A status is about your own change, not about any regulation. This is a measurement for the rule owner, not an audit of your change process.
Changes not linked to our recommendations
None.
Overlaps between rules
Rules that never fired
No rule catalogue was given. A rule that never fired is not in the rule-hit log, so it cannot be listed. Inputs unknown.
Proposals
No written effectiveness threshold was given. The retire line is the fraud rate of all payments with a known outcome.
Effects are backtests on past data, not forecasts.
Add a rule for: Gambling winnings arrive with no deposits, then card transfers follow
Reference REC-16a49a98. Quote this reference in your change record.
Add a missing fraud typeConfidence: lowPriority: 50.0
Rules: none.
Expected effect: no effect estimate for this proposal.
Evidence
- Needs these data fields: customer_id, mcc, operation_type.
- No rule catalogue was given, so we cannot see whether a rule covers it.
- Source: your decision log and fraud labels, 2026-08-03 to 2026-10-01, labels known by 2026-10-01, label delay 14 days.
Caveats
- Selective labels: 95 alerts were on declined payments. Their outcome is unknown, so they are left out of precision and of fraud caught.
- Label maturity: 140 alerts are newer than the 14-day label delay. They are not counted as false alarms.
- Short history: the data covers 3 calendar month(s). Trends need at least 6.
- Synthetic data: the transactions, labels and unit costs are demo values, not a client's data.
- UA-NBU-S07 (Gambling winnings arrive with no deposits, then card transfers follow) is a sign from an NBU draft resolution. It is a draft, not law.
- UA-NBU-S07 (Gambling winnings arrive with no deposits, then card transfers follow)
Add a rule for: Large loans are taken and repaid again and again
Reference REC-fc5701c9. Quote this reference in your change record.
Add a missing fraud typeConfidence: lowPriority: 50.0
Rules: none.
Expected effect: no effect estimate for this proposal.
Evidence
- Needs these data fields: customer_id, operation_type, amount.
- No rule catalogue was given, so we cannot see whether a rule covers it.
- Source: your decision log and fraud labels, 2026-08-03 to 2026-10-01, labels known by 2026-10-01, label delay 14 days.
Caveats
- Selective labels: 95 alerts were on declined payments. Their outcome is unknown, so they are left out of precision and of fraud caught.
- Label maturity: 140 alerts are newer than the 14-day label delay. They are not counted as false alarms.
- Short history: the data covers 3 calendar month(s). Trends need at least 6.
- Synthetic data: the transactions, labels and unit costs are demo values, not a client's data.
- UA-NBU-S05 (Large loans are taken and repaid again and again) is a sign from an NBU draft resolution. It is a draft, not law.
- UA-NBU-S05 (Large loans are taken and repaid again and again)
Add a rule for: One type of operation makes up an unusual share of a user's activity
Reference REC-5ff2bbff. Quote this reference in your change record.
Add a missing fraud typeConfidence: lowPriority: 50.0
Rules: none.
Expected effect: no effect estimate for this proposal.
Evidence
- Needs these data fields: customer_id, operation_type, amount.
- No rule catalogue was given, so we cannot see whether a rule covers it.
- Source: your decision log and fraud labels, 2026-08-03 to 2026-10-01, labels known by 2026-10-01, label delay 14 days.
Caveats
- Selective labels: 95 alerts were on declined payments. Their outcome is unknown, so they are left out of precision and of fraud caught.
- Label maturity: 140 alerts are newer than the 14-day label delay. They are not counted as false alarms.
- Short history: the data covers 3 calendar month(s). Trends need at least 6.
- Synthetic data: the transactions, labels and unit costs are demo values, not a client's data.
- UA-NBU-S02 (One type of operation makes up an unusual share of a user's activity) is a sign from an NBU draft resolution. It is a draft, not law.
- UA-NBU-S02 (One type of operation makes up an unusual share of a user's activity)
Add a rule for: Payment purposes do not fit the merchant's category code
Reference REC-8078ec06. Quote this reference in your change record.
Add a missing fraud typeConfidence: mediumPriority: 50.0
Rules: none.
Expected effect: no effect estimate for this proposal.
Evidence
- Needs these data fields: mcc, payment_purpose.
- No rule catalogue was given, so we cannot see whether a rule covers it.
- It cannot be covered on this data: the transactions file has no category-code or no payment-purpose column (see UA-NBU51-01 (The acquirer passes payee (merchant) name and category code to the issuer. Check that these fields are filled in your data. Most time-relevant item: the deadline is about 20.11.2026)).
- Source: your decision log and fraud labels, 2026-08-03 to 2026-10-01, labels known by 2026-10-01, label delay 14 days.
Caveats
- Selective labels: 95 alerts were on declined payments. Their outcome is unknown, so they are left out of precision and of fraud caught.
- Label maturity: 140 alerts are newer than the 14-day label delay. They are not counted as false alarms.
- Short history: the data covers 3 calendar month(s). Trends need at least 6.
- Synthetic data: the transactions, labels and unit costs are demo values, not a client's data.
- UA-NBU-S11 (Payment purposes do not fit the merchant's category code) is a sign from an NBU draft resolution. It is a draft, not law.
- UA-NBU-S11 (Payment purposes do not fit the merchant's category code)
- UA-NBU51-01 (The acquirer passes payee (merchant) name and category code to the issuer. Check that these fields are filled in your data. Most time-relevant item: the deadline is about 20.11.2026)
Add a rule for: Regular transfers to a personal account look like business activity
Reference REC-05f946b2. Quote this reference in your change record.
Add a missing fraud typeConfidence: lowPriority: 50.0
Rules: none.
Expected effect: no effect estimate for this proposal.
Evidence
- Needs these data fields: account_type, counterparty_id, ts, amount.
- No rule catalogue was given, so we cannot see whether a rule covers it.
- Source: your decision log and fraud labels, 2026-08-03 to 2026-10-01, labels known by 2026-10-01, label delay 14 days.
Caveats
- Selective labels: 95 alerts were on declined payments. Their outcome is unknown, so they are left out of precision and of fraud caught.
- Label maturity: 140 alerts are newer than the 14-day label delay. They are not counted as false alarms.
- Short history: the data covers 3 calendar month(s). Trends need at least 6.
- Synthetic data: the transactions, labels and unit costs are demo values, not a client's data.
- UA-NBU-S08 (Regular transfers to a personal account look like business activity) is a sign from an NBU draft resolution. It is a draft, not law.
- UA-NBU-S08 (Regular transfers to a personal account look like business activity)
Your imported rules
You gave your rule definitions with a consent line from Demo: synthetic data, no real client, dated 2026-11-12. Engine: demo engine (synthetic). The collector replayed each rule on your transactions inside your environment and compared the replay with your engine's hit log, rule ID by rule ID. The replay never replaces a logged number: every number in the other sections comes from your hit log. The replay keeps and uses only the columns your rule file names, and it never opens the labels file.
Agreement = transactions where both fired, divided by transactions where either fired. A what-if needs an agreement of at least 0.95 and at least 10 hits on either side. Rows in a windowed rule's first window are left out (warm-up): the replay has no history from before your file starts. An empty cell makes a check false and is counted.
Format of the rule file: Download an example rule file (JSON, with a note on every key; replace its example consent line before you use it).
Agreement with your engine's log
| Your rule | Action | Logged hits | Replay hits | Both | Logged only | Replay only | Agreement | Status | Rows left out (warm-up) | Rows with empty cells | Unreadable values |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Single transfer above 9000 tp_u21_large_txn_9k (agreement with your engine's log: 88%) | review | 80 | 90 | 80 | 0 | 10 | 88% | Below the agreement floor | 0 | 0 | 0 |
| Transfer above 500 tp_u21_transfer_over_500 (agreement with your engine's log: 100%) | review | 701 | 701 | 701 | 0 | 0 | 100% | Agrees | 0 | 0 | 0 |
| One sender pays five or more receivers in five days tp_u21_fan_out_5_benef_5d (agreement with your engine's log: 100%) | alert | 25 | 25 | 25 | 0 | 0 | 100% | Agrees | 152 | 0 | 0 |
| More than two wires from one sender in a week tp_u21_wire_count_gt2_7d (agreement with your engine's log: 100%) | review | 22 | 22 | 22 | 0 | 0 | 100% | Agrees | 208 | 81 | 0 |
| Wire total from one sender in a week tp_u21_wire_total_gt4999_7d (agreement with your engine's log: 100%) | decline | 89 | 89 | 89 | 0 | 0 | 100% | Agrees | 208 | 81 | 0 |
| Five or more senders pay one account in five days tp_ffiec_fan_in_5_senders_5d (agreement with your engine's log: 100%) | review | 24 | 24 | 24 | 0 | 0 | 100% | Agrees | 152 | 0 | 0 |
| Device risk score from the vendor R-509 (agreement: not replayable) | review | 0 | — | — | — | — | — | Not replayable | — | — | — |
| One device used by several customers R-610 (agreement: not replayable) | alert | 0 | — | — | — | — | — | Not replayable | — | — | — |
| Very large transfer after a quiet spell R-900 (agreement: cannot be compared) | review | 0 | 45 | 0 | 0 | 45 | — | Cannot be compared | 0 | 0 | 0 |
Differences to check
Up to five of your own transaction IDs per kind, so you can find the translation error. They stay in this internal report.
- Single transfer above 9000 tp_u21_large_txn_9k (agreement with your engine's log: 88%): logged only 0; replay only 10 (for example ctx-00522, ctx-00531, ctx-00668, ctx-00725, ctx-01170). Most differences are at exactly the threshold. Check 'more than' against 'at least'.
Not replayable
- Device risk score from the vendor R-509 (agreement: not replayable): You marked this rule as not expressible: Uses the vendor's device score, which is not in our transaction export. Its logged numbers stay as your engine logged them.
- One device used by several customers R-610 (agreement: not replayable): It needs the column device_id, which your transactions file does not have. Its logged numbers stay as your engine logged them.
What-if on past data
Wire total from one sender in a week tp_u21_wire_total_gt4999_7d (agreement with your engine's log: 100%, measured at your current threshold)
Check #2: the total amount with the same sender id in 7 days is more than 4999, counting only transactions where channel is "wire". Your engine's log has 89 hits for this rule after the warm-up; the replay at your current threshold is the base of every change below.
| Threshold | Hits (this rule) | Caught (this rule) | False alarms (this rule) | Not yet mature (this rule) | Outcome unknown (this rule) | Alerts, all rules | False-alarm cost change vs your current threshold (estimate, USD) |
|---|---|---|---|---|---|---|---|
| your current threshold | 89 | 0 | 0 | 15 | 74 | 662 | — |
| 6000 | 70 (-19) | 0 (0) | 0 (0) | 8 (-7) | 62 (-12) | 662 (0) | 5.60 USD |
| 8000 | 37 (-52) | 0 (0) | 0 (0) | 5 (-10) | 32 (-42) | 660 (-2) | 12.00 USD |
Hits, caught, false alarms, not yet mature and outcome unknown count this rule's own hits. "Alerts, all rules" counts every payment any of your rules alerted on, once: a payment another rule alerts anyway does not change it. The cost change prices the alerted good payments of all rules together, as the false-alarm cost in the rest of this report does: analyst time when the payment was not declined, your friction cost when it was, never both. A rule that declines payments declines its hits.
- 6000: Across all your rules, on your history this threshold would have raised the same number of alerts and caught the same fraud cases. Across all your rules, on your history this threshold would have raised the same number of false alarms, and the false-alarm cost would have been 5.60 USD higher (an estimate from your unit costs). On your history, this rule alone would have raised 19 fewer alerts; your other rules alerted on those payments anyway, so the total does not change. This rule declines the payments it hits, so a changed threshold can raise the false-alarm cost. On your history, 7 good payments that this rule declined would not have been declined at this threshold; your other rules alerted on them anyway, so each would have cost analyst time (4.80 USD) instead of your friction cost (4.00 USD).
- 8000: Across all your rules, on your history this threshold would have raised 2 fewer alerts and caught the same fraud cases. Across all your rules, on your history this threshold would have raised the same number of false alarms, and the false-alarm cost would have been 12.00 USD higher (an estimate from your unit costs). On your history, this rule alone would have raised 52 fewer alerts; your other rules alerted on 50 of those payments anyway. This rule declines the payments it hits, so a changed threshold can raise the false-alarm cost. On your history, 20 good payments that this rule declined would not have been declined at this threshold; your other rules alerted on them anyway, so each would have cost analyst time (4.80 USD) instead of your friction cost (4.00 USD). On your history, 1 good payment that only this rule alerted on would not have been alerted on at this threshold, so it would have cost nothing instead of your friction cost (4.00 USD).
Backtest on past data, not a forecast. A changed threshold can change how fraudsters behave.
A payment that was declined has no outcome. Hits on declined payments are counted as 'outcome unknown', not as caught or as false alarms. A rule that declines payments hides the fraud it stopped, so 'caught' can be lower than the real number.
This collector changes no live rule. A threshold change goes through your own change and approval process.
Transfer above 500 tp_u21_transfer_over_500 (agreement with your engine's log: 100%, measured at your current threshold)
Check #1: amount is more than 500. Your engine's log has 701 hits for this rule after the warm-up; the replay at your current threshold is the base of every change below.
| Threshold | Hits (this rule) | Caught (this rule) | False alarms (this rule) | Not yet mature (this rule) | Outcome unknown (this rule) | Alerts, all rules | False-alarm cost change vs your current threshold (estimate, USD) |
|---|---|---|---|---|---|---|---|
| your current threshold | 701 | 41 | 443 | 138 | 71 | 723 | — |
| 1000 | 653 (-48) | 34 (-7) | 410 (-33) | 131 (-7) | 70 (-1) | 684 (-39) | -158.40 USD |
| 2000 | 537 (-164) | 20 (-21) | 344 (-99) | 104 (-34) | 62 (-9) | 602 (-121) | -456.00 USD |
Hits, caught, false alarms, not yet mature and outcome unknown count this rule's own hits. "Alerts, all rules" counts every payment any of your rules alerted on, once: a payment another rule alerts anyway does not change it. The cost change prices the alerted good payments of all rules together, as the false-alarm cost in the rest of this report does: analyst time when the payment was not declined, your friction cost when it was, never both. A rule that declines payments declines its hits.
- 1000: Across all your rules, on your history this threshold would have raised 39 fewer alerts and caught the same fraud cases. Across all your rules, on your history this threshold would have raised 33 fewer false alarms, and the false-alarm cost would have been 158.40 USD lower (an estimate from your unit costs). On your history, this rule alone would have raised 48 fewer alerts, 33 of them false alarms, 7 of them caught fraud cases; your other rules alerted on 9 of those payments anyway.
- 2000: Across all your rules, on your history this threshold would have raised 121 fewer alerts and caught the same fraud cases. Across all your rules, on your history this threshold would have raised 95 fewer false alarms, and the false-alarm cost would have been 456.00 USD lower (an estimate from your unit costs). On your history, this rule alone would have raised 164 fewer alerts, 99 of them false alarms, 21 of them caught fraud cases; your other rules alerted on 43 of those payments anyway.
Backtest on past data, not a forecast. A changed threshold can change how fraudsters behave.
A payment that was declined has no outcome. Hits on declined payments are counted as 'outcome unknown', not as caught or as false alarms. A rule that declines payments hides the fraud it stopped, so 'caught' can be lower than the real number.
This collector changes no live rule. A threshold change goes through your own change and approval process.
Single transfer above 9000 tp_u21_large_txn_9k (agreement with your engine's log: 88%, measured at your current threshold)
What-if refused for Single transfer above 9000: its replay is below the agreement floor, so a what-if would carry the translation error.
Side by side with WoWyrd rules
Each of your rules next to the WoWyrd rule for the same fraud type and the requirement it relates to. A link found by the rule's name says "please check".
| Your rule | Fraud type and WoWyrd rule | Requirement it relates to |
|---|---|---|
| Single transfer above 9000 tp_u21_large_txn_9k (agreement with your engine's log: 88%) Trigger: amount is at least 9000. Action: review. Logged hits: 80. Replay: Below the agreement floor. | No fraud type in our library matches this rule. Add a typology ID to the rule file. | No requirement in our library relates to this rule. |
| Transfer above 500 tp_u21_transfer_over_500 (agreement with your engine's log: 100%) Trigger: amount is more than 500. Action: review. Logged hits: 701. Replay: Agrees. | No fraud type in our library matches this rule. Add a typology ID to the rule file. | No requirement in our library relates to this rule. |
| One sender pays five or more receivers in five days tp_u21_fan_out_5_benef_5d (agreement with your engine's log: 100%) Trigger: the number of different receiver id values with the same sender id in 5 days is at least 5. Action: alert. Logged hits: 25. Replay: Agrees. | One payer sends to far more payees than similar payers UA-NBU-S15 WoWyrd rule available: One account pays many different payees account_many_payees; numbers: not run on your data | UA-NBU-S15 One payer sends to far more payees than similar payers: NBU draft, Annex 2 sign. Status: draft (NBU draft resolution on control of payment operations, published 2026-07-17; not law); as of 2026-11-11. bank.gov.ua |
| More than two wires from one sender in a week tp_u21_wire_count_gt2_7d (agreement with your engine's log: 100%) Trigger: channel is "wire" and the number of transactions with the same sender id in 7 days is more than 2, counting only transactions where channel is "wire". Action: review. Logged hits: 22. Replay: Agrees. | Many operations of one type in a short time, incoming or outgoing UA-NBU-S03 WoWyrd rule available: Many requests from one account in an hour, then a payout payout_velocity_1h; A burst of operations of one type same_operation_type_burst; numbers: not run on your data | UA-NBU-S03 Many operations of one type in a short time, incoming or outgoing: NBU draft, Annex 2 sign. Status: draft (NBU draft resolution on control of payment operations, published 2026-07-17; not law); as of 2026-11-11. bank.gov.ua |
| Wire total from one sender in a week tp_u21_wire_total_gt4999_7d (agreement with your engine's log: 100%) Trigger: channel is "wire" and the total amount with the same sender id in 7 days is more than 4999, counting only transactions where channel is "wire". Action: decline. Logged hits: 95. Replay: Agrees. | No fraud type in our library matches this rule. Add a typology ID to the rule file. | No requirement in our library relates to this rule. |
| Five or more senders pay one account in five days tp_ffiec_fan_in_5_senders_5d (agreement with your engine's log: 100%) Trigger: the number of different sender id values with the same receiver id in 5 days is at least 5. Action: review. Logged hits: 24. Replay: Agrees. | matched by name: please check Many operations of one type in a short time, incoming or outgoing UA-NBU-S03 WoWyrd rule available: Many requests from one account in an hour, then a payout payout_velocity_1h; A burst of operations of one type same_operation_type_burst; numbers: not run on your data Many payers send to one payee, far more than for similar payees UA-NBU-S16 WoWyrd rule available: Many senders pay one account inbound_sender_fan_in; One payee shared by several customers beneficiary_reuse; numbers: not run on your data Many different senders pay one account within a short time OR-01 WoWyrd rule available: Many senders pay one account inbound_sender_fan_in; numbers: not run on your data | matched by name: please check UA-NBU-S03 Many operations of one type in a short time, incoming or outgoing: NBU draft, Annex 2 sign. Status: draft (NBU draft resolution on control of payment operations, published 2026-07-17; not law); as of 2026-11-11. bank.gov.ua UA-NBU-S16 Many payers send to one payee, far more than for similar payees: NBU draft, Annex 2 sign. Status: draft (NBU draft resolution on control of payment operations, published 2026-07-17; not law); as of 2026-11-11. bank.gov.ua UA-16013-01 Passing cards, account access or payment data to fraudsters becomes a crime. Detect and restrict accounts used as drops. Status: adopted by the Rada 28.10.2026; presidential signature not confirmed; as of 2026-11-11. itd.rada.gov.ua FinCEN advisories and alerts (money-mule red flags). Status: public red-flag references; own wording; as of 2026-11-11. No official page confirmed FFIEC BSA/AML Examination Manual, Appendix F (red flags). Status: public red-flag references; own wording; as of 2026-11-11. No official page confirmed Unit21 recommended rules (2024 public copies). Status: public red-flag references; own wording; as of 2026-11-11. No official page confirmed EU-PSR-04 Monitor both outgoing and incoming payments for fraud. Status: agreed, not adopted (EU Payment Services Regulation; adoption expected by 12.2026); as of 2026-11-11. oeil.europarl.europa.eu UK-PSR-01 Sending and receiving providers share APP scam refunds; a claim has a maximum level. Status: in force since 07.10.2024; as of 2026-11-11. psr.org.uk UK-FCA-01 Monitor incoming as well as outgoing payments, with money-mule signs. Status: consultation; comments close 24.12.2026; as of 2026-11-11. fca.org.uk US-NACHA-02 Receiving banks monitor incoming credits, including new, dormant and mule accounts. Status: in force: phase 1 20.03.2026, phase 2 22.06.2026; as of 2026-11-11. nacha.org |
| Device risk score from the vendor R-509 (agreement: not replayable) Trigger: not expressible in the rule file. Action: review. Logged hits: 0. Replay: Not replayable. | matched by name: please check One device fingerprint is used by unrelated users UA-NBU-S13 WoWyrd rule available: One device used by several customers multi_account_device; numbers: not run on your data | matched by name: please check UA-NBU-S13 One device fingerprint is used by unrelated users: NBU draft, Annex 2 sign. Status: draft (NBU draft resolution on control of payment operations, published 2026-07-17; not law); as of 2026-11-11. bank.gov.ua |
| One device used by several customers R-610 (agreement: not replayable) Trigger: the number of different sender id values with the same device id in 1 day is at least 3. Action: alert. Logged hits: 0. Replay: Not replayable. | One device fingerprint is used by unrelated users UA-NBU-S13 WoWyrd rule available: One device used by several customers multi_account_device; numbers: not run on your data | UA-NBU-S13 One device fingerprint is used by unrelated users: NBU draft, Annex 2 sign. Status: draft (NBU draft resolution on control of payment operations, published 2026-07-17; not law); as of 2026-11-11. bank.gov.ua |
| Very large transfer after a quiet spell R-900 (agreement: cannot be compared) Trigger: amount is more than 20000. Action: review. Logged hits: 0. Replay: Cannot be compared. | A dormant account suddenly moves large amounts OR-04 WoWyrd rule: planned | FFIEC BSA/AML Examination Manual, Appendix F (red flags). Status: public red-flag references; own wording; as of 2026-11-11. No official page confirmed UK-FCA-01 Monitor incoming as well as outgoing payments, with money-mule signs. Status: consultation; comments close 24.12.2026; as of 2026-11-11. fca.org.uk US-FFIEC-AF-06 Large amounts soon after opening; a dormant account suddenly active. Status: guidance; as of 2026-11-11. bsaaml.ffiec.gov US-NACHA-02 Receiving banks monitor incoming credits, including new, dormant and mule accounts. Status: in force: phase 1 20.03.2026, phase 2 22.06.2026; as of 2026-11-11. nacha.org |
Not legal advice. This report maps your rules to requirements, in our own short wording. It is not legal advice. You decide how you comply.
Threshold search on your history
For each number threshold of your rules that agree with your engine's log, the collector tried 16 values from half to double the current one. It picked the best value on the earlier 70% of the payments with a known outcome and kept it only if it also did better on the later 30%, which the pick never saw. Target: the most confirmed fraud caught by all your rules, with no more alerts than today.
Backtest on past data, not a forecast. Nothing changes until you approve it in your own engine. A changed threshold can change how fraudsters behave.
| Rule | Check | Current | Proposed | Why |
|---|---|---|---|---|
| REC-d74333d3 tp_u21_transfer_over_500 | amount is more than 500 | 500 | 1000 | Threshold 500 → 1000: catches the same confirmed fraud with 15 fewer alerts, precision 7% → 6%, false-alarm cost 43.20 USD lower on the later 30% of your history. |
tp_u21_transfer_over_500, check 1: amount is more than 500
Earlier part: before 2026-09-03; later part: from that date.
On the later 30%:
| Threshold | Alerts (all rules) | Rule hits | Caught | False alarms | Precision | False-alarm cost (USD) |
|---|---|---|---|---|---|---|
| current 500 | 310 | 301 | 9 | 128 | 6.6% | 688.00 USD |
| best on earlier data 1000 | 295 | 283 | 7 | 119 | 5.6% | 644.80 USD |
Every value tried, on the earlier 70%:
| Threshold | Alerts (all rules) | Rule hits | Caught | False alarms | Precision | False-alarm cost (USD) |
|---|---|---|---|---|---|---|
| 250 | 732 | 728 | 42 | 626 | 6.3% | 3,113.60 USD |
| 300 | 670 | 665 | 41 | 565 | 6.8% | 2,820.80 USD |
| 350 | 607 | 600 | 39 | 504 | 7.2% | 2,532.80 USD |
| 400 | 537 | 527 | 35 | 436 | 7.4% | 2,211.20 USD |
| 425 | 511 | 500 | 33 | 412 | 7.4% | 2,096.00 USD |
| 450 | 478 | 466 | 33 | 378 | 8.0% | 1,937.60 USD |
| 475 | 447 | 434 | 32 | 347 | 8.4% | 1,793.60 USD |
| 500 (current) | 413 | 400 | 32 | 315 | 9.2% | 1,640.00 USD |
| 525 | 411 | 397 | 31 | 313 | 9.0% | 1,630.40 USD |
| 550 | 409 | 395 | 31 | 311 | 9.1% | 1,620.80 USD |
| 575 | 408 | 394 | 31 | 310 | 9.1% | 1,616.00 USD |
| 600 | 408 | 394 | 31 | 310 | 9.1% | 1,616.00 USD |
| 650 | 404 | 390 | 31 | 306 | 9.2% | 1,596.80 USD |
| 750 | 404 | 389 | 30 | 306 | 8.9% | 1,596.80 USD |
| 875 | 397 | 381 | 30 | 299 | 9.1% | 1,563.20 USD |
| 1000 | 389 | 370 | 27 | 291 | 8.5% | 1,524.80 USD |
Not searched
- tp_u21_large_txn_9k: not searched: the replay does not match your engine's hits (below the agreement floor).
- tp_u21_fan_out_5_benef_5d: not searched: an AML rule has no labels below its threshold, so a lower threshold would look free. Use the below-the-line sample for this rule.
- tp_u21_wire_count_gt2_7d: not searched: an AML rule has no labels below its threshold, so a lower threshold would look free. Use the below-the-line sample for this rule.
- tp_u21_wire_total_gt4999_7d, check 1: not searched: only number thresholds compared with more than or less than can be searched.
- tp_u21_wire_total_gt4999_7d, check 2: not searched: too little labelled history (0 hits with a known outcome in the search part, 0 in the check part; at least 30 and 10 are needed).
- tp_ffiec_fan_in_5_senders_5d: not searched: an AML rule has no labels below its threshold, so a lower threshold would look free. Use the below-the-line sample for this rule.
- R-509: not searched: the rule cannot be replayed on your files.
- R-610: not searched: the rule cannot be replayed on your files.
- R-900: not searched: the rule's ID is not in your hit log, so its replay cannot be checked.
Below-the-line sample (AML rules)
AML rules have no labels below the threshold: nobody reviews a payment the rule did not flag. For each number threshold of a rule marked aml, the collector picked up to 20 payments in each of 3 bands just under the line (each band 10% of the threshold). Your analysts review them in btl-sample.csv: write suspicious or not_suspicious in analyst_label. The next run reads the file with --btl-labels and shows the share marked suspicious. The file holds transaction IDs: it stays inside your environment and is never in the summary. The order is fixed by your token key and the seed "wowyrd-btl", so a re-run gives the same list. Nothing here changes a score.
| Rule | Check | Band | Payments in band | Sampled | Suspicious | Suspicious share |
|---|---|---|---|---|---|---|
| tp_u21_fan_out_5_benef_5d | the number of different receiver id values with the same sender id in 5 days is at least 5 | from 4 below 5 | 41 | 20 | 0 of 17 | 0.0% |
| tp_u21_fan_out_5_benef_5d | the number of different receiver id values with the same sender id in 5 days is at least 5 | from 3 below 4 | 141 | 20 | 2 of 20 | 10.0% |
| tp_u21_fan_out_5_benef_5d | the number of different receiver id values with the same sender id in 5 days is at least 5 | from 2 below 3 | 537 | 20 | 0 of 20 | 0.0% |
| tp_u21_wire_count_gt2_7d | the number of transactions with the same sender id in 7 days is more than 2, counting only transactions where channel is "wire" | more than 1 up to 2 | 56 | 20 | — | — |
| tp_ffiec_fan_in_5_senders_5d | the number of different sender id values with the same receiver id in 5 days is at least 5 | from 4 below 5 | 15 | 15 | 1 of 6 | 16.7% |
| tp_ffiec_fan_in_5_senders_5d | the number of different sender id values with the same receiver id in 5 days is at least 5 | from 3 below 4 | 96 | 20 | 4 of 20 | 20.0% |
| tp_ffiec_fan_in_5_senders_5d | the number of different sender id values with the same receiver id in 5 days is at least 5 | from 2 below 3 | 451 | 20 | 1 of 20 | 5.0% |
- tp_u21_wire_count_gt2_7d, check 2: band 2 would start at 0 or below.
Of the reviewed payments just under this threshold, this share was marked suspicious by your analysts. A high share means the threshold may be too high. It is not projected to all payments.
Not sampled
- tp_u21_wire_count_gt2_7d, check 1: not sampled: only number thresholds compared with more than or less than have a line to go under.
Coverage by fraud type
Fraud types present in your labels
| Fraud type | Labelled cases | Caught | Missed | Library types it matches | Your rules' coverage | WoWyrd |
|---|---|---|---|---|---|---|
| One out to many (fan-out) | 13 | 13 | 0 | One payer sends to far more payees than similar payers (UA-NBU-S15), One account pays many new beneficiaries (OR-07) | Covered | WoWyrd rule available: One account pays many different payees account_many_payees |
| Many into one (fan-in) | 12 | 11 | 1 | Many payers send to one payee, far more than for similar payees (UA-NBU-S16), Many different senders pay one account within a short time (OR-01) | Covered | WoWyrd rule available: Many senders pay one account inbound_sender_fan_in; One payee shared by several customers beneficiary_reuse |
| Type not given | 78 | 58 | 20 | none | Unknown | WoWyrd rule: planned |
Not in this dataset, so it cannot be measured here
Your labels hold no case of these types, so this data cannot show whether your rules catch them. This is not a gap in your rules. The rules column lists catalogue rules whose names match the type.
| Library type | ID | Your rules whose names match | Sources | WoWyrd |
|---|---|---|---|---|
| One user holds many more accounts or cards than similar users | UA-NBU-S01 | none | UA-NBU-S01 | WoWyrd rule: planned |
| One type of operation makes up an unusual share of a user's activity | UA-NBU-S02 | none | UA-NBU-S02 | WoWyrd rule available: One type of operation dominates an account operation_type_concentration |
| Many operations of one type in a short time, incoming or outgoing | UA-NBU-S03 | none | UA-NBU-S03 | WoWyrd rule available: Many requests from one account in an hour, then a payout payout_velocity_1h; A burst of operations of one type same_operation_type_burst |
| Balance stays low although large amounts pass through | UA-NBU-S04 | none | UA-NBU-S04 | WoWyrd rule available: Large amounts pass through an account that keeps a low balance low_balance_high_turnover |
| Large loans are taken and repaid again and again | UA-NBU-S05 | none | UA-NBU-S05 | WoWyrd rule available: Large loans taken and repaid again and again loan_issue_repayment_churn |
| Money comes in and leaves fast, little balance stays | UA-NBU-S06 | none | UA-NBU-S06 | WoWyrd rule available: Money leaves soon after it arrives rapid_pass_through |
| Gambling winnings arrive with no deposits, then card transfers follow | UA-NBU-S07 | none | UA-NBU-S07 | WoWyrd rule available: Gambling winnings with no gambling deposits gambling_winnings_without_deposits |
| Regular transfers to a personal account look like business activity | UA-NBU-S08 | none | UA-NBU-S08 | WoWyrd rule available: A personal account paid by many payers personal_account_business_activity |
| A new merchant's volume is far above similar merchants, then the merchant closes | UA-NBU-S09 | none | UA-NBU-S09 | WoWyrd rule available: A merchant with little history suddenly busy low_history_merchant_spike |
| Merchant account inflows mirror its outflows (transit) | UA-NBU-S10 | none | UA-NBU-S10 | WoWyrd rule available: A business account pays out almost all it receives business_account_transit |
| Payment purposes do not fit the merchant's category code | UA-NBU-S11 | none | UA-NBU-S11 | WoWyrd rule available: Payment purpose does not fit the merchant's business payment_purpose_atypical_for_mcc; Card payment to a person without category or purpose acquiring_to_person_missing_data |
| Contacts, device, apps, IMEI, operating system or network address change often | UA-NBU-S12 | none | UA-NBU-S12 | WoWyrd rule available: Payout right after an email or phone change contact_change_to_payout; One session from several network addresses session_multiple_ips; One account used from many devices account_many_devices |
| One device fingerprint is used by unrelated users | UA-NBU-S13 | none | UA-NBU-S13 | WoWyrd rule available: One device used by several customers multi_account_device |
| Two operations are too far apart to travel between in the time | UA-NBU-S14 | none | UA-NBU-S14 | WoWyrd rule available: Locations too far apart for the time between them impossible_travel |
| Most of an incoming amount leaves again within hours | OR-02 | none | FINCEN, EUROPOL-EMMA, FIU-UA | WoWyrd rule available: Money leaves soon after it arrives rapid_pass_through |
| A new account gets large or many credits soon after opening | OR-03 | none | UNIT21, UK-FINANCE | WoWyrd rule available: A new account receives a large credit new_account_large_incoming_credit; A new account paid by several people new_account_inbound_fan_in |
| A dormant account suddenly moves large amounts | OR-04 | none | FFIEC-APPF | WoWyrd rule: planned |
| Repeated amounts just under a reporting or limit threshold | OR-05 | none | FFIEC-APPF | WoWyrd rule available: Payments just under a limit just_below_threshold_payment; Many small payments from one account add up to a large sum amount_structuring_account; Many small payments to one payee add up to a large sum amount_structuring_beneficiary |
| Repeated round amounts with no business reason | OR-06 | none | FFIEC-APPF | WoWyrd rule available: Several payments in large round amounts repeated_round_amounts |
| Outgoing payments per hour above the customer's usual level | OR-08 | none | UNIT21, EBA-GL-2021-02 | WoWyrd rule available: More payments in an hour than the customer usually makes high_customer_velocity_1h; Many requests from one account in an hour, then a payout payout_velocity_1h |
| A large first payment to a payee never paid before | OR-09 | none | UK-FINANCE, UK-PSR | WoWyrd rule available: A large payment to a payee added just before new_payee_high_amount |
| Many senders' accounts share one beneficiary | OR-10 | none | WOWYRD-SPIKE, FINCEN | WoWyrd rule available: One payee shared by several customers beneficiary_reuse |
| Money goes round in a circle between accounts within days | OR-11 | none | FATF-REF, FIU-UA | WoWyrd rule: planned |
| An amount far above the customer's own history | OR-12 | none | EBA-GL-2021-02, NBU-DRAFT | WoWyrd rule available: A payment far above the customer's usual amount amount_far_above_customer_average |
| Many small credits, then one large debit | OR-13 | none | EUROPOL-EMMA, FIU-UA | WoWyrd rule available: Many small credits, then one large payout many_small_credits_then_large_payout |
| Credits from many different banks into one account | OR-14 | none | FIU-UA, FINCEN | WoWyrd rule available: Credits from many different banks into one account inbound_from_many_banks |
| Transfers to or from high-risk jurisdictions | OR-15 | none | FATF-REF, FFIEC-APPF | WoWyrd rule available: A payment to or from a high-risk country high_risk_jurisdiction_counterparty |
| Cash withdrawn right after a credit arrives | OR-16 | none | FINCEN, FIU-UA | WoWyrd rule available: Cash taken out soon after money arrives cash_withdrawal_after_credit |
| The payee name does not match and the payment still goes | OR-17 | none | UK-PSR, EU-PSR | WoWyrd rule available: Paid although the payee name did not match payee_name_mismatch_paid |
| The payment reference mentions investment, crypto or a 'safe account' | OR-18 | none | UK-FINANCE | WoWyrd rule available: The payment reference uses common scam wording payment_reference_scam_wording |
| An older customer suddenly sends a large amount to a new payee | OR-19 | none | FINCEN | WoWyrd rule available: An older customer sends a large amount to a new payee older_customer_new_payee_large |
| Activity at hours unusual for the customer | OR-20 | none | NBU-DRAFT | WoWyrd rule available: Activity at an hour unusual for the customer unusual_hour_for_customer |
New in our reference set since your last report
Our reference set: version v0.7.0, published 2026-11-18 (built into this collector).
Our reference set has not changed since your last report (version v0.7.0).
Incoming payments and money-mule signs
Incoming payments not measured: your transactions file has no direction column (for example direction, with values such as in and out).
Requirements changed since the last review
Your rules are mapped to current requirements. We flag when a new requirement affects them. We follow official publications of regulators in Ukraine, the EU, the UK and the US.
Not legal advice. This report maps your rules to requirements, in our own short wording. It is not legal advice. You decide how you comply.
Requirements library read on 2026-11-11. No change file was given for this run, so no changed requirement is listed here.
Requirements still open since the previous report
Your rules may not cover requirement UA-NBU-01, “Review every rule at least monthly by back-testing it”
Review every rule at least monthly by back-testing it. (NBU draft §16(10), §65-66, status as of 2026-11-11). The previous report (data up to 2026-08-27) found it too (Gap), and it is still open. Now: Partly met.
The data covers 3 calendar month(s), 2026-08-03 to 2026-10-01. A monthly back-test needs one run per month; a trend needs 6 months. This run splits the data by month into 3 periods.
We found no implementation guidance from the regulator for this requirement.
Public cases where a regulator named the same kind of gap (monitoring rules, scenarios or thresholds not reviewed, tested or tuned):
- NBU, 2026: a public case on this kind of gap.
- NYDFS, 2025: a public case on this kind of gap.
- FCA, 2024: a public case on this kind of gap.
Your rules may not cover requirement UA-NBU-02, “Metric 1: compare how often each rule fired with how many payments were actually stopped”
Metric 1: compare how often each rule fired with how many payments were actually stopped. (NBU draft §60(1), status as of 2026-11-11). The previous report (data up to 2026-08-27) found it too (Partly met), and it is still open. Now: Partly met.
Measured: 22.3%; no written threshold to compare against (§62). 6 rules fired 947 times; 211 of those hits were on stopped payments (22.3%). The per-rule table is in the technical report.
We found no implementation guidance from the regulator for this requirement.
Public cases where a regulator named the same kind of gap (transaction monitoring not effective or not calibrated to the firm's risks):
- NBU, 2026: a public case on this kind of gap.
- NBU, 2026: a public case on this kind of gap.
- NYDFS, 2025: a public case on this kind of gap.
Your rules may not cover requirement UA-NBU-03, “Metric 2: share of fraud value that was prevented, out of all fraud value”
Metric 2: share of fraud value that was prevented, out of all fraud value. (NBU draft §60(2), status as of 2026-11-11). The previous report (data up to 2026-08-27) found it too (Partly met), and it is still open. Now: Partly met.
Measured: 37.2%; no written threshold to compare against (§62). Fraud value stopped: 37.2% (239,475.00 of 643,715.00 USD). Fraud value with any alert: 99.2% (638,657.25 USD). Whether reviewed alerts were stopped is not in the decision log.
We found no implementation guidance from the regulator for this requirement.
Public cases where a regulator named the same kind of gap (transaction monitoring not effective or not calibrated to the firm's risks):
- NBU, 2026: a public case on this kind of gap.
- NBU, 2026: a public case on this kind of gap.
- NYDFS, 2025: a public case on this kind of gap.
Your rules may not cover requirement UA-NBU-04, “Set rule-effectiveness thresholds in writing. A rule under its threshold gets an immediate review”
Set rule-effectiveness thresholds in writing. A rule under its threshold gets an immediate review. (NBU draft §62-64, status as of 2026-11-11). The previous report (data up to 2026-08-27) found it too (Gap), and it is still open. Now: Gap.
No written effectiveness threshold was given. As a pointer: 0 of 2 rules with at least 30 alerts with a known outcome catch fraud at or under the base fraud rate of 4.7%.
We found no implementation guidance from the regulator for this requirement.
Public cases where a regulator named the same kind of gap (monitoring rules, scenarios or thresholds not reviewed, tested or tuned; transaction monitoring not effective or not calibrated to the firm's risks):
- NBU, 2026: a public case on this kind of gap.
- NBU, 2026: a public case on this kind of gap.
- NYDFS, 2025: a public case on this kind of gap.
Your rules may not cover requirement UA-NBU-05, “Record a review decision for every rule: keep, adjust or cancel”
Record a review decision for every rule: keep, adjust or cancel. (NBU draft §68-69, status as of 2026-11-11). The previous report (data up to 2026-08-27) found it too (Partly met), and it is still open. Now: Partly met.
No recorded decisions in your files. Recommended decisions: cancel 0, adjust 0, keep 2. 4 rules have fewer than 30 alerts with a known outcome, so no decision is recommended. The decision itself is yours to record.
We found no implementation guidance from the regulator for this requirement.
Public cases where a regulator named the same kind of gap (monitoring rules, scenarios or thresholds not reviewed, tested or tuned):
- NBU, 2026: a public case on this kind of gap.
- NYDFS, 2025: a public case on this kind of gap.
- FCA, 2024: a public case on this kind of gap.
Your rules may not cover requirement UA-NBU-09, “Reconcile payment operations every day”
Reconcile payment operations every day. (NBU draft §16, §18, §2, status as of 2026-11-11). The previous report (data up to 2026-08-27) found it too (Partly met), and it is still open. Now: Partly met.
Days reviewed: 60 (2026-08-03 to 2026-10-01; UTC days). Days with no payments: 0. Payments with no row in the rule-hit log: 0. Rule-hit rows with no payment: 0. No ledger was given, so amounts were not reconciled. Give your settlement or ledger export with --ledger to reconcile them. This checks your exports. Whether your staff reconcile every day is a process question: answer it in the assessment checklist.
We found no implementation guidance from the regulator for this requirement.
We found no public enforcement case for this kind of gap.
Your rules may not cover requirement UA-NBU-11, “Have evidence ready for a supervisor: documents, rules, risk markers”
Have evidence ready for a supervisor: documents, rules, risk markers. (NBU draft §19, status as of 2026-11-11). The previous report (data up to 2026-08-27) found it too (Partly met), and it is still open. Now: Partly met.
This report maps each finding to a requirement ID and its status (20 requirements). Your written rules, risk markers and staff interviews stay with you.
We found no implementation guidance from the regulator for this requirement.
Public cases where a regulator named the same kind of gap (monitoring rules, scenarios or thresholds not reviewed, tested or tuned; transaction monitoring not effective or not calibrated to the firm's risks):
- NBU, 2026: a public case on this kind of gap.
- NBU, 2026: a public case on this kind of gap.
- NYDFS, 2025: a public case on this kind of gap.
Your rules may not cover requirement US-NYDFS-01, “Analyse scenarios, rules and thresholds on an ongoing basis”
Analyse scenarios, rules and thresholds on an ongoing basis. (3 NYCRR 504.3, status as of 2026-11-11). The previous report (data up to 2026-08-27) found it too (Partly met), and it is still open. Now: Partly met.
This run analyses 6 rules over 3 periods by month (3 calendar month(s)). Ongoing analysis means running it every month.
How the regulator describes doing this:
- Final transaction monitoring and filtering program regulation (3 NYCRR 504), press release (NYDFS, 2016-06-30), dfs.ny.gov.
Public cases where a regulator named the same kind of gap (monitoring rules, scenarios or thresholds not reviewed, tested or tuned):
- NYDFS, 2025: a public case on this kind of gap.
- NBU, 2026: a public case on this kind of gap.
- FCA, 2024: a public case on this kind of gap.
Resolved since the previous report
- Requirement UK-FCA-02, “Test and review rules; a poorly calibrated rule set the firm cannot explain is poor practice”: No gap found in your data in this report (was Partly met in the report with data up to 2026-08-27).
- Requirement US-FFIEC-01, “Tune and validate monitoring; justify each threshold”: No gap found in your data in this report (was Partly met in the report with data up to 2026-08-27).
Not legal advice. Your compliance team decides how a requirement applies to you. Public cases show how regulators have treated the same kind of gap; they do not predict an outcome for you.
Regulatory checks by regulator
Not legal advice. This report maps your rules to requirements, in our own short wording. It is not legal advice. You decide how you comply.
Regulatory statuses as of 2026-11-11 (library date). Check them again before you rely on this report.
Checks are grouped by the regulator that sets the requirement. Each row gives the requirement ID, our own short wording of it, the result on your files, and the source's status on the date shown. Our wording is a summary, not the regulator's text.
Checks per NBU requirements
Rows citing the NBU draft of 17.07.2026 — draft, not in force: its text can change before adoption. Resolutions 51, 54 and 116 are in force. Bill 16013 is a Ukrainian bill, shown here with its status.
| Requirement ID | NBU draft paragraph | Our wording | Result | Finding | Evidence | Source status | As of | Official source |
|---|---|---|---|---|---|---|---|---|
| UA-NBU51-01 | — | The acquirer passes payee (merchant) name and category code to the issuer. Check that these fields are filled in your data. Most time-relevant item: the deadline is about 20.11.2026. | Not measurable from these files | Your transactions file has no payee-name or category-code column. Add them to the export so this can be measured. Without them no rule can check payment purpose against category code. | Transactions file: 1,950 rows; columns checked: payee_name, merchant_name, recipient_name, receiver_name, beneficiary_name, mcc, merchant_category_code. | in force since 20.05.2026; 6 months to comply (about 20.11.2026) | 2026-11-11 | bank.gov.ua |
| UA-NBU-01 | §66 | Review every rule at least monthly by back-testing it. (NBU draft §66; library item UA-NBU-01.) | Partly met | The data covers 3 calendar month(s), 2026-08-03 to 2026-10-01. A monthly back-test needs one run per month; a trend needs 6 months. This run splits the data by month into 3 periods. | Source: your decision log and fraud labels, 2026-08-03 to 2026-10-01, labels known by 2026-10-01, label delay 14 days. | draft (NBU draft resolution on control of payment operations, published 2026-07-17; not law) | 2026-11-11 | bank.gov.ua |
| UA-NBU-02 | §60 | Metric 1: compare how often each rule fired with how many payments were actually stopped. (NBU draft §60(1); library item UA-NBU-02.) | Partly met | Measured: 22.3%; no written threshold to compare against (§62). 6 rules fired 947 times; 211 of those hits were on stopped payments (22.3%). The per-rule table is in the technical report. | Source: your decision log and fraud labels, 2026-08-03 to 2026-10-01, labels known by 2026-10-01, label delay 14 days. First rules: tp_ffiec_fan_in_5_senders_5d 24 hits, 0 stopped; tp_u21_fan_out_5_benef_5d 25 hits, 1 stopped; tp_u21_large_txn_9k 80 hits, 12 stopped; tp_u21_transfer_over_500 701 hits, 85 stopped; tp_u21_wire_count_gt2_7d 22 hits, 18 stopped; tp_u21_wire_total_gt4999_7d 95 hits, 95 stopped. | draft (NBU draft resolution on control of payment operations, published 2026-07-17; not law) | 2026-11-11 | bank.gov.ua |
| UA-NBU-03 | §60 | Metric 2: share of fraud value that was prevented, out of all fraud value. (NBU draft §60(2); library item UA-NBU-03.) | Partly met | Measured: 37.2%; no written threshold to compare against (§62). Fraud value stopped: 37.2% (239,475.00 of 643,715.00 USD). Fraud value with any alert: 99.2% (638,657.25 USD). Whether reviewed alerts were stopped is not in the decision log. | Source: your decision log and fraud labels, 2026-08-03 to 2026-10-01, labels known by 2026-10-01, label delay 14 days. 103 payments labelled fraud. | draft (NBU draft resolution on control of payment operations, published 2026-07-17; not law) | 2026-11-11 | bank.gov.ua |
| UA-NBU-04 | §62 | Set rule-effectiveness thresholds in writing. A rule under its threshold gets an immediate review. (NBU draft §62; library item UA-NBU-04.) | Gap | No written effectiveness threshold was given. As a pointer: 0 of 2 rules with at least 30 alerts with a known outcome catch fraud at or under the base fraud rate of 4.7%. | Source: your decision log and fraud labels, 2026-08-03 to 2026-10-01, labels known by 2026-10-01, label delay 14 days. | draft (NBU draft resolution on control of payment operations, published 2026-07-17; not law) | 2026-11-11 | bank.gov.ua |
| UA-NBU-05 | §68–69 | Record a review decision for every rule: keep, adjust or cancel. (NBU draft §68–69; library item UA-NBU-05.) | Partly met | No recorded decisions in your files. Recommended decisions: cancel 0, adjust 0, keep 2. 4 rules have fewer than 30 alerts with a known outcome, so no decision is recommended. The decision itself is yours to record. | Source: your decision log and fraud labels, 2026-08-03 to 2026-10-01, labels known by 2026-10-01, label delay 14 days. | draft (NBU draft resolution on control of payment operations, published 2026-07-17; not law) | 2026-11-11 | bank.gov.ua |
| UA-NBU-11 | — | Have evidence ready for a supervisor: documents, rules, risk markers. | Partly met | This report maps each finding to a requirement ID and its status (20 requirements). Your written rules, risk markers and staff interviews stay with you. | Source: your decision log and fraud labels, 2026-08-03 to 2026-10-01, labels known by 2026-10-01, label delay 14 days. | draft (NBU draft resolution on control of payment operations, published 2026-07-17; not law) | 2026-11-11 | bank.gov.ua |
| UA-NBU51-02 | — | Show the acquirer's full name before an online payment; receipts show payee, commission, purpose and category code. | Not measurable from these files | Your transactions file has no receipt column for the 1,220 acquiring payments (a card channel). Add the category code, payee name, payment purpose and commission to the export so this can be measured. This checks the data in your export, not your payment page or your receipts. Showing the acquirer's full name before an online payment stays a checklist question. | Transactions file; columns looked for: mcc, merchant_category_code, payee_name, merchant_name, recipient_name, receiver_name, beneficiary_name, payment_purpose, purpose, purpose_category, payment_reference, reference, narrative, commission, commission_amount, fee, fee_amount. | in force since 20.05.2026; 6 months to comply (about 20.11.2026) | 2026-11-11 | bank.gov.ua |
| UA-NBU-09 | — | Reconcile payment operations every day. | Partly met | Days reviewed: 60 (2026-08-03 to 2026-10-01; UTC days). Days with no payments: 0. Payments with no row in the rule-hit log: 0. Rule-hit rows with no payment: 0. No ledger was given, so amounts were not reconciled. Give your settlement or ledger export with --ledger to reconcile them. This checks your exports. Whether your staff reconcile every day is a process question: answer it in the assessment checklist. | Transactions file and rule-hit log, by UTC day. | draft (NBU draft resolution on control of payment operations, published 2026-07-17; not law) | 2026-11-11 | bank.gov.ua |
| UA-NBU51-03 | — | Acquiring for natural persons only for allowed categories, with the required data passed to the issuer. | Not measurable from these files | Checklist question. The files do not show this. | Answer in the assessment checklist. | in force since 20.05.2026; 6 months to comply (about 20.11.2026) | 2026-11-11 | bank.gov.ua |
| UA-NBU54-01 | — | Non-bank providers that acquire payments need an authorisation; the sanctions pause has ended. | Not measurable from these files | Checklist question. The files do not show this. | Answer in the assessment checklist. | in force; grace period ended 02.09.2026 | 2026-11-11 | bank.gov.ua |
| UA-NBU116-01 | — | Remote verification of natural persons uses the updated procedure with credit-bureau data. | Not measurable from these files | Checklist question. The files do not show this. | Answer in the assessment checklist. | in force since 10.10.2026; 3 months to align internal procedures (about 10.01.2027) | 2026-11-11 | bank.gov.ua |
| UA-NBU116-02 | — | Reports to the State Financial Monitoring Service go only through the e-cabinet, with numbered registers. | Not measurable from these files | Checklist question. The files do not show this. | Answer in the assessment checklist. | in force since 10.10.2026; 3 months to align internal procedures (about 10.01.2027) | 2026-11-11 | bank.gov.ua |
| UA-16013-01 | — | Passing cards, account access or payment data to fraudsters becomes a crime. Detect and restrict accounts used as drops. | Not measurable from these files | Checklist question. The files do not show this. | Answer in the assessment checklist. | adopted by the Rada 28.10.2026; presidential signature not confirmed | 2026-11-11 | itd.rada.gov.ua |
Checks per EU requirements
| Requirement ID | Our wording | Result | Finding | Evidence | Source status | As of | Official source |
|---|---|---|---|---|---|---|---|
| EU-PSR-05 | When a payee name check fails because of the payee's provider, that provider compensates the payer's provider. Keep evidence of the name check on received transfers. | Not measurable from these files | Your transactions file has no payee name-check result. Add it to measure received transfers with a name mismatch or no check. | Columns looked for: name_check_result, payee_check_result, cop_result, vop_result, name_check. | agreed, not adopted (EU Payment Services Regulation; adoption expected by 12.2026) | 2026-11-11 | oeil.europarl.europa.eu |
Checks per UK requirements
| Requirement ID | Our wording | Result | Finding | Evidence | Source status | As of | Official source |
|---|---|---|---|---|---|---|---|
| UK-PSR-03 | A claim may carry an excess of up to 100 GBP, never for vulnerable customers. | Not measurable from these files | Checklist question. The files do not show this. | Answer in the assessment checklist. | in force | 2026-11-11 | psr.org.uk |
| UK-FCA-02 | Test and review rules; a poorly calibrated rule set the firm cannot explain is poor practice. | No gap found in your data | 0 rules or thresholds look poorly calibrated on your data (none); 0 catalogue rules never fired. Every proposal gives a plain-word reason. | Source: your decision log and fraud labels, 2026-08-03 to 2026-10-01, labels known by 2026-10-01, label delay 14 days. | guidance in force | 2026-11-11 | fca.org.uk |
Checks per US requirements (FinCEN, NYDFS, FFIEC, Nacha, Federal Reserve)
FinCEN and Nacha items are not checked from your files in this run; the rows below are NYDFS and FFIEC items.
| Requirement ID | Our wording | Result | Finding | Evidence | Source status | As of | Official source |
|---|---|---|---|---|---|---|---|
| US-NYDFS-01 | Analyse scenarios, rules and thresholds on an ongoing basis. | Partly met | This run analyses 6 rules over 3 periods by month (3 calendar month(s)). Ongoing analysis means running it every month. | Source: your decision log and fraud labels, 2026-08-03 to 2026-10-01, labels known by 2026-10-01, label delay 14 days. | in force | 2026-11-11 | dfs.ny.gov |
| US-NYDFS-02 | Test rules before and after they go live. | Not measurable from these files | The files do not show when each rule went live, so before-and-after tests cannot be checked. A replay of a new rule on past data can serve as the test before go-live. | Rule catalogue has no go-live date. | in force | 2026-11-11 | dfs.ny.gov |
| US-FFIEC-01 | Tune and validate monitoring; justify each threshold. | No gap found in your data | 0 tuning proposals (none). Each states its evidence on past data. | Source: your decision log and fraud labels, 2026-08-03 to 2026-10-01, labels known by 2026-10-01, label delay 14 days. | guidance | 2026-11-11 | bsaaml.ffiec.gov |
Other checks
No other checks in this run.
Run record
The files, code and parameters of this run, so it can be repeated.
| Item | Value | Source |
|---|---|---|
| Input file: costs | c16cdd870af20772bbd68af79c07cc73a2f8d124f64c743e1132667ff4b97882 | SHA-256 |
| Input file: decisions | 1110cb52381a513a562c9fd1b7c37f694cd1810c754b781153d021a094a03382 | SHA-256 |
| Input file: labels | eded2ebdcc0f80849fefb75cad06bdf5be9afece13ff5772c58f890ec5ac7af6 | SHA-256 |
| Input file: transactions | 6a96e6d94f0d00cfd5a51ab22df5abb094b6c661bfae66dcc63f873af1d3212e | SHA-256 |
| Collector version | v0.7.0-11-gba7bd7b03d5199dd42d1810a9e4294cde08c03a4 | git commit |
| Generation time | 2026-11-20 10:02 UTC | run time |
| cutoff | 2026-10-01T22:21:00+00:00 | client |
| label delay days | 14 | client |
| min cell | 10 | default |
| min join coverage | 0.95 | default |
| period | month | client |
| threshold | not given | not given |
| tokens for all rules | False | default |
| Regulatory library | 2026-11-11 | statuses as of this date |
| Requirement change file | not given | --changes |
| Client rule file | 296c2bb6ecd9cc43c25d19cde0fce613a62000ec94d590eaf74e396b506d1c80 | SHA-256; 9 rules |
| Consent line | Demo: synthetic data, no real client, 2026-11-12: Demo file. A made-up client allows the collector to replay these made-up rule definitions on made-up payments. | the rule file |
| Agreement floor | 0.95 | default |
| What-if values tried | tp_u21_wire_total_gt4999_7d: 6000, 8000; tp_u21_transfer_over_500: 1000, 2000; tp_u21_large_txn_9k: 10000 | --what-if |
| Tool map | not given | --tools or a tool column in the catalogue |
| Ledger export | not given | --ledger |
| Reference set source | built into this collector | |
| Reference set version | v0.7.0 | published 2026-11-18 |
| Reference set file | e96e792983d3748de554ba3b49a15291c8d4110f1778795555caff0fa81ee119 | SHA-256 |
| Previous report state | 5093c9ecc6a39e5c50581f18cd95d93d5f379e1055c8bd7ea0a6d73381c82163 | SHA-256; --previous |
| Below-the-line seed | wowyrd-btl | default |
| Below-the-line payments per band | 20 | default |
| Threshold search target | alerts-cap | --search-target |
| Threshold search grid (x current) | 0.50, 0.60, 0.70, 0.80, 0.85, 0.90, 0.95, 1.00, 1.05, 1.10, 1.15, 1.20, 1.30, 1.50, 1.75, 2.00 | earlier 70% search, later part check |
| Labelled below-the-line sample | 47a4bb93725e00723784fdf985cdfd2067dbac544617e6b5023d9cce3e108f3e | SHA-256; 103 rows; --btl-labels |
Rule glossary
Each rule in this review, in plain words: what it detects, the fraud type it guards against, when it fires, where it comes from, and the data it needs.
tp_ffiec_fan_in_5_senders_5d Rule ID: tp_ffiec_fan_in_5_senders_5d
- What it detects
- No description supplied by the client
- Fraud type it guards against
- Not stated
- When it fires
- Not stated
- Source
- client
- Data fields it needs
- Unknown
tp_u21_fan_out_5_benef_5d Rule ID: tp_u21_fan_out_5_benef_5d
- What it detects
- No description supplied by the client
- Fraud type it guards against
- Not stated
- When it fires
- Not stated
- Source
- client
- Data fields it needs
- Unknown
tp_u21_large_txn_9k Rule ID: tp_u21_large_txn_9k
- What it detects
- No description supplied by the client
- Fraud type it guards against
- Not stated
- When it fires
- Not stated
- Source
- client
- Data fields it needs
- Unknown
tp_u21_transfer_over_500 Rule ID: tp_u21_transfer_over_500
- What it detects
- No description supplied by the client
- Fraud type it guards against
- Not stated
- When it fires
- Not stated
- Source
- client
- Data fields it needs
- Unknown
tp_u21_wire_count_gt2_7d Rule ID: tp_u21_wire_count_gt2_7d
- What it detects
- No description supplied by the client
- Fraud type it guards against
- Not stated
- When it fires
- Not stated
- Source
- client
- Data fields it needs
- Unknown
tp_u21_wire_total_gt4999_7d Rule ID: tp_u21_wire_total_gt4999_7d
- What it detects
- No description supplied by the client
- Fraud type it guards against
- Not stated
- When it fires
- Not stated
- Source
- client
- Data fields it needs
- Unknown
Fraud types in plain words
Each fraud type named in your labels, in one sentence. The word in small grey text is the label as your file gives it.
- Many into one (fan-in) FAN-IN
- Many accounts send money into one account, which collects it.
- One out to many (fan-out) FAN-OUT
- One account sends money out to many accounts, so it spreads thin.
- Type not given no type
- Fraud labels in your file that do not say which kind of fraud it was.