When a customer taps their Visa card at a supermarket terminal, the perception is instantaneous. The reality is a choreographed exchange involving at minimum six distinct systems, two cryptographic operations, and a message protocol designed in 1987 that still processes the majority of the world's card transactions today.
The ISO 8583 Message
ISO 8583 is not an API—it is a binary wire protocol that encodes financial transactions as fixed and variable-length fields. Every POS message carries a Message Type Indicator (MTI), a bitmap declaring which data elements are present, and the data elements themselves.
The MTI is a four-digit BCD (Binary Coded Decimal) value. The first digit encodes the ISO version (0 = 1987, 1 = 1993, 2 = 2003). The second digit is the message class: 1xxx = Authorization, 2xxx = Financial, 4xxx = Reversal. The third digit encodes function (Request / Response / Advice), and the fourth describes the transaction originator.
| MTI | Meaning | Direction |
|---|---|---|
| 0100 | Authorization Request | Acquirer → Issuer |
| 0110 | Authorization Response | Issuer → Acquirer |
| 0200 | Financial Transaction Request | Acquirer → Issuer |
| 0210 | Financial Transaction Response | Issuer → Acquirer |
| 0400 | Reversal Request | Acquirer → Issuer |
| 0420 | Reversal Advice | Issuer → Acquirer |
Following the MTI is the primary bitmap—64 bits indicating which of Data Elements 1–64 are present. If bit 1 is set, a secondary bitmap extends coverage to DE 65–128. A tertiary bitmap (private use, DE 129–192) exists in some proprietary implementations.
def parse_bitmap(raw_hex: str) -> list[int]: """Return list of present DE numbers from hex bitmap string.""" bitmap_bytes = bytes.fromhex(raw_hex) present = [] for byte_idx, byte in enumerate(bitmap_bytes): for bit_pos in range(8): if byte & (0x80 >> bit_pos): de_number = byte_idx * 8 + bit_pos + 1 present.append(de_number) return present # Example: F220000012800000 (primary bitmap only) # DE 1,2,3,6,10,11,12,13,22,26,32,38,39,41,42 are present print(parse_bitmap("F220000012800000")) # → [1, 2, 3, 6, 10, 11, 12, 13, 22, 26, 32, 38, 39, 41, 42]
The Authorization Flow
A contactless payment triggers the following sequence. Timing figures below are representative of a European four-party scheme (Visa/Mastercard) with an acquiring bank in the same country as the issuer.
The most variable stage is Core Banking Query, which must perform an account lookup, balance check, velocity checks, and—for chip transactions—a dynamic CVV2 (iCVV) verification. Banks running legacy COBOL cores on mainframes typically exhibit p99 latencies in the 200–350ms range here; modern cloud-native issuers can achieve under 80ms at p99.
"The p99 latency of your core banking is the ceiling of your customer experience. You cannot build a 1-second payment on a 600ms mainframe call."
Key Data Elements in an Authorization
The critical data elements in a 0100 Authorization Request are:
| DE | Name | Format | Purpose |
|---|---|---|---|
| DE-2 | Primary Account Number | LLVAR n..19 | The PAN (card number). Never stored in plaintext post-auth. |
| DE-3 | Processing Code | n6 | Transaction type. 00xxxx = Purchase, 01xxxx = Withdrawal, 20xxxx = Credit. |
| DE-4 | Amount, Transaction | n12 | In minor currency units (cents). €10.50 = 000000001050. |
| DE-22 | Point of Service Entry Mode | n3 | 051 = chip+PIN, 071 = contactless NFC, 010 = manual. |
| DE-35 | Track 2 Equivalent Data | LLVAR Z..37 | PAN + Expiry + Service Code. Present in mag-stripe; absent in chip. |
| DE-55 | ICC Data (EMV Tags) | LLLVAR b..510 | Chip data block: ATC, ARQC, TVR, CVR, amount authorised. |
Decline Reason Distribution
Not all transactions are approved. Understanding the decline split is essential for fraud and risk teams—and for product teams measuring checkout abandonment caused by technical failures.
Clearing vs. Settlement
Authorization is a hold—it does not move money. The actual fund movement happens in two further stages:
Clearing (T+0 to T+1): The acquirer submits a clearing file to the card scheme, containing all captured transactions from the authorization day. This is typically a batch process running at the end-of-business cut-off, though some schemes now support near-real-time clearing.
Settlement (T+1 to T+2): The card scheme performs multilateral netting across all acquirers and issuers, then initiates net settlement payments through central bank systems (TARGET2 in Europe, Fedwire in the US). Issuers debit cardholder accounts; acquirers credit merchant accounts.
Reconciliation and the Break Problem
Every cleared transaction must be reconciled against the original authorization. Three types of breaks are common:
Amount mismatches arise when the final cleared amount differs from the authorized amount—common in hospitality (tips, fuel pre-auth). The scheme permits partial clearance or over-clearance within defined tolerances.
Orphan clears occur when a clearing record has no matching authorization—caused by voice authorization, system failures, or mis-routed transactions.
Expired authorizations happen when the clearing arrives after the authorization has lapsed (typically 5–7 days). The issuer may not honour the clearing, creating a liability dispute.
The EMV Chip: Cryptographic Guarantee
Modern chip cards perform a challenge-response authentication with the terminal and issuer. The card generates an Application Cryptogram (ARQC) using a session key derived from the card's master key and a diversification factor. The issuer verifies this cryptogram, providing a cryptographic guarantee that the physical card was present—not just the card number.
from Crypto.Cipher import DES3 from Crypto.Hash import CMAC def derive_session_key(icc_master_key: bytes, atc: int) -> bytes: """Derive per-transaction key using ATC diversification (EMV 4.3 A1.3).""" diversification = atc.to_bytes(2, 'big') + b'\xf0' + b'\x00' * 5 diversification += atc.to_bytes(2, 'big') + b'\x0f' + b'\x00' * 5 cipher = DES3.new(icc_master_key, DES3.MODE_ECB) left = cipher.encrypt(diversification[:8]) right = cipher.encrypt(diversification[8:]) return left + right def verify_arqc(session_key: bytes, pdol_data: bytes, arqc: bytes) -> bool: """Verify the Application Request Cryptogram using 3DES-MAC.""" mac = CMAC.new(session_key, ciphermod=DES3) mac.update(pdol_data) expected = mac.digest()[:8] return expected == arqc
Implications for Data Engineering
For anyone building data pipelines over payment data, several architectural considerations arise directly from this flow:
Event duality: Authorization and settlement are separate events with potentially different amounts, timestamps, and system identifiers. A naive schema that models "transactions" as a single entity will produce incorrect balance calculations.
PAN handling: PCI DSS requires that full PANs are never stored in analytical systems. Pipelines must tokenize or truncate (first 6 + last 4) at the point of ingestion. The BIN (first 6–8 digits) retains analytical value for card scheme, product type, and issuing bank identification.
Latency asymmetry: Authorization events arrive in real-time (sub-second); settlement arrives as batch files hours or days later. Any analytical system that joins these must handle late-arriving data gracefully—ideally with a watermark strategy rather than a fixed batch join.