Designing fraud-prevention systems that keep analysts in the loop

A technical article extending the Nerdearla Chile 2026 talk

Author

Affiliation

José P. Barrantes

IBM Cloud, Fraud Science and Engineering

Companion article for the Nerdearla Chile 2026 talk, Santiago, April 17, 2026.

Repository: Fraud System Architecture

Abstract

Fraud-prevention outcomes depend on rules, machine learning, operational policy, analyst work, and platform engineering. This article extends the Nerdearla Chile 2026 talk “Fraud prevention, machine learning, and design patterns: keep your analysts in the loop” with an architectural account of how these components interact.

We propose a design pattern for ML-enabled human-in-the-loop (HIL) triage. Rules handle explicit cases, machine learning estimates risk and prioritizes cases, a policy layer selects actions, and analysts investigate ambiguous cases and return structured feedback. The proposal draws on research in software architecture for ML systems, HIL design, AI design patterns, learning to defer, alert prioritization, reliable ML, and platform engineering. Human participation includes review, collaboration, and teaching throughout the system lifecycle (Mosqueira-Rey et al., 2023).

1 Introduction

This article develops the architecture, design-pattern rationale, operational consequences, and MLOps practices behind the conference talk. We focus on how to integrate automated decisions with expert review in fraud operations.

Fraud systems combine deterministic controls, statistical scoring, decision policy, and human judgment¹. Traffic volume and attack speed limit how much work analysts can handle manually. Ambiguous cases, changing attacker behavior, incomplete labels, asymmetric costs, and governance requirements constrain automation. The architecture needs to account for both sets of limits.

Production ML requires data pipelines, serving infrastructure, monitoring, deployment processes, and coordination across engineering roles (Amershi, Begel, et al., 2019; Lewis, Ozkaya, et al., 2021; Nazir et al., 2024; Sculley et al., 2015). Fraud operations add decision queues, analyst interfaces, escalation paths, and review outcomes. These components form a socio-technical system²: model outputs, human judgment, workflows, and organizational decisions jointly determine which actions are taken (Dobbe & Wolters, 2024; Salwei & Carayon, 2022).

Figure 1: Rules, machine learning, and expert analysts contribute complementary capabilities to fraud prevention.

In Figure 1, rules encode explicit knowledge and controls, machine learning combines heterogeneous signals into scores or rankings, and analysts contribute contextual judgment, novelty detection, and feedback. Their interfaces and allocation of responsibilities determine how they work together.

2 Architectural requirements in fraud prevention

Fraud systems commonly combine rules, supervised models, anomaly detection, and investigation to handle novelty, delayed labels, and operational constraints. Carcillo et al. (2021) describe a layered control structure with automated and human-supervised components. Hernandez Aros et al. (2024) document supervised, unsupervised, deep-learning, and other approaches used in financial-fraud detection.

Adversarial adaptation. Fraudsters change strategies when detection methods become known³ (Bolton & Hand, 2002). In credit-card fraud, this occurs alongside changes in legitimate customer behavior (Carcillo et al., 2021). Lunghi et al. (2023) examine the resulting challenges of concept drift, streaming constraints, limited observability, and hostile adaptation.

Delayed and imperfect labels. Labels often become available after a customer complaint or investigation⁴ (Carcillo et al., 2021; Lunghi et al., 2023). Unreported fraud can remain labeled as legitimate, contaminating the training data (Bolton & Hand, 2002; Carcillo et al., 2021). Fraud also includes different forms of abuse with distinct indicators (Bolton & Hand, 2002; Hernandez Aros et al., 2024). Combining these into one positive class can discard information useful for triage and investigation. The system’s decisions affect which cases are investigated and therefore which labels become available for learning and evaluation.

Asymmetric error costs. Missed fraud causes direct loss; false alarms create customer friction, investigation expense, and potential reputational harm⁵. Investigating every case is too expensive, so suspicion scores help allocate review effort (Bolton & Hand, 2002). Detection policy must balance those costs with timely intervention (Hernandez Aros et al., 2024).

Time sensitivity. Earlier detection can reduce the cost of a fraud event⁶ (Hilal et al., 2022). Suspicion ranking helps investigators prioritize urgent cases (Bolton & Hand, 2002), while online processing imposes timing constraints on detection and response (Lunghi et al., 2023). The architecture determines which decisions can be automated, which require escalation, and how analyst time is allocated (Carcillo et al., 2021).

These requirements affect the full lifecycle: development, monitoring, maintenance, and evolution (Lewis, Ozkaya, et al., 2021; Muccini & Vaidhyanathan, 2021). They also require explicit decisions about infrastructure, quality attributes, and integration (Nazir et al., 2024).

3 Production components and development planning

Sculley et al. (2015) identify data dependencies, feature extraction, configuration, serving, monitoring, and process tooling as sources of hidden technical debt⁷. Each creates maintenance work and dependencies that need to be considered when designing a fraud system.

Figure 2: Production components in fraud operations: data flows, software, serving infrastructure, monitoring, process tools, and expert review support the model. Adapted from Sculley et al. (2015).

Our adaptation in Figure 2 includes human review alongside the software and data infrastructure. Failures at component boundaries can impair decisions: a feature outage, an integration error, a misleading analyst view, or missing monitoring can compromise an otherwise useful prediction. Lin & Ryaboy (2013) describe related engineering requirements for repeatable large-scale analysis, including workflows, recurring jobs, and supporting infrastructure.

Figure 3: Planning an ML-enabled fraud system across business requirements, software development, model development, expert review, and platform infrastructure. Adapted from Lewis, Ozkaya, et al. (2021), Andersen & Maalej (2024), and Kästner (2025).

Figure 3 shows how these components should be planned together⁸. In the planning model of Lewis, Ozkaya, et al. (2021), architecture receives business context and system requirements, defines component responsibilities, and coordinates their development. Problems identified in software engineering, data pipelines, modeling, quality assurance, or operations feed back into architectural decisions and can require changes across teams.

Conventional software development covers code, APIs, business logic, integrations, and interfaces. ML development covers model requirements, data engineering, training, and quality assurance, producing both a trained model and its supporting data pipeline. Co-versioning⁹ links the deployed model to its data, parameters, and evaluation artifacts (Lewis, Ozkaya, et al., 2021). Our adaptation adds expert review as a component whose workflow and interfaces are specified during planning.

4 Design-pattern rationale

We propose this architecture as a reusable response to fraud operations with automated decisions and limited review capacity. A design-pattern description¹⁰ specifies its context, competing requirements, solution, and consequences (Konieczny, 2025; Lakshmanan et al., 2020).

Existing work provides several ways to document this design knowledge¹¹. Washizaki et al. (2020) catalog architecture patterns and anti-patterns for ML systems; Heiland et al. (2023) extend the repository of patterns for AI-based systems. Järvenpää et al. (2024) examine reusable architectural tactics, and Cruz et al. (2023) address architecture rationale and evaluation.

Andersen & Maalej (2024) catalog HIL patterns¹² for data preparation, training, operation, monitoring, and explanation. These patterns give human participation defined responsibilities within the system. The broader HIL-ML literature includes active learning, interactive ML, and machine teaching¹³, with different forms and timing of expert intervention (Mosqueira-Rey et al., 2023). In deployed systems, recommendation support, active moderation, corrective feedback, continuous learning, and explanation are relevant to fraud review queues, analyst overrides, escalation, and feedback capture (Andersen & Maalej, 2024).

Figure 4: ML-enabled HIL fraud triage as a design pattern: context, competing requirements, solution, consequences, and related patterns. Synthesized from Lakshmanan et al. (2020), Andersen & Maalej (2024), Heiland et al. (2023), Cruz et al. (2023), and Järvenpää et al. (2024).

The context in Figure 4 is a fraud operation where review demand exceeds analyst capacity, some cases require immediate action, attackers adapt, and errors have different costs (Bolton & Hand, 2002; Carcillo et al., 2021; Ghadermazi et al., 2024; Jalalvand et al., 2024). Response time, customer friction, detection coverage, human oversight, and available expertise impose competing requirements (Alves et al., 2025; Bolton & Hand, 2002; Jalalvand et al., 2024).

The solution combines rules, scoring, policy, analyst review, and feedback (Andersen & Maalej, 2024; Mosqueira-Rey et al., 2023). Expected benefits include more efficient use of analyst time, lower friction on low-risk traffic, and more transparent decision governance (Alves et al., 2025; Andersen & Maalej, 2024; Ghadermazi et al., 2024). Adoption requires calibration, queue management, traceability, versioning, and additional integration work (Cruz et al., 2023; Lewis, Ozkaya, et al., 2021; Nazir et al., 2024). Documenting these consequences helps teams assess the pattern against their operating conditions (Richards & Ford, 2025).

5 Pattern components and routing

The model consumes features and estimates risk. The policy layer¹⁴ combines that estimate with operating goals and constraints. The system then approves, blocks, escalates, or routes the case to review (Figure 5).

Figure 5: Risk scoring, decision policy, and action: the model estimates risk, the policy layer applies operational constraints, and the system executes the selected action.

A separate policy layer makes the rules governing business actions inspectable (Chen et al., 2022). It can account for guardrails, analyst capacity, time sensitivity, customer value, regional policy, and legal requirements (Ghadermazi et al., 2024; Jalalvand et al., 2024; Kästner, 2025). Teams can revise these decisions independently of model training.

Figure 6: Reference architecture of ML-enabled HIL triage: rules handle explicit cases, ML scores the remaining cases, and policy routes them to approval, analyst review, or automatic mitigation.

In Figure 6, deterministic rules handle cases covered by explicit controls near the entry point. The remaining cases reach the scoring layer. Policy may approve low-risk cases automatically, mitigate clear high-risk cases, and route ambiguous cases to analysts. This gives escalation logic a defined location and separates risk estimation from decision governance.

Learning to defer¹⁵ studies the allocation of cases between automated prediction and expert review. Its treatment of expert accuracy, availability, and capacity provides a basis for investigating routing policies in this architecture (Alves et al., 2025).

6 Analyst workflow and feedback

Analyst review is a runtime component¹⁶ with specified inputs, decision authority, escalation paths, and outputs (Andersen & Maalej, 2024; Ding et al., 2023; Mosqueira-Rey et al., 2023). Its design determines what evidence reviewers receive and how their conclusions affect the system.

Figure 7: Collaboration between analysts and ML: models prioritize cases and reduce search effort; analysts provide contextual judgment, escalation decisions, and feedback for calibration and learning.

Figure 7 shows this exchange. Models help analysts focus their attention. Analysts interpret evidence, handle exceptions, and provide feedback during production operation. Depending on the intervention, they act as reviewers, collaborators, or teachers (Mosqueira-Rey et al., 2023). Tools for anomaly reasoning and management also need to support explanation, action, and iterative investigation (Ding et al., 2023).

Figure 8: Feedback paths in HIL triage: analyst outcomes support rule maintenance and model improvement, including review of cases resolved automatically.

Review outcomes should support relabeling, rule creation, threshold revision, and post-incident analysis. Kadam (2024) examine how analyst feedback can be propagated to related transactions¹⁷, allowing sparse annotations to contribute to model improvement. This is useful when attacks are weakly characterized or previously unseen. Expert review in open-set recognition¹⁸ offers a related approach to handling novelty and uncertainty (Yuan et al., 2026).

The yellow return paths in Figure 8 connect automatically approved and mitigated cases to retrospective analyst review¹⁹. Sampling these cases helps identify false positives and false negatives, then assess whether labels, thresholds, rules, or routing logic need revision. Errors may become visible after deployment through edge cases, data drift, or changed operating conditions (Chen et al., 2022; Tan Wei Hao et al., 2026). Structured review supplies corrections and additional labels (Andersen & Maalej, 2024), including when only a subset of predictions can be verified manually (Chen et al., 2022). Kadam (2024) report improvements in robustness, recall, and performance on unseen fraud patterns through reuse of human feedback.

Figure 9: Operational learning in HIL triage: observed outcomes and analyst feedback inform updates to rules and models.

Figure 9 summarizes the feedback cycle. Once outcomes become available, they inform rule maintenance and model development. The architecture needs a return path from operational decisions to the components that can be revised.

Figure 10: Analyst workbench for HIL fraud triage: ranked cases, risk bands, reason codes, service-level timers, action controls, and structured feedback fields. Synthesized from Jalalvand et al. (2024), Ghadermazi et al. (2024), and Alves et al. (2025).

The workbench²⁰ in Figure 10 presents the case information needed for investigation and captures its outcome. Alert-prioritization research identifies workload, context, analyst skill, assignment, and review efficiency as operational concerns (Ghadermazi et al., 2024; Jalalvand et al., 2024). Learning-to-defer research also accounts for differences in expert performance and availability (Alves et al., 2025).

The interface should expose case priority, contributing signals, deadlines, and available actions. It should communicate uncertainty and make oversight and intervention understandable at the point of use (Amershi, Weld, et al., 2019). Uber’s Project RADAR provides an industry example of human review used to validate and put candidate fraud rules into operation (Zelvenskiy et al., 2022).

7 System evaluation

Evaluation should cover the interactions between model outputs, analyst interpretation, workflow, and organizational action (Dobbe & Wolters, 2024; Lewis, Ozkaya, et al., 2021; Nazir et al., 2024). These interactions determine how an ML component performs within the larger work system (Salwei & Carayon, 2022). We organize the measures into four areas²¹, with policy behavior assessed across them.

Figure 11: Evaluation of data quality, model and decision quality, queue operations, runtime health, and policy behavior. Synthesized from Lewis, Ozkaya, et al. (2021), Jalalvand et al. (2024), Ghadermazi et al. (2024), Alves et al. (2025), Chen et al. (2022), and Tan Wei Hao et al. (2026).

Figure 11 links technical measures to the conditions under which the system is used (Dobbe & Wolters, 2024; Fazelpour, 2025). Data and input quality includes distribution shifts, missing features, and stale inputs, all of which can impair predictions (Chen et al., 2022; Kästner, 2025; Lewis, Ozkaya, et al., 2021). Model and decision quality includes precision among the cases selected for review²², recall on confirmed fraud, calibration, and deferral rate²³ (Alves et al., 2025; Chen et al., 2022).

Queue and analyst operations includes backlog, review latency, analyst agreement, workload, and SLA compliance (Alves et al., 2025; Ghadermazi et al., 2024; Jalalvand et al., 2024). Runtime and platform quality includes latency, error rates, uptime, and observability (Lewis, Ozkaya, et al., 2021; Tan Wei Hao et al., 2026).

Approval, review, and block rates describe policy behavior²⁴ (Chen et al., 2022). The organization sets the policy that converts risk estimates into actions under its operating constraints (Kästner, 2025; Lewis, Ozkaya, et al., 2021). Tracking these rates alongside threshold sensitivity and queue saturation helps assess the consequences of policy changes.

The trade-offs in Figure 11 depend on the operating context (Fazelpour, 2025). Faster responses allow less time for investigation; higher detection coverage may increase customer friction; more human review requires capacity; frequent adaptation increases the work needed for auditability (Alves et al., 2025; Chen et al., 2022; Jalalvand et al., 2024). Architectural decisions should document how these requirements are balanced and evaluated (Cruz et al., 2023; Kästner, 2025; Lewis, Ozkaya, et al., 2021; Nazir et al., 2024; Richards & Ford, 2025).

8 MLOps and platform engineering

The pattern has a lifecycle²⁵ in which monitoring and analyst feedback inform maintenance, retraining, and system evolution (Lewis, Ozkaya, et al., 2021). These signals also support data curation, feature maintenance, and policy revision (Kästner, 2025; Tan Wei Hao et al., 2026).

Figure 12: MLOps lifecycle for HIL fraud systems: monitoring and analyst feedback support data curation, model improvement, and policy revision. Adapted from Lewis, Ozkaya, et al. (2021), extended with platform-engineering concepts from Tan Wei Hao et al. (2026), and specialized for fraud feedback using Kadam (2024).

The upper part of Figure 12 covers data capture, feature generation, training, and evaluation. The lower part covers deployment, monitoring, analyst review, and feedback. Its return paths distinguish updates to datasets and feature pipelines, improvements to models, and revisions to rules, thresholds, or routing policy.

These paths require dependable data infrastructure. Lin & Ryaboy (2013) describe workflows, recurring jobs, and infrastructure for repeatable large-scale analysis. Konieczny (2025) document patterns for ingestion, transformation, lineage, and pipeline quality. The distinction between annotation, interactive steering, and teaching in HIL systems also helps identify how different feedback signals should be used (Mosqueira-Rey et al., 2023).

Monitorability²⁶ supports maintenance and evolution (Lewis, Ozkaya, et al., 2021). Teams also need to identify mismatches between the assumptions of different roles and components (Lewis, Bellomo, et al., 2021). Platform support can provide repeatable processes for deployment, observability, and governance (Tan Wei Hao et al., 2026), while fraud-specific feedback methods connect analyst outcomes to model maintenance (Kadam, 2024).

The shared platform²⁷ in Figure 12 provides versioning, feature storage, a model registry, observability, and governance. These services support traceability and reduce duplicated infrastructure work across teams.

9 Expected benefits and implementation costs

The pattern is intended to allocate analyst attention to cases where expert judgment is useful (Alves et al., 2025; Jalalvand et al., 2024). Automatic handling of clear low-risk and high-risk cases can reduce customer friction and intervention delays (Jalalvand et al., 2024; Kästner, 2025). Separating policy from inference makes decision logic easier to inspect (Chen et al., 2022; Lewis, Ozkaya, et al., 2021), and structured review outcomes provide inputs for rule maintenance and model improvement (Andersen & Maalej, 2024; Mosqueira-Rey et al., 2023).

Adoption adds components and integration work (Lewis, Ozkaya, et al., 2021; Nazir et al., 2024). Queue monitoring must cover backlog, latency, workload, and assignment quality (Ghadermazi et al., 2024; Jalalvand et al., 2024). Decision logic needs versioning and an audit trail, and feedback data needs defined capture and governance practices (Cruz et al., 2023; Lewis, Ozkaya, et al., 2021). Software engineers, data scientists, risk operators, and analysts must coordinate changes to shared system behavior (Nazir et al., 2024; Richards & Ford, 2025). These costs belong in the adoption decision.

9.1 Common failure modes

Coupling scoring and policy.²⁸ When a model output directly triggers an action, the governing business logic can become difficult to inspect (Lewis, Ozkaya, et al., 2021; Washizaki et al., 2020). Tight coupling also complicates diagnosis: failures may arise from the model, the business logic, or their interaction (Washizaki et al., 2020). The separate policy layer described in Section 5 makes these responsibilities explicit and allows decision logic to be revised independently (Kästner, 2025; Washizaki et al., 2020).

Unstructured review outcomes.²⁹ Free-text notes alone make analyst conclusions difficult to reuse (Mosqueira-Rey et al., 2023). Structured fields for outcome, reason code, confidence, and suggested action make feedback available to operation, monitoring, and subsequent improvement (Andersen & Maalej, 2024; Mosqueira-Rey et al., 2023). Notes can retain the context needed to interpret those fields.

Incomplete operational design. Queue design, escalation logic, and the analyst interface require attention during development. Alert volume can overwhelm reviewers, and workload, fatigue, skill, and case context affect review quality (Jalalvand et al., 2024). These requirements need to be coordinated with model development and system maintenance (Lewis, Ozkaya, et al., 2021; Nazir et al., 2024).

Insufficient monitoring. Lewis, Ozkaya, et al. (2021) identify monitorability as an architectural driver for maintenance and evolution. Queue overload, rising analyst disagreement, and calibration drift can indicate degradation while the service remains available (Jalalvand et al., 2024; Lewis, Ozkaya, et al., 2021). Monitoring should cover these conditions from deployment.

Unmanaged integration complexity.³⁰ Washizaki et al. (2020) describe the ML anti-patterns “glue code” and “pipeline jungles.” In fraud systems, feature joins, rule engines, scoring services, case queues, and alerting tools can accumulate under delivery pressure. Shared interfaces, traceability, and versioning help keep these dependencies maintainable (Kästner, 2025; Lewis, Ozkaya, et al., 2021; Washizaki et al., 2020).

These failure modes identify requirements for adopting the pattern: explicit policy, structured feedback, operational design, monitoring, and maintainable integrations (Andersen & Maalej, 2024; Lewis, Ozkaya, et al., 2021; Washizaki et al., 2020). Documenting the responsibilities and failure conditions supports architectural review across software, data, policy, and analyst work (Cruz et al., 2023; Richards & Ford, 2025).

10 Contribution to ML-enabled software architecture

Muccini & Vaidhyanathan (2021) call for better architecting practices and clearer standards for ML-based systems. Nazir et al. (2024) consolidate design challenges, practices, and architectural decisions. Existing work covers hidden technical debt, architecture mismatch, HIL patterns, monitorability, evaluation, and internal platforms; documenting their application to specific domains can help teams use that knowledge.

Fraud operations make the interactions between these concerns visible. Decisions affect financial loss, customer experience, investigation workload, escalation, and organizational risk. A socio-technical account includes human and institutional decisions within the system boundary (Dobbe & Wolters, 2024), considers workflow, users, technology, and organizational context together (Salwei & Carayon, 2022), and treats ML as part of an operational product (Kästner, 2025). HIL research provides methods for specifying how people participate in that system (Andersen & Maalej, 2024; Mosqueira-Rey et al., 2023).

Our contribution is to document the combination of rules, scoring, policy, analyst review, and feedback as a reusable architecture for fraud operations. It builds on established HIL approaches and ML pattern catalogs (Andersen & Maalej, 2024; Washizaki et al., 2020). The description identifies component responsibilities, routing and feedback mechanisms, governance requirements, and adoption costs, following production ML engineering and architectural practice (Kästner, 2025; Richards & Ford, 2025).

11 Open questions

Policy optimization. How should routing policy adapt to risk scores, analyst capacity, SLA pressure, and business value over time?

Feedback quality. Which analyst signals lead to useful system improvements, and how should analyst disagreement be represented? Learning-to-defer research considers expert heterogeneity and capacity constraints (Alves et al., 2025). Their interaction with fraud-specific roles and queue dynamics needs further study. Expert review for unknown attack detection through open-set recognition offers a related direction to investigate (Yuan et al., 2026).

Architecture evaluation. How should scenario-based evaluation be adapted to fraud operations? Existing work addresses architecture evaluation for ML systems (Cruz et al., 2023; Nazir et al., 2024). Fraud-specific scenarios should exercise queue saturation, feature outages, delayed labels, and adversarial shifts to assess behavior after deployment (Chen et al., 2022; Jalalvand et al., 2024; Lunghi et al., 2023).

Internal platforms. Which capabilities should be shared across fraud and risk teams? Fragmented scripts, dashboards, and services can make coordination, orchestration, and operational visibility difficult. Lin & Ryaboy (2013) describe these infrastructure problems across data workflows and frameworks. Internal ML platforms can support automation, monitoring, deployment, and developer experience (Tan Wei Hao et al., 2026). Orchestration, observability, provenance, and infrastructure maintenance are relevant requirements for such a platform (Kästner, 2025).

LLMs in analyst workflows.³¹ How can case summarization, natural-language explanations, and conversational investigation support be integrated with structured feedback and audit requirements? LLM components introduce requirements for interface design, validation, observability, guardrails, and evaluation (Tan Wei Hao et al., 2026). Their role in this pattern requires further design and assessment, particularly where generated summaries or explanations inform analyst decisions.

12 Conclusion

ML-enabled HIL triage assigns explicit responsibilities to rules, models, policy, analysts, and platform services. Rules apply known controls, models estimate risk, policy selects actions, and analysts investigate cases and record structured outcomes. Feedback supports updates to data, rules, models, and routing logic.

The pattern requires evaluation of data quality, decision quality, review operations, runtime health, and policy behavior. Its usefulness depends on whether the organization can maintain those components and coordinate changes over time. We offer it as a documented architecture for teams designing fraud systems with automated decisions and expert review.

Contact

I’d be glad to hear your questions, disagreements, or experiences with fraud review systems.

• LinkedIn: José Pablo Barrantes
• BlueSky: doggofan77.bsky.social

References

Alves, J. V., Leitão, D., Jesus, S., Sampaio, M. O. P., Liébana, J., Saleiro, P., Figueiredo, M. A. T., & Bizarro, P. (2025). A benchmarking framework and dataset for learning to defer in human-AI decision-making. Scientific Data, 12, 506. https://doi.org/10.1038/s41597-025-04664-y

Amershi, S., Begel, A., Bird, C., DeLine, R., Gall, H., Kamar, E., Nagappan, N., & Nushi, B. (2019). Software engineering for machine learning: A case study. 2019 IEEE/ACM 41st International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP), 291–300. https://doi.org/10.1109/ICSE-SEIP.2019.00042

Amershi, S., Weld, D., Vorvoreanu, M., Fourney, A., Nushi, B., Collisson, P., Suh, J., Iqbal, S. T., Bennett, P. N., Inkpen, K., Teevan, J., Kikin-Gil, R., & Horvitz, E. (2019). Guidelines for human-AI interaction. Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3290605.3300233

Andersen, J., & Maalej, W. (2024). Design patterns for machine learning-based systems with humans in the loop. IEEE Software, 41(4), 151–159. https://doi.org/10.1109/MS.2023.3340256

Bolton, R. J., & Hand, D. J. (2002). Statistical fraud detection: A review. Statistical Science, 17(3), 235–255. https://doi.org/10.1214/ss/1042727940

Carcillo, F., Le Borgne, Y.-A., Caelen, O., Kessaci, Y., Oblé, F., & Bontempi, G. (2021). Combining unsupervised and supervised learning in credit card fraud detection. Information Sciences, 557, 317–331. https://doi.org/10.1016/j.ins.2019.05.042

Chen, C., Murphy, N. R., Parisa, K., Sculley, D., & Underwood, T. (2022). Reliable machine learning: Applying SRE principles to ML in production. O’Reilly Media.

Cruz, P., Ulloa, G., San Martin, D., & Veloz, A. (2023). Software architecture evaluation of a machine learning enabled system: A case study. 2023 42nd IEEE International Conference of the Chilean Computer Science Society (SCCC). https://doi.org/10.1109/SCCC59417.2023.10315755

Ding, X., Seleznev, N., Kumar, S., Bruss, C. B., & Akoglu, L. (2023). From detection to action: A human-in-the-loop toolkit for anomaly reasoning and management. Proceedings of the Fourth ACM International Conference on AI in Finance, 279–287. https://doi.org/10.1145/3604237.3626872

Dobbe, R., & Wolters, A. (2024). Toward sociotechnical AI: Mapping vulnerabilities for machine learning in context. Minds and Machines, 34(2), Article 12. https://doi.org/10.1007/s11023-024-09668-y

Fazelpour, S. (2025). Disciplining deliberation: A socio-technical perspective on machine learning trade-offs. The British Journal for the Philosophy of Science. https://doi.org/10.1086/734552

Ghadermazi, J., Shah, A., & Jajodia, S. (2024). A machine learning and optimization framework for efficient alert management in a cybersecurity operations center. Digital Threats: Research and Practice, 5(2), Article 19. https://doi.org/10.1145/3644393

Heiland, L., Hauser, M., & Bogner, J. (2023). Design patterns for AI-based systems: A multivocal literature review and pattern repository. 2023 IEEE/ACM 2nd International Conference on AI Engineering—Software Engineering for AI (CAIN). https://doi.org/10.1109/CAIN58948.2023.00034

Hernandez Aros, L., Bustamante Molano, L. X., Gutierrez-Portela, F., Moreno Hernandez, J. J., & Rodríguez Barrero, M. S. (2024). Financial fraud detection through the application of machine learning techniques: A literature review. Humanities and Social Sciences Communications, 11(1), 1–22. https://doi.org/10.1057/s41599-024-03606-0

Hilal, W., Gadsden, S. A., & Yawney, J. (2022). Financial fraud: A review of anomaly detection techniques and recent advances. Expert Systems with Applications, 193, 116429. https://doi.org/10.1016/j.eswa.2021.116429

Jalalvand, F., Chhetri, M. B., Nepal, S., & Paris, C. (2024). Alert prioritisation in security operations centres: A systematic survey on criteria and methods. ACM Computing Surveys, 57(2), Article 42. https://doi.org/10.1145/3695462

Järvenpää, H., Lago, P., Bogner, J., Lewis, G. A., Muccini, H., & Ozkaya, I. (2024). A synthesis of green architectural tactics for ML-enabled systems. Proceedings of the 46th International Conference on Software Engineering: Software Engineering in Society (ICSE-SEIS ’24), 130–141. https://doi.org/10.1145/3639475.3640111

Kadam, P. (2024). Enhancing financial fraud detection with human-in-the-loop feedback and feedback propagation. 2024 International Conference on Machine Learning and Applications (ICMLA), 1198–1203. https://doi.org/10.1109/ICMLA61862.2024.00185

Kästner, C. (2025). Machine learning in production: From models to products. MIT Press. https://mlip-cmu.github.io/book/

Konieczny, B. (2025). Data engineering design patterns: Recipes for solving the most common data engineering problems. O’Reilly Media.

Lakshmanan, V., Robinson, S., & Munn, M. (2020). Machine learning design patterns. O’Reilly Media.

Lewis, G. A., Bellomo, S., & Ozkaya, I. (2021). Characterizing and detecting mismatch in machine-learning-enabled systems. 2021 IEEE/ACM 1st Workshop on AI Engineering—Software Engineering for AI (WAIN), 133–140. https://doi.org/10.1109/WAIN52551.2021.00028

Lewis, G. A., Ozkaya, I., & Xu, X. (2021). Software architecture challenges for ML systems. 2021 IEEE International Conference on Software Maintenance and Evolution (ICSME), 634–638. https://doi.org/10.1109/ICSME52107.2021.00071

Lin, J., & Ryaboy, D. (2013). Scaling big data mining infrastructure: The Twitter experience. ACM SIGKDD Explorations Newsletter, 14(2), 6–19. https://doi.org/10.1145/2481244.2481247

Lunghi, D., Simitsis, A., Caelen, O., & Bontempi, G. (2023). Adversarial learning in real-world fraud detection: Challenges and perspectives. Proceedings of the Second ACM Data Economy Workshop, 27–33. https://doi.org/10.1145/3600046.3600051

Mosqueira-Rey, E., Hernández-Pereira, E., Alonso-Ríos, D., Bobes-Bascarán, J., & Fernández-Leal, Á. (2023). Human-in-the-loop machine learning: A state of the art. Artificial Intelligence Review, 56, 3005–3054. https://doi.org/10.1007/s10462-022-10246-w

Muccini, H., & Vaidhyanathan, K. (2021). Software architecture for ML-based systems: What exists and what lies ahead. 2021 IEEE/ACM 1st Workshop on AI Engineering—Software Engineering for AI (WAIN), 121–128. https://doi.org/10.1109/WAIN52551.2021.00026

Nazir, R., Bucaioni, A., & Pelliccione, P. (2024). Architecting ML-enabled systems: Challenges, best practices, and design decisions. Journal of Systems and Software, 207, 111860. https://doi.org/10.1016/j.jss.2023.111860

Richards, M., & Ford, N. (2025). Fundamentals of software architecture: A modern engineering approach (2nd ed.). O’Reilly Media.

Salwei, M. E., & Carayon, P. (2022). A sociotechnical systems framework for the application of artificial intelligence in health care delivery. Journal of Cognitive Engineering and Decision Making, 16(4), 194–206. https://doi.org/10.1177/15553434221097357

Sculley, D., Holt, G., Golovin, D., Davydov, E., Phillips, T., Ebner, D., Chaudhary, V., Young, M., Crespo, J.-F., & Dennison, D. (2015). Hidden technical debt in machine learning systems. Advances in Neural Information Processing Systems, 28.

Tan Wei Hao, B., Padmanabhan, S., & Mallya, V. (2026). Machine learning platform engineering: Build an internal developer platform for ML and AI systems. Manning.

Washizaki, H., Uchida, H., Khomh, F., & Guéhéneuc, Y.-G. (2020). Machine learning architecture and design patterns. IEEE Software, 37(4), 76–84. https://doi.org/10.1109/MS.2019.2961960

Yuan, X., Yu, P., Liu, S., Sun, Z., Zhang, Y., & Xu, J. (2026). An expert-in-the-loop framework for unknown attack detection via open-set recognition. Journal of Computer Security. https://doi.org/10.1177/0926227X251414058

Zelvenskiy, S., Harisinghani, G., Yu, T., Ng, E., & Wei, R. (2022). Project RADAR: Intelligent early fraud detection system with humans in the loop. Uber Blog. https://www.uber.com/en-CR/blog/project-radar-intelligent-early-fraud-detection/

Footnotes

1. Hybrid system: A fraud system combining deterministic rules, ML scoring, decision policy, and expert human judgment.

2. Socio-technical system: A system whose behavior is shaped by technical components, such as models and infrastructure, and human work, such as analyst review and organizational decisions.

3. Adversarial adaptation: Fraudsters change tactics in response to known defenses, requiring continued monitoring and revision of detection methods.

4. Label delay and imperfection: Confirmation depends on complaints and investigations. Undetected fraud can contaminate the legitimate class, and a binary label can obscure different forms of abuse.

5. Asymmetric error costs: False negatives and false positives have different financial and operational consequences. Policy must reflect those differences.

6. Time sensitivity: The usefulness of a fraud decision depends on whether it arrives in time to support intervention.

7. Hidden technical debt: Maintenance burden arising from data dependencies, pipelines, configuration, serving, monitoring, and integration code around an ML model.

8. Co-architecting: Coordinated design of the runtime system and the model-development pipeline.

9. Co-versioning: Traceable versions of data, features, models, evaluation artifacts, and policy thresholds allow post-incident analysis to reconstruct the decision state.

10. Design pattern: A reusable solution to a recurring problem in a specific context, documented with its forces, solution, and consequences.

11. ML architecture patterns: Washizaki et al. identify 33 ML patterns and anti-patterns. Heiland et al. document 70 AI-system patterns.

12. HIL patterns: Reusable designs for human participation in data preparation, training, operation, monitoring, and explanation.

13. HIL-ML approaches: Active learning, interactive ML, and machine teaching differ in how and when human expertise contributes to learning.

14. Decision policy: Explicit logic that combines model scores with business and operational constraints to select an action.

15. Learning to defer (L2D): A framework for deciding whether to handle a case automatically or route it to an expert, accounting for their expected performance.

16. Analyst workflow: A review process with structured inputs, such as ranked cases and reason codes, and structured outputs, such as decisions and feedback.

17. Feedback propagation: Reuse of analyst annotations across related observations, such as transactions connected in a graph.

18. Open-set recognition: Identifying observations outside the known classes. Expert review can help characterize these unfamiliar cases.

19. Retrospective audit: Review of automatically resolved cases to detect errors and assess changes to labels, thresholds, rules, or routing logic.

20. Analyst workbench: An interface with case rankings, risk bands, reason codes, service-level agreement (SLA) timers, action controls, and structured feedback fields.

21. Evaluation areas: Data and input quality; model and decision quality; queue and analyst operations; runtime and platform quality. Policy behavior and trade-offs connect these areas.

22. Precision@k: Precision among the top-k ranked cases. Choosing k to match review capacity makes the measure relevant to the analyst queue.

23. Deferral rate: The fraction of cases routed to human review. This rate affects analyst workload and is controlled through routing policy.

24. Policy behavior: The traffic split across approval, review, and blocking, together with its sensitivity to thresholds and capacity limits.

25. MLOps lifecycle: Data capture, feature generation, training, evaluation, deployment, monitoring, analyst review, and feedback. Return paths support data curation, model improvement, and policy revision.

26. Monitorability: The ability to observe data quality, model quality, service health, and queue health during operation.

27. ML platform: Shared infrastructure for versioning, feature storage, model registration, observability, and governance.

28. Scoring–policy coupling: Direct dependence of business actions on model outputs without explicit, separately maintained decision logic.

29. Unstructured review: Review records that lack consistent outcome fields, limiting their use in evaluation and maintenance.

30. Pipeline jungle: A collection of interdependent processing steps and services whose interfaces and maintenance responsibilities are poorly defined.

31. LLM integration: Potential support for case summaries, natural-language reason codes, and conversational investigation, subject to evaluation of feedback quality, auditability, and governance.