Mayur Thakur, Managing Director, Goldman Sachs, NYC 2017
summary
Built a Hadoop-based platform that flattens diverse firm data into a few canonical tables (one aims for one HBase table for orders) and runs MapReduce pipelines and Map-only surveillances daily so expensive joins happen once per day. A spoofing model uses six enforcement cases, flags order imbalance >2.5 and rapid cancellations (<1s) as strong signals. Search over sensitive data stays inside the YARN cluster to index 1 billion documents and ~500 billion tokens with a slow and fast index to target sub-second latency.
AI-written summary · about
chapters (4)
- 0:00Start
- 10:37Spoof Orders
- 12:56Common Factors in Indicative of Spoofing Behavior
- 13:09Order Imbalance
related talks
05mInterview
18mBuilding a Scalable, Resilient System for Real-Time Fraud Detection
42mIT Around Data Management: Finance Industry
49mBuilding a Data-Driven Bank
27mManish Gupta: Director, American Express at Cypher 2017 - Part 1/3
51mReal-time Fraud Detection: Challenges and Solutions
47mReal-time Fraud Detection Challenges and Solutions
45mSolving Real & Big (Data) Problems Using Hadoop