Redefining Test Data Strategy in the Gen AI Era

on YouTube ↗38m

" summary

Claims a repeatable AI-driven framework that turns intent into executable scenarios then into exact records, cutting provisioning time and production dependency while raising coverage, reusability and defect detection. Shows a concrete walkthrough for validate high-risk payment failure, an architecture that uses a metadata context layer and separate reasoning from data access so LLMs plan but APIs and generators execute, and a test data factory that selects AI vs deterministic generators and runs schema, referential integrity and business‑rule validation before use. Metrics: two down, three up.

AI-written summary · about

" chapters (60)

1. 0:00Start
2. 0:08Session Opens - Redefining Test Data Strategy in the Gen AI Era
3. 0:38Speaker Intro - Manideep Singireddy, Senior Manager SDET, Equiniox
4. 1:38The Part of Quality Engineering Nobody Talks About
5. 2:10Test Data Processes Designed for a Much Slower Engineering Model
6. 2:40Rethinking Test Data as an Engineering Capability
7. 3:40The Question: AI Changed - Did Your Test Data Strategy?
8. 4:10Pipelines Run in Minutes; Getting the Right Data Takes Hours or Days
9. 4:41A 20-Minute Regression Suite That Really Takes All Day
10. 5:11The Real Measure: Time From Test Intent to Usable Data
11. 5:43Force One - Speed; Force Two - Complexity
12. 6:16Force Three - AI: Prompts, Context, Retrieval and Probabilistic Responses
13. 6:48Faster Data, Richer Data, Context-Aware Data
14. 7:18The Traditional Model: Copy Production, Mask, Move Down, Consume
15. 7:49The Constraints: Slow Refreshes, Access Control, Masking Overhead
16. 8:19Production Reflects What Already Happened, Not What You Need to Test
17. 8:50The Real Limitation: You Can't Engineer Coverage Intentionally
18. 9:52Business Meaning Scattered Across Code, Queries, and Individual Heads
19. 10:22What Changes When the System Under Test Uses Generative AI
20. 10:53The Input Is No Longer Just the Prompt
21. 11:25You Are No Longer Testing a Request - You Are Testing the Context
22. 11:58The New Category: Prompts, Context, Conversations, Memory, Adversarial Cases
23. 12:30A Record Says What Exists; A Scenario Says What You're Reproducing
24. 13:00The Fundamental Question: "What Do We Have?" vs "What Scenario Do We Need?"
25. 13:31Scenario-Driven Reverses the Logic: Intent, Scenario, Then Data
26. 14:01Building the Scenario Instead of Searching Tables for a Match
27. 14:35How Does AI Know What Data Is Relevant? Not Magically
28. 15:07It Knows What an Account Is - Not What It Means in Your Platform
29. 15:37Exposing Enterprise Knowledge to the Reasoning Layer
30. 16:07Should You Just Give the Model All Your Databases? No
31. 16:39Separate Reasoning From Data Access
32. 17:10Permissions, Read-Only, Masking, Auditing Enforced by the Services
33. 17:40The Model Decides What It Needs; The System Decides How Safely
34. 18:12A Concrete Walkthrough: Validate High-Risk Payment Failure
35. 18:44Turning a Test Goal Into a Clear, Repeatable Scenario
36. 19:16The Design Principle: Don't Generate Data First
37. 19:47Volume Does Not Improve Testing - Generate Scenarios First
38. 20:17Then Produce Exactly the Records Each Case Requires
39. 20:47Don't Let the LLM Become the Entire Platform
40. 21:19Once the Blueprint Exists, Deterministic Systems Take Over
41. 21:49An API Creates an Account More Safely Than Database Access
42. 22:22The Intelligent Test Data Factory Starts With a Scenario
43. 22:52Choosing the Right Generator: AI, Traditional Tooling, or Existing Frameworks
44. 23:22The Intelligence Is Selecting the Generator, Not Generating
45. 23:53Synthetic Data Is Not Yet Reusable Test Data
46. 24:24Validating Schema, Referential Integrity, Business Rules, Distribution
47. 24:54Production Signals, Test Execution and Quality Engineering Insight
48. 25:26A New Production Journey Becomes a New Test Scenario
49. 25:56The Complete Architecture as a Continuous Loop
50. 26:27Intent, Context, Reasoning, Scenario Engine
51. 26:57QE Intelligence: Past Defects, Coverage Gaps, Prior Results
52. 27:57Proving It Creates Value: Two Metrics Down, Three Up
53. 28:28Down: Provisioning Time and Production Dependency; Up: Coverage, Reusability, Detection
54. 31:01Agentic Workflows Last - and Five Shifts Worth Remembering
55. 31:33One: Production-First to Synthetic-First; Two: Records to Scenarios
56. 32:05Three: Static Data to Data Intelligence; Four: Manual to Governed Agentic Provisioning
57. 32:36Five: Standalone TDM to an Autonomous Ecosystem
58. 35:11Q&A: Does Synthetic Data Solve the Privacy Problem or Move It?
59. 35:41Q&A: Can Defects Be Prevented?
60. 36:11Q&A: How Often Should Test Data Be Updated to Stop Model Drift?

" related talks

• 32mQuality engineering in the AI age: From testing AI to leveraging GenAIVy Nguyen · XConf Lite Vietnam 2025
• 47mTest Data Key to Effective Test CoverageAshok Kumar · TestMu Conference 2025
• 44mHow AI is Exposing the Existential Crisis of Software TestingLalitkumar, Udita · TestMu Conference 2025
• 14mDesigning Secure, Compliant, & Scalable Test Data Pipelines for EnterpriseArfi Siddik Mollashaik · Commit Your Code 2025
• 43mInfluence Of Artificial Intelligence & Machine Learning In Test Data ManagementPraveen Bagare · PNSQC 2024
• 01h 05mPanel Discussion: Building Confidence in AI-Driven Testing DecisionsIdan Gazit, Asaf Saar, Guljeet Nagpaul, 𝐂𝐨𝐭𝐲 𝐑𝐨𝐬𝐞𝐧𝐛𝐥𝐚𝐭𝐡 · TestMu Conference 2025
• 09mTesting Strategies for Generative AI ApplicationsSumit Mundhada · TestFlix 2023
• 06mTriple Threat: AI Technologies Reduce Testing CostsJyotheeswara Reddy Gottam · Conf42 Observability 2025