Beyond Manual Hunting: OSINT Copilot with LLMs & Evidence-Based Collection, Enrichment ...
summary
A pipeline turns chaotic forum and marketplace scraping into searchable intelligence: HTTP clients and browser automation handle scale and bot defenses, deterministic extractors pull IOCs and entities, semantic LLMs produce summaries, risk flags and search vectors, and a graph+vector index links events for deep queries. Uses open tooling (benchmark on ~10,000 events), runs small LMs for cost and privacy control like vllm and MPT, stores documents and vectors in Elasticsearch, and agents query the DB to build relation graphs and prioritize high-risk, monetized events.
AI-written summary · about
chapters (14)
- 0:00OSINT Copilot intro
- 1:49Chaotic data to intelligence
- 3:56Lightweight vs browser automation
- 6:08Entity extraction analysis
- 6:50Semantic analysis with LLMs
- 8:14Small LLMs scale better
- 9:59Structuring events enrichment
- 11:04Elasticsearch indexing
- 12:49Prioritizing high-risk events
- 13:35Bootstrap monitoring setups
- 14:59Deep search with graphs
- 19:19Pipeline collection to reporting
- 20:45Benchmarking small LLMs
- 21:27Q&A
related talks
28mSupercharging Threat Intelligence with AI
48mHunting Stealth Adversaries with Graphs & AI
22mAgentic Hybrid Retrieval: Orchestrating LLM Agents, Hybrid Search & Explainability in OpenSearch
28mSecuring LLMs: Insights into OWASP top 10
38mNext-Gen Detection: Harnessing LLMs for Sigma Rule Automation
30mCybersecurity Risk Assessment Using LLM Agents and Graph Data Science
17mOSINT to Compromise
28mAI Agents in Action: Harnessing MCP and OpenSearch for Modern Use Cases