~/talk

Live-Coding: Become a Web Scraping Ninja with Scrapoxy!

on YouTube 43m
TalkinFrenchat DevFest Toulouse 2024 · on Thursday, 2024-11-07· [security]

speaker

name:Fabien Vauchelles
website:https://scrapoxy.io

resources

video:https://youtu.be/aUqj0O4-qbE
slides:none on file

summary

Hands-on live-coding session demonstrating legal web-scraping techniques using Scrapoxy (an open-source proxy aggregator) and Playwright for browser automation. Covers bypassing anti-bot protections and fingerprints, reverse-engineering tracking mechanisms, and managing proxies/browser farms to collect data for feeding AI models.

AI-written summary · about

chapters (11)

  1. 0:00Web scraping intro
  2. 3:17Target site and goals
  3. 8:34Basic protections
  4. 11:45Proxies to avoid blocks
  5. 16:01Scrapoxy and orchestration
  6. 19:55Advanced protections
  7. 22:24AntiBot bypass
  8. 26:19Playwright with Scrapy
  9. 29:14Deobfuscating AntiBot JS
  10. 36:22Techniques summary and demo
  11. 36:42Q&A

related talks

~/search
type to search 6,249 conferences, 138,987 speakers and 185,172 talks
~/help
cmdctrl k
search every conference, speaker and series
/
focus the filter search
h
home
c
conferences
s
speakers
e
series
t
light / dark — the status bar button does the same
?
this help
esc
close this, then clear every filter — or just the search, while you are typing in it

Ignored while you are typing. Pages without a filter rail have no search to focus.