You don't always need NumPy

on YouTube ↗32m

" summary

Claim: many non-numeric parts of a workflow are faster to build and more robust with standard Python or small focused tools than forcing everything into NumPy. Concrete examples: use lists, dicts, sets, tuples and collections.Counter for bookkeeping; use generators to stream huge CSVs line‑by‑line and avoid GBs of in‑memory data; use SQLite durable transactions for resumable long jobs. For heavy domain needs pick proven C libraries or databases (roaring bitmaps, sortedcontainers, Neo4j, Elasticsearch, PostGIS) rather than inventing array indirections.

AI-written summary · about

" chapters (20)

1. 0:00Introduction
2. 2:55Getting stuck
3. 3:57The fun part
4. 4:38The friction
5. 6:03The practical part
6. 6:38Bag of words
7. 8:41Inbuilt data structures
8. 9:08Lists
9. 10:19Sets
10. 11:24Dictionaries
11. 11:46Tuples
12. 12:47Semantics
13. 13:28Standard Library
14. 15:11Generators
15. 17:34Generator
16. 19:04Database
17. 20:25SQLite
18. 21:25Databases
19. 22:55Smaller libraries
20. 24:00Wrapping up

" related talks

• 26mSmall Big Data: using NumPy and Pandas when your data doesn't fit ...Itamar Turner Trauring · PyCon US 2020
• 34mSmall Big Data: using NumPy and Pandas when your data...Itamar Turner Trauring · PyData New York City 2019
• 37mThe Python Ecosystem for Data Science: A Guided TourChristian Staudt · PyCon.DE 2017
• 01mSmall Big Data: using NumPy and PandasItamar Turner Trauring · Data Love 2021
• 01h 27mMaking computation easier with cool Numpy tricksKirit Thadaka · PyData London 2018
• 45mPython for Speed, Scale, and ScienceTravis E. Oliphant · Python fwdays'20 online conference
• 29mI wish I learnt that earlier!Rachel Bunder · PyCon AU 2016
• 43mWhy do I need to know Python- I'm a pandas userJames Powell · PyData NYC 2022