Decoding Binary Files - a Primer (specifically optical spectra for analysis)
summary
Uses Python (struct, pandas, numpy, glob) and Jupyter to reverse‑engineer spectrophotometer binaries by comparing exported CSVs as ground truth, finds little-endian float32 spectral values beginning at start at byte 120 and continuing to file end (files ~1.3k bytes, last value near byte 1320), reconstructs wavelengths with np.linspace, handles reversed order and float rounding, decodes about six instruments, and the parsed spectra matches CSV exactly, producing a working parser for production analysis.
AI-written summary · about
chapters (17)
- 0:00Introduction and speaker
- 1:48Text vs binary files
- 3:55Binary file properties
- 5:19Spectrophotometer data overview
- 6:50Examining binary files
- 9:01Loading data in Python
- 11:09Detecting endianness and data type
- 13:41Scanning binary data
- 15:45Finding wavelength patterns
- 17:14Decoding transmittance values
- 18:42Applying pattern to other files
- 20:29Verifying decoded data
- 20:52Binary file structure notes
- 21:35Q&A session begins
- 21:56Advantages of binary over CSV
- 23:21Challenges faced
- 26:14Handling encryption in files
related talks
30mSignal Processing in Electrochemistry with Python: Applications to the US Opioids Crisis
37mUsing Python to visualize and analyze ion beam experiment data at a particle accelerator facility
35mWorking with data in Python for the Aspiring Data Scientists
01h 53mAnalyze your data in Python
21mPolars: A highly optimized dataframe library
32mEDA Techniques for Data Analysis
25mEmpowering weather-satellite data users with FOSS: the Pytroll case
41mFootball's data analytics in python