Strong Data Processing Inequalities and Estimation with Constraints
summary
Claim: a likelihood ratio bound anywhere in a Markov chain forces a strong data processing inequality that contracts mutual information by roughly (e^α−1)^2 and yields concrete KL and estimation lower bounds. Tools are mutual information, KL/Pinsker/Le Cam testing reductions and coordinate independence. Example: estimating a 1-sparse ±δ mean under B-bit communication per sample gives error Ω(min{1, D/(B N)}) for small δ, so bits must scale nearly linearly in D.
AI-written summary · about
chapters (7)
- 0:00Introduction to problem
- 0:37Constrained estimation model
- 5:19Likelihood ratio bounds
- 10:58Statistical hardness theorem
- 17:11Sparse mean estimation
- 24:05Communication constraints
- 27:51Discussion and Q&A
related talks
53mOptimality in Locally Private Estimation and Learning
59mCompositional Properties of Statistical Procedures: An Information-Theoretic View
15mFundamental Limits Of Learning In Data-Driven Problems
31mStatistical Estimation on Graphs
29mComputational Barriers to Estimation from Low-Degree Polynomials
45mTesting and Learning Distributions Under Local Information Constraints
01h 30mHigh-Dimensional Statistics I
01h 05mDistribution Estimation and Applications III