Anusua Trivedi, Data Scientist at Texas Advanced Computing Center (TACC), UT Austin
summary
Built a PubMed recommender that turns articles into TF‑IDF vectors, computes cosine similarity to pick top‑3 documents for a user profile, then applies collaborative filtering (Alternating Least Squares) on Wrangler; evaluation reports lower RMSE/MAE and faster training for PySpark ALS vs scikit‑learn and Mahout/Hadoop compared to baselines. Also runs GPU deep learning on Maverick with NVIDIA DIGITS/Caffe and cuDNN for MRI classification achieving 72–82% accuracy. Core tools: Vector Space model + TF‑IDF + cosine similarity.
AI-written summary · about
chapters (19)
- 0:00Intro
- 0:50Some Basic Observations
- 1:33Data Intensive Computing & Big Data
- 2:57High Performance Computing (HPC)
- 3:31Sample TACC Computing Cluster
- 4:11Combine HPC & Data intensive computing
- 7:13TACC Ecosystem Goals
- 8:32Why use TACC Supercomputers?
- 11:51Two Sample ML workflows in TACC Supercomputers
- 13:04Deep Learning on Images
- 13:29Deep Learning on MRI using TACC Supercomputers
- 15:25What is a Recommendation System?
- 16:00Using Vector Space Model (VSM) for Pubmed
- 17:52Message Passing in HPC
- 18:25Why MPI is not the best tool for ML
- 19:08Choosing Hadoop over MPI
- 20:13Limitations of Hadoop in HPC
- 21:50Our Recommendation Model
- 23:08Performance Evaluation of Pubmed Recommendation Model
related talks
01h 21mUnlocking Personalization: A Deep Dive into Modern Recommendation Algorithms
36mAutomatic Citation generation with Natural Language Processing
38mRecommender systems with Tensorflow
28mThe Power of Personalized Customer Experiences Through AI Innovations
19mMichael Alcorn, Sr. Software Engineer, Red Hat Inc.
37mBuild a Recommender System in Apache Spark and Integrate it Using Akka
31mCreating recommender systems with Spark, Scala and Prediction.Io
37mA primer on recommendation systems.