Design Twitter trends

1. trending topic detection
-simple counting: tokennization(n-gram, dictionary based, noun-phares, hashtag), sliding window, remove stop words, compare with historical statistics,
-chi-square test
-topic specific models


2. event detection
- online clsustering
- online clustering using MinHash

3. Natural language processing for tweets

Reference

Trend Detection In Twitter Social Data https://www.youtube.com/watch?v=duHxpSTmwW0
slides http://blogs.ischool.berkeley.edu/i290-abdt-s12/files/2012/08/Kostas_Trends_Sept_13_2012.pdf

 Predicting the Present with Google Trends http://people.ischool.berkeley.edu/~hal/Papers/2011/ptp.pdf

• Implementing Real-Time Trending Topics With a Distributed Rolling Count Algorithm in Storm
• Early detection of Twitter trends explained