Echo 📣 detection of nearly-identical tweets

In this project, a locality-sensitive hashing over minhash signatures was implemented through a Map Reduce architecture (PySpark).

Get started

Resources

context

https://help.twitter.com/en/rules-and-policies/platform-manipulation

dataset

https://www.kaggle.com/datasets/bwandowando/ukraine-russian-crisis-twitter-dataset-1-2-m-rows