PoC (Semantic Shift, Topic Relationships, Visualization)

Notes

• Data: The data we worked with is from d23 onwards. (2007 and after)
• Data Pipeline: Web scraping → Elasticsearch → Topic modeling / Semantic Analysis → Visualization
• Topic Modeling: BERTopic
• Semantic Analysis: What is Done is Done / WiDiD
• Initial try at Visualizations: Parliament Galaxy (3D topic-MP relationships), Heatmaps and t-SNE plots for topic analysis

Setup

1. Install dependencies: pip install -r requirements.txt
2. Setup Elasticsearch (see docs/build_elasticsearch.md)
3. Run analysis pipeline (see README.md)

Outputs

• Semantic analysis visualizations
• Topic analysis results
• MP/ - Topic engagement