live
Global Financial Pulse
Prices, headlines, and a sentiment score, on a schedule.
Global Financial Pulse is an end-to-end ELT pipeline that tracks global market sentiment.
Pipeline
- Ingests Yahoo Finance prices and News API headlines with async Python, using 10 concurrent workers.
- Scores each headline with TextBlob, on a scale from -1 to +1.
- Dagster runs the pipeline every 30 minutes during US market hours, plus a daily retention cleanup.
- dbt transforms the data into a Bronze / Silver / Gold medallion architecture in PostgreSQL.
- The stack is containerized with Docker and self-hosted on a Synology NAS.
- A Streamlit dashboard reads the curated data.