Data Ingestion Flow
CSV Files
AS400 / Email / Upload
→
Inbox Folder
data/inbox/stock/
→
Pipeline
Validate & Deduplicate
→
SQLite Database
data/reports.db
Processed Folder
data/processed/
←
Pipeline
Moves files after processing
←
Failed Folder
data/failed/
Remote Access Flow (Cloudflare Tunnel)
Web Browser
Dashboard & Upload
Power BI
/api/data/stock
Excel
/export/stock
↓
Cloudflare Tunnel
HTTPS (encrypted)
↓
Flask Web Server
localhost:5000
↔
SQLite Database
data/reports.db
Automatic Processing Flow (Power Automate)
Email Arrives
With CSV attachment
→
Power Automate
Saves attachment
→
OneDrive
Syncs to server PC
→
File Watcher
Auto-detects new files
→
Database
Data ready!
Connection Reference
Web Dashboard
Local: http://localhost:5000
Remote: https://your-tunnel-url
Power BI (JSON API)
Get Data → Web
/api/data/stock
/api/data/inbound
/api/data/outbound
Excel (CSV Export)
Data → From Web
/export/stock?format=csv
/export/stock?format=excel
Health Check
Monitor server status:
/api/health
/api/status
System Components
Database
Files / External
Client Applications