Repository navigation
Claude/trusting knuth 3zpn12 - #1
Merged
Merged
Conversation
- analysis/: reproducible scripts (data_profile, model_eval, verify_dashboard_numbers) with pinned requirements and outputs - docs/: ARCHITECTURE, DATA_REPORT, MODEL_REPORT, ISSUES - No existing project file is modified (Phase 5 checkpoint pending) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018R6EHZMccpcZ9X9YoY1uUq
The value was a real-looking key, not a placeholder. Replacing it here does not remove it from git history: the key must be revoked at fred.stlouisfed.org. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018R6EHZMccpcZ9X9YoY1uUq
- pipeline/volatility.py: one feature/target definition (trailing 5/22/60d realised vol, target = std of next 5 log returns), HAR log-linear model serialised as JSON (no pickle/sklearn at inference), input validation - pipeline/stats.py: circular-shift permutation test (valid under autocorrelation) and calibrated AR(1) null - pipeline/market_dynamics.py: MFI, shocks and regimes on real data only, with burn-in, past-only thresholds and within-spell autocorrelation - pipeline/sentiment.py: FinBERT polarity resolved by label name (fixes P(neutral)-P(positive) bug), cleaning that keeps numbers/symbols - tests/: 29 unit tests incl. no-look-ahead and false-positive-rate checks Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018R6EHZMccpcZ9X9YoY1uUq
- run_pipeline.py: purged walk-forward (TimeSeriesSplit gap=5) comparing HAR, Random Forest, persistence and mean baselines; fits and saves the served HAR models as JSON; computes MFI and shock propagation on REAL prices; recomputes GMSI significance under a calibrated AR(1) null - legacy sentiment RF: drop rows only for used columns, document the 5-day target, purge gap, refuse to train when n < 2*min_samples_leaf (the cause of the constant models/rf_btc.pkl, now deleted) - remove dead pipeline/dataset_builder.py - analysis/ reads audited artefacts from commit e1eb270 via git show - pinned research requirements (Python 3.12+) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018R6EHZMccpcZ9X9YoY1uUq
scripts/08 silently fell back to seeded synthetic GARCH data because its input files are produced by no code in the repo; those synthetic figures and numbers were published as real BTC/NIFTY results. It now reads data/raw prices, fails loudly on missing inputs, runs regime statistics only when a real GMSI file is passed (--gmsi), and simulates only under an explicit --demo flag into reports/figures/demo/ with a watermark. Regenerated fig1/fig5/fig6 from real prices; deleted fig3/fig4 (regime results that need the uncommitted GMSI series). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018R6EHZMccpcZ9X9YoY1uUq
- validate_gmsi.py: circular-shift permutation test (keeps autocorrelation) replaces the i.i.d. shuffle, which flags independent persistent series as significant ~55% of the time; i.i.d. p still printed for reference - regime_analysis.py: expanding past-only regime and shock thresholds, lag-1 autocorrelation only over consecutive days within a regime spell - both fail with a clear message when the uncommitted GDELT-derived inputs are missing Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018R6EHZMccpcZ9X9YoY1uUq
- repo-relative paths everywhere (scripts mixed repo-root and ../ CWDs) - GDELT download/filter scripts get a main() and CLI date/quarter args instead of running at import with hard-coded dates - construct_vsi_full: full rolling windows (min_periods=1 gave stds from 1-2 points) and literal '\\n' in log output - reconstruct_gmsi: document the real method (equal-weight expanding z-scores, no PCA) and print component coverage Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018R6EHZMccpcZ9X9YoY1uUq
- annualise crypto vol with sqrt(365), others with sqrt(252) - never serve the in-progress month from the parquet cache - switch deprecated quandl client to nasdaq-data-link Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018R6EHZMccpcZ9X9YoY1uUq
Each notebook gets a header listing its known issues and stating that its saved outputs predate these fixes (inputs in data/processed/ are not committed, so they cannot be re-run here). Code fixes: - 03: FinBERT polarity by label name (was P(neutral) - P(positive)) - 02: cleaner keeps numbers/symbols and drops non-English headlines - 06: target is std of r[t+1..t+5] (was 4/5 already-observed days) - 07: target scaler fit on the training part only - 10: missing numpy import; NIFTY lead-lag cell echoed BTC - 11: EventRootCode compared as strings (conflict/economic/political counts were always zero) Removed notebooks/08_market_dynamics_analysis.ipynb (unexecuted copy of the old synthetic-fallback script) and the empty Untitled.ipynb. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018R6EHZMccpcZ9X9YoY1uUq
- remove all simulated data, hard-coded results and the fabricated null distribution; every number is read from run_pipeline.py outputs - new Volatility forecast page: serves the HAR model (cached with st.cache_resource) for any date in the data or user-pasted closes, with friendly validation errors, plus walk-forward model comparison - GMSI page reports corrected significance; MFI and shock pages use real prices - landing page renders in ~2-4 s (was 185 s: 620 quadratic add_vrect calls); split into data layer + one module per page - pinned dashboard requirements (tested on Python 3.11-3.14), dark theme via .streamlit/config.toml, removed ignored runtime.txt and the duplicated dashboard/assets copies of reports/figures - AppTest suite (all pages, input validation) and a Playwright smoke test Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018R6EHZMccpcZ9X9YoY1uUq
- delete empty src/*.py and docs/methodology.md placeholders - de-duplicate the repeated 2025-11/12 rows in nifty_google_trends.csv - merge the two overlapping report walkthroughs into one .md with relative image paths and a correction banner - reports/README.md: generator, data and status of every figure Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018R6EHZMccpcZ9X9YoY1uUq
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018R6EHZMccpcZ9X9YoY1uUq
- README states results computed from real data and the corrections made - docs/DEPLOY.md: step-by-step Streamlit Community Cloud deployment with local verification evidence and troubleshooting - docs/ARCHITECTURE.md Part A: current architecture, training/inference boundary and request path; ISSUES.md: resolution table per issue - root requirements renamed to requirements-research.txt so Cloud can only install dashboard/requirements.txt; ignore .streamlit/secrets.toml Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018R6EHZMccpcZ9X9YoY1uUq
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018R6EHZMccpcZ9X9YoY1uUq
- INTERVIEW_DEFENSE.md: 2-minute and 30-second pitches, 48 questions with answers grounded in pipeline outputs, honest weak-spot lines, glossary - docs/README.md: index, recommended reading order, reproduction commands Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018R6EHZMccpcZ9X9YoY1uUq
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
This branch had an error being deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.