Projects
Methods for high-dimensional, noisy data, carried over from neuroscience to manufacturing, finance and language, with hands-on work from data pipelines and model evaluation to production operations. Public projects come with full code and analysis reports, and every project was built with the help of Claude Code.
Citation-Grounded RAG System / Dissertation RAG Copilot
A RAG assistant with strict citation grounding for an academic literature corpus, with built-in measurement of the hallucination rate. Retrieval is the easy half: being able to measure the hallucination rate is what makes the system trustworthy.
- Prompt and context engineering: the generator uses an XML prompt that enforces citations, so every claim must carry a passage ID and a verbatim quote, or the model can only return a fixed refusal; the judge sees only the claim and the passages, not the generator's reasoning, to avoid self-endorsement. The judge prompt was revised version by version against errors measured on a validation set (v1 → v2 false-accept rate 27% → 4%). Later experiments were decided by the same standard: a generator prompt v2 improved trap questions but increased format errors on the 7B model, and judge prompts v4/v5 that added a classification step raised the false-accept rate, so neither was adopted. For 7–14B models, deterministic rule checks proved more reliable than ever-longer prompts.
- Hybrid retrieval (dense embeddings ⊕ BM25, fused with RRF): academic text is full of exact terms such as
ISFCandTFCEthat pure semantic vectors blur. - Chunking ablation: section-aware chunking reached Hit@5 100% / MRR 0.952 on a 26-question hand-built golden set, against 88% / 0.794 for fixed-length chunks.
- Verification layered from cheap to expensive: a verbatim quote must appear in the retrieved passage or the claim is marked fabricated; an LLM judge then rules on the claim; a separate rule check catches "planned work written up as results" and sends it to human review.
- The judge is validated too: models and prompts were compared on a known-answer set with deliberately planted errors, and phi4 was chosen (false-accept rate 5%, 95% CI 2–15%); a generator grading its own output was the weakest option.
- A claim-by-claim audit of system errors found that some misjudgments came from context rather than the model (passages missing section headings, quotes spanning passages); fixing them raised strict citation precision from 84% to 97% (30/31). Every rate is reported with a Wilson confidence interval, noting that most differences at this sample size are not significant. No false claim has been shown to the user as "supported" so far.
- Runs fully offline (local Ollama) and also supports journal-PDF corpora.
Video Feature Extraction and Temporal Alignment / fMRI Video Feature Analysis
A quantitative comparison of low-level visual, temporal, semantic and object-level information in "nature" and "urban" naturalistic videos, extracting entropy and energy measures for every TR (0.752 s).
- Built a multi-level feature pipeline with OpenCV, HOG features and YOLOv8.
- Turned raw video into regressor time series that hold up to statistical testing, for use in fMRI analysis.
Chinese Conversation Risk Triage / LINE Group Health Triage
A health-triage system for LINE work groups that analyzes conversations, scores risk and produces a prioritized PDF report, turning unstructured Chinese chat logs into ranked, actionable risk signals.
- Six indicators: age of unanswered questions, customer response delay P90 (both in business hours), oldest unresolved issue (extracted by an LLM), share of negative sentiment, message volume, and a semantic-entropy time series that tracks topic complexity.
- Non-compensatory tripwires: escalation signals such as refunds, complaints or asking for a manager trigger directly and cannot be averaged away by other healthy indicators.
- Data sources: LINE chat exports, real-time LINE Messaging API webhooks, and Telegram group exports.
- Synthetic evaluation: 100 manager groups (2,667 messages) generated with programmatically planted ground truth (risks and decoys); a local LLM only rewrites them into colloquial Taiwanese Mandarin, so scoring is against the answer key rather than another AI's opinion. The rule system is strong on timing and phi4 on semantics; combined, risk-level accuracy is 80% (rules 48%, phi4 65%). The rule system was then tuned on a tuning set and the held-out set scored only once: 94% risk-level accuracy on 50 held-out groups with zero false alarms on decoys. Thresholds were calibrated on synthetic data; the next step is re-validation on a fresh batch of groups.
- Stock-community mode (about 1.09 million Telegram messages): 165 hand-labeled messages validate the LLM's bullish/bearish labels, with LLM–human agreement of Cohen's κ up to 0.42 (random sample) / 0.55 (enriched sample); a ticker-entropy time series predicts topic shifts in the next window, AUC 0.62 → 0.71 (95% CI excludes 0). Along the way I caught a sample-size confound that explained most of an earlier AUC of 0.82.
Conference Registration and Payment System / PACLIC 40 Registration
The online registration and payment system for the 40th PACLIC (Pacific Asia Conference on Language, Information and Computation, held at National Taiwan University in December 2026), live since October 1, 2026. Built on an existing conference registration system with a team; I was responsible for backend features and production operations.
- Flow: Google sign-in → registration form → student-ID upload for students → card payment through NewebPay; organizers review student IDs and export attendee lists as CSV.
- Table-based pricing: the early-bird/regular gap in the new fee schedule differs across tiers, so no single discount formula fits; early-bird eligibility is set by payment completion time, and registration and payment close automatically at the deadline.
- Deliberate design: amounts are computed only on the backend from the registration contents, and any amount sent from the frontend is ignored; student-ID review is decoupled from payment so review never blocks it.
- Privacy: student IDs are deleted automatically after the conference according to the retention period.
- Deployment and operations: Docker Compose + Nginx, GitHub Actions CI, a production deploy that requires manual confirmation, daily encrypted database backups synced to the cloud, and restore drills.
Semiconductor Quality and Yield Prediction / Semiconductor Quality ML
Anomaly detection and yield prediction across three public manufacturing datasets: static snapshots (SECOM), process time series (PHM 2016 CMP) and production-route graphs (Bosch). The core argument: method choice should follow the structure of the data, not the trend.
- MSE 6.24 / MAE 1.82 on the official PHM CMP competition test set.
- Production-line graph analysis on Bosch located station L3_S32 with a failure rate 7.8× the overall rate: graph structure helps attribution but not prediction.
- A time-ordered deployment simulation on SECOM compared four retraining strategies before go-live and quantified whether the data could support the decision (only 41 failures across 8 windows), avoiding retraining costs whose benefit cannot be verified.
- Diagnostics gate model selection: for example, the CMP virtual-metrology R² of 0.993 drops to 0.897 within the main MRR cluster, and a 1D-CNN that was best on validation was worst on test with a 49% spread across seeds, keeping gains that come from data distribution or overfitting out of deployment.
Precision-Weighted Multi-Agent Investment Research / Precision-Weighted Multi-Agent
Borrows precision weighting from the predictive-processing framework in neuroscience: expert agents for financial statements, news, macro and fund flows, supply chain (knowledge graph) and technicals each produce a judgment with a confidence value, and an arbiter weights them dynamically by each agent's track record (the inverse of a Brier-score EMA). LLM reasoning can run on Claude or a local qwen3:8b.
- Survivorship-bias-free universe: 271 stocks, the top 100 by trading value each year including delisted names, for 8,475 walk-forward predictions over 2019–2026; financial statements carry a publication lag to rule out look-ahead.
- Key finding: precision weighting only works if each source's accuracy can be estimated reliably; measured per-period IC has a standard deviation four times its mean, so estimation error is as large as the signal, and the system now centers on the consistently performing financial-statement agent. Benchmarked against 0050 on risk-adjusted returns, the stock-selection strategy's Sharpe has not yet beaten the market.
- Working signal: within-sector selection by the financial-statement agent (IC +0.029, t 2.16) is positive in both sub-periods and across all 21 sampling start points, and a sector-neutral long portfolio earns +0.26% per period after costs; the out-of-sample effect is smaller and is being confirmed by forward validation with frozen rules.
- Monthly revenue surprise (SUE): chosen from the Taiwan-market literature and aligned to filing deadlines, it is the first signal significant in both 2019–2022 and 2023–2026 (within-sector IC +0.032 / +0.059, t 2.0 / 2.9) and still works with entry delayed 1–10 days; mixed half-and-half with a 0050 timing position, its Sharpe beats 0050 in both periods. It is frozen as the second forward specification (fv2) and recorded automatically every month from October 13, 2026; adding earnings surprise or holding longer did not help.
- Overfitting control: rules were frozen on October 9, 2026 with the evaluation plan registered in advance; any change to the code fingerprint blocks predictions, and records are append-only. The formal forward record accumulates from November 2026. Windows Task Scheduler runs prediction and settlement unattended every month and rebuilds the research dashboard when new results arrive.
🎮 Side Projects
Taiwan Gold Volley / Browser Volleyball Game
A client-only 2D volleyball game starring 16 Taiwanese animals, playable against the computer, two players on one device, or online, with an animal guide built on TaiCOL conservation data. It supports keyboard, touch and gamepad, and installs as a PWA for offline play.
- Deterministic physics engine with rollback netcode: both peers run the same engine, exchange only 1 byte of input per frame and reconcile with a state hash every 30 frames, connected peer to peer over WebRTC (PeerJS) with no backend.
- Automated quality gates: unit tests cover determinism, physics, rollback and key bindings, and Playwright end-to-end tests run on Chromium and WebKit; every push to main is built and deployed to GitHub Pages only after all tests pass.
More code and work in progress: github.com/JamesLeeCY