Skip to content

USER_GUIDE.md

Latest commit

 

History

History
353 lines (261 loc) · 23.8 KB

USER_GUIDE.md

File metadata and controls

353 lines (261 loc) · 23.8 KB

User Guide

Welcome to the CairnIQ User Guide. This document provides a complete walkthrough of the application's capabilities, its high-performance interfaces, background operations, cognitive systems, and data infrastructure.

Table of Contents

  1. Core Capabilities & Navigation
  2. Chat Interface: Toggles & Quick Actions
  3. File Uploads & Portfolio Management
  4. Thesis Journal & Active Theses
  5. Active Thesis Health & Live Alerts
  6. Context Graph (Semantic Knowledge Base)
  7. User Memory & Custom Instructions ("My Instructions")
  8. Live News Feed & Market Pulse
  9. AI Provider Setup & Keychain Security
  10. Data Source Integration Matrix
  11. Troubleshooting & Diagnostics

1. Core Capabilities & Navigation

CairnIQ is a private, local-first financial advisor agent framework that combines real-time market data, deep cognitive analysis, and structured memory tracking. It operates as a multi-agent system where a Supervisor routes user queries to specialized reasoning engines (news, fundamentals, portfolio metrics, risk profiles, macro indicators).

Interface Layout

The dashboard is split into three high-density components:

  • Left Navigation Sidebar: Toggle between primary application views:
    • Chat: The main conversational workspace.
    • Context: The semantic knowledge graph, user memory facts, and custom instruction sets.
    • Journal: The trade journal showing active theses, catalysts, and conviction tracking.
    • Portfolio Editor: The manual holdings grid, where you can modify rows, upload ledger CSVs, or download templates.
    • Settings: Infrastructure management panel for credentials, API tokens, region locales, base currencies, and model IDs.
  • Center Panel: Displays the active workspace (the interactive chat interface, context editor, journal logs, or holdings sheet).
  • Right Information Sidebar:
    • Portfolio vs. Benchmark Chart: A real-time sparkline comparing your portfolio's performance against the S&P 500 (SPX).
    • Market Pulse Briefing: A snapshot of current market regime, streak days, volatility indicators (VIX), and overall Fear & Greed levels.
    • Primary News Feed: Real-time summaries of market events affecting your current holdings.

2. Chat Interface: Toggles & Quick Actions

The chat workspace is designed for rapid execution and deep strategy lookup. It features specialized modes and quick action macros to automate common analysis workflows.

+--------------------------------------------------------------------------------+
|  [Input Area]                                                                  |
|  [ Type your question or stock symbol here...                       ]          |
|                                                                                |
|  [ + Attach File ]               [ GHOST (Off) ]   [ DEEP (Off) ]   [ Send ]   |
+--------------------------------------------------------------------------------+

Analysis Mode Toggles

Directly below the chat input box, there are two primary toggles:

  • DEEP Toggle: Enables Deep Analysis mode.
    • Functionality: Maps the query to Detailed (Deep Analysis) response parameters. It routes tasks to the primary, high-capacity reasoning model slot (AIDLC_MODEL_ID) rather than fast models.
    • Use Case: Use when conducting macro portfolio stress tests, multi-year asset allocations, or searching for hidden correlations.
    • UI Indicator: Highlights in solid primary green/emerald with a subtle outer shadow when enabled.
  • GHOST Toggle: Enables Privacy Mode.
    • Functionality: Skips the memory write path entirely. No facts, preferences, or profile changes are saved to user_memory.json or mirrored to the knowledge graph.
    • Alternative Triggers: You can also force Ghost Mode by typing tags like @Private, @Ghost, [Private], or No capture directly inside your message text.
    • UI Indicator: Highlights in amber when enabled, displaying a status badge: 🛡️ Ghost Mode Active: Conversation will not be recorded in memory.

Chat Quick Actions

Quick Actions are macro buttons positioned above the input box that auto-inject complex, pre-formatted instructions into the agent loop.

  1. Today's Priority (btn-priority):
    • No Ticker Input: Executes a 4-stage portfolio triage check:
      • Stage 1 (Regime & Secular Themes): Resolves the current market posture and maps it against user-defined structural themes (secular holdings).
      • Stage 2 (Portfolio Health): Verifies asset distributions (Equities, Fixed Income, Cash) using bounds dynamically calculated from your risk tolerance and retirement horizon (e.g., Aggressive/15yr+ target is 70–85% equities, shifting down 10% in defensive regimes).
      • Stage 3 (Triage): Applies prioritised decision rules: checks for structural risk (secular trim triggers met, concentration >8%, thesis-breakers), identifies captured opportunities (high cash levels + positive technical triggers), executes lazy rebalancing, or defaults to "DO NOTHING".
      • Stage 4 (Position Sizing): Computes specific dollar allocations, shares to trade, and stop levels based on a strict 2% risk limit.
    • With Ticker Input (e.g., "AAPL" in chat input + click): Appends a FOCUS directive that runs the same full framework, but specifically asks whether acting on that ticker today is the single highest-priority action given your current holdings and market conditions.
    • Output structure: A single ⭐ TODAY'S PRIORITY action (in bold caps, or DO NOTHING — which is the most common, intended answer), a 2-bullet 🎯 WHY, an 📋 EXECUTION table (omitted when DO NOTHING), an 👀 OPPORTUNITY RADAR (up to two awareness-only items), and a 🔔 NEXT-CHECK TRIGGER (one observable level/event that should make you revisit). Trim/sell candidates must pass verify_portfolio_holdings, and tax language is suppressed when all accounts are tax-sheltered.
  2. Analyze (btn-analyze):
    • Sends: [MarketAnalyst lens=portfolio_audit] Audit my current holdings for uncompensated risk.
    • Audits your actual holdings (via assess_portfolio_risk, check_portfolio_allocation, analyze_fx_risks, check_portfolio_correlation, and get_market_pulse_data) and leads with the single biggest dollar-at-risk finding. It deliberately suppresses external screen picks unless your cash is above band (and even then, one suggestion max).
  3. Scan (btn-scan):
    • Sends: [MarketAnalyst lens=external_screen] Find new external tickers worth attention right now.
    • The mirror image of Analyze: hunts for new external tickers (via screen_stocks, scan_opportunities, and detect_sector_rotation), leads with the single highest-conviction pick and a "why now," and tags each with a one-line portfolio-fit note (held / watchlist / sector-gap / overlap). It does not produce a portfolio audit.
  4. Guru Picks (btn-guru):
    • Sends: [MarketAnalyst lens=guru_validation] Validate which Media Guru picks survived the opportunity pipeline.
    • (Appears only when guru picks are enabled) Runs scan_guru_picks, shows a count_cleared / count_scanned tally, and for each cleared pick lists the guru source, signal, freshness, pipeline status, and foundation/headwind result. Picks that failed the pipeline are not presented as passing.
  5. Dip Plan (btn-dip):
    • Sends: [MarketAnalyst lens=watchlist_dip] Identify entry levels for watchlisted tickers pulling back into actionable zones.
    • For each watchlisted candidate (using run_technical_analysis and analyze_patterns) it gives an entry zone, a structural stop anchored to support / 40-week MA / ATR, and what would invalidate the dip thesis — quoting available cash. It will not pitch tickers you don't own and haven't watchlisted.
  6. System Health (btn-health):
    • Sends: Check system health and portfolio integrity.
    • Synthesizes live connection statuses, key expirations, and logs fallback database paths (FAISS vs. BM25).
  7. Market News (btn-news):
    • Sends: [NewsAnalyst] Summarize the most impactful market events from the last 24 hours.
    • Triggers the News Analyst to compile macro summaries and specific sector headlines.
  8. Trump Yap (btn-trump):
    • Invokes the Political & Social Media Market Impact Analyst via the Deep Reasoning engine.
    • No Input: Calls the get_latest_trump_yaps tool to pull Donald Trump's most recent Truth Social posts in real time (falling back to a web search for recent statements if the feed is empty).
    • With Input (a pasted post, headline, or policy announcement in the chat box + click): Analyzes that specific statement directly instead of fetching the feed.
    • Output: Maps the statement to market and supply-chain connections — identifying Winners (beneficiaries) and Losers (vulnerable targets) across affected sectors, commodities, and tickers (e.g. tariff threats, chip policy, energy deregulation, EV incentives, defense budget, FX moves). It then audits your portfolio for exposure to the affected names and proposes sized, stop-anchored trigger moves (hedges, trims, or opportunistic buys).
    • If no recent post or catalyst is found, it returns Data Unavailable: No recent social statements or news catalysts found. rather than fabricating a thesis.
  9. Systemic Risk (btn-risk):
    • Sends: [RiskManager] Calculate my total systemic risk exposure.
    • Calculates beta exposure, correlation clusters, and macroeconomic interest rate/inflation sensitivities.

3. File Uploads & Portfolio Management

CairnIQ supports two distinct upload mechanics: chat attachments for ad-hoc context loading, and structured portfolio ledger uploads for tracker updates.

Chat File Attachments

You can supply external context directly inside the chat interface:

  1. Click the paperclip icon (#chat-attach) next to the composer or paste an image directly from your operating system clipboard.
  2. Supported formats include PDFs (application/pdf), Text files (text/plain), and Images.
  3. The UI encodes files as Base64 strings and appends them to the /api/chat request body. The agent parses documents using its fast reasoning slot, injecting their text contents directly into the chat prompt context.

Portfolio CSV Uploads

To upload your holdings in bulk:

  1. Navigate to the Portfolio Editor page.
  2. Click Download Template to download the default schema, or construct a CSV file with the following header:
Symbol,Shares,Purchase Price,Account
AAPL,100,150.00,TFSA
MSFT,50,300.00,RRSP
GOOGL,25,140.00,Taxable
  • Symbol: Stock ticker symbol (e.g., AAPL, VTI, or Canadian assets like SHOP.TO).
  • Shares: Decimal or integer quantity held.
  • Purchase Price: Average price paid per share (quoted in your profile's base currency).
  • Account: Free-text grouping label (e.g. TFSA, RRSP, 401k, IRA, Taxable). This allows the agent to identify tax shelters and adjust its suggestions (e.g., suppressing tax-loss harvesting language if all positions reside in tax-sheltered accounts).
  1. Click Upload CSV to upload your file. It replaces user_data/my_portfolio.csv and flushes the portfolio cache instantly.

Important

Live Broker Precedence (Deduplication): If you have connected a live brokerage (Questrade or Alpaca) in settings, its holdings are pulled at runtime and merged with your manual CSV file. To prevent double-counting, live broker positions take precedence. If a ticker is found in both the manual CSV and the live broker sync, the live broker's position size and basis will override the manual CSV entry.

4. Thesis Journal & Active Theses

The Thesis Journal serves as a record of your active strategies. It helps prevent short-term market noise from disrupting long-term investment goals.

graph LR
    A[AI Response containing Recommendation] -->|User clicks PIN icon| B[LLM extracts Structured Thesis]
    B --> C[Active Theses Memory]
    C --> D[Thesis Journal UI]
    C --> E[Injected into Agent System Prompts]
Loading

Pinning a Thesis

When the AI agent provides a stock recommendation or strategic setup:

  1. Hover over the AI's response message block.
  2. Click the Pin (Thumbtack) icon on the message toolbar.
  3. The system calls the /api/memory/extract_thesis endpoint, which prompts an LLM via a structured DSPy interface to isolate the key decision factors from the text.
  4. The extracted data is stored in the active_theses list inside user_memory.json and immediately renders on the Journal page.

Extracted Thesis Fields

Every pinned thesis contains the following fields:

  • symbol: The target ticker (e.g., AAPL).
  • action: The planned action (BUY, SELL, HOLD).
  • quantity: Target shares or dollar amount (if specified).
  • conditions: The tactical entry rules or valuation criteria (e.g., "Buy if price drops below $170").
  • catalyst: The core driving event (e.g., "Q3 earnings acceleration", "Product launch").
  • catalyst_date: Expected date of the catalyst.
  • stop_loss: Stop-loss level (e.g. "$155 close").
  • target_price: Expected upside exit target.
  • expiry_date: Time horizon for the thesis, after which it should be reviewed.
  • notes: Background information and conversational logs.

5. Active Thesis Health & Live Alerts

Once saved, active theses are not just static records. The system actively monitors their health by enriching them with real-time price context.

Dynamic Drift & Volatility-Adjusted Thresholds

The system continuously evaluates your active theses against live stock data:

  1. Extracting Target Prices: Parses dollar figures from your saved thesis conditions and stop-loss entries.
  2. Tracking Drift: Calculates how far the current price has drifted from your specified entry zones: $$\text{Drift %} = \frac{\text{Current Price} - \text{Target Entry}}{\text{Target Entry}} \times 100$$
  3. Volatility (Beta) Adjustments: To prevent false alarms on high-beta names, the system adjusts the "Entry Missed" threshold based on the stock's beta:
    • Low Volatility ($\text{Beta} < 0.9$): Tight 5% threshold.
    • High Volatility ($\text{Beta} > 1.2$): Wide 15% threshold.
    • Standard Assets ($0.9 \le \text{Beta} \le 1.2$): Default 8% threshold.

Thesis Health Flags

Based on these calculations, the system appends diagnostic flags:

  • 🚨 ENTRY MISSED: The price is above the target entry zone by more than your volatility-adjusted threshold.
  • ⚠️ HOVERING ABOVE ENTRY: The price is above the entry zone but remains within the threshold, indicating the entry may trigger soon.
  • 🚨 STALE: The thesis has not been updated in over 30 days.

System Prompt Injection

These health flags are injected into system prompts under the === 📌 ACTIVE INVESTMENT THESES (PRIORITY) === header.

When you ask the AI a question (especially when using the Today's Priority quick action), the Supervisor reads these flags. If a thesis is flagged as ENTRY MISSED or STALE, the agent is instructed to proactively point this out and recommend either updating the thesis, adjusting the entry zone, or closing it.

6. Context Graph (Semantic Knowledge Base)

The Context page displays an interactive, semantic knowledge graph powered by NetworkX and rendered via D3.js. This graph maps how your holdings, sectors, macroeconomic events, and preferences connect.

       [Sector: Technology]
              ^
              | IN_SECTOR
          [Stock: AAPL] <=== CORRELATED_WITH ===> [Stock: MSFT]
              ^
              | EXPOSED_TO
       [Portfolio: User] -- HAS_RISK_TOLERANCE --> [Aggressive]

Automation & Graph Maintenance

  • Portfolio Integration: The system automatically adds relationships like Stock --IN_SECTOR--> Sector and Portfolio --EXPOSED_TO--> Sector based on your holdings. It also calculates top correlations and links highly correlated assets (correlation coefficient $&gt;0.7$) with a CORRELATED_WITH edge.
  • Staleness Pruning: Edges can be configured with a expiration date (using a stale_after_days parameter). The graph manager regularly runs pruning tasks to expire old edges.
  • Orphan Cleanups: When edges expire, any nodes that have no remaining connections, no "owned" flags, and are classified as "Unknown" or "Theme" are automatically removed to keep the graph clean.

Manual Customization in the UI

You can manually edit the graph from the Context view:

  • Create Node: Input an Entity Name (e.g., AMD or Inflation) and select its Entity Type (Ticker, Sector, Event, Macro Indicator).
  • Create Relationship: Connect two existing nodes by selecting a Source node, a Target node, and inputting a Relation Type (e.g., Competes with, Depends on, Correlated with).
  • Inspector Tools: Click on any node or edge in the graph canvas to open the Inspector Panel. You can view details, delete the entity, or break specific relationship links.

7. User Memory & Custom Instructions

CairnIQ uses a dual-memory system: a structured memory file for user profile facts, and a custom instructions list for user-defined guardrails.

User Memory Structure

The flat memory manager loads and saves data to user_data/user_memory.json. It tracks:

  • user_profile: Demographics (Age, Income, Risk Tolerance, Goals, Base Currency).
  • key_facts: Bulleted details about your preferences (e.g., "Prefers value plays", "Avoids biotech").
  • past_recommendations: Logs previous recommendations to keep responses consistent.
  • conversation_summaries: High-level summaries of past chats.

Custom Instructions ("My Instructions")

Located under the My Instructions feed on the Context page, this tool allows you to log rules and preferences that override default agent behaviors.

  • How it Works: Corrective lessons or rules you add in the UI are saved to the lessons_learned array in user_memory.json. The memory manager formats these as -- USER LESSONS -- and injects them directly into the agent's system prompts.
  • Examples of Custom Rules:
    • "Never recommend trimming a stock unless single-name concentration exceeds 8%." (Overrides default rebalancing calculations).
    • "Suppress tax loss harvesting suggestions." (Forces the agent to ignore tax-related triggers).
    • "Prioritize cash accumulation over buying new equities." (Modifies the portfolio triage logic).

8. Live News Feed & Market Pulse

To keep the UI responsive, expensive market calculations and news summaries are run by background workers and cached daily.

Live News Feed

  • Background Worker: The news-feed-worker thread runs the @NewsAnalyst agent in the background. It fetches headlines for all your holdings, filters out noise, and compiles an Intelligence Report.
  • Caching & Polling: The report is saved to user_data/daily_cache/. The web UI polls the /api/news-feed endpoint every 4 seconds. If the worker is still generating, the UI displays a loading spinner showing the progress (e.g. (3/20)).
  • Forcing Refresh: Click the refresh button (btn-refresh-news) on the News panel to clear the cache and trigger a fresh run.

Market Pulse

  • Background Worker: The market-pulse-worker generates a daily macro briefing.
  • Calculations: Scans indexes to determine the current macro regime (Defensive, Balanced, Constructive, Euphoric), calculates a regime score out of 100, checks the VIX, and pulls the Fear & Greed index.
  • UI Updates: The briefing displays in the right sidebar. If the market regime changes, it adjusts your portfolio allocation bands accordingly (e.g., reducing equity bands during a Defensive regime).

9. AI Provider Setup & Keychain Security

CairnIQ splits reasoning tasks between two model slots to optimize response times and API costs.

Dual-Model Setup

  • Primary Model (AIDLC_MODEL_ID): High-capacity models (e.g., Claude 3.5 Opus, GPT-4o) handle complex tasks like portfolio triage, risk assessments, and rebalancing recommendations.
  • Fast Model (AIDLC_SONNET_MODEL_ID): Faster models (e.g., Claude 3.5 Sonnet, GPT-4o-mini) handle routine tasks like news summarization, health checks, and semantic searches.

Key Providers

  1. AWS Bedrock (Recommended for enterprise / data privacy):
    • ARN-style IDs: e.g., global.anthropic.claude-3-5-sonnet-v2-0.
    • Uses cross-region inference profiles (global.*) to optimize rate limits.
  2. Anthropic (Direct API):
    • Bare IDs: e.g., claude-3-5-sonnet-latest.
  3. OpenAI (Direct API):
    • IDs: e.g., gpt-4o, gpt-4o-mini.

Keychain Security

To protect your API keys:

  • Keychain Access: By default, CairnIQ encrypts and stores sensitive API keys in your OS keychain (macOS Keychain Access, Windows Credential Manager, Linux Secret Service).
  • Plaintext Fallback: If keychain access is unavailable (e.g., running on headless servers, Docker containers), credentials fall back to user_data/.env in plaintext.

10. Data Source Integration Matrix

CairnIQ uses several financial data APIs to populate the dashboard and feed analytical tools.

Provider Purpose Status Cost Notes
AlphaVantage Market quotes, FX rates, charts Required Free / Paid Free key (25 req/day) covers basic needs.
FMP Fundamentals, SEC filings, transcripts Nice-to-Have Free / Paid Unlocks deep earnings and insider trading analysis.
FRED Macroeconomic indicators Nice-to-Have Free Unlocks interest rates, CPI, inflation. Falls back to cached data if missing.
Finnhub Sentiment & recommendations Nice-to-Have Free Unlocks social sentiment and analyst target prices.
Polygon.io Options chains & technical indicators Nice-to-Have Free Unlocks advanced indicators (RSI, MACD) and options.
Tavily Search engine for current events Nice-to-Have Free Unlocks general web searches for the news analyst.
Questrade Syncing Canadian holdings Nice-to-Have Free Connects to Questrade via OAuth refresh tokens.
Alpaca Syncing US holdings & paper trading Nice-to-Have Free Connects to Alpaca paper/live accounts.

11. Troubleshooting & Diagnostics

If you encounter issues, follow these steps to isolate the cause:

System Diagnostics

  1. Open the Chat page.
  2. Select the Health quick action button, or type: 
    @DeepReasoning Run health check
  3. The health suite will verify your API configurations, check network latency, and output a diagnostics table.

Common Solutions

  • API Rate Limits: If your FMP or AlphaVantage keys are hit by rate limits, the credential manager will place them on a 5-minute (300s) cooldown. You can configure backup keys in user_data/.env (e.g., FMP_API_KEY_2, FMP_API_KEY_3) to enable automatic key rotation.
  • Vector DB Fallbacks: If compiling the FAISS vector database fails on your system architecture, the system will automatically fall back to BM25 keyword search for logs and chat history.
  • Log Locations:
    • Server operations: logs/server/server.jsonl
    • Tool operations: logs/tools/tools.jsonl
    • Agent steps: logs/agent/