Skip to content

TROUBLESHOOTING.md

Latest commit

 

History

History
867 lines (591 loc) · 19.3 KB

TROUBLESHOOTING.md

File metadata and controls

867 lines (591 loc) · 19.3 KB
Raw

Troubleshooting Guide

OpenVox GUI Version 3.7.1-beta1

This guide helps you solve common problems with OpenVox GUI. Think of it as your "fix-it" manual - we'll start with the most common issues and work our way to more complex ones.

Table of Contents

  1. Quick Fixes (Try These First!)
  2. Login and Access Problems
  3. Service Won't Start
  4. Connection Problems
  5. Performance Issues
  6. Display and UI Problems
  7. Data and Report Issues
  8. Certificate Problems
  9. Update and Deployment Issues
  10. Getting More Help

Quick Fixes (Try These First!)

Before diving into specific problems, try these common fixes that solve 80% of issues:

1. Restart the Service

sudo systemctl restart openvox-gui

2. Check the Service Status

sudo systemctl status openvox-gui

Look for:

  • ✅ Green "active (running)" status
  • ❌ Red "failed" or "inactive" status

3. Clear Your Browser Cache

  • Windows/Linux: Press Ctrl + F5
  • Mac: Press Cmd + Shift + R

4. Check the Logs

# View recent errors
sudo journalctl -u openvox-gui -p err -n 50

# View all recent logs
sudo journalctl -u openvox-gui -n 100

5. Verify Network Connectivity

# Can you reach OpenVoxDB?
ping -c 2 openvox.yourcompany.com

# Is the port accessible?
telnet openvox.yourcompany.com 8081

If these don't fix your problem, continue to the specific sections below.

Login and Access Problems

Problem: Can't Access the Web Interface

Symptoms:

  • Browser shows "This site can't be reached"
  • Connection timeout errors

Solutions:

  1. Check if the service is running:

    sudo systemctl status openvox-gui
# Should show "active (running)"

  2. Check if the port is open:

    # See if something is listening on port 4567
sudo ss -tlnp | grep 4567

  3. Check firewall settings:

    # For Red Hat/CentOS:
sudo firewall-cmd --list-all

# For Ubuntu:
sudo ufw status

  4. Open the firewall port if needed:

    # For Red Hat/CentOS:
sudo firewall-cmd --permanent --add-port=4567/tcp
sudo firewall-cmd --reload

# For Ubuntu:
sudo ufw allow 4567/tcp

  5. Try accessing locally first:

    curl -k https://localhost:4567/health
# Should return: {"status":"ok","version":"3.7.1-beta1"}

Problem: Forgot Admin Password

Solution:

Reset the admin password:

sudo /opt/openvox-gui/venv/bin/python /opt/openvox-gui/scripts/manage_users.py passwd admin
# Enter new password when prompted

Or create a new admin user:

sudo /opt/openvox-gui/venv/bin/python /opt/openvox-gui/scripts/manage_users.py add newadmin --role admin

Problem: "Invalid Credentials" Error

Solutions:

  1. Check if caps lock is on (seriously, it happens!)

  2. Verify the username exists:

    sudo /opt/openvox-gui/venv/bin/python /opt/openvox-gui/scripts/manage_users.py list

  3. Reset the password:

    sudo /opt/openvox-gui/venv/bin/python /opt/openvox-gui/scripts/manage_users.py passwd username

  4. Check if authentication is enabled:

    grep AUTH_BACKEND /opt/openvox-gui/config/.env
# Should show: AUTH_BACKEND=local

Problem: Certificate Warning in Browser

This is normal! OpenVox GUI uses a self-signed certificate by default.

Solution:

  1. Click "Advanced" or "Show Details"
  2. Click "Proceed to site" or "Accept Risk and Continue"
  3. The warning will appear each time unless you add an exception

To use a real certificate, see the Configuration documentation.

Service Won't Start

Problem: Service Fails to Start

Symptoms:

● openvox-gui.service - OpenVox GUI
   Loaded: loaded
   Active: failed

Solutions:

  1. Check for detailed errors:

    sudo journalctl -u openvox-gui -n 100 --no-pager

  2. Common causes and fixes:

    Port already in use:

    # Find what's using the port
sudo ss -tlnp | grep 4567

# Either stop the other service or change OpenVox GUI port
sudo nano /opt/openvox-gui/config/.env
# Change APP_PORT=4567 to another port

    Permission problems:

    # Fix ownership
sudo chown -R puppet:puppet /opt/openvox-gui

# Fix permissions
sudo chmod 755 /opt/openvox-gui
sudo chmod 600 /opt/openvox-gui/config/.env

    Python dependency issues:

    cd /opt/openvox-gui
source venv/bin/activate
pip install --upgrade -r backend/requirements.txt
deactivate

  3. Test configuration manually:

    cd /opt/openvox-gui
source venv/bin/activate
python -c "from backend.app.config import settings; print('Config OK')"
deactivate

Problem: Service Starts Then Immediately Stops

Solutions:

  1. Check for configuration errors:

    sudo nano /opt/openvox-gui/config/.env
# Verify all settings are correct

  2. Check SSL certificates exist:

    ls -la /etc/puppetlabs/puppet/ssl/certs/
# Should show certificate files

  3. Verify database is accessible:

    ls -la /opt/openvox-gui/data/
# Should show openvox_gui.db

Connection Problems

Problem: "Cannot Connect to OpenVoxDB" Errors

Solutions:

  1. Verify OpenVoxDB is running:

    # On the OpenVoxDB server:
sudo systemctl status puppetdb

  2. Test connectivity:

    # From OpenVox GUI server:
ping openvoxdb.yourcompany.com
telnet openvoxdb.yourcompany.com 8081

  3. Check SSL certificates:

    # Verify certificates exist
ls -la /etc/puppetlabs/puppet/ssl/certs/*.pem
ls -la /etc/puppetlabs/puppet/ssl/private_keys/*.pem

  4. Test OpenVoxDB connection manually:

    curl --cert /etc/puppetlabs/puppet/ssl/certs/$(hostname -f).pem \
     --key /etc/puppetlabs/puppet/ssl/private_keys/$(hostname -f).pem \
     --cacert /etc/puppetlabs/puppet/ssl/certs/ca.pem \
     https://openvoxdb.yourcompany.com:8081/pdb/query/v4/nodes

  5. Check configuration:

    grep PUPPETDB /opt/openvox-gui/config/.env
# Verify hostname and port are correct

Problem: "Cannot Connect to OpenVox Server" Errors

Solutions:

  1. Verify OpenVox Server is running:

    # On the OpenVox Server:
sudo systemctl status puppetserver

  2. Check configuration:

    grep PUPPET_SERVER /opt/openvox-gui/config/.env

  3. Test connection:

    curl -k https://openvox.yourcompany.com:8140/puppet/v3/environments

Performance Issues

Problem: Dashboard Loads Slowly

Solutions:

  1. Increase worker processes:

    sudo nano /opt/openvox-gui/config/.env
# Change UVICORN_WORKERS=2 to 4 or 8
sudo systemctl restart openvox-gui

  2. Check system resources:

    # CPU and memory usage
top

# Disk space
df -h

  3. Check database size:

    ls -lh /opt/openvox-gui/data/openvox_gui.db
# If very large (>100MB), consider cleanup

  4. Optimize OpenVoxDB queries:

    • Reduce the time range for report queries
    • Limit the number of nodes displayed

Problem: High Memory Usage

Solutions:

  1. Reduce worker count:

    sudo nano /opt/openvox-gui/config/.env
# Change UVICORN_WORKERS to 1 or 2

  2. Restart service to clear memory:

    sudo systemctl restart openvox-gui

  3. Check for memory leaks:

    # Monitor memory over time
watch -n 5 'ps aux | grep openvox-gui'

Display and UI Problems

Problem: Page Shows Old Version After Update

Solutions:

  1. Hard refresh your browser:

    • Windows/Linux: Ctrl + F5
    • Mac: Cmd + Shift + R

  2. Clear all browser data:

    • Chrome: Settings → Privacy → Clear browsing data
    • Firefox: Settings → Privacy → Clear Data

  3. Try incognito/private mode to confirm it's a cache issue

Problem: Charts or Graphs Not Displaying

Solutions:

  1. Check browser console for errors:

    • Press F12 to open developer tools
    • Go to Console tab
    • Look for red error messages

  2. Verify data is being returned:

    curl -k https://localhost:4567/api/dashboard/stats

  3. Check for JavaScript errors:

    • Disable browser extensions
    • Try a different browser

Problem: Theme Not Switching

Solutions:

  1. Clear local storage:

    • Open browser developer tools (F12)
    • Go to Application/Storage tab
    • Clear Local Storage for the site

  2. Manually set theme:

    // In browser console:
localStorage.setItem("theme", "formal"); // or 'casual'
location.reload();

Data and Report Issues

Problem: No Nodes Appearing in Dashboard

Solutions:

  1. Verify OpenVoxDB has data:

    # Check OpenVoxDB directly
curl -k https://localhost:8081/pdb/query/v4/nodes

  2. Check time synchronization:

    # Ensure time is correct
date

# Sync time if needed
sudo ntpdate -s time.nist.gov

  3. Verify OpenVox agents are reporting:

    # On an OpenVox agent:
sudo puppet agent -t

Problem: Reports Missing or Incomplete

Solutions:

  1. Check report processor on OpenVox Server:

    grep reports /etc/puppetlabs/puppet/puppet.conf
# Should include "puppetdb"

  2. Verify reports are being stored:

    # Query OpenVoxDB for recent reports
curl -k https://localhost:8081/pdb/query/v4/reports?limit=10

  3. Check report retention settings in OpenVoxDB

Problem: Facts Not Showing in Fact Explorer

Solutions:

  1. Refresh fact cache:

    # On OpenVox agents:
sudo puppet facts upload

  2. Check fact terminus:

    grep facts_terminus /etc/puppetlabs/puppet/puppet.conf
# Should be "puppetdb"

Certificate Problems

Problem: Can't Sign Certificates

Solutions:

  1. Verify sudo permissions:

    sudo cat /etc/sudoers.d/openvox-gui
# Should allow puppetserver ca commands

  2. Check OpenVox CA service:

    sudo puppetserver ca list --all

  3. Manually sign certificates:

    sudo puppetserver ca sign --certname node.example.com

Problem: Certificate Expiration Warnings

Solutions:

  1. Check certificate dates:

    sudo puppetserver ca list --all
openssl x509 -in /path/to/cert.pem -noout -dates

  2. Regenerate expiring certificates:

    # On the agent:
sudo puppet ssl clean
sudo puppet agent -t

# On the server:
sudo puppetserver ca sign --certname node.example.com

Update and Deployment Issues

Problem: Code Deployment Fails

Solutions:

  1. Check r10k configuration:

    sudo r10k deploy display

  2. Verify Git access:

    # Test Git repository access
git ls-remote https://your-git-repo.com/control-repo.git

  3. Check sudo permissions:

    grep r10k /etc/sudoers.d/openvox-gui

  4. Run r10k manually to see errors:

    sudo r10k deploy environment -pv

Problem: Orchestration (OpenBolt) Not Working

Solutions:

  1. Verify OpenBolt is installed:

    which bolt
bolt --version

  2. Install OpenBolt if missing:

    # Red Hat/CentOS:
sudo yum install openbolt

# Ubuntu/Debian:
sudo apt install openbolt

  3. Check OpenBolt project configuration:

    cat /opt/openvox-gui/bolt-project.yaml

Agent Installer Problems (3.6.0+)

For issues specific to the local OpenVox package mirror and the curl ... | sudo bash agent install workflow, see docs/INSTALLER.md -- it has the full feature guide plus a dedicated troubleshooting section. Quick reference for the most common gotchas:

Problem: curl https://server:8140/packages/install.bash returns ~378 bytes of HTML

Puppetserver wasn't restarted after the openvox-gui upgrade dropped its static-content mount config. The HTML is puppetserver's default "unknown path" page.

sudo systemctl restart puppetserver
sudo systemctl is-active puppetserver

After the restart, the URL should return the install.bash script (~17 KB).

Problem: bash: --server: invalid option

You appended args to the one-liner without bash -s -- between bash and them. Bash interpreted --server as one of its own options. The fix is to insert -s -- so bash treats trailing tokens as positional args for the script:

curl -k --noproxy <server> https://<server>:8140/packages/install.bash | sudo bash -s -- --server <fqdn>

The GUI's published one-liner doesn't pass extra args (the script auto-discovers the FQDN), so this only trips you if you're overriding behavior.

Problem: curl: (56) CONNECT tunnel failed, response 407

Your agent host is behind a corporate proxy and the bootstrap curl tried to tunnel through it. Use the GUI's published one-liner -- it includes --noproxy <fqdn> to bypass the proxy for the puppetserver host:

curl -k --noproxy <fqdn> https://<fqdn>:8140/packages/install.bash | sudo bash

For the bare curl ... | bash form to work without --noproxy, set no_proxy in /etc/environment on the host once.

Problem: Certificate verification failed: The certificate is NOT trusted

You're seeing this on a follow-up apt-get update or dnf upgrade after the agent install completed. install.bash installs the puppet CA into the system trust store automatically, so this should only happen if the CA install step failed (CA endpoint unreachable from the agent, update-ca-certificates missing, unsupported OS family). Re-run install.bash to retry, or install the CA manually:

# Debian/Ubuntu
sudo curl -ksLf https://<fqdn>:8140/puppet-ca/v1/certificate/ca \
    -o /usr/local/share/ca-certificates/openvox-puppet-ca.crt
sudo update-ca-certificates

# RHEL family
sudo curl -ksLf https://<fqdn>:8140/puppet-ca/v1/certificate/ca \
    -o /etc/pki/ca-trust/source/anchors/openvox-puppet-ca.crt
sudo update-ca-trust extract

Problem: Agent Install page shows "Mirror size: 0 B" / "Last sync: never"

The local mirror at /opt/openvox-pkgs/ is empty. Either:

  • Click Sync now on Infrastructure -> Agent Install in the GUI (admin/operator role required)
  • Or trigger the systemd service from CLI: sudo systemctl start openvox-repo-sync.service
  • Or just wait for the 02:30 nightly timer

The first sync downloads ~1-2 GB and takes 15-45 minutes.

Problem: Install script dies with Could not determine the puppetserver FQDN

In normal operation install.bash discovers the FQDN automatically from the kernel's TCP state (the curl connection lingers in /proc/net/tcp after the bootstrap download completes) plus reverse DNS. When all four resolution paths fail (--server arg / env var, /proc/net/tcp discovery, server-side rendered placeholder, existing puppet.conf), this error fires. Most likely cause: reverse DNS for the puppetserver IP returns nothing or returns a name that's not the puppetserver's actual FQDN. Workaround:

curl -k --noproxy <fqdn> https://<fqdn>:8140/packages/install.bash \
    | sudo bash -s -- --server <fqdn>

Problem: Agent install gets through repo setup but dnf install openvox-agent fails with 404s

The mirror exists but doesn't have packages for your agent's specific OS / architecture. Either:

  • The first sync hasn't covered that platform yet (check Installer page -> Per-platform breakdown)
  • Or the platform isn't in the mirror's allowlist (check the systemd unit's environment overrides at /etc/sysconfig/openvox-repo-sync or /etc/default/openvox-repo-sync)

Re-run the sync limited to your platform:

sudo /opt/openvox-gui/scripts/sync-openvox-repo.sh \
    --platforms yum --el-releases 9 --arches x86_64

Problem: Sync errors with "A sync is already running"

A previous sync was killed without cleaning up its lock file:

sudo rm -f /opt/openvox-pkgs/.sync.lock

Getting More Help

Collect Diagnostic Information

When asking for help, gather this information:

# Create a diagnostic report
cd /opt/openvox-gui
mkdir -p /tmp/openvox-diag

# Version info
curl -k https://localhost:4567/health > /tmp/openvox-diag/version.txt

# Service status
sudo systemctl status openvox-gui > /tmp/openvox-diag/service.txt

# Recent logs
sudo journalctl -u openvox-gui -n 500 > /tmp/openvox-diag/logs.txt

# Configuration (remove passwords!)
grep -v PASSWORD /opt/openvox-gui/config/.env > /tmp/openvox-diag/config.txt

# System info
uname -a > /tmp/openvox-diag/system.txt
python3 --version >> /tmp/openvox-diag/system.txt

# Create archive
tar czf /tmp/openvox-diagnostic.tar.gz /tmp/openvox-diag/

Where to Get Help

  1. GitHub Issues: https://github.com/cvquesty/openvox-gui/issues

    • Search existing issues first
    • Include diagnostic information
    • Describe what you tried

  2. GitHub Discussions: For questions and community help

  3. Documentation:

Emergency Recovery

If everything is broken and you need to start fresh:

# 1. Backup your data
sudo cp -r /opt/openvox-gui/data /backup/
sudo cp -r /opt/openvox-gui/config /backup/

# 2. Uninstall
sudo ./install.sh --uninstall

# 3. Reinstall
sudo ./install.sh

# 4. Restore data
sudo cp /backup/config/.env /opt/openvox-gui/config/
sudo cp -r /backup/data/* /opt/openvox-gui/data/
sudo chown -R puppet:puppet /opt/openvox-gui

# 5. Restart
sudo systemctl restart openvox-gui

Common Error Messages

"Failed to fetch dynamically imported module"

Cause: Browser has old version cached after an update

Fix: Hard refresh (Ctrl+F5 or Cmd+Shift+R)

"Connection refused" or "ECONNREFUSED"

Cause: Service isn't running or port is blocked

Fix: Start service and check firewall

"SSL: CERTIFICATE_VERIFY_FAILED"

Cause: SSL certificate problem with OpenVoxDB

Fix: Check certificate paths and permissions

"Permission denied"

Cause: File ownership or permission issues

Fix: sudo chown -R puppet:puppet /opt/openvox-gui

"Database is locked"

Cause: Multiple processes trying to access SQLite database

Fix: Restart service: sudo systemctl restart openvox-gui

Remember: Most problems have simple solutions. Start with the Quick Fixes, then work through the specific section for your issue. When in doubt, check the logs - they usually tell you exactly what's wrong!

Still stuck? Don't hesitate to ask for help on GitHub Issues!

This document was created with the assistance of AI (Grok, xAI). All technical content has been reviewed and verified by human contributors.