Skip to content

Commit

Permalink
Browse files Browse the repository at this point in the history
Overview
- Updated the version of Python to 3.11.3
- Updated to bundle OpenSSL v3.0.8 in all packages (Windows currently at v1.1.1t, but will use v3+ soon.)
- Updated to bundle zLib v1.2.13 in all packages
- Simplified building - building on CentOS 7 provides packages that will run on almost all Linux distributions 
- The listener and passive services/daemons are now combined into one service/daemon called ncpa
- Can optionally run only Listener or Passive 
- Improved logging and installation output 
- Systems with systemd now use a service definition instead of init.d script 
- Listener web UI Admin section now accurately reflects current configuration 
- Updated jQuery to 3.6.4

Edit commits
* Replaced obsolete unicode() with equivalent for py v3 ,str()

* Added remaining config terms to config defaults array.

* Multiple logs for the processes implemented.

* Handle closing of websockets properly. Make logging output more readable

* Update github build to create git hash file. TODO: change code to work in workflows

* update jquery

* Fixed some Mac uninstaller issues

* fix misc errors in build for windows

* Removed patch that was resolved in setup.py

* Add code needed for NSI in Windows

* Moved NSI setup code again to bottom of file with if nt condition

* In Windows, can't inject logging object into subprocess. So, create loggers in subprocesses.

* Refactored logging in listener and passive class set

* Changed to use program files vs 32 bit program files. Updated copy right date.

* update spec and service files to use systemctl

* changed service start mode to -n

* updated linux packaging to retain .rpm when building on ubuntu, so we can use one server for all builds

* fixed issue with copying cx-freeze libs

* remove python executable from pkg. It used to need this to work, but doesn't any longer.

* made it so py and ssl version are printed at info level of logging

* added full path to libssletc. in setup

* another update centos 7 epel install

* added openssl version check for deb distros

* skip openSSL install, fix epel repos for CentOS 7

* adjust epel processing

* Fixed cx lib link conditional

* Minor improvements to source install functions. Moved some functions from setup.sh

* Consolidated literals (version numbers), modified package installatoins, add OpenSSL source installation.

* add scripts for openSSL install

* Numerous updates to OpenSSL and Python installers. Still does not work correctly

* Source installers work on cent8 and ubuntu 20. For testing running as standalone, install some of ffi, sqlite reprequisites normally installed by setup script.

* Updated ncpa to print OpenSSL version in debug logging mode

* Updated installers to use either passed params or globals. Updated Py installer to configure source without flag pointing to custom OpenSSL, if none is installed.

* bug: openssl version sed - OS compatibility

* Print Py version in debug mode

* Print out OpenSSL path after install, allow passsing new SSL path to python as arg for runing as stand-alone

* Bring OpenSSL install build into ncpa build folder

* install openssl to generic folder name in build/resources. Update setup.py to try to include libssl etc.

* updated to put py and ssl version in log

* added bin paths to setup.py

* Added code for copying _ssl if needed

* updated python version in agent/setup.py

* fixed nsi copying

* added dynamic python launcher to handle different windows python launcher commands for build_windows.py

* fixed language issue in ncpa.nsi

* added build dir when not existing for windows

* sym-linked cx_freeze lib-dynload directory to instealled Python. Can't seem to find it by itself. TODO-Ph: see if there s a way to make cx-freeze copy from this dir

* setup.sh - skip openssl pkg install if we are going to build from source.

* update to find original libssl.so on ubuntu 16

* reverted windows executable to be a service

* updated build_windows.py to use the right build command for cx_Freeze to build a windows process

* add --allow-unauthenticated flag to linux set up apt commands so they will work on older distros

* removed unecessary sudo cmmands that cause trouble on debian 8. You nned to be root to run the script anyway.

* added apt-get update - helps install complete on old distros.

* Update installer to support some more distros

* fixed opensuse zypper options

* updated setup.sh so suse distros take openssl building into account

* suppressed install of epel on non-centos because it fails on rhel8, and isn't needed.

* cleaning up and upgrading log to show process name instead of being ambiguous

* added freeze support to multiprocessing so that it will work while frozen

* set it to create an executable instead of a service

* Rewrote WinService to handle creating of Windows service -- cx_Freeze should now create an exe (NOT a service) and NSIS should use sc create NCPA binPath= "path/to/ncpa.exe"

* altered cx_Freeze setup to create exe (not a service) and include necessary pywin32 packages

* reverted cx_Freeze launch to use build_exe instead of build (we are no longer trying to make a service through cx_Freeze)

* updated pypiwin32(deprecated) to pywin32 to allow windows service mangement

* updated NSIS script for new install/management process

* added empty logs to repo

* updated default python launcher to py (python3 is usually py on windows)

* fixing BUILDING.rst directory slash orientation

* updating to use parameters on windows

* added message about not using setup_config.py anymore (cx_Freeze service config file)

* Minor bash changes to make Solaris compatible

* Update so cx_freeze libs that aren't included in python are moved into python lib-dynload, before linking to it.

* Only load windows only modules on Windows

* simplified lib-dynload udpate

* simplified cx dynlib so it no longer uses linking

* wrap WinService class definition in conditional, so it doesn't break linux

* fixing some errors with passive and NRDP code

* updated NSIS script to ask user if they want to save their configuration files when deleting NCPA

* organizing pywin32 imports

* chown installed passive log file before daemon runs to avoid permission error

* Add .deb creation when building on CentOS 7.

* fixing kafka

* update service shutdown for older distros during upgrade

* updated linux install service handling to unload v2.4.1 services

* updated build_windows.py to write git hash (also added some coloration to the build and a build-complete message

* adding logging to debug plugins on linux

* got kafka working on linux

* updated NSIS script to set NCPA service startup to auto instead of delayed-auto

* updated install script to remove NCPA 2 services

* updated NSIS installer to install into the Program Files folder since the binary is a 64-bit program. This may break 32-bit compatibility, but nothing is 32-bit anymore so it shouldn't be a problem.

* Added Python and OpenSSL version to listener UI dashboard

* Refactored Listener UI Admin section to pull values from actual configuration rather than making new assignments with a separeate set of defaults to maintain. Also, updated to match current configuration options.

* Brought changes.rst into sync with v2.4.1, and added v3 info

* Add'l changes to CHANGES

* added git hash to ship with build

* Set passive process loop dlay back to 1 sec

* Updated changes.rst

* Update README.rst for v3.0.0

* Updated BUILDING.rst for v3

* Removed bash color changes - they conflict with non-black backgrounds.

* Updates to incorporate ms vc runtime.

* Updated Windows installer to reflect current config

* Changed nrpd default hostname from NCPA 2 to NCPA. This shouldn't have version info in it.

* Fix grammar ran -> run

* Further clean up of config to match v3, and make some stuff more clear.

* Removed change log section and replaced with link to change log. No need to duplicate this info here, and it is very difficult to maintain.

* Update Listener UI Help section to match current release.

* added .vscode folder to git ignore

---------

Co-authored-by: Dylan Anderson <[email protected]>
Co-authored-by: bb <[email protected]>
Co-authored-by: bbahner <[email protected]>
Co-authored-by: ne-bbahn <[email protected]>
  • Loading branch information
5 people authored Jun 27, 2023
1 parent 79e3904 commit b9a5c51
Show file tree
Hide file tree
Showing 63 changed files with 2,654 additions and 1,464 deletions.
20 changes: 15 additions & 5 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,16 +2,18 @@ name: Build

on:
push:
branches: [ 'dev-v3' ]
branches: [ 'dev-v3-app-changes' ]

jobs:

build:

runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [ubuntu-latest, macos-latest, windows-latest]
# os: [ubuntu-latest, macos-latest, windows-latest]
# os: [ubuntu-latest, macos-latest]
os: [ubuntu-20.04]

steps:
- uses: actions/checkout@v2
Expand All @@ -29,5 +31,13 @@ jobs:
if: runner.os == 'Windows'
run: python -m pip install pypiwin32 cx_Logging
- name: Build the NCPA binaries
run: python setup.py build_exe
working-directory: agent
run: |
python setup.py build_exe
pwd
../build/build_github.sh
working-directory: agent
- name: Upload ${{ matrix.os }} artifact
uses: actions/upload-artifact@v2
with:
name: NCPA-3.0.0-${{ matrix.os }}
path: build/ncpa_3.0.0-*.*
25 changes: 22 additions & 3 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -7,12 +7,31 @@
.idea/
.vagrant
.coverage
.cover
.cache/
agent/build/*
.vscode/
docs/_build
*.exe
build/resources/*.tgz
agent/plugins/*
ncpa.db
prereqs.installed
.python-version
launch.json

agent/build
agent/plugins/*
# agent/var/log/*
agent/var/run/*.pid

build/resources/*.tgz
build/resources/Python*/*
build/resources/openssl/*
build/bin
build/include
build/lib
build/ncpa.spec
build/ncpa
build/ncpa-3*/
build/ncpa-3*/
build/*.deb
build/*.rpm
build/share
143 changes: 97 additions & 46 deletions BUILDING.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2,79 +2,80 @@
Building NCPA
=============

*This document is a work in progress for Python 3 and NCPA 3.*
This document contains instructions for:

* `Building on Windows <https://github.com/NagiosEnterprises/ncpa/blob/master/BUILDING.rst#building-on-windows>`_

* `Building on Linux <https://github.com/NagiosEnterprises/ncpa/blob/master/BUILDING.rst#building-on-linux>`_

* `Building on MacOS <https://github.com/NagiosEnterprises/ncpa/blob/master/BUILDING.rst#building-on-macos>`_

*WARNING*: DO THIS ON A DEDICATED VM OR A NON-PRODUCTION SYSTEM!

THE BUILD SCRIPT WILL MAKE CHANGES TO THE SYSTEM THAT MAY BE INCOMPATIBLE WITH OTHER SOFTWARE

Building on Windows
===================

*Note: The current Windows pre-build script is written in batch and
must be executed by cmd.exe. For this reason, any Windows commands
listed in this document will be written with cmd.exe compatibility
in mind.*

Prerequisites
-------------
**Prerequisites for Windows** (Installing some of these prerequisites requires admin rights)
-------------------------

* `Git for Windows <https://git-scm.com/download/win>`_
* Python 3.9.x (32-Bit) (`Download <https://www.python.org/downloads/>`_)
* OpenSSL for Windows (32-bit) (`Download <https://slproweb.com/download/Win32OpenSSL-1_1_1a.exe>`_) *(Requires admin rights)*
* `Microsoft Visual C++ Compiler Build Tools <https://wiki.python.org/moin/WindowsCompilers>`_ *(Requires admin rights/version used is based on version of python installed)*
* `NSIS 3 <http://nsis.sourceforge.net/Download>`_ *Requires admin rights*
* `Python 3.11.x <https://www.python.org/downloads/>`_
* `NSIS 3 <http://nsis.sourceforge.net/Download>`_

Configure the Build Environment
Configure the Windows Build Environment
-------------------------------

Install Prerequisites
~~~~~~~~~~~~~~~~~~~~~

* Python

1. Download and install Python 3.x. (`see prerequisites <#prerequisites>`_)
2. Execute the installer as usual.
* Git

* OpenSSL
1. Download and install Git for Windows. (`see prerequisites <#prerequisites>`_)

1. Download and install the OpenSSL package. (`see prerequisites <#prerequisites>`_)
2. Be sure to make a not of the installation directory while installing.

* Microsoft Visual C++ Compiler` Build Tools
* Python

1. Download and run the installer. (`see prerequisites <#prerequisites>`_)
2. Follow the instructions outlined in the article in prerequisite section to
ensure you install the proper version for your python version
1. Download and install Python 3.x. (`see prerequisites <#prerequisites>`_)
2. Execute the installer as usual, making sure to check the box to add Python to your PATH (on the first page).

* NSIS

1. Download and run the installer. (`see prerequisites <https://github.com/NagiosEnterprises/ncpa/blob/master/BUILDING.rst#prerequisites>`_)

* pip

* Pip is installed by default but should be updated before continuing::

"%pydir%" -m pip install --upgrade pip
* Pip is installed alongside python by default but should be updated before continuing::

Install the Last Modules
~~~~~~~~~~~~~~~~~~~~~~~~
py -m pip install --upgrade pip

* Install the full list of python modules

"%pydir%\python" -m pip install --upgrade -r build/resources/require.win.txt
Note: py should be the command to run python 3. If it is not, you may need to use the full path to the python executable.

Build NCPA
~~~~~~~~~~
----------

In your Git Bash terminal (or cmd.exe with ``C:\Program Files\Git\usr\bin`` added to your PATH), run the following commands:

Navigate to your desired build directory and clone the repository::

cd /c/desired/build/directory
git clone https://github.com/NagiosEnterprises/ncpa.git

In a Command Prompt/Terminal (cmd.exe) terminal with admin rights, run the following commands::

Run the build script::
cd C:\desired\build\directory\ncpa
py build\build_windows.py

"%pydir%\python" build\build_windows.py
This will create a file called ``ncpa-<version>.exe`` in the ``build`` directory.
This is the installer for NCPA and can be used to install NCPA on a Windows system.


Building on Linux
=================

Building from most Linux distros is much less complicated than Windows. We have a
couple helpful scripts that make it much easier. *We will assume you have wget and git installed*
Building on CentOS 7 is the easiest way to get a working package for all Linux distributions. When you build on CentOS 7, both a .deb as well as an .rpm package are built.

*WARNING: DO THIS ON A VM OR NOT A PRODUCTION SYSTEM*
In most cases, building on the distribution that is targeted, e.g. building on Ubuntu 20.04 to deploy on Ubuntu 20.04, will work, but the resulting package will not be as portable.

To start, clone the repository in your directory::

Expand All @@ -83,20 +84,70 @@ To start, clone the repository in your directory::

Now run the setup scripts to install the requirements::

cd ncpa/build/scripts
cd ncpa/build
./build.sh

Follow the prompts to setup the system. When running the build.sh script it will setup
the system and build the ncpa binary.


Building on Mac OS X
====================
**Install on the target Linux server**
--------------------------------

Copy the resulting ~/ncpa/build/ncpa-3.x.x-x.x86_64.rpm or ncpa_3.x.x-x_amd64.deb to the desired server and install using the appropriate package system:

On CentOS/RHEL/Oracle/Amazon/Rocky::

yum install ./ncpa-3.x.x-1.x86_64.rpm

On Ubuntu 16+/Debian 9+::

apt install ./ncpa_3.0.0-1._amd64.deb

Working on this section. It's basically the same as Linux, however you may need to
On Ubuntu 14/Debian 8 (not supported, but may work)::

dpkg --force-depends -i ./ncpa_3.0.0-1._amd64.deb

On OpenSuSE/SLES::

zypper install ./ncpa-3.x.x-1.x86_64.rpm


Building on MacOS
=================

*Not updated for v3, yet.*
It's basically the same as Linux, however you may need to
install the libraries and python differently, due to it being macOS. You must have
python3 installed prior to running it. You'll also have to use the following command
to build the dmg::
python3, wget and git installed prior to building NCPA v3.x.:


**Clone the repository into your directory**::

cd ~
git clone https://github.com/NagiosEnterprises/ncpa

**Make your user root, and install the xcode command line tools**::

sudo su -
xcode-select --install

**Execute the build script**::

cd ncpa/build/scripts
cd ~/ncpa/build
./build.sh

Note that there may be some difficulty with installing this on other machines without Apple Developer credentials, and with the enhanced system security in newer versions. Please see `Installing on Nagios NCPA v 2.4 Agent on MacOS <https://nagiosenterprises.my.site.com/support/s/article/Installing-the-Nagios-NCPA-v-2-4-Agent-on-MacOS-7ec3e7de>`_ for more information.

Building Tips
=============

There are plenty of derivative operating systems that will not work by following just
the instructions given in this document. NCPA is capable of being built on any system
that supports Python, so not to worry - it is possible!

The common problem is going to be getting the libraries for all the python modules
to be compiled and behave correctly with Python. We recommend compiling them from
source if you must, and compiling Python from source too - with any changes you need
to give the Python build process for library locations. Once that's done, you can
continue by installing the required `pip` modules and trying the build process.
Loading

0 comments on commit b9a5c51

Please sign in to comment.