Separate out documentation into separate files

Author: netniV

Command-Line-Scripts.md

@@ -0,0 +1,47 @@
+Data Input Methods
+==================
+
+Data input methods allow Cacti to retrieve data to insert into data sources and ultimately put on a graph. There are different ways for Cacti to retrieve data, the most popular being through an external script or from SNMP.
+
+Creating a Data Input Method
+----------------------------
+
+To create a new data input method, select the Data Input Methods option under the Management heading. Once on that screen, click Add on the right. You will be presented with a few fields to populate on the following screen.
+
+||
+
+When you are finished filling in all necessary fields, click the Create button to continue. You will be redirected back to the same page, but this time with two new boxes, Input Fields and Output Fields. The Input Fields box is used to define any fields that require information from the user. Any input fields referenced to in the input string must be defined here. The Output Fields box is used to define each field that you expect back from the script. *All data input methods must have at least one output field defined*, but may have more for a script.
+
+### Data Input Fields
+
+To define a new field, click Add next to the input or output field boxes. You will be presented with some or all of the fields below depending on whether you are adding an input or output field.
+
+||
+
+When you are finished filling in all necessary fields, click the Create button to continue. You will be redirected back to the data input method edit page. From here you can continue to add additional fields, or click Save on this screen when finished.
+
+Making Your Scripts Work With Cacti
+-----------------------------------
+
+The simplest way to extend Cacti's data gathering functionality is through external scripts. Cacti comes with a number of scripts out of the box which are located in the `scripts/` directory. These scripts are used by the data input methods that are present in a new installation of Cacti.
+
+To have Cacti call an external script to gather data you must create a new data input method, making sure to specify Script/Command for the Input Type field. See the previous section, [Creating a Data Input Method](data_input_methods.html) for more information about how to create a data input method. To gather data using your data input method, Cacti simply executes the shell command specified in the Input String field. Because of this, you can have Cacti run any shell command or call any script which can be written in almost any language.
+
+What Cacti is concerned with is the output of the script. When you define your data input method, you are required to define one or more output fields. The number of output fields that you define here is important to your script's output. For a data input method with only one output field, your script should output its value in the following format:
+
+    <value_1>
+
+So if I wrote a script that outputs the number of running processes, its output might look like the following:
+
+`67`
+
+Data input methods with more than one output field are handled a bit differently when writing scripts. Scripts that output more than one value should be formatted like the following:
+
+    <fieldname_1>:<value_1> <fieldname_2>:<value_2> ... <fieldname_n>:<value_n>
+
+Lets say that I write a script that outputs the 1, 5, and 10 minute load average of a Unix machine. In Cacti, I name the output fields '1min', '5min', and '10min', respectively. Based on these two things, the output of the script should look like the following:
+
+`1min:0.40 5min:0.32 10min:0.01`
+
+One last thing to keep in mind when writing scripts for Cacti is that they will be executed as the user the data gatherer runs as. Sometimes a script may work correctly when executed as root, but fails due to permissions problems when executed as a less privileged user.
+

Data-Input-Methods.md

@@ -0,0 +1,86 @@
+Data Queries
+============
+
+Data queries are not a replacement for data input methods in Cacti. Instead they provide an easy way to query, or list data based upon an index, making the data easier to graph. The most common use of a data query within Cacti is to retrieve a list of network interfaces via SNMP. If you want to graph the traffic of a network interface, first Cacti must retrieve a list of interfaces on the host. Second, Cacti can use that information to create the necessary graphs and data sources. Data queries are only concerned with the first step of the process, that is obtaining a list of network interfaces and not creating the graphs/data sources for them. While listing network interfaces is a common use for data queries, they also have other uses such as listing partitions, processors, or even cards in a router.
+
+One requirement for any data query in Cacti, is that it has some unique value that defines each row in the list. This concept follows that of a 'primary key' in SQL, and makes sure that each row in the list can be uniquely referenced. Examples of these index values are 'ifIndex' for SNMP network interfaces or the device name for partitions.
+
+There are two types of data queries that you will see referred to throughout Cacti. They are script queries and SNMP queries. Script and SNMP queries are virtually identical in their functionality and only differ in how they obtain their information. A script query will call an external command or script and an SNMP query will make an SNMP call to retrieve a list of data.
+
+All data queries have two parts, the XML file and the definition within Cacti. An XML file must be created for each query, that defines where each piece of information is and how to retrieve it. This could be thought of as the actual query. The second part is a definition within Cacti, which tells Cacti where to find the XML file and associates the data query with one or more graph templates.
+
+Creating a Data Query
+---------------------
+
+Once you have created the XML file that defines your data query, you must add the data query within Cacti. To do this you must click on Data Queries under the Data Gathering heading, and select Add. You will be prompted for some basic information about the data query, described in more detail below.
+
+||
+
+When you are finished filling in all necessary fields, click the Create button to continue. You will be redirected back to the same page, but this time with some additional information to fill in. If you receive a red warning that says 'XML File Does Not Exist', correct the value specified in the 'XML Path' field.
+
+### Associated Graph Templates
+
+Every data query must have at least one graph template associated with it, and possibly more depending on the number of output fields specified in the XML file. This is where you get to choose what kind of graphs to generate from this query. For instance, the interface data query has multiple graph template associations, used to graph traffic, errors, or packets. To add a new graph template association, simply click Add at the right of the Associated Graph Templates box. You will be presented with a few fields to fill in:
+
+||
+
+When you are finished filling in these fields, click the Create button. You will be redirected back to the same page with some additional information to fill in. Cacti will make a list of each data template referenced to in your selected graph template and display them under the Associated Data Templates box. For each data source item listed, you must selected the data query output field that corresponds with it. *Do not forget to check the checkbox to the right of each selection, or your settings will not be saved.*
+
+The Suggested Values box gives you a way to control field values of data sources and graphs created using this data query. If you specify multiple suggested values for the same field, Cacti will evaluate them in order which you can control using the up or down arrow icons. For more information about valid field names and variables, read the section on suggested values.
+
+When you are finished filling in all necessary fields on this form, click the Save button to return to the data queries edit screen. Repeat the steps under this heading as many times as necessary to represent all data in your XML file. When you are finished with this, you should be ready to start adding your data query to hosts.
+
+SNMP Query XML Syntax
+---------------------
+
+    <query>
+       <name>Get SNMP Interfaces</name>
+       <description>Queries a host for a list of monitorable interfaces</description>
+       <oid_uptime>.1.3.x.x.x</oid_uptime>
+       <oid_index>.1.3.6.1.2.1.2.2.1.1</oid_index>
+       <oid_index_parse>OID/REGEXP:.*.([0-9]{1,3}.[0-9]{1,3})$</oid_index_parse>
+       <oid_num_indexes>.1.3.6.1.2.1.2.1.0</oid_num_indexes>
+       <index_order>ifDescr:ifName:ifIndex</index_order>
+       <index_order_type>numeric</index_order_type>
+       <index_title_format>|chosen_order_field|</index_title_format>
+
+       <fields>
+          <ifIndex>
+             <name>Index</name>
+             <method>walk</method>
+             <source>value</source>
+             <direction>input</direction>
+             <oid>.1.3.6.1.2.1.2.2.1.1</oid>
+          </ifIndex>
+       </fields>
+    </query>
+
+||
+
+Script Query XML Syntax
+-----------------------
+
+    <query>
+       <name>Get Unix Mounted Partitions</name>
+       <description>Queries a list of mounted partitions on a unix-based host with the 'df' command.</description>
+       <script_path>perl |path_cacti|/scripts/query_unix_partitions.pl</script_path>
+       <arg_index>index</arg_index>
+       <arg_query>query</arg_query>
+       <arg_get>get</arg_get>
+       <arg_num_indexes>num_indexes</arg_num_indexes>
+       <output_delimeter>:</output_delimeter>
+       <index_order>dskDevice:dskMount</index_order>
+       <index_order_type>alphabetic</index_order_type>
+       <index_title_format>|chosen_order_field|</index_title_format>
+
+       <fields>
+          <dskDevice>
+             <name>Device Name</name>
+             <direction>input</direction>
+             <query_name>device</query_name>
+          </dskDevice>
+       </fields>
+    </query>
+
+||
+

Data-Queries.md

@@ -0,0 +1,162 @@
+Debugging
+=========
+
+Cacti users sometimes complain about NaN's in their graphs. Unfortunately, there are several reasons for this result. The following is a step-by-step procedure recommended for debugging.
+
+Check Cacti Log File
+--------------------
+
+Please have a look at your cacti log file. Usually, you'll find it at `<path_cacti>/log/cacti.log`. Else see `Settings`, `Paths`. Check for this kind of error:
+
+    CACTID: Host[...] DS[....] WARNING: SNMP timeout detected [500 ms], ignoring host '........'
+
+For "reasonable" timeouts, this may be related to a snmpbulkwalk issue. To change this, see `Settings`, `Poller` and lower the value for `The Maximum SNMP OID's Per SNMP Get Request`. Start at a value of 2 and increase it again, if the poller starts working. (1 or less disables snmpbulkwalk) Some agent's don't have the horsepower to deliver that many OID's at a time. Therefore, we can reduce the number for those older/underpowered devices.
+
+Check Basic Data Gathering
+--------------------------
+
+For scripts, run them as cactiuser from cli to check basic functionality. E.g. for a perl script named `your-perl-script.pl` with parameters "p1 p2" under *nix this would look like:
+
+    su - cactiuser
+    /full/path/to/perl your-perl-script.pl p1 p2
+    ... (check output)
+
+For snmp, snmpget the _exact_ OID you're asking for, using same community string and snmp version as defined within cacti. For an OID of `.1.3.6.1.4.something`, community string of `very-secret` and version 2 for target host `target-host` this would look like
+
+    snmpget -c very-secret -v 2c target-host .1.3.6.1.4.something
+    .... (check output)
+
+Check cacti's poller
+--------------------
+
+First make sure that crontab always shows poller.php. This program will either call cmd.php, the PHP based poller _or_ cactid, the fast alternative, written in C. Define the poller you're using at `Settings`, `Poller`. Cactid has to be implemented seperately, it does not come with cacti by default.
+
+Now, clear `./log/cacti.log` (or rename it to get a fresh start)
+
+Then, change `Settings`, `Poller Logging Level` to DEBUG for _one_ polling cycle. You may rename this log as well to avoid more stuff added to it with subsequent polling cycles.
+
+Now, find the host/data source in question. The `Host[<id>]` is given numerically, the <id> being a specific number for that host. Find this <id> from the `Devices` menue when editing the host: The url contains a string like
+
+    id=<id>
+
+Check, whether the output is as expected. If not, check your script (e.g. `/full/path/to/perl`). If ok, proceed to next step
+
+This procedure may be replaced by running the poller manually for the failing host only. To do so, you need the <id>, again. If you're using cmd.php, set the DEBUG logging level as defined above and run
+
+    php -q cmd.php <id> <id>
+
+If you're using cactid, you may override logging level when calling the poller:
+
+    ./cactid --verbosity=5 <id> <id>
+
+All output is printed to STDOUT in both cases. This procdure allows for repeated tests without waiting for the next polling interval. And there's no need to manually search for the failing host between hundreds of lines of output.
+
+Check MySQL updating
+--------------------
+
+In most cases, this step make be skipped. You may want to return to this step, if the next one fails (e.g. no rrdtool update to be found)
+
+From debug log, please find the MySQL update statement for that host concerning table `poller_output`. On very rare occasions, this will fail. So please copy that sql statement and paste it to a mysql session started from cli. This may as well be done from some tool like phpMyAdmin. Check the sql return code.
+
+Check rrd file updating
+-----------------------
+
+Down in the same log, you should find some
+
+    rrdtool update <filename> --template ...
+
+You should find exactly one update statement for each file.
+
+RRD files should be created by the poller. If it does not create them, it will not fill them either. If it does, please check your `Poller Cache` from Utilities and search for your target. Does the query show up here?
+
+Check rrd file ownership
+------------------------
+
+If rrd files were created e.g. with root ownership, a poller running as cactiuser will not be able to update those files
+
+    cd /var/www/html/cacti/rra
+    ls -l localhost*
+    -rw-r--r--  1 root      root      463824 May 31 12:40 localhost_load_1min_5.rrd
+    -rw-r--r--  1 cactiuser cactiuser 155584 Jun  1 17:10 localhost_mem_buffers_3.rrd
+    -rw-r--r--  1 cactiuser cactiuser 155584 Jun  1 17:10 localhost_mem_swap_4.rrd
+    -rw-r--r--  1 cactiuser cactiuser 155584 Jun  1 17:10 localhost_proc_7.rrd
+    -rw-r--r--  1 cactiuser cactiuser 155584 Jun  1 17:10 localhost_users_6.rrd
+
+Run the following command to cure this problem
+
+    chown cactiuser:cactiuser *.rrd
+
+Check rrd file numbers
+----------------------
+
+You're perhaps wondering about this step, if the former was ok. But due to data sources MINIMUM and MAXIMUM definitions, it is possible, that valid updates for rrd files are suppressed, because MINIMUM was not reached or MAXIMUM was exceeded.
+
+Assuming, you've got some valid `rrdtool update` in step 3, perform a
+
+    rrdtool fetch <rrd file> AVERAGE
+
+and look at the last 10-20 lines. If you find NaN's there, perform
+
+    rrdtool info <rrd file>
+
+and check the `ds[...].min` and `ds[...].max` entries, e.g.
+
+    ds[loss].min = 0.0000000000e+00
+    ds[loss].max = 1.0000000000e+02
+
+In this example, MINIMUM = 0 and MAXIMUM = 100. For a `ds.[...].type=GAUGE` verify, that e.g. the number returned by the script does not exceed `ds[...].MAX` (same holds for MINIMUM, respectively).
+
+If you run into this, please do not only update the data source definition within the Data Template, but perform a
+
+    rrdtool tune <rrd file> --maximum <ds-name>:<new ds maximum>
+
+for all existing rrd files belonging to that Data Template.
+
+At this step, it is wise to check `step` and `heartbeat` of the rrd file as well. For standard 300 seconds polling intervals (step=300), it is wise to set `minimal_heartbeat` to 600 seconds. If a single update is missing and the next one occurs in less than 600 seconds from the last one, rrdtool will interpolate the missing update. Thus, gaps are "filled" automatically by interpolation. Be aware of the fact, that this is no "real" data! Again, this must be done in the Data Template itself and by using rrdtool tune for all existing rrd files of this type.
+
+Check rrdtool graph statement
+-----------------------------
+
+Last resort would be to check, that the correct data sources are used. Goto `Graph Management` and select your Graph. Enable DEBUG Mode to find the whole `rrdtool graph` statement. You should notice the `DEF` statements. They specify the rrd file and data source to be used. You may check, that all of them are as wanted.
+
+Miscellaneous
+-------------
+
+Up to current cacti 0.8.6h, table `poller_output` may increase beyond reasonable size.
+
+This is commonly due to php.ini's memory settings of 8MB default. Change this to at least 64 MB.
+
+To check this, please run following sql from mysql cli (or phpmyadmin or the like)
+
+    select count(*) from poller_output;
+
+If the result is huge, you may get rid of those stuff by
+
+    truncate table poller_output;
+
+As of current SVN code for upcoming cacti 0.9, I saw measures were taken on both issues (memory size, truncating poller_output).
+
+RPM Installation?
+-----------------
+
+Most rpm installations will setup the crontab entry now. If you've followed the installation instructions to the letter (which you should always do ;-) ), you may now have two poller running. That's not a good thing, though. Most rpm installations will setup cron in `/etc/cron.d/cacti`
+
+Now, please check all your crontabs, especially `/etc/crontab` and crontabs of users root and cactiuser. Leave only one poller entry for all of them. Personally, I've chosen `/etc/cron.d/cacti` to avoid problems when updating rpm's. Mosten often, you won't remember this item when updating lots of rpm's, so I felt more secure to put it here. And I've made some slight modifications, see
+
+    shell> vi /etc/cron.d/cacti
+    */5 * * * *     cactiuser       /usr/bin/php -q /var/www/html/cacti/poller.php > /var/local/log/poller.log 2>&1
+
+This will produce a file `/var/local/log/poller.log`, which includes some additional informations from each poller's run, such as rrdtool errors. It occupies only some few bytes and will be overwritten each time.
+
+If you're using the crontab of user "cactiuser" instead, this will look like
+
+    shell> crontab -e -u cactiuser
+    */5 * * * *     /usr/bin/php -q /var/www/html/cacti/poller.php > /var/local/log/poller.log 2>&1
+
+Not NaN, but 0 (zero) values?
+-----------------------------
+
+Pay attention to custom scripts. It is required, that external commands called from there are in the `$PATH` of the cactiuser running the poller. It is therefor recommended to provide `/full/path/to/external/command`
+
+User "criggie" reported an issue with running smartctl. It was complaining "you are not root" so a quick `chmod +s` on the script fixed that problem.
+