Skip to content

Commit

Permalink
Merge pull request #409 from ngageoint/doc-updates-0.8.7
Browse files Browse the repository at this point in the history
Final tweaks to documentation before release
  • Loading branch information
jwomeara committed May 28, 2015
2 parents 8918fbf + a99a51c commit 7d7525a
Show file tree
Hide file tree
Showing 4 changed files with 82 additions and 12 deletions.
26 changes: 15 additions & 11 deletions docs/content/085-accumulo-config.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,11 @@

=== Overview

The two high level tasks to configure Accumulo for use with GeoWave are to ensure the memory allocations for the master
and tablet server processes are adequate and to add the GeoWave Accumulo iterator to a classloader. The iterator is a rather
large file so ensure the Accumulo Master process has at least 512m of heap space and the Tablet Server processes have at least
1g of heap space.

The recommended Accumulo configuration for GeoWave requires several manual configuration steps but isolates the GeoWave
libraries in application specific classpath(s) reducing the possibility of dependency conflict issues. A single user for
all of geowave data or a user per data type are two of the many local configuration options just ensure each namespace
Expand All @@ -15,25 +20,24 @@ containing GeoWave tables is configured to pick up the geowave-accumulo.jar.
. Grant the user ownership permissions on all tables created within the application namespace
. Create an application or data set specific classpath
. Configure all tables within the namespace to use the application classpath
. Configure GeoWave transaction support

[source, bash]
----
accumulo shell -u root
createuser geowave // <1>
createnamespace geowave
grant NameSpace.CREATE_TABLE -ns geowave -u geowave
config -s general.vfs.context.classpath.geowave=hdfs://NAME_NODE_FQDN:8020/accumulo/classpath/geowave/[^.].*.jar
config -ns geowave -s table.classpath.context=geowave
exit // <2>
# From the node with the GeoWave application server installed
java -cp "/usr/local/geowave/geoserver/webapps/geoserver/WEB-INF/lib/*" mil.nga.giat.geowave.vector.plugin.GeoWaveGTDataStore -z ZOOKEEPER_NODE:2181 -i accumulo -u root -p ROOT_PASSWORD -n geowave -m 20 -r geowave
grant NameSpace.CREATE_TABLE -ns geowave -u geowave <2>
config -s general.vfs.context.classpath.geowave=hdfs://NAME_NODE_FQDN:8020/ACCUMULO_ROOT/classpath/geowave/VERSION_AND_VENDOR_VERSION/[^.].*.jar <3>
config -ns geowave -s table.classpath.context=geowave <4>
exit
----
<1> You'll be prompted for a password
<2> Done with Accumulo specific configs
<2> Ensure the user has ownership of all tables created within the namespace
<3> The Accumulo root path in HDFS varies between hadoop vendors. Cloudera is /accumulo and Hortonworks is /apps/accumulo
<4> Link the namespace with the application classpath, adjust the labels as needed if you've used different user or appilcation names

These manual configuration steps have to be performed once after the first install of GeoWave. After the initial install you
may elect to do further user and namespace creation and configuring to provide isolation between groups and data sets
These manual configuration steps have to be performed before attempting to create GeoWave index tables. After the initial configuration
you may elect to do further user and namespace creation and configuring to provide isolation between groups and data sets.

=== Versioning

Expand Down Expand Up @@ -78,7 +82,7 @@ sudo find /tmp -name "*geowave-accumulo.jar" -exec unzip -p {} build.properties
.Example
[source, bash]
----
[spohnae@c1-node-03 ~]$ sudo -u hdfs hadoop fs -cat /accumulo/classpath/geowave/geowave-accumulo-build.properties | grep scm.revision | sed s/project.scm.revision=//
[spohnae@c1-node-03 ~]$ sudo -u hdfs hadoop fs -cat /ACCUMULO_ROOT/classpath/geowave/VERSION_AND_VENDOR_VERSION/geowave-accumulo-build.properties | grep scm.revision | sed s/project.scm.revision=//
294ffb267e6691de3b9edc80e312bf5af7b2d23f <1>
[spohnae@c1-node-03 ~]$ sudo find /tmp -name "*geowave-accumulo.jar" -exec unzip -p {} build.properties \; | grep scm.revision | sed s/project.scm.revision=//
294ffb267e6691de3b9edc80e312bf5af7b2d23f <2>
Expand Down
3 changes: 3 additions & 0 deletions docs/content/105-puppet.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,7 @@ class { 'geowave::repo':
repo_refresh_md => 0,
} ->
class { 'geowave':
geowave_version => '0.8.7',
hadoop_vendor_version => 'cdh5',
install_accumulo => true,
install_app => true,
Expand All @@ -79,6 +80,7 @@ node 'c1-master' {
repo_enabled => 1,
} ->
class { 'geowave':
geowave_version => '0.8.7',
hadoop_vendor_version => 'cdh5',
install_accumulo => true,
install_app => true,
Expand All @@ -92,6 +94,7 @@ node 'c1-app-01' {
repo_enabled => 1,
} ->
class { 'geowave':
geowave_version => '0.8.7',
hadoop_vendor_version => 'cdh5',
install_app_server => true,
http_port => '8888',
Expand Down
63 changes: 63 additions & 0 deletions docs/content/manpages/geowave-zktx.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
//:geowave-zktx(1)
//:===============
//::doctype: manpage

NAME
//:----

geowave-zktx - Reserve transaction IDs for long running WFS-T transactions

SYNOPSIS
//:--------

*geowave -zkTx* <options>

DESCRIPTION
//:-----------

The geowave-zkTx(1) operator will create the specified number of transaction IDs for use with long running WFS-T transactions with auto-commit turned off.
These transaction IDs are used as visibility labels in Accumulo to hide ongoing changes until a commit point when the label is removed.

OPTIONS
//:-------

-h,--help::
Display help

-i, --instance-id <arg>::
The Accumulo instance ID

-m,--maximum <arg>::
Maximum number of simultaneous transactions

-n, --namespace <arg>::
The table namespace (optional; default is no namespace)

-p, --password <arg>::
The password for the user

-r,--recipient <arg>::
Recipient application user account for the set of transactions

-u, --user <arg>::
A valid Accumulo user ID

-z, --zookeepers <arg>::
A comma-separated list of zookeeper servers that an Accumulo instance is using

EXAMPLES
//:--------

geowave -zkTx \
-i hdp-accumulo-instance \
-m NUM_SIMULTANEOUS_TRANSACTIONS \
-n NAMESPACE \
-r NAMESPACE_USER \
-u root \
-p ROOT_PWD \
-z ZK_SERVER:2181

This command should output a logging message stating if it added transaction IDs ex: "Added 20 useable transaction IDs". If transactions
have already been allocated it may return a message like "Added 0 useable transactions IDs" which absent any error messages is OK. You
can verify the transactions have been created using a zookeeper client with a command like, "zookeeper-client -server c3-master:2181 ls /geowave/tx"
which should return a list of transaction IDs (guids)
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ public static void main(
"m",
"maximum",
true,
"Maximum number of simulataneous transactions"));
"Maximum number of simultaneous transactions"));
options.addOption(new Option(
"r",
"recipient",
Expand Down

0 comments on commit 7d7525a

Please sign in to comment.