Skip to content

Latest commit

 

History

History
138 lines (93 loc) · 5.15 KB

elk-install-aws.md

File metadata and controls

138 lines (93 loc) · 5.15 KB
Raw

This is one of the options listed at elk-install.md.

Amazon announced its ElastiSearch Service offering October 1st 2015. See: https://aws.amazon.com/blogs/aws/new-amazon-elasticsearch-service/

After a month working with it, http://kirankoduru.github.io/elasticsearch/moving-from-aws-elasticsearch-service.html identified reasons why he got away from it:

  • AWS's service does not consume credits companies accumulate.

  • AWS does not provide a selection of region, which may cause some latency.

  • AWS's services uses a back version of Elasticsearch, such as 1.5 when version 2.0 is available. This is important for those who don't want to miss out on those bug fixes and shiny new feature releases.

  • AWS's IAM policies are the only way to configure access to its Elasticsearch service. That is a good way to secure inbound connections. But some prefer using security groups.

  • AWS turns off dynamic scripting and does not allow script upload into the scripts directory.

  • AWS performs backups only after receiving an email to AWS support rather than allowing the elasticsearch-aws-cloud plugin to configure s3 repositories from the elasticsearch service dashboard.

  • AWS does not include Shield, the Elastic commercial security plugin that handles RBAC of cluster and indexes.

  • AWS is not supported by Elastic Support.

  • AWS controls the elasticsearch.yml file containing settings to tweak. The AWS elasticsearch service makes sure everything works perfectly.

The ironic thing is that Elastic's own Found service is run on AWS.'

## Install Steps

Below are the steps I took to create a public instance within AWS.

  1. Use Firefox to get in EC2 dashboard at http://aws.amazon.com/ec2/

  2. Scroll down to the bottom of the Analytics section to click on the Elasticsearch Service link. The default for me is US West 2, which is in Oregon.

https://us-west-2.console.aws.amazon.com/es/home?region=us-west-2#

  1. If you want to use a different region, click on "Oregon" (or another) at the upper right corner.

  2. Clicking on the Getting Starte Guide opens a new window tab.

  3. Click on the blue Get Started button.

  4. PROTIP: When specifying domain, keep to lower case characters and perhaps a number at the end for versioning. For example:

test1

  1. For Instance type, since I don't have much data yet during experimentation, I use Free Tier eligible t2.micro.elasticsearch.

  2. Hover over the (i) icon to the right of Enable dedicated master. It says: "We recommend that you allocate at least three dedicated master nodes for each production Elastisearch domain."

I leave blank anyways during experimentation.

  1. Hover over the (i) icon to the right of Enable zone awareness. It says: "Distributes nodes across two Availability Zones..."

  2. For Storage, I select EBS with 10 GB of General Purpose (SSD) drives.

PROTIP: Free tier users gets up to 10 gigabytes of Magnetic or SSD-Backed EBS storage at no charge for up to 750 hours per month.

  1. After clicking Next, for access policy I select "Allow open access to the domain" so anyone can upload documents.

The version in the file is of AWS access code API version:

"Version": "2012-10-17",
  1. Click Next.
  2. Confirm and create. It takes several minutes for Domain status to go from Loading to Ready. Below is a sample screen after provisioning:

screen shot 2015-10-13 at 8 31 10 am

These sample links are no longer active, of course. But when it was for me, clicking on https://search-test1-da54anmy3esch22sskcuukwf6i.us-west-2.es.amazonaws.com/ got me this:

{
"status" : 200,
"name" : "Ikthalon",
"cluster_name" : "495629083449:test1",
"version" : {
  "number" : "1.5.2",
  "build_hash" : "62ff9868b4c8a0c45860bebb259e21980778ab1c",
  "build_timestamp" : "2015-04-27T09:21:06Z",
  "build_snapshot" : false,
  "lucene_version" : "4.10.4"
},
"tagline" : "You Know, for Search"
}

Instead of the web UI, use AWS ES CLI.

  1. Click on the Kibana link, such as:
https://search-test1-da54anmy3esch22sskcuukwf6i.us-west-2.es.amazonaws.com/_plugin/kibana/

screen shot 2015-10-13 at 9 42 48 am

  1. To make use of the instance using Python in my Mac terminal window:
pip install elasticsearch
  1. For more about using Python for Elastisearch, this article (from November 2014):

http://bitquabit.com/post/having-fun-python-and-elasticsearch-part-1/