Skip to content

Latest commit

 

History

History
67 lines (47 loc) · 3.51 KB

03-heavyweight.md

File metadata and controls

67 lines (47 loc) · 3.51 KB

🔍 Searching Data, the DataMade way

  1. Lightweight
    • DataTables
    • django-filter
    • django-autocomplete-light
  2. Middleweight
    • Custom SQL (Postgres)
  3. Heavyweight
    • Solr & Haystack
  4. Glossary

Heavyweight

Do you need more control of fuzzy matches than Postgres search vectors provide? Do calculated values slow your queries to a crawl? Is your search corpus just plain huge? While a more involved solution than issuing SQL or handling search client side, an implementation of powerful search engine ElasticSearch may be your best bet.

If you’re implementing search for a Django application, Haystack is an extension that connects your application to a custom search engine, e.g., ElasticSearch. While it’s not without pitfalls, Haystack provides a familiar API for defining your search fields, issuing queries, and retrieving results and so can smooth your transition to a more complex search setup.

You should under no circumstances write custom adapter code to interact with ElasticSearch. You will end up reimplementing much of HayStack badly.

Beware: Configuring and administering ElasticSearch is of intermediate to advanced difficulty. When you need a fancy search, however, it can be well worth the effort (and we have some experienced hands on deck who are happy to help).

Pros

Cons

  • ElasticSearch is infinitely configurable.
  • ElasticSearch process must be managed separately from your application, though this is made less of a problem with containerization (e.g., Docker).

Getting started

Run ElasticSearch

  1. Create a docker-compose.yml file at the root of your project directory.
  elasticsearch:
    image: elasticsearch:7.14.2
    container_name: chi-councilmatic-elasticsearch
    ports:
      - 9200:9200
    environment:
      - discovery.type=single-node
      - logger.org.elasticsearch.discovery=DEBUG
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    mem_limit: 1g
    volumes:
      - yourapp-es-data:/usr/share/elasticsearch/data

    volumes:
      yourapp-es-data:

Examples

chi-councilmatic (Django)

Query large bodies of text. Return faceted & highlighted results.