Skip to content

Instantly share code, notes, and snippets.

@reyjrar
Last active May 12, 2023 11:58
Show Gist options
  • Save reyjrar/4364063 to your computer and use it in GitHub Desktop.
Save reyjrar/4364063 to your computer and use it in GitHub Desktop.
ElasticSearch config for a write-heavy cluster
##################################################################
# /etc/elasticsearch/elasticsearch.yml
#
# Base configuration for a write heavy cluster
#
# Cluster / Node Basics
cluster.name: logng
# Node can have abritrary attributes we can use for routing
node.name: logsearch-01
node.datacenter: amsterdam
# Force all memory to be locked, forcing the JVM to never swap
bootstrap.mlockall: true
## Threadpool Settings ##
# Search pool
threadpool.search.type: fixed
threadpool.search.size: 20
threadpool.search.queue_size: 100
# Bulk pool
threadpool.bulk.type: fixed
threadpool.bulk.size: 60
threadpool.bulk.queue_size: 300
# Index pool
threadpool.index.type: fixed
threadpool.index.size: 20
threadpool.index.queue_size: 100
# Indices settings
indices.memory.index_buffer_size: 30%
indices.memory.min_shard_index_buffer_size: 12mb
indices.memory.min_index_buffer_size: 96mb
# Cache Sizes
indices.fielddata.cache.size: 15%
indices.fielddata.cache.expire: 6h
indices.cache.filter.size: 15%
indices.cache.filter.expire: 6h
# Indexing Settings for Writes
index.refresh_interval: 30s
index.translog.flush_threshold_ops: 50000
# Minimum nodes alive to constitute an operational cluster
discovery.zen.minimum_master_nodes: 2
# Unicast Discovery (disable multicast)
discovery.zen.ping.multicast.enabled: false
discovery.zen.ping.unicast.hosts: [ "logsearch-01", "logsearch-02", "logsearch-03" ]
@cdenneen
Copy link

Is this a good working ES config or something you're testing?

@qiaohaijun
Copy link

I will try

@meash-nrel
Copy link

@ebuildy
Copy link

ebuildy commented Apr 19, 2018

there is no "good settings", it always depends of your data and use cases, never copy/paste something you found on a blog with title "how handle billion document with elasticsearch" ^^

Try -> benchmark -> try -> benchmark

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment