elasticsearch - Multiple Logstash instances vs Filebeats -


i'm trying establish best architecture our elastic stack implementation.

we have 2 distinct networks (lets call them internal , external) , several web / db / application servers (approx 10) on each of these networks.

i consume iis logs, our rabbitmq messages , other bits , bobs machines in both networks , send them single server on internal network elastic , kibana installation located.

for servers on both internal , external networks can see 2 main ways logs sent elastic.

  1. setup logstash on each server , send output elastic server on internal network.
  2. setup filebeats on each server , send logs single server running logstash (this same box hosts elastic , kibana)

i'm unsure of pros , cons of these approaches @ moment. believe correct approach use filebeats, i'm unaware why wouldn't put logstash in multiple places seems better distributing processing of logs. again, perhaps having 1 logstash 20-30 inputs isn't problem?

interested in thoughts or guidance in area.

from read in documentation, logstash more demanding in term of memory filebeat, if kind of treatment on logs (like grok parsing). logstash represent @ least jvm (with jruby). filebeat, assume footprint smaller, since it's optimized shipping logs (i never used it, can't say).

also complicates update want logstash instances or configurations.

for centralized logstash, advantage easy change adress of elasticsearch instance, redirect cache redis or add output. found logstash (in version 2.+) required frequent restart, that's easier if have 1 instance deal with.

i have never used logstash multiple inputs, can't say.


in job responsible of log centralisation system, used beaver (a filebeat equivalent) ship logs redis server , had 2 or 3 logstash server sending elasticsearch. of comments above comes period.


Comments

Popular posts from this blog

html - How to set bootstrap input responsive width? -

javascript - Highchart x and y axes data from json -

javascript - Get js console.log as python variable in QWebView pyqt -