多个远程主机的rsyslog服务器模板注意事项--->链接到先前回答的问题
@ meuh,我发现此帖子非常有用,因为目前正在使用此配置。
我已经完成了上面提到的步骤,并且工作正常。
我现在有一个ELK设置,其中rsyslog将日志转发给它。
我的模板是:
$template
templmesg,"/data01/RemoteLogs/DLF/%$YEAR%/%$MONTH%/%HOSTNAME%/%HOSTNAME%-%$DAY%-%$MONTH%-%$YEAR%.log"
$template mylogsec,"/data01/RemoteLogs/Logserver/%$YEAR%/%$MONTH%/%HOSTNAME%/%HOSTNAME%-%$DAY%-%$MONTH%-%$YEAR%.log"
if $fromhost startswith "10.100.10" then ?templmesg
& stop
if $fromhost startswith "10.100.112" then ?mylogsec
& stop
因此我有两个位置分别存储了日志。
由于日志存储(例如DLF和Logserver)位于多个位置。(ELK)的Kibana不显示从rsyslog接收到的日志。它仅从DLF / dir而非Logserver的一个日志位置读取。
现在,我陷入困境,不知道如何将rsyslog日志从多个位置转发到ELK,并使其显示在kibana中。或者,在rsyslog中是否需要进行任何特定的配置?
以下是rsyslog配置文件:
# For more information see /usr/share/doc/rsyslog-*/rsyslog_conf.html
# If you experience problems, see http://www.rsyslog.com/doc/troubleshoot.html
#### MODULES ####
# The imjournal module bellow is now used as a message source instead of imuxsock.
$ModLoad imuxsock # provides support for local system logging (e.g. via logger command)
$ModLoad imjournal # provides access to the systemd journal
$ModLoad imklog # reads kernel messages (the same are read from journald)
#$ModLoad immark # provides --MARK-- message capability
# Provides UDP syslog reception
$ModLoad imudp
$UDPServerRun 514
# Provides TCP syslog reception
$ModLoad imtcp
$InputTCPServerRun 514
#### GLOBAL DIRECTIVES ####
# Where to place auxiliary files
$WorkDirectory /var/lib/rsyslog
# Use default timestamp format
$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat
# File syncing capability is disabled by default. This feature is usually not required,
# not useful and an extreme performance hit
#$ActionFileEnableSync on
# Include all config files in /etc/rsyslog.d/
$IncludeConfig /etc/rsyslog.d/*.conf
# Turn off message reception via local log socket;
# local messages are retrieved through imjournal now.
$OmitLocalLogging on
# File to store the position in the journal
$IMJournalStateFile imjournal.state
#### RULES ####
# Log all kernel messages to the console.
# Logging much else clutters up the screen.
#kern.* /dev/console
$template templmesg,"/data01/RemoteLogs/DLF/%$YEAR%/%$MONTH%/%HOSTNAME%/%HOSTNAME%-%$DAY%-%$MONTH%-%$YEAR%.log"
$template mylogsec,"/data01/RemoteLogs/DLF/Logserver/%$YEAR%/%$MONTH%/%HOSTNAME%/%HOSTNAME%-%$DAY%-%$MONTH%-%$YEAR%.log"
#if $fromhost startswith "10.100.10" then ?templmesg
#& stop
if $fromhost startswith "10.100.112" then ?mylogsec
& stop
local0.* ?templmesg
local1.* ?templmesg
local2.* ?templmesg
local3.* ?templmesg
local4.* ?templmesg
local5.* ?templmesg
local6.* ?templmesg
template(name="json-template"
type="list") {
constant(value="{")
constant(value="\"@timestamp\":\"") property(name="timereported" dateFormat="rfc3339")
constant(value="\",\"@version\":\"1")
constant(value="\",\"message\":\"") property(name="msg" format="json")
constant(value="\",\"sysloghost\":\"") property(name="hostname")
constant(value="\",\"severity\":\"") property(name="syslogseverity-text")
constant(value="\",\"facility\":\"") property(name="syslogfacility-text")
constant(value="\",\"programname\":\"") property(name="programname")
constant(value="\",\"procid\":\"") property(name="procid")
constant(value="\"}\n")
}
# Log anything (except mail) of level info or higher.
# Don't log private authentication messages!
#
#$createDirs on
*.info;mail.none;authpriv.none;cron.none;local0.none;local1.none;local2.none;local3.none;local4.none;local5.none;local6.none ?templmesg
# The authpriv file has restricted access.
authpriv.* ?templmesg
# Log all the mail messages in one place.
mail.* ?templmesg
# Log cron stuff
cron.* ?templmesg
# Everybody gets emergency messages
#*.emerg :omusrmsg:*
# Save news errors of level crit and higher in a special file.
uucp,news.crit /var/log/spooler
# Save boot messages also to boot.log
local7.* ?templmesg
# ### begin forwarding rule ###
# The statement between the begin ... end define a SINGLE forwarding
# rule. They belong together, do NOT split them. If you create multiple
# forwarding rules, duplicate the whole block!
# Remote Logging (we use TCP for reliable delivery)
#
# An on-disk queue is created for this action. If the remote host is
# down, messages are spooled to disk and sent when it is up again.
#$ActionQueueFileName fwdRule1 # unique name prefix for spool files
#$ActionQueueMaxDiskSpace 1g # 1gb space limit (use as much as possible)
#$ActionQueueSaveOnShutdown on # save messages to disk on shutdown
#$ActionQueueType LinkedList # run asynchronously
#$ActionResumeRetryCount -1 # infinite retries if host is down
# remote host is: name/ip:port, e.g. 192.168.0.1:514, port optional
*.* @10.100.10.30:10514;json-template
# ### end of the forwarding rule ###
由于rsyslog.conf
配置文件是从上到下解析的,因此将按照与文件中定义的顺序相同的顺序对每个消息依次执行操作。在您的情况下,将$fromhost startswith "10.100.112"
处理与测试匹配的消息(即,写入“ mylogsec”模板指定的日志文件),然后由该stop
语句丢弃。
解决这个问题的方法很简单。在rsyslog删除消息之前,您必须将其转发到远程Logstash服务器。您可以如下所示修改过滤器:
if $fromhost startswith "10.100.112" then ?mylogsec
& @10.100.10.30:10514;json-template
& stop
由于您使用JSON模板在此处进行转发,因此您还需要将该模板的定义移到过滤器表达式之前。因此,最终结构将如下所示:
$template templmesg...
$template mylogsec...
template(name="json-template"...
if $fromhost startswith "10.100.112" then ?mylogsec
& @10.100.10.30:10514;json-template
& stop
完成后,重新启动rsyslog
守护程序以使更改生效。
本文收集自互联网,转载请注明来源。
如有侵权,请联系 [email protected] 删除。
我来说两句