Writing forwarded rsyslogs to journald to be able to filter them by SYSLOG_IDENTIFIER?
Hello StackOverflow Gods/Regulars/Users !
I am currently working on a logging system for two applications running on my servers.
Here is the context :
- Server dev1 : Ubuntu server 18.04 (freshly installed)
- is running a systemd.service algo-ep
[Unit]
Description="Algo EP"
[Service]
Type=simple
User=me
WorkingDirectory=/home/me/bin
ExecStart=/home/me/bin/AlgoEp
StandardOutput=syslog
StandardError=syslog
SyslogIdentifier=dev1_algo_ep
[Install]
WantedBy=multi-user.target
- is running another systemd.service algo-mdw
[Unit]
Description="Algo MDW"
[Service]
Type=simple
User=me
WorkingDirectory=/home/me/bin
ExecStart=/home/me/bin/AlgoMdw
StandardOutput=syslog
StandardError=syslog
SyslogIdentifier=dev1_algo_mdw
[Install]
WantedBy=multi-user.target
- Server dev2 : Ubuntu server 18.04 (freshly installed)
- is running a systemd.service algo-ep
[Unit]
Description="Algo EP"
[Service]
Type=simple
User=me
WorkingDirectory=/home/me/bin
ExecStart=/home/me/bin/AlgoEp
StandardOutput=syslog
StandardError=syslog
SyslogIdentifier=dev2_algo_ep
[Install]
WantedBy=multi-user.target
I wanted to be able to read the log of each service when I ssh on dev1 using journalctl (systemd-journal).
Like this:journalctl -t dev1_algo_ep -t dev1_algo_mdw -t dev2_algo_ep
So, I added a rsyslog.d/algo-ep.conf
on dev2:
if $programname == 'dev2_algo_ep' then {
action(type="omfwd"
queue.type="linkedlist"
queue.filename="algo_fwd"
queue.saveOnShutdown="on"
action.resumeRetryCount="-1"
target="dev1" port="514" protocol="tcp"
)
}
and added rsyslog.d/algo.conf
on dev1:
module(load="imtcp")
module(load="omjournal")
ruleset(name="remote-dev2") {
action(type="omjournal")
}
input(type="imtcp" port="514" ruleset="remote-dev2")
At this point, no problem, I got the line in journalctl with journalctl -r
:
Nov 23 13:27:47 dev1 dev2_algo_ep[3142]:[15246]: Ep Server listening on localhost:10001...
Nov 23 13:27:47 dev1 dev2_algo_ep[2421]:[15246]: Ep Server stops...
[...]
But when I try journalctl -t dev2_algo_ep
:
me@dev1:~$ journalctl -t dev2_algo_ep
-- Logs begin at Fri 2018-06-01 13:54:11 CEST, end at Fri 2018-11-23 13:27:47 CET. --
me@dev1:~$
Because received log's SYSLOG_IDENTIFIER is set as dev2_algo_ep[3142]:
instead of dev2_algo_ep
.
So, my question : Is there a way, magical or obvious
- to export the log from dev2 to dev1 with a specific SYSLOG_IDENTIFIER ?
- or to receive the log on dev1 and to set a specific SYSLOG_IDENTIFIER before sending it to journald ?
- or simply to do this ?
Thanks in advance for your advice, your help and your information !
[Edit]
It seems that the mix rsyslog + journald is very little known. I didn't found anything in the man page (except the possibility to create a template to rebuild the log at reception on dev1, but looks pretty odd to me).
logging systemd ubuntu-server rsyslog systemd-journald
add a comment |
Hello StackOverflow Gods/Regulars/Users !
I am currently working on a logging system for two applications running on my servers.
Here is the context :
- Server dev1 : Ubuntu server 18.04 (freshly installed)
- is running a systemd.service algo-ep
[Unit]
Description="Algo EP"
[Service]
Type=simple
User=me
WorkingDirectory=/home/me/bin
ExecStart=/home/me/bin/AlgoEp
StandardOutput=syslog
StandardError=syslog
SyslogIdentifier=dev1_algo_ep
[Install]
WantedBy=multi-user.target
- is running another systemd.service algo-mdw
[Unit]
Description="Algo MDW"
[Service]
Type=simple
User=me
WorkingDirectory=/home/me/bin
ExecStart=/home/me/bin/AlgoMdw
StandardOutput=syslog
StandardError=syslog
SyslogIdentifier=dev1_algo_mdw
[Install]
WantedBy=multi-user.target
- Server dev2 : Ubuntu server 18.04 (freshly installed)
- is running a systemd.service algo-ep
[Unit]
Description="Algo EP"
[Service]
Type=simple
User=me
WorkingDirectory=/home/me/bin
ExecStart=/home/me/bin/AlgoEp
StandardOutput=syslog
StandardError=syslog
SyslogIdentifier=dev2_algo_ep
[Install]
WantedBy=multi-user.target
I wanted to be able to read the log of each service when I ssh on dev1 using journalctl (systemd-journal).
Like this:journalctl -t dev1_algo_ep -t dev1_algo_mdw -t dev2_algo_ep
So, I added a rsyslog.d/algo-ep.conf
on dev2:
if $programname == 'dev2_algo_ep' then {
action(type="omfwd"
queue.type="linkedlist"
queue.filename="algo_fwd"
queue.saveOnShutdown="on"
action.resumeRetryCount="-1"
target="dev1" port="514" protocol="tcp"
)
}
and added rsyslog.d/algo.conf
on dev1:
module(load="imtcp")
module(load="omjournal")
ruleset(name="remote-dev2") {
action(type="omjournal")
}
input(type="imtcp" port="514" ruleset="remote-dev2")
At this point, no problem, I got the line in journalctl with journalctl -r
:
Nov 23 13:27:47 dev1 dev2_algo_ep[3142]:[15246]: Ep Server listening on localhost:10001...
Nov 23 13:27:47 dev1 dev2_algo_ep[2421]:[15246]: Ep Server stops...
[...]
But when I try journalctl -t dev2_algo_ep
:
me@dev1:~$ journalctl -t dev2_algo_ep
-- Logs begin at Fri 2018-06-01 13:54:11 CEST, end at Fri 2018-11-23 13:27:47 CET. --
me@dev1:~$
Because received log's SYSLOG_IDENTIFIER is set as dev2_algo_ep[3142]:
instead of dev2_algo_ep
.
So, my question : Is there a way, magical or obvious
- to export the log from dev2 to dev1 with a specific SYSLOG_IDENTIFIER ?
- or to receive the log on dev1 and to set a specific SYSLOG_IDENTIFIER before sending it to journald ?
- or simply to do this ?
Thanks in advance for your advice, your help and your information !
[Edit]
It seems that the mix rsyslog + journald is very little known. I didn't found anything in the man page (except the possibility to create a template to rebuild the log at reception on dev1, but looks pretty odd to me).
logging systemd ubuntu-server rsyslog systemd-journald
add a comment |
Hello StackOverflow Gods/Regulars/Users !
I am currently working on a logging system for two applications running on my servers.
Here is the context :
- Server dev1 : Ubuntu server 18.04 (freshly installed)
- is running a systemd.service algo-ep
[Unit]
Description="Algo EP"
[Service]
Type=simple
User=me
WorkingDirectory=/home/me/bin
ExecStart=/home/me/bin/AlgoEp
StandardOutput=syslog
StandardError=syslog
SyslogIdentifier=dev1_algo_ep
[Install]
WantedBy=multi-user.target
- is running another systemd.service algo-mdw
[Unit]
Description="Algo MDW"
[Service]
Type=simple
User=me
WorkingDirectory=/home/me/bin
ExecStart=/home/me/bin/AlgoMdw
StandardOutput=syslog
StandardError=syslog
SyslogIdentifier=dev1_algo_mdw
[Install]
WantedBy=multi-user.target
- Server dev2 : Ubuntu server 18.04 (freshly installed)
- is running a systemd.service algo-ep
[Unit]
Description="Algo EP"
[Service]
Type=simple
User=me
WorkingDirectory=/home/me/bin
ExecStart=/home/me/bin/AlgoEp
StandardOutput=syslog
StandardError=syslog
SyslogIdentifier=dev2_algo_ep
[Install]
WantedBy=multi-user.target
I wanted to be able to read the log of each service when I ssh on dev1 using journalctl (systemd-journal).
Like this:journalctl -t dev1_algo_ep -t dev1_algo_mdw -t dev2_algo_ep
So, I added a rsyslog.d/algo-ep.conf
on dev2:
if $programname == 'dev2_algo_ep' then {
action(type="omfwd"
queue.type="linkedlist"
queue.filename="algo_fwd"
queue.saveOnShutdown="on"
action.resumeRetryCount="-1"
target="dev1" port="514" protocol="tcp"
)
}
and added rsyslog.d/algo.conf
on dev1:
module(load="imtcp")
module(load="omjournal")
ruleset(name="remote-dev2") {
action(type="omjournal")
}
input(type="imtcp" port="514" ruleset="remote-dev2")
At this point, no problem, I got the line in journalctl with journalctl -r
:
Nov 23 13:27:47 dev1 dev2_algo_ep[3142]:[15246]: Ep Server listening on localhost:10001...
Nov 23 13:27:47 dev1 dev2_algo_ep[2421]:[15246]: Ep Server stops...
[...]
But when I try journalctl -t dev2_algo_ep
:
me@dev1:~$ journalctl -t dev2_algo_ep
-- Logs begin at Fri 2018-06-01 13:54:11 CEST, end at Fri 2018-11-23 13:27:47 CET. --
me@dev1:~$
Because received log's SYSLOG_IDENTIFIER is set as dev2_algo_ep[3142]:
instead of dev2_algo_ep
.
So, my question : Is there a way, magical or obvious
- to export the log from dev2 to dev1 with a specific SYSLOG_IDENTIFIER ?
- or to receive the log on dev1 and to set a specific SYSLOG_IDENTIFIER before sending it to journald ?
- or simply to do this ?
Thanks in advance for your advice, your help and your information !
[Edit]
It seems that the mix rsyslog + journald is very little known. I didn't found anything in the man page (except the possibility to create a template to rebuild the log at reception on dev1, but looks pretty odd to me).
logging systemd ubuntu-server rsyslog systemd-journald
Hello StackOverflow Gods/Regulars/Users !
I am currently working on a logging system for two applications running on my servers.
Here is the context :
- Server dev1 : Ubuntu server 18.04 (freshly installed)
- is running a systemd.service algo-ep
[Unit]
Description="Algo EP"
[Service]
Type=simple
User=me
WorkingDirectory=/home/me/bin
ExecStart=/home/me/bin/AlgoEp
StandardOutput=syslog
StandardError=syslog
SyslogIdentifier=dev1_algo_ep
[Install]
WantedBy=multi-user.target
- is running another systemd.service algo-mdw
[Unit]
Description="Algo MDW"
[Service]
Type=simple
User=me
WorkingDirectory=/home/me/bin
ExecStart=/home/me/bin/AlgoMdw
StandardOutput=syslog
StandardError=syslog
SyslogIdentifier=dev1_algo_mdw
[Install]
WantedBy=multi-user.target
- Server dev2 : Ubuntu server 18.04 (freshly installed)
- is running a systemd.service algo-ep
[Unit]
Description="Algo EP"
[Service]
Type=simple
User=me
WorkingDirectory=/home/me/bin
ExecStart=/home/me/bin/AlgoEp
StandardOutput=syslog
StandardError=syslog
SyslogIdentifier=dev2_algo_ep
[Install]
WantedBy=multi-user.target
I wanted to be able to read the log of each service when I ssh on dev1 using journalctl (systemd-journal).
Like this:journalctl -t dev1_algo_ep -t dev1_algo_mdw -t dev2_algo_ep
So, I added a rsyslog.d/algo-ep.conf
on dev2:
if $programname == 'dev2_algo_ep' then {
action(type="omfwd"
queue.type="linkedlist"
queue.filename="algo_fwd"
queue.saveOnShutdown="on"
action.resumeRetryCount="-1"
target="dev1" port="514" protocol="tcp"
)
}
and added rsyslog.d/algo.conf
on dev1:
module(load="imtcp")
module(load="omjournal")
ruleset(name="remote-dev2") {
action(type="omjournal")
}
input(type="imtcp" port="514" ruleset="remote-dev2")
At this point, no problem, I got the line in journalctl with journalctl -r
:
Nov 23 13:27:47 dev1 dev2_algo_ep[3142]:[15246]: Ep Server listening on localhost:10001...
Nov 23 13:27:47 dev1 dev2_algo_ep[2421]:[15246]: Ep Server stops...
[...]
But when I try journalctl -t dev2_algo_ep
:
me@dev1:~$ journalctl -t dev2_algo_ep
-- Logs begin at Fri 2018-06-01 13:54:11 CEST, end at Fri 2018-11-23 13:27:47 CET. --
me@dev1:~$
Because received log's SYSLOG_IDENTIFIER is set as dev2_algo_ep[3142]:
instead of dev2_algo_ep
.
So, my question : Is there a way, magical or obvious
- to export the log from dev2 to dev1 with a specific SYSLOG_IDENTIFIER ?
- or to receive the log on dev1 and to set a specific SYSLOG_IDENTIFIER before sending it to journald ?
- or simply to do this ?
Thanks in advance for your advice, your help and your information !
[Edit]
It seems that the mix rsyslog + journald is very little known. I didn't found anything in the man page (except the possibility to create a template to rebuild the log at reception on dev1, but looks pretty odd to me).
logging systemd ubuntu-server rsyslog systemd-journald
logging systemd ubuntu-server rsyslog systemd-journald
edited Nov 27 '18 at 13:34
Psyko
asked Nov 23 '18 at 13:02
PsykoPsyko
14
14
add a comment |
add a comment |
0
active
oldest
votes
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53447223%2fwriting-forwarded-rsyslogs-to-journald-to-be-able-to-filter-them-by-syslog-ident%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
0
active
oldest
votes
0
active
oldest
votes
active
oldest
votes
active
oldest
votes
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53447223%2fwriting-forwarded-rsyslogs-to-journald-to-be-able-to-filter-them-by-syslog-ident%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown