Writing forwarded rsyslogs to journald to be able to filter them by SYSLOG_IDENTIFIER?












0















Hello StackOverflow Gods/Regulars/Users !



I am currently working on a logging system for two applications running on my servers.



Here is the context :




  • Server dev1 : Ubuntu server 18.04 (freshly installed)


    • is running a systemd.service algo-ep






[Unit]
Description="Algo EP"
[Service]
Type=simple
User=me
WorkingDirectory=/home/me/bin
ExecStart=/home/me/bin/AlgoEp
StandardOutput=syslog
StandardError=syslog
SyslogIdentifier=dev1_algo_ep
[Install]
WantedBy=multi-user.target




  • is running another systemd.service algo-mdw




[Unit]
Description="Algo MDW"
[Service]
Type=simple
User=me
WorkingDirectory=/home/me/bin
ExecStart=/home/me/bin/AlgoMdw
StandardOutput=syslog
StandardError=syslog
SyslogIdentifier=dev1_algo_mdw
[Install]
WantedBy=multi-user.target




  • Server dev2 : Ubuntu server 18.04 (freshly installed)


    • is running a systemd.service algo-ep






[Unit]
Description="Algo EP"
[Service]
Type=simple
User=me
WorkingDirectory=/home/me/bin
ExecStart=/home/me/bin/AlgoEp
StandardOutput=syslog
StandardError=syslog
SyslogIdentifier=dev2_algo_ep
[Install]
WantedBy=multi-user.target



I wanted to be able to read the log of each service when I ssh on dev1 using journalctl (systemd-journal).



Like this:journalctl -t dev1_algo_ep -t dev1_algo_mdw -t dev2_algo_ep



So, I added a rsyslog.d/algo-ep.conf on dev2:



if $programname == 'dev2_algo_ep' then {          
action(type="omfwd"
queue.type="linkedlist"
queue.filename="algo_fwd"
queue.saveOnShutdown="on"
action.resumeRetryCount="-1"
target="dev1" port="514" protocol="tcp"
)
}


and added rsyslog.d/algo.conf on dev1:



module(load="imtcp")
module(load="omjournal")

ruleset(name="remote-dev2") {
action(type="omjournal")
}

input(type="imtcp" port="514" ruleset="remote-dev2")


At this point, no problem, I got the line in journalctl with journalctl -r:



Nov 23 13:27:47 dev1 dev2_algo_ep[3142]:[15246]:  Ep Server listening on localhost:10001...
Nov 23 13:27:47 dev1 dev2_algo_ep[2421]:[15246]: Ep Server stops...
[...]


But when I try journalctl -t dev2_algo_ep:



me@dev1:~$ journalctl -t dev2_algo_ep
-- Logs begin at Fri 2018-06-01 13:54:11 CEST, end at Fri 2018-11-23 13:27:47 CET. --
me@dev1:~$


Because received log's SYSLOG_IDENTIFIER is set as dev2_algo_ep[3142]: instead of dev2_algo_ep.



So, my question : Is there a way, magical or obvious




  1. to export the log from dev2 to dev1 with a specific SYSLOG_IDENTIFIER ?

  2. or to receive the log on dev1 and to set a specific SYSLOG_IDENTIFIER before sending it to journald ?

  3. or simply to do this ?


Thanks in advance for your advice, your help and your information !



[Edit]
It seems that the mix rsyslog + journald is very little known. I didn't found anything in the man page (except the possibility to create a template to rebuild the log at reception on dev1, but looks pretty odd to me).










share|improve this question





























    0















    Hello StackOverflow Gods/Regulars/Users !



    I am currently working on a logging system for two applications running on my servers.



    Here is the context :




    • Server dev1 : Ubuntu server 18.04 (freshly installed)


      • is running a systemd.service algo-ep






    [Unit]
    Description="Algo EP"
    [Service]
    Type=simple
    User=me
    WorkingDirectory=/home/me/bin
    ExecStart=/home/me/bin/AlgoEp
    StandardOutput=syslog
    StandardError=syslog
    SyslogIdentifier=dev1_algo_ep
    [Install]
    WantedBy=multi-user.target




    • is running another systemd.service algo-mdw




    [Unit]
    Description="Algo MDW"
    [Service]
    Type=simple
    User=me
    WorkingDirectory=/home/me/bin
    ExecStart=/home/me/bin/AlgoMdw
    StandardOutput=syslog
    StandardError=syslog
    SyslogIdentifier=dev1_algo_mdw
    [Install]
    WantedBy=multi-user.target




    • Server dev2 : Ubuntu server 18.04 (freshly installed)


      • is running a systemd.service algo-ep






    [Unit]
    Description="Algo EP"
    [Service]
    Type=simple
    User=me
    WorkingDirectory=/home/me/bin
    ExecStart=/home/me/bin/AlgoEp
    StandardOutput=syslog
    StandardError=syslog
    SyslogIdentifier=dev2_algo_ep
    [Install]
    WantedBy=multi-user.target



    I wanted to be able to read the log of each service when I ssh on dev1 using journalctl (systemd-journal).



    Like this:journalctl -t dev1_algo_ep -t dev1_algo_mdw -t dev2_algo_ep



    So, I added a rsyslog.d/algo-ep.conf on dev2:



    if $programname == 'dev2_algo_ep' then {          
    action(type="omfwd"
    queue.type="linkedlist"
    queue.filename="algo_fwd"
    queue.saveOnShutdown="on"
    action.resumeRetryCount="-1"
    target="dev1" port="514" protocol="tcp"
    )
    }


    and added rsyslog.d/algo.conf on dev1:



    module(load="imtcp")
    module(load="omjournal")

    ruleset(name="remote-dev2") {
    action(type="omjournal")
    }

    input(type="imtcp" port="514" ruleset="remote-dev2")


    At this point, no problem, I got the line in journalctl with journalctl -r:



    Nov 23 13:27:47 dev1 dev2_algo_ep[3142]:[15246]:  Ep Server listening on localhost:10001...
    Nov 23 13:27:47 dev1 dev2_algo_ep[2421]:[15246]: Ep Server stops...
    [...]


    But when I try journalctl -t dev2_algo_ep:



    me@dev1:~$ journalctl -t dev2_algo_ep
    -- Logs begin at Fri 2018-06-01 13:54:11 CEST, end at Fri 2018-11-23 13:27:47 CET. --
    me@dev1:~$


    Because received log's SYSLOG_IDENTIFIER is set as dev2_algo_ep[3142]: instead of dev2_algo_ep.



    So, my question : Is there a way, magical or obvious




    1. to export the log from dev2 to dev1 with a specific SYSLOG_IDENTIFIER ?

    2. or to receive the log on dev1 and to set a specific SYSLOG_IDENTIFIER before sending it to journald ?

    3. or simply to do this ?


    Thanks in advance for your advice, your help and your information !



    [Edit]
    It seems that the mix rsyslog + journald is very little known. I didn't found anything in the man page (except the possibility to create a template to rebuild the log at reception on dev1, but looks pretty odd to me).










    share|improve this question



























      0












      0








      0








      Hello StackOverflow Gods/Regulars/Users !



      I am currently working on a logging system for two applications running on my servers.



      Here is the context :




      • Server dev1 : Ubuntu server 18.04 (freshly installed)


        • is running a systemd.service algo-ep






      [Unit]
      Description="Algo EP"
      [Service]
      Type=simple
      User=me
      WorkingDirectory=/home/me/bin
      ExecStart=/home/me/bin/AlgoEp
      StandardOutput=syslog
      StandardError=syslog
      SyslogIdentifier=dev1_algo_ep
      [Install]
      WantedBy=multi-user.target




      • is running another systemd.service algo-mdw




      [Unit]
      Description="Algo MDW"
      [Service]
      Type=simple
      User=me
      WorkingDirectory=/home/me/bin
      ExecStart=/home/me/bin/AlgoMdw
      StandardOutput=syslog
      StandardError=syslog
      SyslogIdentifier=dev1_algo_mdw
      [Install]
      WantedBy=multi-user.target




      • Server dev2 : Ubuntu server 18.04 (freshly installed)


        • is running a systemd.service algo-ep






      [Unit]
      Description="Algo EP"
      [Service]
      Type=simple
      User=me
      WorkingDirectory=/home/me/bin
      ExecStart=/home/me/bin/AlgoEp
      StandardOutput=syslog
      StandardError=syslog
      SyslogIdentifier=dev2_algo_ep
      [Install]
      WantedBy=multi-user.target



      I wanted to be able to read the log of each service when I ssh on dev1 using journalctl (systemd-journal).



      Like this:journalctl -t dev1_algo_ep -t dev1_algo_mdw -t dev2_algo_ep



      So, I added a rsyslog.d/algo-ep.conf on dev2:



      if $programname == 'dev2_algo_ep' then {          
      action(type="omfwd"
      queue.type="linkedlist"
      queue.filename="algo_fwd"
      queue.saveOnShutdown="on"
      action.resumeRetryCount="-1"
      target="dev1" port="514" protocol="tcp"
      )
      }


      and added rsyslog.d/algo.conf on dev1:



      module(load="imtcp")
      module(load="omjournal")

      ruleset(name="remote-dev2") {
      action(type="omjournal")
      }

      input(type="imtcp" port="514" ruleset="remote-dev2")


      At this point, no problem, I got the line in journalctl with journalctl -r:



      Nov 23 13:27:47 dev1 dev2_algo_ep[3142]:[15246]:  Ep Server listening on localhost:10001...
      Nov 23 13:27:47 dev1 dev2_algo_ep[2421]:[15246]: Ep Server stops...
      [...]


      But when I try journalctl -t dev2_algo_ep:



      me@dev1:~$ journalctl -t dev2_algo_ep
      -- Logs begin at Fri 2018-06-01 13:54:11 CEST, end at Fri 2018-11-23 13:27:47 CET. --
      me@dev1:~$


      Because received log's SYSLOG_IDENTIFIER is set as dev2_algo_ep[3142]: instead of dev2_algo_ep.



      So, my question : Is there a way, magical or obvious




      1. to export the log from dev2 to dev1 with a specific SYSLOG_IDENTIFIER ?

      2. or to receive the log on dev1 and to set a specific SYSLOG_IDENTIFIER before sending it to journald ?

      3. or simply to do this ?


      Thanks in advance for your advice, your help and your information !



      [Edit]
      It seems that the mix rsyslog + journald is very little known. I didn't found anything in the man page (except the possibility to create a template to rebuild the log at reception on dev1, but looks pretty odd to me).










      share|improve this question
















      Hello StackOverflow Gods/Regulars/Users !



      I am currently working on a logging system for two applications running on my servers.



      Here is the context :




      • Server dev1 : Ubuntu server 18.04 (freshly installed)


        • is running a systemd.service algo-ep






      [Unit]
      Description="Algo EP"
      [Service]
      Type=simple
      User=me
      WorkingDirectory=/home/me/bin
      ExecStart=/home/me/bin/AlgoEp
      StandardOutput=syslog
      StandardError=syslog
      SyslogIdentifier=dev1_algo_ep
      [Install]
      WantedBy=multi-user.target




      • is running another systemd.service algo-mdw




      [Unit]
      Description="Algo MDW"
      [Service]
      Type=simple
      User=me
      WorkingDirectory=/home/me/bin
      ExecStart=/home/me/bin/AlgoMdw
      StandardOutput=syslog
      StandardError=syslog
      SyslogIdentifier=dev1_algo_mdw
      [Install]
      WantedBy=multi-user.target




      • Server dev2 : Ubuntu server 18.04 (freshly installed)


        • is running a systemd.service algo-ep






      [Unit]
      Description="Algo EP"
      [Service]
      Type=simple
      User=me
      WorkingDirectory=/home/me/bin
      ExecStart=/home/me/bin/AlgoEp
      StandardOutput=syslog
      StandardError=syslog
      SyslogIdentifier=dev2_algo_ep
      [Install]
      WantedBy=multi-user.target



      I wanted to be able to read the log of each service when I ssh on dev1 using journalctl (systemd-journal).



      Like this:journalctl -t dev1_algo_ep -t dev1_algo_mdw -t dev2_algo_ep



      So, I added a rsyslog.d/algo-ep.conf on dev2:



      if $programname == 'dev2_algo_ep' then {          
      action(type="omfwd"
      queue.type="linkedlist"
      queue.filename="algo_fwd"
      queue.saveOnShutdown="on"
      action.resumeRetryCount="-1"
      target="dev1" port="514" protocol="tcp"
      )
      }


      and added rsyslog.d/algo.conf on dev1:



      module(load="imtcp")
      module(load="omjournal")

      ruleset(name="remote-dev2") {
      action(type="omjournal")
      }

      input(type="imtcp" port="514" ruleset="remote-dev2")


      At this point, no problem, I got the line in journalctl with journalctl -r:



      Nov 23 13:27:47 dev1 dev2_algo_ep[3142]:[15246]:  Ep Server listening on localhost:10001...
      Nov 23 13:27:47 dev1 dev2_algo_ep[2421]:[15246]: Ep Server stops...
      [...]


      But when I try journalctl -t dev2_algo_ep:



      me@dev1:~$ journalctl -t dev2_algo_ep
      -- Logs begin at Fri 2018-06-01 13:54:11 CEST, end at Fri 2018-11-23 13:27:47 CET. --
      me@dev1:~$


      Because received log's SYSLOG_IDENTIFIER is set as dev2_algo_ep[3142]: instead of dev2_algo_ep.



      So, my question : Is there a way, magical or obvious




      1. to export the log from dev2 to dev1 with a specific SYSLOG_IDENTIFIER ?

      2. or to receive the log on dev1 and to set a specific SYSLOG_IDENTIFIER before sending it to journald ?

      3. or simply to do this ?


      Thanks in advance for your advice, your help and your information !



      [Edit]
      It seems that the mix rsyslog + journald is very little known. I didn't found anything in the man page (except the possibility to create a template to rebuild the log at reception on dev1, but looks pretty odd to me).







      logging systemd ubuntu-server rsyslog systemd-journald






      share|improve this question















      share|improve this question













      share|improve this question




      share|improve this question








      edited Nov 27 '18 at 13:34







      Psyko

















      asked Nov 23 '18 at 13:02









      PsykoPsyko

      14




      14
























          0






          active

          oldest

          votes











          Your Answer






          StackExchange.ifUsing("editor", function () {
          StackExchange.using("externalEditor", function () {
          StackExchange.using("snippets", function () {
          StackExchange.snippets.init();
          });
          });
          }, "code-snippets");

          StackExchange.ready(function() {
          var channelOptions = {
          tags: "".split(" "),
          id: "1"
          };
          initTagRenderer("".split(" "), "".split(" "), channelOptions);

          StackExchange.using("externalEditor", function() {
          // Have to fire editor after snippets, if snippets enabled
          if (StackExchange.settings.snippets.snippetsEnabled) {
          StackExchange.using("snippets", function() {
          createEditor();
          });
          }
          else {
          createEditor();
          }
          });

          function createEditor() {
          StackExchange.prepareEditor({
          heartbeatType: 'answer',
          autoActivateHeartbeat: false,
          convertImagesToLinks: true,
          noModals: true,
          showLowRepImageUploadWarning: true,
          reputationToPostImages: 10,
          bindNavPrevention: true,
          postfix: "",
          imageUploader: {
          brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
          contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
          allowUrls: true
          },
          onDemand: true,
          discardSelector: ".discard-answer"
          ,immediatelyShowMarkdownHelp:true
          });


          }
          });














          draft saved

          draft discarded


















          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53447223%2fwriting-forwarded-rsyslogs-to-journald-to-be-able-to-filter-them-by-syslog-ident%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown

























          0






          active

          oldest

          votes








          0






          active

          oldest

          votes









          active

          oldest

          votes






          active

          oldest

          votes
















          draft saved

          draft discarded




















































          Thanks for contributing an answer to Stack Overflow!


          • Please be sure to answer the question. Provide details and share your research!

          But avoid



          • Asking for help, clarification, or responding to other answers.

          • Making statements based on opinion; back them up with references or personal experience.


          To learn more, see our tips on writing great answers.




          draft saved


          draft discarded














          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53447223%2fwriting-forwarded-rsyslogs-to-journald-to-be-able-to-filter-them-by-syslog-ident%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown





















































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown

































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown







          Popular posts from this blog

          404 Error Contact Form 7 ajax form submitting

          How to know if a Active Directory user can login interactively

          TypeError: fit_transform() missing 1 required positional argument: 'X'