sed to fix a messy nginx.conf












4












$begingroup$


I have a few servers to fix that repeatedly get their nginx.conf file messed up due to a rogue deploy script.



These are the three sed commands I'm using to fix at present:



sed -i '' 's/^([[:blank:]]*)#([[:blank:]]*)server_names_hash_bucket_size/1server_names_hash_bucket_size/g' nginx.conf


The first finds any kind of blank space at the start of a line followed by a pound symbol and the string server_names_hash_bucket_size. Then it replaces it with the original blank space (to honour any indentations) and the string without the pound.



sed -i '' '/^([[:blank:]]*)server_names_hash_bucket_size/s/[0-9]//g' nginx.conf


The second gets rid of any numbers:



sed -i '' '/^([[:blank:]]*)server_names_hash_bucket_size/s/;/64;/g' nginx.conf


The third finds the semi colon and replaces it with the necessary value 64;



I'm currently testing this in terminal OSX, but ultimately will be used on Linux machines.



Does anyone know how to tidy this up and can point out any potential Linux gotchas with sed?










share|improve this question









New contributor




mar627 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.







$endgroup$

















    4












    $begingroup$


    I have a few servers to fix that repeatedly get their nginx.conf file messed up due to a rogue deploy script.



    These are the three sed commands I'm using to fix at present:



    sed -i '' 's/^([[:blank:]]*)#([[:blank:]]*)server_names_hash_bucket_size/1server_names_hash_bucket_size/g' nginx.conf


    The first finds any kind of blank space at the start of a line followed by a pound symbol and the string server_names_hash_bucket_size. Then it replaces it with the original blank space (to honour any indentations) and the string without the pound.



    sed -i '' '/^([[:blank:]]*)server_names_hash_bucket_size/s/[0-9]//g' nginx.conf


    The second gets rid of any numbers:



    sed -i '' '/^([[:blank:]]*)server_names_hash_bucket_size/s/;/64;/g' nginx.conf


    The third finds the semi colon and replaces it with the necessary value 64;



    I'm currently testing this in terminal OSX, but ultimately will be used on Linux machines.



    Does anyone know how to tidy this up and can point out any potential Linux gotchas with sed?










    share|improve this question









    New contributor




    mar627 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
    Check out our Code of Conduct.







    $endgroup$















      4












      4








      4





      $begingroup$


      I have a few servers to fix that repeatedly get their nginx.conf file messed up due to a rogue deploy script.



      These are the three sed commands I'm using to fix at present:



      sed -i '' 's/^([[:blank:]]*)#([[:blank:]]*)server_names_hash_bucket_size/1server_names_hash_bucket_size/g' nginx.conf


      The first finds any kind of blank space at the start of a line followed by a pound symbol and the string server_names_hash_bucket_size. Then it replaces it with the original blank space (to honour any indentations) and the string without the pound.



      sed -i '' '/^([[:blank:]]*)server_names_hash_bucket_size/s/[0-9]//g' nginx.conf


      The second gets rid of any numbers:



      sed -i '' '/^([[:blank:]]*)server_names_hash_bucket_size/s/;/64;/g' nginx.conf


      The third finds the semi colon and replaces it with the necessary value 64;



      I'm currently testing this in terminal OSX, but ultimately will be used on Linux machines.



      Does anyone know how to tidy this up and can point out any potential Linux gotchas with sed?










      share|improve this question









      New contributor




      mar627 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
      Check out our Code of Conduct.







      $endgroup$




      I have a few servers to fix that repeatedly get their nginx.conf file messed up due to a rogue deploy script.



      These are the three sed commands I'm using to fix at present:



      sed -i '' 's/^([[:blank:]]*)#([[:blank:]]*)server_names_hash_bucket_size/1server_names_hash_bucket_size/g' nginx.conf


      The first finds any kind of blank space at the start of a line followed by a pound symbol and the string server_names_hash_bucket_size. Then it replaces it with the original blank space (to honour any indentations) and the string without the pound.



      sed -i '' '/^([[:blank:]]*)server_names_hash_bucket_size/s/[0-9]//g' nginx.conf


      The second gets rid of any numbers:



      sed -i '' '/^([[:blank:]]*)server_names_hash_bucket_size/s/;/64;/g' nginx.conf


      The third finds the semi colon and replaces it with the necessary value 64;



      I'm currently testing this in terminal OSX, but ultimately will be used on Linux machines.



      Does anyone know how to tidy this up and can point out any potential Linux gotchas with sed?







      regex linux sed






      share|improve this question









      New contributor




      mar627 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
      Check out our Code of Conduct.











      share|improve this question









      New contributor




      mar627 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
      Check out our Code of Conduct.









      share|improve this question




      share|improve this question








      edited 25 mins ago









      Jamal

      30.3k11119227




      30.3k11119227






      New contributor




      mar627 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
      Check out our Code of Conduct.









      asked 2 days ago









      mar627mar627

      235




      235




      New contributor




      mar627 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
      Check out our Code of Conduct.





      New contributor





      mar627 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
      Check out our Code of Conduct.






      mar627 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
      Check out our Code of Conduct.






















          1 Answer
          1






          active

          oldest

          votes


















          3












          $begingroup$

          In the general case, the answer to any question regarding sed is "use something else."



          Not that sed is not useful! It definitely is! It's just that it's best for simple tasks that have compact, obvious solutions.



          In particular, Perl is good at replacing sed scripts, thanks to its more expressive regular expressions and uncomplicated access to conventional programming constructs. It even comes with an automated tool to do the translation from sed to Perl, called s2p.



          perl -pli.bak -e'
          next unless s/^(s*)(#s*)?(?=server_names_hash_bucket_size)/$1/;
          tr/d//d;
          s/;/64;/;
          ' nginx.conf


          The newlines are optional and included for readability. Line by line:






            • -p means "print each line after running this code against it." This should be familiar since sed works exactly the same way.


            • -l means "remove newlines before running the code and reinstate them before printing."


            • -i.bak means "edit in-place and save the original as original.bak".


            • -e means the same thing as in sed.



            • We're only interested in lines with server_names_hash_bucket_size, so skip to the next line if the replace fails (the unmodified line still prints; we're skipping the rest of this code but not the printing).

            • We need to include lines that had no #, so make the # match optional with ? after (#s*).


            • (?=...) is a lookahead; the text must exist but is not included in the left side of the substitution, so we don't need to include it on the right side. Only the (optional) leading whitespace ($1) needs to be reinstated on the RHS.


          1. tr works like y in sed. This line is equivalent to s/[0-9]//g (as far as we're concerned. Technically, there are digits other than 0-9 but they aren't going to appear in your nginx.conf.)


          2. Same as sed.





          You can simplify even further by recognizing that the only significant parts of a server_names_hash_bucket_size line, after you throw away the number, are the leading whitespace and terminal comments.



          Preserve the indent as before, throw away as many non-comment characters as possible ([^#]*) and rewrite the line in one replace operation:



          perl -pli.bak -e' s/^(s*)(?:#s*)?(server_names_hash_bucket_size)[^#]*/$1$2 64;/ ' nginx.conf


          (?: makes that group non-capturing, so the second capture ($2) will be server_names_hash_bucket_size.





          At this point, we're back in "simple enough for sed" territory:



          s/^([[:blank:]]*)(#[[:blank:]]*)*(server_names_hash_bucket_size)[^#]*/13 64;/






          share|improve this answer











          $endgroup$









          • 1




            $begingroup$
            Fantastic write up @oh-my-goodness, I'm going to adopt some of Perls more intuitive syntax for complex adjustments. The final 'simple enough for sed' solution works perfectly too; commented matches of the string are ignored correctly, indentations are honoured, and the 64 value is successfully placed where needed. Thank you
            $endgroup$
            – mar627
            yesterday






          • 1




            $begingroup$
            thanks! Note I made a small fix to the sed after your comment, to allow matching more than one space after #.
            $endgroup$
            – Oh My Goodness
            19 hours ago











          Your Answer





          StackExchange.ifUsing("editor", function () {
          return StackExchange.using("mathjaxEditing", function () {
          StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
          StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["\$", "\$"]]);
          });
          });
          }, "mathjax-editing");

          StackExchange.ifUsing("editor", function () {
          StackExchange.using("externalEditor", function () {
          StackExchange.using("snippets", function () {
          StackExchange.snippets.init();
          });
          });
          }, "code-snippets");

          StackExchange.ready(function() {
          var channelOptions = {
          tags: "".split(" "),
          id: "196"
          };
          initTagRenderer("".split(" "), "".split(" "), channelOptions);

          StackExchange.using("externalEditor", function() {
          // Have to fire editor after snippets, if snippets enabled
          if (StackExchange.settings.snippets.snippetsEnabled) {
          StackExchange.using("snippets", function() {
          createEditor();
          });
          }
          else {
          createEditor();
          }
          });

          function createEditor() {
          StackExchange.prepareEditor({
          heartbeatType: 'answer',
          autoActivateHeartbeat: false,
          convertImagesToLinks: false,
          noModals: true,
          showLowRepImageUploadWarning: true,
          reputationToPostImages: null,
          bindNavPrevention: true,
          postfix: "",
          imageUploader: {
          brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
          contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
          allowUrls: true
          },
          onDemand: true,
          discardSelector: ".discard-answer"
          ,immediatelyShowMarkdownHelp:true
          });


          }
          });






          mar627 is a new contributor. Be nice, and check out our Code of Conduct.










          draft saved

          draft discarded


















          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fcodereview.stackexchange.com%2fquestions%2f214508%2fsed-to-fix-a-messy-nginx-conf%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown

























          1 Answer
          1






          active

          oldest

          votes








          1 Answer
          1






          active

          oldest

          votes









          active

          oldest

          votes






          active

          oldest

          votes









          3












          $begingroup$

          In the general case, the answer to any question regarding sed is "use something else."



          Not that sed is not useful! It definitely is! It's just that it's best for simple tasks that have compact, obvious solutions.



          In particular, Perl is good at replacing sed scripts, thanks to its more expressive regular expressions and uncomplicated access to conventional programming constructs. It even comes with an automated tool to do the translation from sed to Perl, called s2p.



          perl -pli.bak -e'
          next unless s/^(s*)(#s*)?(?=server_names_hash_bucket_size)/$1/;
          tr/d//d;
          s/;/64;/;
          ' nginx.conf


          The newlines are optional and included for readability. Line by line:






            • -p means "print each line after running this code against it." This should be familiar since sed works exactly the same way.


            • -l means "remove newlines before running the code and reinstate them before printing."


            • -i.bak means "edit in-place and save the original as original.bak".


            • -e means the same thing as in sed.



            • We're only interested in lines with server_names_hash_bucket_size, so skip to the next line if the replace fails (the unmodified line still prints; we're skipping the rest of this code but not the printing).

            • We need to include lines that had no #, so make the # match optional with ? after (#s*).


            • (?=...) is a lookahead; the text must exist but is not included in the left side of the substitution, so we don't need to include it on the right side. Only the (optional) leading whitespace ($1) needs to be reinstated on the RHS.


          1. tr works like y in sed. This line is equivalent to s/[0-9]//g (as far as we're concerned. Technically, there are digits other than 0-9 but they aren't going to appear in your nginx.conf.)


          2. Same as sed.





          You can simplify even further by recognizing that the only significant parts of a server_names_hash_bucket_size line, after you throw away the number, are the leading whitespace and terminal comments.



          Preserve the indent as before, throw away as many non-comment characters as possible ([^#]*) and rewrite the line in one replace operation:



          perl -pli.bak -e' s/^(s*)(?:#s*)?(server_names_hash_bucket_size)[^#]*/$1$2 64;/ ' nginx.conf


          (?: makes that group non-capturing, so the second capture ($2) will be server_names_hash_bucket_size.





          At this point, we're back in "simple enough for sed" territory:



          s/^([[:blank:]]*)(#[[:blank:]]*)*(server_names_hash_bucket_size)[^#]*/13 64;/






          share|improve this answer











          $endgroup$









          • 1




            $begingroup$
            Fantastic write up @oh-my-goodness, I'm going to adopt some of Perls more intuitive syntax for complex adjustments. The final 'simple enough for sed' solution works perfectly too; commented matches of the string are ignored correctly, indentations are honoured, and the 64 value is successfully placed where needed. Thank you
            $endgroup$
            – mar627
            yesterday






          • 1




            $begingroup$
            thanks! Note I made a small fix to the sed after your comment, to allow matching more than one space after #.
            $endgroup$
            – Oh My Goodness
            19 hours ago
















          3












          $begingroup$

          In the general case, the answer to any question regarding sed is "use something else."



          Not that sed is not useful! It definitely is! It's just that it's best for simple tasks that have compact, obvious solutions.



          In particular, Perl is good at replacing sed scripts, thanks to its more expressive regular expressions and uncomplicated access to conventional programming constructs. It even comes with an automated tool to do the translation from sed to Perl, called s2p.



          perl -pli.bak -e'
          next unless s/^(s*)(#s*)?(?=server_names_hash_bucket_size)/$1/;
          tr/d//d;
          s/;/64;/;
          ' nginx.conf


          The newlines are optional and included for readability. Line by line:






            • -p means "print each line after running this code against it." This should be familiar since sed works exactly the same way.


            • -l means "remove newlines before running the code and reinstate them before printing."


            • -i.bak means "edit in-place and save the original as original.bak".


            • -e means the same thing as in sed.



            • We're only interested in lines with server_names_hash_bucket_size, so skip to the next line if the replace fails (the unmodified line still prints; we're skipping the rest of this code but not the printing).

            • We need to include lines that had no #, so make the # match optional with ? after (#s*).


            • (?=...) is a lookahead; the text must exist but is not included in the left side of the substitution, so we don't need to include it on the right side. Only the (optional) leading whitespace ($1) needs to be reinstated on the RHS.


          1. tr works like y in sed. This line is equivalent to s/[0-9]//g (as far as we're concerned. Technically, there are digits other than 0-9 but they aren't going to appear in your nginx.conf.)


          2. Same as sed.





          You can simplify even further by recognizing that the only significant parts of a server_names_hash_bucket_size line, after you throw away the number, are the leading whitespace and terminal comments.



          Preserve the indent as before, throw away as many non-comment characters as possible ([^#]*) and rewrite the line in one replace operation:



          perl -pli.bak -e' s/^(s*)(?:#s*)?(server_names_hash_bucket_size)[^#]*/$1$2 64;/ ' nginx.conf


          (?: makes that group non-capturing, so the second capture ($2) will be server_names_hash_bucket_size.





          At this point, we're back in "simple enough for sed" territory:



          s/^([[:blank:]]*)(#[[:blank:]]*)*(server_names_hash_bucket_size)[^#]*/13 64;/






          share|improve this answer











          $endgroup$









          • 1




            $begingroup$
            Fantastic write up @oh-my-goodness, I'm going to adopt some of Perls more intuitive syntax for complex adjustments. The final 'simple enough for sed' solution works perfectly too; commented matches of the string are ignored correctly, indentations are honoured, and the 64 value is successfully placed where needed. Thank you
            $endgroup$
            – mar627
            yesterday






          • 1




            $begingroup$
            thanks! Note I made a small fix to the sed after your comment, to allow matching more than one space after #.
            $endgroup$
            – Oh My Goodness
            19 hours ago














          3












          3








          3





          $begingroup$

          In the general case, the answer to any question regarding sed is "use something else."



          Not that sed is not useful! It definitely is! It's just that it's best for simple tasks that have compact, obvious solutions.



          In particular, Perl is good at replacing sed scripts, thanks to its more expressive regular expressions and uncomplicated access to conventional programming constructs. It even comes with an automated tool to do the translation from sed to Perl, called s2p.



          perl -pli.bak -e'
          next unless s/^(s*)(#s*)?(?=server_names_hash_bucket_size)/$1/;
          tr/d//d;
          s/;/64;/;
          ' nginx.conf


          The newlines are optional and included for readability. Line by line:






            • -p means "print each line after running this code against it." This should be familiar since sed works exactly the same way.


            • -l means "remove newlines before running the code and reinstate them before printing."


            • -i.bak means "edit in-place and save the original as original.bak".


            • -e means the same thing as in sed.



            • We're only interested in lines with server_names_hash_bucket_size, so skip to the next line if the replace fails (the unmodified line still prints; we're skipping the rest of this code but not the printing).

            • We need to include lines that had no #, so make the # match optional with ? after (#s*).


            • (?=...) is a lookahead; the text must exist but is not included in the left side of the substitution, so we don't need to include it on the right side. Only the (optional) leading whitespace ($1) needs to be reinstated on the RHS.


          1. tr works like y in sed. This line is equivalent to s/[0-9]//g (as far as we're concerned. Technically, there are digits other than 0-9 but they aren't going to appear in your nginx.conf.)


          2. Same as sed.





          You can simplify even further by recognizing that the only significant parts of a server_names_hash_bucket_size line, after you throw away the number, are the leading whitespace and terminal comments.



          Preserve the indent as before, throw away as many non-comment characters as possible ([^#]*) and rewrite the line in one replace operation:



          perl -pli.bak -e' s/^(s*)(?:#s*)?(server_names_hash_bucket_size)[^#]*/$1$2 64;/ ' nginx.conf


          (?: makes that group non-capturing, so the second capture ($2) will be server_names_hash_bucket_size.





          At this point, we're back in "simple enough for sed" territory:



          s/^([[:blank:]]*)(#[[:blank:]]*)*(server_names_hash_bucket_size)[^#]*/13 64;/






          share|improve this answer











          $endgroup$



          In the general case, the answer to any question regarding sed is "use something else."



          Not that sed is not useful! It definitely is! It's just that it's best for simple tasks that have compact, obvious solutions.



          In particular, Perl is good at replacing sed scripts, thanks to its more expressive regular expressions and uncomplicated access to conventional programming constructs. It even comes with an automated tool to do the translation from sed to Perl, called s2p.



          perl -pli.bak -e'
          next unless s/^(s*)(#s*)?(?=server_names_hash_bucket_size)/$1/;
          tr/d//d;
          s/;/64;/;
          ' nginx.conf


          The newlines are optional and included for readability. Line by line:






            • -p means "print each line after running this code against it." This should be familiar since sed works exactly the same way.


            • -l means "remove newlines before running the code and reinstate them before printing."


            • -i.bak means "edit in-place and save the original as original.bak".


            • -e means the same thing as in sed.



            • We're only interested in lines with server_names_hash_bucket_size, so skip to the next line if the replace fails (the unmodified line still prints; we're skipping the rest of this code but not the printing).

            • We need to include lines that had no #, so make the # match optional with ? after (#s*).


            • (?=...) is a lookahead; the text must exist but is not included in the left side of the substitution, so we don't need to include it on the right side. Only the (optional) leading whitespace ($1) needs to be reinstated on the RHS.


          1. tr works like y in sed. This line is equivalent to s/[0-9]//g (as far as we're concerned. Technically, there are digits other than 0-9 but they aren't going to appear in your nginx.conf.)


          2. Same as sed.





          You can simplify even further by recognizing that the only significant parts of a server_names_hash_bucket_size line, after you throw away the number, are the leading whitespace and terminal comments.



          Preserve the indent as before, throw away as many non-comment characters as possible ([^#]*) and rewrite the line in one replace operation:



          perl -pli.bak -e' s/^(s*)(?:#s*)?(server_names_hash_bucket_size)[^#]*/$1$2 64;/ ' nginx.conf


          (?: makes that group non-capturing, so the second capture ($2) will be server_names_hash_bucket_size.





          At this point, we're back in "simple enough for sed" territory:



          s/^([[:blank:]]*)(#[[:blank:]]*)*(server_names_hash_bucket_size)[^#]*/13 64;/







          share|improve this answer














          share|improve this answer



          share|improve this answer








          edited 23 hours ago

























          answered yesterday









          Oh My GoodnessOh My Goodness

          70029




          70029








          • 1




            $begingroup$
            Fantastic write up @oh-my-goodness, I'm going to adopt some of Perls more intuitive syntax for complex adjustments. The final 'simple enough for sed' solution works perfectly too; commented matches of the string are ignored correctly, indentations are honoured, and the 64 value is successfully placed where needed. Thank you
            $endgroup$
            – mar627
            yesterday






          • 1




            $begingroup$
            thanks! Note I made a small fix to the sed after your comment, to allow matching more than one space after #.
            $endgroup$
            – Oh My Goodness
            19 hours ago














          • 1




            $begingroup$
            Fantastic write up @oh-my-goodness, I'm going to adopt some of Perls more intuitive syntax for complex adjustments. The final 'simple enough for sed' solution works perfectly too; commented matches of the string are ignored correctly, indentations are honoured, and the 64 value is successfully placed where needed. Thank you
            $endgroup$
            – mar627
            yesterday






          • 1




            $begingroup$
            thanks! Note I made a small fix to the sed after your comment, to allow matching more than one space after #.
            $endgroup$
            – Oh My Goodness
            19 hours ago








          1




          1




          $begingroup$
          Fantastic write up @oh-my-goodness, I'm going to adopt some of Perls more intuitive syntax for complex adjustments. The final 'simple enough for sed' solution works perfectly too; commented matches of the string are ignored correctly, indentations are honoured, and the 64 value is successfully placed where needed. Thank you
          $endgroup$
          – mar627
          yesterday




          $begingroup$
          Fantastic write up @oh-my-goodness, I'm going to adopt some of Perls more intuitive syntax for complex adjustments. The final 'simple enough for sed' solution works perfectly too; commented matches of the string are ignored correctly, indentations are honoured, and the 64 value is successfully placed where needed. Thank you
          $endgroup$
          – mar627
          yesterday




          1




          1




          $begingroup$
          thanks! Note I made a small fix to the sed after your comment, to allow matching more than one space after #.
          $endgroup$
          – Oh My Goodness
          19 hours ago




          $begingroup$
          thanks! Note I made a small fix to the sed after your comment, to allow matching more than one space after #.
          $endgroup$
          – Oh My Goodness
          19 hours ago










          mar627 is a new contributor. Be nice, and check out our Code of Conduct.










          draft saved

          draft discarded


















          mar627 is a new contributor. Be nice, and check out our Code of Conduct.













          mar627 is a new contributor. Be nice, and check out our Code of Conduct.












          mar627 is a new contributor. Be nice, and check out our Code of Conduct.
















          Thanks for contributing an answer to Code Review Stack Exchange!


          • Please be sure to answer the question. Provide details and share your research!

          But avoid



          • Asking for help, clarification, or responding to other answers.

          • Making statements based on opinion; back them up with references or personal experience.


          Use MathJax to format equations. MathJax reference.


          To learn more, see our tips on writing great answers.




          draft saved


          draft discarded














          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fcodereview.stackexchange.com%2fquestions%2f214508%2fsed-to-fix-a-messy-nginx-conf%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown





















































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown

































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown







          Popular posts from this blog

          404 Error Contact Form 7 ajax form submitting

          How to know if a Active Directory user can login interactively

          TypeError: fit_transform() missing 1 required positional argument: 'X'