Perl record separator -











up vote
0
down vote

favorite












I'm stuck on a seemingly trivial problem but not sure what is it that I'm missing. Need help.



I have a file that is delimited by the standard field separator (0x1f) and record separator (0x1e) characters. (https://en.wikipedia.org/wiki/Delimiter#ASCII_delimited_text)



I don't need to parse out the fields but interested in getting the records.



I read about Perl's record separator special variable and tried using that to parse the file.



The file looks like this. ^ represents the field separator and ^^ represents the record separator (in vim). On sublime these will show up as the relevant hex codes.



ID^_NAME^_PARENTID^_Prov ID^_Pat_ID^_Another ID^_Program1^_Program2^_Status^_Date^_Reason^_Added^_Sn Length^_ze Reason^_StAge^_EnAge^_Notes^^NUMBER^_VARCHAR^_NUMBER^_    NUMBER^_NUMBER^_NUMBER^_VARCHAR^_VARCHAR^_VARCHAR^_DATE^_VARCHAR^_VARCHAR^_VARCHAR^_VARCHAR^_VARCHAR^_VARCHAR^_VARCHAR^^12^_40^_12^_^_12^_12^_200^_200^_12^_^_200^_1^_    4000^_4000^_2000^_2000^_4000^^0^_^_0^_^_0^_0^_^_^_^_^_^_^_^_^_^_^_^^


Following is the code that I wrote to parse the records out. Issue is, whatever I do, the entire file is read into the $row scalar.



I initially assumed that perl expects the $/ to be set to a string type. Doing that also doesn't seem to work and I'm stuck.



Appreciate any help. Thanks.



#local $/ = sprintf("%s",chr("0xa"));
local $/ = chr(0xa);

open my $fh, "<", $file or die "$file: $!";

print("reading recordsn");

while (my $row = <$fh>) {
print("Record:", $row, "n");
}









share|improve this question






















  • You said in your question the record separator is 0x1e, but your code is using 0xa (newline). Which is it?
    – Schwern
    Nov 20 at 4:31








  • 1




    @Schwern Sorry about that. I was tweaking the code to see how this behaves with setting $/ with newline. Inadvertently posted that WIP code. Thanks for spotting.
    – prabhu
    Nov 20 at 4:43















up vote
0
down vote

favorite












I'm stuck on a seemingly trivial problem but not sure what is it that I'm missing. Need help.



I have a file that is delimited by the standard field separator (0x1f) and record separator (0x1e) characters. (https://en.wikipedia.org/wiki/Delimiter#ASCII_delimited_text)



I don't need to parse out the fields but interested in getting the records.



I read about Perl's record separator special variable and tried using that to parse the file.



The file looks like this. ^ represents the field separator and ^^ represents the record separator (in vim). On sublime these will show up as the relevant hex codes.



ID^_NAME^_PARENTID^_Prov ID^_Pat_ID^_Another ID^_Program1^_Program2^_Status^_Date^_Reason^_Added^_Sn Length^_ze Reason^_StAge^_EnAge^_Notes^^NUMBER^_VARCHAR^_NUMBER^_    NUMBER^_NUMBER^_NUMBER^_VARCHAR^_VARCHAR^_VARCHAR^_DATE^_VARCHAR^_VARCHAR^_VARCHAR^_VARCHAR^_VARCHAR^_VARCHAR^_VARCHAR^^12^_40^_12^_^_12^_12^_200^_200^_12^_^_200^_1^_    4000^_4000^_2000^_2000^_4000^^0^_^_0^_^_0^_0^_^_^_^_^_^_^_^_^_^_^_^^


Following is the code that I wrote to parse the records out. Issue is, whatever I do, the entire file is read into the $row scalar.



I initially assumed that perl expects the $/ to be set to a string type. Doing that also doesn't seem to work and I'm stuck.



Appreciate any help. Thanks.



#local $/ = sprintf("%s",chr("0xa"));
local $/ = chr(0xa);

open my $fh, "<", $file or die "$file: $!";

print("reading recordsn");

while (my $row = <$fh>) {
print("Record:", $row, "n");
}









share|improve this question






















  • You said in your question the record separator is 0x1e, but your code is using 0xa (newline). Which is it?
    – Schwern
    Nov 20 at 4:31








  • 1




    @Schwern Sorry about that. I was tweaking the code to see how this behaves with setting $/ with newline. Inadvertently posted that WIP code. Thanks for spotting.
    – prabhu
    Nov 20 at 4:43













up vote
0
down vote

favorite









up vote
0
down vote

favorite











I'm stuck on a seemingly trivial problem but not sure what is it that I'm missing. Need help.



I have a file that is delimited by the standard field separator (0x1f) and record separator (0x1e) characters. (https://en.wikipedia.org/wiki/Delimiter#ASCII_delimited_text)



I don't need to parse out the fields but interested in getting the records.



I read about Perl's record separator special variable and tried using that to parse the file.



The file looks like this. ^ represents the field separator and ^^ represents the record separator (in vim). On sublime these will show up as the relevant hex codes.



ID^_NAME^_PARENTID^_Prov ID^_Pat_ID^_Another ID^_Program1^_Program2^_Status^_Date^_Reason^_Added^_Sn Length^_ze Reason^_StAge^_EnAge^_Notes^^NUMBER^_VARCHAR^_NUMBER^_    NUMBER^_NUMBER^_NUMBER^_VARCHAR^_VARCHAR^_VARCHAR^_DATE^_VARCHAR^_VARCHAR^_VARCHAR^_VARCHAR^_VARCHAR^_VARCHAR^_VARCHAR^^12^_40^_12^_^_12^_12^_200^_200^_12^_^_200^_1^_    4000^_4000^_2000^_2000^_4000^^0^_^_0^_^_0^_0^_^_^_^_^_^_^_^_^_^_^_^^


Following is the code that I wrote to parse the records out. Issue is, whatever I do, the entire file is read into the $row scalar.



I initially assumed that perl expects the $/ to be set to a string type. Doing that also doesn't seem to work and I'm stuck.



Appreciate any help. Thanks.



#local $/ = sprintf("%s",chr("0xa"));
local $/ = chr(0xa);

open my $fh, "<", $file or die "$file: $!";

print("reading recordsn");

while (my $row = <$fh>) {
print("Record:", $row, "n");
}









share|improve this question













I'm stuck on a seemingly trivial problem but not sure what is it that I'm missing. Need help.



I have a file that is delimited by the standard field separator (0x1f) and record separator (0x1e) characters. (https://en.wikipedia.org/wiki/Delimiter#ASCII_delimited_text)



I don't need to parse out the fields but interested in getting the records.



I read about Perl's record separator special variable and tried using that to parse the file.



The file looks like this. ^ represents the field separator and ^^ represents the record separator (in vim). On sublime these will show up as the relevant hex codes.



ID^_NAME^_PARENTID^_Prov ID^_Pat_ID^_Another ID^_Program1^_Program2^_Status^_Date^_Reason^_Added^_Sn Length^_ze Reason^_StAge^_EnAge^_Notes^^NUMBER^_VARCHAR^_NUMBER^_    NUMBER^_NUMBER^_NUMBER^_VARCHAR^_VARCHAR^_VARCHAR^_DATE^_VARCHAR^_VARCHAR^_VARCHAR^_VARCHAR^_VARCHAR^_VARCHAR^_VARCHAR^^12^_40^_12^_^_12^_12^_200^_200^_12^_^_200^_1^_    4000^_4000^_2000^_2000^_4000^^0^_^_0^_^_0^_0^_^_^_^_^_^_^_^_^_^_^_^^


Following is the code that I wrote to parse the records out. Issue is, whatever I do, the entire file is read into the $row scalar.



I initially assumed that perl expects the $/ to be set to a string type. Doing that also doesn't seem to work and I'm stuck.



Appreciate any help. Thanks.



#local $/ = sprintf("%s",chr("0xa"));
local $/ = chr(0xa);

open my $fh, "<", $file or die "$file: $!";

print("reading recordsn");

while (my $row = <$fh>) {
print("Record:", $row, "n");
}






perl parsing text-processing






share|improve this question













share|improve this question











share|improve this question




share|improve this question










asked Nov 20 at 4:24









prabhu

4591925




4591925












  • You said in your question the record separator is 0x1e, but your code is using 0xa (newline). Which is it?
    – Schwern
    Nov 20 at 4:31








  • 1




    @Schwern Sorry about that. I was tweaking the code to see how this behaves with setting $/ with newline. Inadvertently posted that WIP code. Thanks for spotting.
    – prabhu
    Nov 20 at 4:43


















  • You said in your question the record separator is 0x1e, but your code is using 0xa (newline). Which is it?
    – Schwern
    Nov 20 at 4:31








  • 1




    @Schwern Sorry about that. I was tweaking the code to see how this behaves with setting $/ with newline. Inadvertently posted that WIP code. Thanks for spotting.
    – prabhu
    Nov 20 at 4:43
















You said in your question the record separator is 0x1e, but your code is using 0xa (newline). Which is it?
– Schwern
Nov 20 at 4:31






You said in your question the record separator is 0x1e, but your code is using 0xa (newline). Which is it?
– Schwern
Nov 20 at 4:31






1




1




@Schwern Sorry about that. I was tweaking the code to see how this behaves with setting $/ with newline. Inadvertently posted that WIP code. Thanks for spotting.
– prabhu
Nov 20 at 4:43




@Schwern Sorry about that. I was tweaking the code to see how this behaves with setting $/ with newline. Inadvertently posted that WIP code. Thanks for spotting.
– prabhu
Nov 20 at 4:43












1 Answer
1






active

oldest

votes

















up vote
3
down vote



accepted










You can use chr(0xNN), but it's simpler to write a hex character as "xNN". A string containing record separator is "x1e".



#!/usr/bin/env perl

use strict;
use warnings;
use v5.10;

my $file = shift;
open my $fh, "<", $file or die "$file: $!";

say "reading records";

local $/ = "x1e";
while (my $row = <$fh>) {
say("Record:", join ",", split /x1f/, $row);
}





share|improve this answer





















    Your Answer






    StackExchange.ifUsing("editor", function () {
    StackExchange.using("externalEditor", function () {
    StackExchange.using("snippets", function () {
    StackExchange.snippets.init();
    });
    });
    }, "code-snippets");

    StackExchange.ready(function() {
    var channelOptions = {
    tags: "".split(" "),
    id: "1"
    };
    initTagRenderer("".split(" "), "".split(" "), channelOptions);

    StackExchange.using("externalEditor", function() {
    // Have to fire editor after snippets, if snippets enabled
    if (StackExchange.settings.snippets.snippetsEnabled) {
    StackExchange.using("snippets", function() {
    createEditor();
    });
    }
    else {
    createEditor();
    }
    });

    function createEditor() {
    StackExchange.prepareEditor({
    heartbeatType: 'answer',
    convertImagesToLinks: true,
    noModals: true,
    showLowRepImageUploadWarning: true,
    reputationToPostImages: 10,
    bindNavPrevention: true,
    postfix: "",
    imageUploader: {
    brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
    contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
    allowUrls: true
    },
    onDemand: true,
    discardSelector: ".discard-answer"
    ,immediatelyShowMarkdownHelp:true
    });


    }
    });














    draft saved

    draft discarded


















    StackExchange.ready(
    function () {
    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53386203%2fperl-record-separator%23new-answer', 'question_page');
    }
    );

    Post as a guest















    Required, but never shown

























    1 Answer
    1






    active

    oldest

    votes








    1 Answer
    1






    active

    oldest

    votes









    active

    oldest

    votes






    active

    oldest

    votes








    up vote
    3
    down vote



    accepted










    You can use chr(0xNN), but it's simpler to write a hex character as "xNN". A string containing record separator is "x1e".



    #!/usr/bin/env perl

    use strict;
    use warnings;
    use v5.10;

    my $file = shift;
    open my $fh, "<", $file or die "$file: $!";

    say "reading records";

    local $/ = "x1e";
    while (my $row = <$fh>) {
    say("Record:", join ",", split /x1f/, $row);
    }





    share|improve this answer

























      up vote
      3
      down vote



      accepted










      You can use chr(0xNN), but it's simpler to write a hex character as "xNN". A string containing record separator is "x1e".



      #!/usr/bin/env perl

      use strict;
      use warnings;
      use v5.10;

      my $file = shift;
      open my $fh, "<", $file or die "$file: $!";

      say "reading records";

      local $/ = "x1e";
      while (my $row = <$fh>) {
      say("Record:", join ",", split /x1f/, $row);
      }





      share|improve this answer























        up vote
        3
        down vote



        accepted







        up vote
        3
        down vote



        accepted






        You can use chr(0xNN), but it's simpler to write a hex character as "xNN". A string containing record separator is "x1e".



        #!/usr/bin/env perl

        use strict;
        use warnings;
        use v5.10;

        my $file = shift;
        open my $fh, "<", $file or die "$file: $!";

        say "reading records";

        local $/ = "x1e";
        while (my $row = <$fh>) {
        say("Record:", join ",", split /x1f/, $row);
        }





        share|improve this answer












        You can use chr(0xNN), but it's simpler to write a hex character as "xNN". A string containing record separator is "x1e".



        #!/usr/bin/env perl

        use strict;
        use warnings;
        use v5.10;

        my $file = shift;
        open my $fh, "<", $file or die "$file: $!";

        say "reading records";

        local $/ = "x1e";
        while (my $row = <$fh>) {
        say("Record:", join ",", split /x1f/, $row);
        }






        share|improve this answer












        share|improve this answer



        share|improve this answer










        answered Nov 20 at 4:40









        Schwern

        87.6k16101229




        87.6k16101229






























            draft saved

            draft discarded




















































            Thanks for contributing an answer to Stack Overflow!


            • Please be sure to answer the question. Provide details and share your research!

            But avoid



            • Asking for help, clarification, or responding to other answers.

            • Making statements based on opinion; back them up with references or personal experience.


            To learn more, see our tips on writing great answers.





            Some of your past answers have not been well-received, and you're in danger of being blocked from answering.


            Please pay close attention to the following guidance:


            • Please be sure to answer the question. Provide details and share your research!

            But avoid



            • Asking for help, clarification, or responding to other answers.

            • Making statements based on opinion; back them up with references or personal experience.


            To learn more, see our tips on writing great answers.




            draft saved


            draft discarded














            StackExchange.ready(
            function () {
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53386203%2fperl-record-separator%23new-answer', 'question_page');
            }
            );

            Post as a guest















            Required, but never shown





















































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown

































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown







            Popular posts from this blog

            404 Error Contact Form 7 ajax form submitting

            How to know if a Active Directory user can login interactively

            Refactoring coordinates for Minecraft Pi buildings written in Python