Loading ...
Sorry, an error occurred while loading the content.

problem parsing CSV

Expand Messages
  • johnethompson2001
    Note: I m new to using the mailing list - my apologies if I m doing anything incorrectly. I m having a problem with parsing some CSV data. I m using
    Message 1 of 1 , Jul 22, 2009
    View Source
    • 0 Attachment
      Note: I'm new to using the mailing list - my apologies if I'm doing
      anything incorrectly.

      I'm having a problem with parsing some CSV data. I'm using
      Text::CSV_XS.

      I need to be able to process CSV files in the 2 formats in these sample
      files (not with the same script)

      FIle 1
      "My name", " is ""Joe""
      ", "Who are you?"

      File 2
      "My name", " is \"Joe\" \n ", "Who are you?"

      The first format uses "" for a quote inside a field and has an embedded
      newline in the 2nd field The second format uses \" and has the character
      string "\n" rather than a newline.

      I've written a test program that reads these files using csv->getline()
      and prints out the fields.

      Using the following options
      my $csv = Text::CSV_XS->new( { binary => 1, eol => $/, allow_whitespace
      => 1 } );

      I have no problem with file 1 - output is

      field 0 - My name
      field 1 - is "Joe"

      field 2 - Who are you?

      but file 2 gives

      Error parsing file2 - last line 0
      Error: EIQ - QUO character not allowed
      Error line: "My name", " is \"Joe\" \n ", "Who are you?"

      I've found various suggestions for different options and I've tried a
      number of them. I can get the file to read, but not to parse the \n
      properly (I'd like it converted to an actual newline). Some of the
      options I've tried and the output for file 2 are below:

      my $csv = Text::CSV_XS->new( { binary => 1, eol => $/, allow_whitespace
      => 1, allow_loose_escapes => 1, ( escape_char => "\\" ) } );

      my $csv = Text::CSV_XS->new( { binary => 1, allow_whitespace => 1,
      allow_loose_escapes => 1, allow_loose_quotes =>1, ( escape_char => "\\"
      ) } );

      my $csv = Text::CSV_XS->new( { binary => 1, eol => $/, allow_whitespace
      => 1, allow_loose_escapes => 1, allow_loose_quotes =>1, ( escape_char =>
      "\\" ) } );

      These all give this output

      field 1 - My name
      field 2 - is "Joe" n
      field 3 - Who are you?


      my $csv = Text::CSV_XS->new( { binary => 1, allow_loose_quotes => 1,
      blank_is_undef => 1, escape_char => undef } );

      my $csv = Text::CSV_XS->new( { binary => 1, eol => $/, allow_whitespace
      => 1, allow_loose_escapes => 1, allow_loose_quotes =>1, ( escape_char =>
      undef ) } )

      my $csv = Text::CSV_XS->new( { binary => 1, allow_whitespace => 1,
      allow_loose_escapes => 1, allow_loose_quotes =>1, ( escape_char => undef
      ) }

      These all give this output

      field 1 - My name
      field 2 - " is \"Joe\" \n "
      field 3 - "Who are you?"

      Any help or suggestions greatly appreciated.
      John

      Here's my test script
      -----------------------------------------
      #!/usr/bin/perl -w

      use strict;
      use Text::CSV_XS;

      while ( my $file = shift @ARGV ) {
      parse_file( $file );
      }

      sub parse_file {
      my ( $href, $d, $u, $q );
      my ( $file ) = @_;

      # Create our CSV object
      #my $csv = Text::CSV_XS->new( { binary => 1, eol => $/,
      allow_whitespace
      => 1 } );
      #my $csv = Text::CSV_XS->new( { binary => 1, eol => $/,
      allow_whitespace
      => 1, allow_loose_escapes => 1, ( escape_char => "\\" ) } );
      #my $csv = Text::CSV_XS->new( { binary => 1, allow_loose_quotes
      => 1, bl
      ank_is_undef => 1, escape_char => undef } );
      #my $csv = Text::CSV_XS->new( { binary => 1, eol => $/,
      allow_whitespace
      => 1, allow_loose_quotes => 1, escape_char=> undef } );
      #my $csv = Text::CSV_XS->new( { binary => 1, eol => $/,
      allow_whitespace
      => 1, allow_loose_escapes => 1, allow_loose_quotes =>1, ( escape_char
      => "\\" )
      } );
      #my $csv = Text::CSV_XS->new( { binary => 1, allow_whitespace =>
      1, allo
      w_loose_escapes => 1, allow_loose_quotes =>1, ( escape_char => "\\" ) }
      );
      #my $csv = Text::CSV_XS->new( { binary => 1, eol => $/,
      allow_whitespace
      => 1, allow_loose_escapes => 1, allow_loose_quotes =>1, ( escape_char
      => undef
      ) } );
      my $csv = Text::CSV_XS->new( { binary => 1, allow_whitespace =>
      1, allow
      _loose_escapes => 1, allow_loose_quotes =>1, ( escape_char => undef ) }
      );

      my $i;
      open( LOG, $file ) || die $!;
      while ( my $row = $csv->getline( *LOG ) ) {
      for ($i = 0 ; $i <= 100 ; $i++ ) {
      last if ( !defined $row->[$i] );
      print "field $i - $row->[$i]\n";
      }
      print "\n";
      }

      if ( !$csv->eof() ) {
      print "Error parsing $file\n";
      print "Error: " . $csv->error_diag() . "\n";
      print "Error line: " . $csv->error_input() . "\n";
      exit( -1 );
      }

      close( LOG );
      }
    Your message has been successfully submitted and would be delivered to recipients shortly.