Loading ...
Sorry, an error occurred while loading the content.
 

Re: saving decoded unicode value to Session causes error

Expand Messages
  • Tsirkin Evgeny
    It looks like that SDBM_File file handler or something else in it s internals does not know that you are storing (writing) utf8 records into it. I don t know
    Message 1 of 4 , Aug 29, 2007
      It looks like that SDBM_File file handler or something else in it's internals does not
      know that you are storing (writing) utf8 records into it.
      I don't know how to force it to know this :( maybe some SDBM_File expert here
      will know.
      However the workaround for this is to NOT mark the string you get as utf8 i.e.
      don't do Encode::decode_utf8. Just set your pages/server to send to the browser
      header saying that you need utf8 as request charset and you will be fine.

      <meta http-equiv="Content-Type" content="text/html; charset=utf-8">
      will do this or
      $Response->AddHeader()

      What this solution does NOT do - it does not validate the input to be legal utf8.
      But most of the time you will be fine.
      Hope this helps.
      Evgeny

      On 8/29/07, Richard Yen <dba@...> wrote:
      Hi,

      Wondering if anyone has ever encountered this issue.

      Trying to internationalize my site, so at every input, I pass it
      through Encode::decode_utf8.

      However, when I attempt to put the values into Session, I get the
      following message:
      > Errors Output
      > Wide character in subroutine entry at /usr/local/perl/lib/site_perl/
      > 5.8.7/MLDBM/Sync/SDBM_File.pm line 80, <GEN10> line 21. , /usr/
      > local/perl/lib/site_perl/5.8.7/Apache/ASP.pm line 1521
      > Debug Output
      > Use of uninitialized value in string ne at ../global/lib/
      > lib_filesystem.pm line 41.
      > Using an array as a reference is deprecated at ../global/lib/
      > lib_peer_review.pm line 389, line 21.
      > Wide character in subroutine entry at /usr/local/perl/lib/site_perl/
      > 5.8.7/MLDBM/Sync/SDBM_File.pm line 80, line 21. , /usr/local/perl/
      > lib/site_perl/5.8.7/Apache/ASP.pm line 1521

      An example is if I use the capital Omega character (option-z on mac
      keyboard), this error occurs when attempting to store into Session:

      > my $secret_answer = Encode::decode_utf8($main::Request->Form
      > ('secret_answer'));
      > $main::Session->{secret_answer} = $secret_answer;

      I've discovered that if I put the decoded value into a hash, and then
      put the hash into Session, it behaves normally.  Also, if I don't
      decode at all, it behaves normally also.

      Would anyone know why this happens, or how to fix it?

      Thanks!
      --Richard

      ---------------------------------------------------------------------
      To unsubscribe, e-mail: asp-unsubscribe@...
      For additional commands, e-mail: asp-help@...


    • Richard Yen
      ... Hmm, I thought that perhaps it was something with Apache::ASP, and that SDBM_File would be a dependency that was being passed bad data? ... I suppose I
      Message 2 of 4 , Aug 29, 2007
        On Aug 29, 2007, at 12:28 AM, Tsirkin Evgeny wrote:

        > It looks like that SDBM_File file handler or something else in it's
        > internals does not
        > know that you are storing (writing) utf8 records into it.
        > I don't know how to force it to know this :( maybe some SDBM_File
        > expert here
        > will know.
        Hmm, I thought that perhaps it was something with Apache::ASP, and
        that SDBM_File would be a dependency that was being passed bad data?

        > However the workaround for this is to NOT mark the string you get
        > as utf8 i.e.
        > don't do Encode::decode_utf8. Just set your pages/server to send to
        > the browser
        > header saying that you need utf8 as request charset and you will be
        > fine.
        >
        > <meta http-equiv="Content-Type" content="text/html; charset=utf-8">
        > will do this or
        > $Response->AddHeader()
        I suppose I misunderstood the purpose of this <meta> tag...so not
        only output, but inputs get encoded as UTF-8 here? Cool stuff....
        When, then, would I need to use decode_utf8? I suppose it's for file-
        reading or command-line input then?

        > What this solution does NOT do - it does not validate the input to
        > be legal utf8.
        True. Will keep that in mind.

        > But most of the time you will be fine.
        > Hope this helps.
        > Evgeny
        Thanks!
        --Richard



        >
        >
        > On 8/29/07, Richard Yen <dba@...> wrote: Hi,
        >
        > Wondering if anyone has ever encountered this issue.
        >
        > Trying to internationalize my site, so at every input, I pass it
        > through Encode::decode_utf8.
        >
        > However, when I attempt to put the values into Session, I get the
        > following message:
        > > Errors Output
        > > Wide character in subroutine entry at /usr/local/perl/lib/site_perl/
        > > 5.8.7/MLDBM/Sync/SDBM_File.pm line 80, <GEN10> line 21. , /usr/
        > > local/perl/lib/site_perl/5.8.7/Apache/ASP.pm line 1521
        > > Debug Output
        > > Use of uninitialized value in string ne at ../global/lib/
        > > lib_filesystem.pm line 41.
        > > Using an array as a reference is deprecated at ../global/lib/
        > > lib_peer_review.pm line 389, line 21.
        > > Wide character in subroutine entry at /usr/local/perl/lib/site_perl/
        > > 5.8.7/MLDBM/Sync/SDBM_File.pm line 80, line 21. , /usr/local/perl/
        > > lib/site_perl/5.8.7/Apache/ASP.pm line 1521
        >
        > An example is if I use the capital Omega character (option-z on mac
        > keyboard), this error occurs when attempting to store into Session:
        >
        > > my $secret_answer = Encode::decode_utf8($main::Request->Form
        > > ('secret_answer'));
        > > $main::Session->{secret_answer} = $secret_answer;
        >
        > I've discovered that if I put the decoded value into a hash, and then
        > put the hash into Session, it behaves normally. Also, if I don't
        > decode at all, it behaves normally also.
        >
        > Would anyone know why this happens, or how to fix it?
        >
        > Thanks!
        > --Richard
        >
        > ---------------------------------------------------------------------
        > To unsubscribe, e-mail: asp-unsubscribe@...
        > For additional commands, e-mail: asp-help@...
        >
        >


        ---------------------------------------------------------------------
        To unsubscribe, e-mail: asp-unsubscribe@...
        For additional commands, e-mail: asp-help@...
      • Tsirkin Evgeny
        Hi Richard! First of all i need to apologize: It is possible that i gave you a bad advice. I am not a Apache::ASP developer, just a user that is monitoring the
        Message 3 of 4 , Aug 30, 2007
          Hi Richard!
          First of all i need to apologize:
          It is possible that i gave you a bad advice.
          I am not a Apache::ASP developer, just a user that is monitoring the list
          to follow the development (if any is making place ,i hope :) ) and
          I have tried to help  .As NOT a developer i hope it is allowed for me to make
          mistakes.

          I will try to explain :
          If you setup your server and/or application to send the header saying that the
          data is utf8 encoded then what you get from client will also be utf8 encoded.
          That IS right .So,if you just use
          $Request->Form()
          to get the data posted you will get the utf8 encoded data and you probably
          WILL be fine ,the same as i have said in the prev. reply.
          However there IS a problem:
          The string you will get from $Request object although utf8 encoded will NOT be
          marked as utf8 encoded .In other words - you get the data fine and if you for example
          store it in DBMS you are fine,but perl does not know that it is utf8 encoded.For
          perl the data is just an array of bytes
          Why does it matter for perl to mark strings as utf8?If you try to use regular expr.
          or some string functions (like length ,i think) it may not work because perl will
          assume to be character 8 bit long which not always true for utf8.
          So ,my advise of just NOT using Encode::decode_utf8 may cause actually problems
          if you try to do some processing on the data you get.
          Actually, i personally do exactly what i have advised to you :
          Taking the data just like i get it from $Request and store it.
          In the rare cases when i need to do some processing one the localized
          data (like regular expressions match ) i am marking it as utf8.


          > Hmm, I thought that perhaps it was something with Apache::ASP, and
          > that SDBM_File would be a dependency that was being passed bad data?

          I have made a test and it worked just fine,here is the code:
             <html>
          <head>
          </head>
          <body>
          <form action="" method=post>
          <input type=text name=v>
          <input type=submit>
          </form>
          </body>
          </html>
          <%

              use Encode;
          my $v=Encode::decode_utf8($Request->Form('v'));
          $Session->{v}=$v;
          print $Session->{v};
          %>
          I am also using SDBM_File  .Thought i am obviously have another version of perl :(
          Why i think that this an SDBM issue?
          The error you are getting :
          "Wide character in subroutine entry"
          means that you are trying to output/call function with wide utf8 character
          while this file handler is not setup to get this kind of data .
          A quick google search for "MLDBM utf8" approves this. :(
          So i guess you need a workaround or just try to use another DBM.

          > I suppose I misunderstood the purpose of this <meta> tag...so not
          > only output, but inputs get encoded as UTF-8 here?  Cool stuff....

          Sometimes the meta tag get ignored and you need the header,also make sure your
          server does not send such a header by his own (Apache sometimes does).

          > When, then, would I need to use decode_utf8?  I suppose it's for file-
          > reading or command-line input then?

          If you need perl to know it is  utf8  ,such as some of string manipulations.
          Again,You may also try another DBM ,change your StateDB setting .
          Maybe this is the best solution .
          Evgeny.

        Your message has been successfully submitted and would be delivered to recipients shortly.