Loading ...
Sorry, an error occurred while loading the content.

saving decoded unicode value to Session causes error

Expand Messages
  • Richard Yen
    Hi, Wondering if anyone has ever encountered this issue. Trying to internationalize my site, so at every input, I pass it through Encode::decode_utf8. However,
    Message 1 of 4 , Aug 28, 2007
    • 0 Attachment
      Hi,

      Wondering if anyone has ever encountered this issue.

      Trying to internationalize my site, so at every input, I pass it
      through Encode::decode_utf8.

      However, when I attempt to put the values into Session, I get the
      following message:
      > Errors Output
      > Wide character in subroutine entry at /usr/local/perl/lib/site_perl/
      > 5.8.7/MLDBM/Sync/SDBM_File.pm line 80, <GEN10> line 21. , /usr/
      > local/perl/lib/site_perl/5.8.7/Apache/ASP.pm line 1521
      > Debug Output
      > Use of uninitialized value in string ne at ../global/lib/
      > lib_filesystem.pm line 41.
      > Using an array as a reference is deprecated at ../global/lib/
      > lib_peer_review.pm line 389, line 21.
      > Wide character in subroutine entry at /usr/local/perl/lib/site_perl/
      > 5.8.7/MLDBM/Sync/SDBM_File.pm line 80, line 21. , /usr/local/perl/
      > lib/site_perl/5.8.7/Apache/ASP.pm line 1521

      An example is if I use the capital Omega character (option-z on mac
      keyboard), this error occurs when attempting to store into Session:

      > my $secret_answer = Encode::decode_utf8($main::Request->Form
      > ('secret_answer'));
      > $main::Session->{secret_answer} = $secret_answer;

      I've discovered that if I put the decoded value into a hash, and then
      put the hash into Session, it behaves normally. Also, if I don't
      decode at all, it behaves normally also.

      Would anyone know why this happens, or how to fix it?

      Thanks!
      --Richard

      ---------------------------------------------------------------------
      To unsubscribe, e-mail: asp-unsubscribe@...
      For additional commands, e-mail: asp-help@...
    • Tsirkin Evgeny
      It looks like that SDBM_File file handler or something else in it s internals does not know that you are storing (writing) utf8 records into it. I don t know
      Message 2 of 4 , Aug 29, 2007
      • 0 Attachment
        It looks like that SDBM_File file handler or something else in it's internals does not
        know that you are storing (writing) utf8 records into it.
        I don't know how to force it to know this :( maybe some SDBM_File expert here
        will know.
        However the workaround for this is to NOT mark the string you get as utf8 i.e.
        don't do Encode::decode_utf8. Just set your pages/server to send to the browser
        header saying that you need utf8 as request charset and you will be fine.

        <meta http-equiv="Content-Type" content="text/html; charset=utf-8">
        will do this or
        $Response->AddHeader()

        What this solution does NOT do - it does not validate the input to be legal utf8.
        But most of the time you will be fine.
        Hope this helps.
        Evgeny

        On 8/29/07, Richard Yen <dba@...> wrote:
        Hi,

        Wondering if anyone has ever encountered this issue.

        Trying to internationalize my site, so at every input, I pass it
        through Encode::decode_utf8.

        However, when I attempt to put the values into Session, I get the
        following message:
        > Errors Output
        > Wide character in subroutine entry at /usr/local/perl/lib/site_perl/
        > 5.8.7/MLDBM/Sync/SDBM_File.pm line 80, <GEN10> line 21. , /usr/
        > local/perl/lib/site_perl/5.8.7/Apache/ASP.pm line 1521
        > Debug Output
        > Use of uninitialized value in string ne at ../global/lib/
        > lib_filesystem.pm line 41.
        > Using an array as a reference is deprecated at ../global/lib/
        > lib_peer_review.pm line 389, line 21.
        > Wide character in subroutine entry at /usr/local/perl/lib/site_perl/
        > 5.8.7/MLDBM/Sync/SDBM_File.pm line 80, line 21. , /usr/local/perl/
        > lib/site_perl/5.8.7/Apache/ASP.pm line 1521

        An example is if I use the capital Omega character (option-z on mac
        keyboard), this error occurs when attempting to store into Session:

        > my $secret_answer = Encode::decode_utf8($main::Request->Form
        > ('secret_answer'));
        > $main::Session->{secret_answer} = $secret_answer;

        I've discovered that if I put the decoded value into a hash, and then
        put the hash into Session, it behaves normally.  Also, if I don't
        decode at all, it behaves normally also.

        Would anyone know why this happens, or how to fix it?

        Thanks!
        --Richard

        ---------------------------------------------------------------------
        To unsubscribe, e-mail: asp-unsubscribe@...
        For additional commands, e-mail: asp-help@...


      • Richard Yen
        ... Hmm, I thought that perhaps it was something with Apache::ASP, and that SDBM_File would be a dependency that was being passed bad data? ... I suppose I
        Message 3 of 4 , Aug 29, 2007
        • 0 Attachment
          On Aug 29, 2007, at 12:28 AM, Tsirkin Evgeny wrote:

          > It looks like that SDBM_File file handler or something else in it's
          > internals does not
          > know that you are storing (writing) utf8 records into it.
          > I don't know how to force it to know this :( maybe some SDBM_File
          > expert here
          > will know.
          Hmm, I thought that perhaps it was something with Apache::ASP, and
          that SDBM_File would be a dependency that was being passed bad data?

          > However the workaround for this is to NOT mark the string you get
          > as utf8 i.e.
          > don't do Encode::decode_utf8. Just set your pages/server to send to
          > the browser
          > header saying that you need utf8 as request charset and you will be
          > fine.
          >
          > <meta http-equiv="Content-Type" content="text/html; charset=utf-8">
          > will do this or
          > $Response->AddHeader()
          I suppose I misunderstood the purpose of this <meta> tag...so not
          only output, but inputs get encoded as UTF-8 here? Cool stuff....
          When, then, would I need to use decode_utf8? I suppose it's for file-
          reading or command-line input then?

          > What this solution does NOT do - it does not validate the input to
          > be legal utf8.
          True. Will keep that in mind.

          > But most of the time you will be fine.
          > Hope this helps.
          > Evgeny
          Thanks!
          --Richard



          >
          >
          > On 8/29/07, Richard Yen <dba@...> wrote: Hi,
          >
          > Wondering if anyone has ever encountered this issue.
          >
          > Trying to internationalize my site, so at every input, I pass it
          > through Encode::decode_utf8.
          >
          > However, when I attempt to put the values into Session, I get the
          > following message:
          > > Errors Output
          > > Wide character in subroutine entry at /usr/local/perl/lib/site_perl/
          > > 5.8.7/MLDBM/Sync/SDBM_File.pm line 80, <GEN10> line 21. , /usr/
          > > local/perl/lib/site_perl/5.8.7/Apache/ASP.pm line 1521
          > > Debug Output
          > > Use of uninitialized value in string ne at ../global/lib/
          > > lib_filesystem.pm line 41.
          > > Using an array as a reference is deprecated at ../global/lib/
          > > lib_peer_review.pm line 389, line 21.
          > > Wide character in subroutine entry at /usr/local/perl/lib/site_perl/
          > > 5.8.7/MLDBM/Sync/SDBM_File.pm line 80, line 21. , /usr/local/perl/
          > > lib/site_perl/5.8.7/Apache/ASP.pm line 1521
          >
          > An example is if I use the capital Omega character (option-z on mac
          > keyboard), this error occurs when attempting to store into Session:
          >
          > > my $secret_answer = Encode::decode_utf8($main::Request->Form
          > > ('secret_answer'));
          > > $main::Session->{secret_answer} = $secret_answer;
          >
          > I've discovered that if I put the decoded value into a hash, and then
          > put the hash into Session, it behaves normally. Also, if I don't
          > decode at all, it behaves normally also.
          >
          > Would anyone know why this happens, or how to fix it?
          >
          > Thanks!
          > --Richard
          >
          > ---------------------------------------------------------------------
          > To unsubscribe, e-mail: asp-unsubscribe@...
          > For additional commands, e-mail: asp-help@...
          >
          >


          ---------------------------------------------------------------------
          To unsubscribe, e-mail: asp-unsubscribe@...
          For additional commands, e-mail: asp-help@...
        • Tsirkin Evgeny
          Hi Richard! First of all i need to apologize: It is possible that i gave you a bad advice. I am not a Apache::ASP developer, just a user that is monitoring the
          Message 4 of 4 , Aug 30, 2007
          • 0 Attachment
            Hi Richard!
            First of all i need to apologize:
            It is possible that i gave you a bad advice.
            I am not a Apache::ASP developer, just a user that is monitoring the list
            to follow the development (if any is making place ,i hope :) ) and
            I have tried to help  .As NOT a developer i hope it is allowed for me to make
            mistakes.

            I will try to explain :
            If you setup your server and/or application to send the header saying that the
            data is utf8 encoded then what you get from client will also be utf8 encoded.
            That IS right .So,if you just use
            $Request->Form()
            to get the data posted you will get the utf8 encoded data and you probably
            WILL be fine ,the same as i have said in the prev. reply.
            However there IS a problem:
            The string you will get from $Request object although utf8 encoded will NOT be
            marked as utf8 encoded .In other words - you get the data fine and if you for example
            store it in DBMS you are fine,but perl does not know that it is utf8 encoded.For
            perl the data is just an array of bytes
            Why does it matter for perl to mark strings as utf8?If you try to use regular expr.
            or some string functions (like length ,i think) it may not work because perl will
            assume to be character 8 bit long which not always true for utf8.
            So ,my advise of just NOT using Encode::decode_utf8 may cause actually problems
            if you try to do some processing on the data you get.
            Actually, i personally do exactly what i have advised to you :
            Taking the data just like i get it from $Request and store it.
            In the rare cases when i need to do some processing one the localized
            data (like regular expressions match ) i am marking it as utf8.


            > Hmm, I thought that perhaps it was something with Apache::ASP, and
            > that SDBM_File would be a dependency that was being passed bad data?

            I have made a test and it worked just fine,here is the code:
               <html>
            <head>
            </head>
            <body>
            <form action="" method=post>
            <input type=text name=v>
            <input type=submit>
            </form>
            </body>
            </html>
            <%

                use Encode;
            my $v=Encode::decode_utf8($Request->Form('v'));
            $Session->{v}=$v;
            print $Session->{v};
            %>
            I am also using SDBM_File  .Thought i am obviously have another version of perl :(
            Why i think that this an SDBM issue?
            The error you are getting :
            "Wide character in subroutine entry"
            means that you are trying to output/call function with wide utf8 character
            while this file handler is not setup to get this kind of data .
            A quick google search for "MLDBM utf8" approves this. :(
            So i guess you need a workaround or just try to use another DBM.

            > I suppose I misunderstood the purpose of this <meta> tag...so not
            > only output, but inputs get encoded as UTF-8 here?  Cool stuff....

            Sometimes the meta tag get ignored and you need the header,also make sure your
            server does not send such a header by his own (Apache sometimes does).

            > When, then, would I need to use decode_utf8?  I suppose it's for file-
            > reading or command-line input then?

            If you need perl to know it is  utf8  ,such as some of string manipulations.
            Again,You may also try another DBM ,change your StateDB setting .
            Maybe this is the best solution .
            Evgeny.

          Your message has been successfully submitted and would be delivered to recipients shortly.