Loading ...
Sorry, an error occurred while loading the content.
 

Re: [soaplite] Base64 and Cyrillic

Expand Messages
  • Paul Kulchenko
    Hi, Sergei! It really depends on what (and how) you re trying to do that. Here is some information that can be helpful. First, if you do NOT specify type, your
    Message 1 of 12 , Oct 12, 2001
      Hi, Sergei!

      It really depends on what (and how) you're trying to do that.

      Here is some information that can be helpful.

      First, if you do NOT specify type, your string that has cyrillic
      characters might be encoded as base64 which doesn't hurt, but on
      other side (after decoding) encoding on this string (original) might
      be different from the rest of the message (which is utf8 for
      SOAP::Lite server).

      Second, you can always specify encoding using ->encoding() method on
      serializer or SOAP::Lite objects as described in cookbook
      (http://cookbook.soaplite.com/#internationalization and encoding).

      There is also recommendation on how to transcode your data from
      iso-8859-1 into utf8:

      (my $utf = '������') =~ tr/\0-\x{ff}//CU;

      which works only for 5.6.0. For 5.6.1 and later (and even for 5.6.0)
      you need to use pack:

      my $utf8 = pack('U*', unpack('C*', '������'));

      (there are other options, such as Unicode::String, Unicode::Map8,
      Encode, and others)

      There is a problem using *REAL* utf8 with 5.6.1 and later. You may
      NOT be able to send message that has utf8 chars in it properly. Here
      is why. LWP and other lowlevel libraries define length of the string
      using length() function that counts *chars* on multibyte characters
      (as in your case) instead of *bytes*. But on wire we're sending
      *bytes* regardless of used encoding. As the result, specified
      content-length is smaller than it should be (it counts chars instead
      of bytes) and there is no way to change it on high level ("use bytes"
      has lexical scope). After long discussion on p5p list where I was
      arguing that it should be done on LWP or lower level and Gisle
      arguing that it should be done on mine ;) or I shoudln't be sending
      utf8 at all, I decided to downgrade utf8 strings to bytes for all
      HTTP transports, so low-level libraries will work fine. I still think
      that it shouldn't be done on my level and you can switch it off if
      you like, but that's how it is now and will be included with the next
      version. There is no such problem with perl 5.6.0 where length
      calculates bytes. If you're curious about implementation details,
      it's done with

      $envelope = pack('C0A*', $envelope);

      but only if char length not equal byte length, so there is no penalty
      in most cases.

      Completing the picture, utf8 that XML::Parser returns is *not* a real
      utf8 from Perl's perspective. Strings are not marked as *utf* (at
      least not in current version). Which means that they are not handled
      as utf8 strings. Not a problem if you don't use utf8 characters, but
      you do. For you strings must be promoted into utf8 strings with:

      $string = pack('U0A*', $string);

      or something similar.

      I do NOT include this functionality for two reasons: XML::Parser
      should do that (I hope at some point it will) and, second, if you do
      it using Encode interface there is no performance penalty in
      upgrading strings into utf8 (string IS already utf8 encoded, just not
      marked as such), comparing to pack method.

      > What information have I to provide to understand what is wrong?
      As Aaron pointed, debug info will be helpful as well as information
      what you *expect* to get.

      Let me know if you have any comments, questions or suggestions. Thank
      you.

      Best wishes, Paul.

      --- Sergei Dolmatov <sergei@...> wrote:
      > Greetings!
      >
      > I test SOAP::Lite for a few days - and got that it's really cool
      > tool (and looks like it's more simple that Delphi implementation ;)
      >
      > But today got weird problem - when I return string with cyrillic
      > symbols it sometimes encodes it not properly.
      > So I have three functions that returns cyrillic strings - two of
      > them works correct (looks like always).
      >
      > What information have I to provide to understand what is wrong?
      >
      > --
      > Best regards,
      > Sergei Dolmatov.
      >
      > ------------------------ Yahoo! Groups Sponsor
      >
      > To unsubscribe from this group, send an email to:
      > soaplite-unsubscribe@yahoogroups.com
      >
      >
      >
      > Your use of Yahoo! Groups is subject to
      > http://docs.yahoo.com/info/terms/
      >
      >


      __________________________________________________
      Do You Yahoo!?
      Make a great connection at Yahoo! Personals.
      http://personals.yahoo.com
    • Paul Kulchenko
      Hi, Aaron! ... I would prefer iso-8859-1 for this matter ;) ... They still use it, but if it s not available XML::Parser::Lite will be use instead. ::Lite
      Message 2 of 12 , Oct 12, 2001
        Hi, Aaron!

        --- "Aaron E. Ross" <ross@...> wrote:
        > $soap->encoding('cyrillic')
        I would prefer 'iso-8859-1' for this matter ;)

        > problem too, since XML::Parser uses UTF8 internally. Although
        > IIRC newer versions of SOAP::Lite don't use XML::Parser.
        They still use it, but if it's not available XML::Parser::Lite will
        be use instead. ::Lite parser doesn't do utf8 decoding, so you'll get
        your data in original encoding.

        Best wishes, Paul.


        __________________________________________________
        Do You Yahoo!?
        Make a great connection at Yahoo! Personals.
        http://personals.yahoo.com
      • Sergei Dolmatov
        ... I m very stupid, but I didn t found a way how to do trace... :( I use on_debug(sub{print@_}) method - isn t it correct? But problem now is solved by adding
        Message 3 of 12 , Oct 16, 2001
          On Fri, Oct 12, 2001 at 09:15:31AM -0400, Aaron E. Ross wrote:
          > On Fri, Oct 12, 2001 at 03:45:18PM +0900, Sergei Dolmatov wrote:
          > >
          > > But today got weird problem - when I return string with cyrillic
          > > symbols it sometimes encodes it not properly.
          >
          > A dump of the request/response that is not working would be helpful. try
          >
          > use SOAP::Lite +trace => transport

          I'm very stupid, but I didn't found a way how to do trace... :(
          I use on_debug(sub{print@_}) method - isn't it correct?
          But problem now is solved by adding encoding('iso-8859-5') property as described below.

          And looks like name of encoding doesn't matter - when I change it to 'iso-8859-1' nothing changes in output.

          And how can I do something similar says in Delphi? Or I have to convert all my strings in UTF8 before sending to server?

          >
          > > So I have three functions that returns cyrillic strings - two of them
          > > works correct (looks like always).
          >
          > what error do you get? how do you know it's not working?
          >
          > what encoding are you using? if it's not UTF8 you'll need to tell the
          > serializer:
          >
          > $soap->encoding('cyrillic')
          >
          > I'm not sure what the equivalent is on the server side.
          >
          > if there isn't a round trip conversion with unicode, you've got another
          > problem too, since XML::Parser uses UTF8 internally. Although IIRC newer
          > versions of SOAP::Lite don't use XML::Parser.
          >
          > HTH, Aaron
          >

          --
          Best regards,
          Sergei Dolmatov.
        • Paul Kulchenko
          Hi, Sergei! ... It s correct. It does exactly the same as use SOAP::Lite +trace = debug = sub {print@_}; ... Right. You need to provide your strings in
          Message 4 of 12 , Oct 17, 2001
            Hi, Sergei!

            --- Sergei Dolmatov <sergei@...> wrote:
            > I'm very stupid, but I didn't found a way how to do trace... :(
            > I use on_debug(sub{print@_}) method - isn't it correct?
            It's correct. It does exactly the same as

            use SOAP::Lite +trace => debug => sub {print@_};

            > And looks like name of encoding doesn't matter - when I change it
            > to 'iso-8859-1' nothing changes in output.
            Right. You need to provide your strings in proper encoding and than
            specify what encoding to put in XML Declaration (and content-type)
            using encoding(). SOAP::Lite will not do transcoding for you, for a
            couple of reasons: there are different methods to do it for different
            Perl versions (support for utf8 is different also), you may want to
            use different methods depending on your performance requirements, you
            need to specify what is your current encoding to transcode, and
            moreover, SOAP::Lite will apply transcoding to the whole message,
            whereas you can apply it to specific string that needs to be
            transcoded.

            For your specific needs it's easy to add something like this
            (untested):

            sub SOAP::Serialized::as_iso_8859 {
            my $self = shift;
            my $value = decode_from_iso_8859_into_utf8(shift);
            $self->as_string($value, @_);
            }

            and then use SOAP::Data->type(iso_8859 => '������') instead of
            SOAP::Data->type(string => ....)

            Or even register this function in autotype lookup, so it'll detect
            128-255 characters and transcode those if required.

            > And how can I do something similar says in Delphi? Or I have to
            > convert all my strings in UTF8 before sending to server?
            Don't quite understand your question about Delphi. Did you mean COM
            interface? Something else?

            you may also find helpful my message on this topic
            (http://groups.yahoo.com/group/soaplite/message/904) if you missed
            it.

            Best wishes, Paul.


            __________________________________________________
            Do You Yahoo!?
            Make a great connection at Yahoo! Personals.
            http://personals.yahoo.com
          • Sergei Dolmatov
            ... Well, my english is very poor :( I d try to describe it more detailed - in perl client using SOAP::Lite I set encoding( iso-8859-1 ) property in request.
            Message 5 of 12 , Oct 19, 2001
              On Wed, Oct 17, 2001 at 03:13:24PM -0700, Paul Kulchenko wrote:
              > Hi, Sergei!
              >
              > > And how can I do something similar says in Delphi? Or I have to
              > > convert all my strings in UTF8 before sending to server?
              > Don't quite understand your question about Delphi. Did you mean COM
              > interface? Something else?

              Well, my english is very poor :(
              I'd try to describe it more detailed - in perl client using SOAP::Lite I set encoding('iso-8859-1') property in request.
              AFAIK, other clients may have not this value (so, I still didn't find something like that in Delphi's HTTPRio).
              And I got error 500 - wide characters in request.
              So question is - how can I set on server's side encoding property? And is it even possible?

              >
              > you may also find helpful my message on this topic
              > (http://groups.yahoo.com/group/soaplite/message/904) if you missed
              > it.

              Yes, it's helpful, but as far as I understood, it's again about client side?..

              --
              Regards,
              Sergei Dolmatov.
            • Paul Kulchenko
              Hi, Sergei! ... Unfortunately can t tell you anything about Delphi, but you definitely can change encoding on server side for SOAP::Lite. use: my $server =
              Message 6 of 12 , Oct 22, 2001
                Hi, Sergei!

                --- Sergei Dolmatov <sergei@...> wrote:
                > I'd try to describe it more detailed - in perl client using
                > SOAP::Lite I set encoding('iso-8859-1') property in request.
                > AFAIK, other clients may have not this value (so, I still didn't
                > find something like that in Delphi's HTTPRio).
                > And I got error 500 - wide characters in request.
                > So question is - how can I set on server's side encoding property?
                > And is it even possible?
                Unfortunately can't tell you anything about Delphi, but you
                definitely can change encoding on server side for SOAP::Lite. use:

                my $server = SOAP::Transport::HTTP::CGI # or whatever server you have
                -> encoding('isao-8859-1')
                -> dispatch_to(...)
                ;

                You can also transcode your data to utf8 and use default encoding.

                > Yes, it's helpful, but as far as I understood, it's again about
                > client side?..
                There is no big difference between client and server side. Interface
                is *slightly* different, but almost all that was said about client
                side is true about server side also.

                Best wishes, Paul.

                __________________________________________________
                Do You Yahoo!?
                Make a great connection at Yahoo! Personals.
                http://personals.yahoo.com
              • Sergei Dolmatov
                ... Well, I m really stupid... :( Output from error_log: Can t locate object method encoding via package SOAP::Transport::HTTP::CGI (perhaps you forgot to
                Message 7 of 12 , Oct 22, 2001
                  On Mon, Oct 22, 2001 at 07:46:26AM -0700, Paul Kulchenko wrote:
                  > Hi, Sergei!
                  >
                  > --- Sergei Dolmatov <sergei@...> wrote:
                  > > I'd try to describe it more detailed - in perl client using
                  > > SOAP::Lite I set encoding('iso-8859-1') property in request.
                  > > AFAIK, other clients may have not this value (so, I still didn't
                  > > find something like that in Delphi's HTTPRio).
                  > > And I got error 500 - wide characters in request.
                  > > So question is - how can I set on server's side encoding property?
                  > > And is it even possible?
                  > Unfortunately can't tell you anything about Delphi, but you
                  > definitely can change encoding on server side for SOAP::Lite. use:
                  >
                  > my $server = SOAP::Transport::HTTP::CGI # or whatever server you have
                  > -> encoding('isao-8859-1')
                  > -> dispatch_to(...)
                  > ;
                  >

                  Well, I'm really stupid... :(

                  Output from error_log:

                  Can't locate object method "encoding" via package "SOAP::Transport::HTTP::CGI" (perhaps you forgot to load "SOAP::Transport::HTTP::CGI"?) at /home/sergei/public_html/SoapShop.cgi line 5.

                  Where am I wrong?

                  --
                  Regards,
                  Sergei Dolmatov.
                • Paul Kulchenko
                  Hi, Sergei! ... No, you are not ;). Sorry, should read: my $server = SOAP::Transport::HTTP::CGI - dispatch_to(...) ;
                  Message 8 of 12 , Oct 23, 2001
                    Hi, Sergei!

                    > > my $server = SOAP::Transport::HTTP::CGI # or whatever server you
                    > have
                    > > -> encoding('isao-8859-1')
                    > > -> dispatch_to(...)
                    > > ;
                    > >
                    >
                    > Well, I'm really stupid... :(
                    No, you are not ;). Sorry, should read:

                    my $server = SOAP::Transport::HTTP::CGI
                    -> dispatch_to(...)
                    ;
                    $server->serializer->encoding('iso-8859-1');
                    $server->handle;

                    Best wishes, Paul.

                    --- Sergei Dolmatov <sergei@...> wrote:
                    > On Mon, Oct 22, 2001 at 07:46:26AM -0700, Paul Kulchenko wrote:
                    > > Hi, Sergei!
                    > >
                    > > --- Sergei Dolmatov <sergei@...> wrote:
                    > > > I'd try to describe it more detailed - in perl client using
                    > > > SOAP::Lite I set encoding('iso-8859-1') property in request.
                    > > > AFAIK, other clients may have not this value (so, I still
                    > didn't
                    > > > find something like that in Delphi's HTTPRio).
                    > > > And I got error 500 - wide characters in request.
                    > > > So question is - how can I set on server's side encoding
                    > property?
                    > > > And is it even possible?
                    > > Unfortunately can't tell you anything about Delphi, but you
                    > > definitely can change encoding on server side for SOAP::Lite.
                    > use:
                    > >
                    > > my $server = SOAP::Transport::HTTP::CGI # or whatever server you
                    > have
                    > > -> encoding('isao-8859-1')
                    > > -> dispatch_to(...)
                    > > ;
                    > >
                    >
                    > Well, I'm really stupid... :(
                    >
                    > Output from error_log:
                    >
                    > Can't locate object method "encoding" via package
                    > "SOAP::Transport::HTTP::CGI" (perhaps you forgot to load
                    > "SOAP::Transport::HTTP::CGI"?) at
                    > /home/sergei/public_html/SoapShop.cgi line 5.
                    >
                    > Where am I wrong?
                    >
                    > --
                    > Regards,
                    > Sergei Dolmatov.
                    >
                    > ------------------------ Yahoo! Groups Sponsor
                    >
                    > To unsubscribe from this group, send an email to:
                    > soaplite-unsubscribe@yahoogroups.com
                    >
                    >
                    >
                    > Your use of Yahoo! Groups is subject to
                    > http://docs.yahoo.com/info/terms/
                    >
                    >


                    __________________________________________________
                    Do You Yahoo!?
                    Make a great connection at Yahoo! Personals.
                    http://personals.yahoo.com
                  • Bill Jones
                    Hi All :) I have a basic question, I ve been playing with only a few days now but I am not sure how to implement the few examples I ve seen both here and on
                    Message 9 of 12 , Oct 23, 2001
                      Hi All :)

                      I have a basic question, I've been playing with only a few days now but I am
                      not sure how to 'implement' the few examples I've seen both here and on the
                      WWW.

                      I want to talk via SSL using basic authentication to:

                      http://207.203.95.95/bill/BrokerEmployee.asmx/ClassRoster?strBrokerID=ETB251
                      &strClassReferenceNumber=138887&strTerm=20021

                      (Sorry for the long URL, you may need to fix formatting errors caused by
                      posting.)

                      I posted a NTLM question the other day and got two very good responses about
                      various NTLM sites to get the protocol for implementing it in Perl (which I
                      am following up on :)

                      The remote server supports SOAP, GET, and POST - however, I am at a loss as
                      to how to use SOAP to do this. Ultimately I want to parse the XML - but I
                      am not sure I am even talking to the server correctly to start with.

                      Any thoughts or references would be most appreciated :)

                      -Sx-
                      William C (Bill) Jones
                      Lead, Courseware Support Analyst
                      (Lead e-Systems Developer)
                      Florida Community College at Jacksonville
                      501 West State Street, Rm 229
                      Jacksonville, Florida 32202-4030
                      wcjones@...
                      PHONE (904) 632-3089
                      FAX (904) 632-3007
                      __________________________________________________________________
                      "I know you believe you understand what you think I said, but I am
                      not sure you realize that what you heard is not what I meant."
                      ::: Richard Nixon
                    • alexukua
                      Example how to working with Ukraine text utf8 my $utf8= Åëåêòðîííà á³áë³îòåêà ; my $utf8 = pack U* , unpack U0U* , $utf8; my
                      Message 10 of 12 , Feb 18, 2009
                        Example how to working with Ukraine text utf8

                        my $utf8='Åëåêòðîííà á³áë³îòåêà';

                        my $utf8 = pack 'U*', unpack 'U0U*', $utf8;

                        my $title=SOAP::Data->name('title')->type('string')->value($utf8);
                        --- In soaplite@yahoogroups.com, Bill Jones <sneex@...> wrote:
                        >
                        > Hi All :)
                        >
                        > I have a basic question, I've been playing with only a few days now
                        but I am
                        > not sure how to 'implement' the few examples I've seen both here and
                        on the
                        > WWW.
                        >
                        > I want to talk via SSL using basic authentication to:
                        >
                        >
                        http://207.203.95.95/bill/BrokerEmployee.asmx/ClassRoster?strBrokerID=ETB251
                        > &strClassReferenceNumber=138887&strTerm=20021
                        >
                        > (Sorry for the long URL, you may need to fix formatting errors caused by
                        > posting.)
                        >
                        > I posted a NTLM question the other day and got two very good
                        responses about
                        > various NTLM sites to get the protocol for implementing it in Perl
                        (which I
                        > am following up on :)
                        >
                        > The remote server supports SOAP, GET, and POST - however, I am at a
                        loss as
                        > to how to use SOAP to do this. Ultimately I want to parse the XML -
                        but I
                        > am not sure I am even talking to the server correctly to start with.
                        >
                        > Any thoughts or references would be most appreciated :)
                        >
                        > -Sx-
                        > William C (Bill) Jones
                        > Lead, Courseware Support Analyst
                        > (Lead e-Systems Developer)
                        > Florida Community College at Jacksonville
                        > 501 West State Street, Rm 229
                        > Jacksonville, Florida 32202-4030
                        > wcjones@...
                        > PHONE (904) 632-3089
                        > FAX (904) 632-3007
                        > __________________________________________________________________
                        > "I know you believe you understand what you think I said, but I am
                        > not sure you realize that what you heard is not what I meant."
                        > ::: Richard Nixon
                        >
                      Your message has been successfully submitted and would be delivered to recipients shortly.