Loading ...
Sorry, an error occurred while loading the content.

Base64 and Cyrillic

Expand Messages
  • Sergei Dolmatov
    Greetings! I test SOAP::Lite for a few days - and got that it s really cool tool (and looks like it s more simple that Delphi implementation ;) But today got
    Message 1 of 12 , Oct 11, 2001
    • 0 Attachment
      Greetings!

      I test SOAP::Lite for a few days - and got that it's really cool tool (and looks like it's more simple that Delphi implementation ;)

      But today got weird problem - when I return string with cyrillic symbols it sometimes encodes it not properly.
      So I have three functions that returns cyrillic strings - two of them works correct (looks like always).

      What information have I to provide to understand what is wrong?

      --
      Best regards,
      Sergei Dolmatov.
    • Aaron E. Ross
      ... A dump of the request/response that is not working would be helpful. try use SOAP::Lite +trace = transport ... what error do you get? how do you know it s
      Message 2 of 12 , Oct 12, 2001
      • 0 Attachment
        On Fri, Oct 12, 2001 at 03:45:18PM +0900, Sergei Dolmatov wrote:
        >
        > But today got weird problem - when I return string with cyrillic
        > symbols it sometimes encodes it not properly.

        A dump of the request/response that is not working would be helpful. try

        use SOAP::Lite +trace => transport

        > So I have three functions that returns cyrillic strings - two of them
        > works correct (looks like always).

        what error do you get? how do you know it's not working?

        what encoding are you using? if it's not UTF8 you'll need to tell the
        serializer:

        $soap->encoding('cyrillic')

        I'm not sure what the equivalent is on the server side.

        if there isn't a round trip conversion with unicode, you've got another
        problem too, since XML::Parser uses UTF8 internally. Although IIRC newer
        versions of SOAP::Lite don't use XML::Parser.

        HTH, Aaron
      • Paul Kulchenko
        Hi, Sergei! It really depends on what (and how) you re trying to do that. Here is some information that can be helpful. First, if you do NOT specify type, your
        Message 3 of 12 , Oct 12, 2001
        • 0 Attachment
          Hi, Sergei!

          It really depends on what (and how) you're trying to do that.

          Here is some information that can be helpful.

          First, if you do NOT specify type, your string that has cyrillic
          characters might be encoded as base64 which doesn't hurt, but on
          other side (after decoding) encoding on this string (original) might
          be different from the rest of the message (which is utf8 for
          SOAP::Lite server).

          Second, you can always specify encoding using ->encoding() method on
          serializer or SOAP::Lite objects as described in cookbook
          (http://cookbook.soaplite.com/#internationalization and encoding).

          There is also recommendation on how to transcode your data from
          iso-8859-1 into utf8:

          (my $utf = '������') =~ tr/\0-\x{ff}//CU;

          which works only for 5.6.0. For 5.6.1 and later (and even for 5.6.0)
          you need to use pack:

          my $utf8 = pack('U*', unpack('C*', '������'));

          (there are other options, such as Unicode::String, Unicode::Map8,
          Encode, and others)

          There is a problem using *REAL* utf8 with 5.6.1 and later. You may
          NOT be able to send message that has utf8 chars in it properly. Here
          is why. LWP and other lowlevel libraries define length of the string
          using length() function that counts *chars* on multibyte characters
          (as in your case) instead of *bytes*. But on wire we're sending
          *bytes* regardless of used encoding. As the result, specified
          content-length is smaller than it should be (it counts chars instead
          of bytes) and there is no way to change it on high level ("use bytes"
          has lexical scope). After long discussion on p5p list where I was
          arguing that it should be done on LWP or lower level and Gisle
          arguing that it should be done on mine ;) or I shoudln't be sending
          utf8 at all, I decided to downgrade utf8 strings to bytes for all
          HTTP transports, so low-level libraries will work fine. I still think
          that it shouldn't be done on my level and you can switch it off if
          you like, but that's how it is now and will be included with the next
          version. There is no such problem with perl 5.6.0 where length
          calculates bytes. If you're curious about implementation details,
          it's done with

          $envelope = pack('C0A*', $envelope);

          but only if char length not equal byte length, so there is no penalty
          in most cases.

          Completing the picture, utf8 that XML::Parser returns is *not* a real
          utf8 from Perl's perspective. Strings are not marked as *utf* (at
          least not in current version). Which means that they are not handled
          as utf8 strings. Not a problem if you don't use utf8 characters, but
          you do. For you strings must be promoted into utf8 strings with:

          $string = pack('U0A*', $string);

          or something similar.

          I do NOT include this functionality for two reasons: XML::Parser
          should do that (I hope at some point it will) and, second, if you do
          it using Encode interface there is no performance penalty in
          upgrading strings into utf8 (string IS already utf8 encoded, just not
          marked as such), comparing to pack method.

          > What information have I to provide to understand what is wrong?
          As Aaron pointed, debug info will be helpful as well as information
          what you *expect* to get.

          Let me know if you have any comments, questions or suggestions. Thank
          you.

          Best wishes, Paul.

          --- Sergei Dolmatov <sergei@...> wrote:
          > Greetings!
          >
          > I test SOAP::Lite for a few days - and got that it's really cool
          > tool (and looks like it's more simple that Delphi implementation ;)
          >
          > But today got weird problem - when I return string with cyrillic
          > symbols it sometimes encodes it not properly.
          > So I have three functions that returns cyrillic strings - two of
          > them works correct (looks like always).
          >
          > What information have I to provide to understand what is wrong?
          >
          > --
          > Best regards,
          > Sergei Dolmatov.
          >
          > ------------------------ Yahoo! Groups Sponsor
          >
          > To unsubscribe from this group, send an email to:
          > soaplite-unsubscribe@yahoogroups.com
          >
          >
          >
          > Your use of Yahoo! Groups is subject to
          > http://docs.yahoo.com/info/terms/
          >
          >


          __________________________________________________
          Do You Yahoo!?
          Make a great connection at Yahoo! Personals.
          http://personals.yahoo.com
        • Paul Kulchenko
          Hi, Aaron! ... I would prefer iso-8859-1 for this matter ;) ... They still use it, but if it s not available XML::Parser::Lite will be use instead. ::Lite
          Message 4 of 12 , Oct 12, 2001
          • 0 Attachment
            Hi, Aaron!

            --- "Aaron E. Ross" <ross@...> wrote:
            > $soap->encoding('cyrillic')
            I would prefer 'iso-8859-1' for this matter ;)

            > problem too, since XML::Parser uses UTF8 internally. Although
            > IIRC newer versions of SOAP::Lite don't use XML::Parser.
            They still use it, but if it's not available XML::Parser::Lite will
            be use instead. ::Lite parser doesn't do utf8 decoding, so you'll get
            your data in original encoding.

            Best wishes, Paul.


            __________________________________________________
            Do You Yahoo!?
            Make a great connection at Yahoo! Personals.
            http://personals.yahoo.com
          • Sergei Dolmatov
            ... I m very stupid, but I didn t found a way how to do trace... :( I use on_debug(sub{print@_}) method - isn t it correct? But problem now is solved by adding
            Message 5 of 12 , Oct 16, 2001
            • 0 Attachment
              On Fri, Oct 12, 2001 at 09:15:31AM -0400, Aaron E. Ross wrote:
              > On Fri, Oct 12, 2001 at 03:45:18PM +0900, Sergei Dolmatov wrote:
              > >
              > > But today got weird problem - when I return string with cyrillic
              > > symbols it sometimes encodes it not properly.
              >
              > A dump of the request/response that is not working would be helpful. try
              >
              > use SOAP::Lite +trace => transport

              I'm very stupid, but I didn't found a way how to do trace... :(
              I use on_debug(sub{print@_}) method - isn't it correct?
              But problem now is solved by adding encoding('iso-8859-5') property as described below.

              And looks like name of encoding doesn't matter - when I change it to 'iso-8859-1' nothing changes in output.

              And how can I do something similar says in Delphi? Or I have to convert all my strings in UTF8 before sending to server?

              >
              > > So I have three functions that returns cyrillic strings - two of them
              > > works correct (looks like always).
              >
              > what error do you get? how do you know it's not working?
              >
              > what encoding are you using? if it's not UTF8 you'll need to tell the
              > serializer:
              >
              > $soap->encoding('cyrillic')
              >
              > I'm not sure what the equivalent is on the server side.
              >
              > if there isn't a round trip conversion with unicode, you've got another
              > problem too, since XML::Parser uses UTF8 internally. Although IIRC newer
              > versions of SOAP::Lite don't use XML::Parser.
              >
              > HTH, Aaron
              >

              --
              Best regards,
              Sergei Dolmatov.
            • Paul Kulchenko
              Hi, Sergei! ... It s correct. It does exactly the same as use SOAP::Lite +trace = debug = sub {print@_}; ... Right. You need to provide your strings in
              Message 6 of 12 , Oct 17, 2001
              • 0 Attachment
                Hi, Sergei!

                --- Sergei Dolmatov <sergei@...> wrote:
                > I'm very stupid, but I didn't found a way how to do trace... :(
                > I use on_debug(sub{print@_}) method - isn't it correct?
                It's correct. It does exactly the same as

                use SOAP::Lite +trace => debug => sub {print@_};

                > And looks like name of encoding doesn't matter - when I change it
                > to 'iso-8859-1' nothing changes in output.
                Right. You need to provide your strings in proper encoding and than
                specify what encoding to put in XML Declaration (and content-type)
                using encoding(). SOAP::Lite will not do transcoding for you, for a
                couple of reasons: there are different methods to do it for different
                Perl versions (support for utf8 is different also), you may want to
                use different methods depending on your performance requirements, you
                need to specify what is your current encoding to transcode, and
                moreover, SOAP::Lite will apply transcoding to the whole message,
                whereas you can apply it to specific string that needs to be
                transcoded.

                For your specific needs it's easy to add something like this
                (untested):

                sub SOAP::Serialized::as_iso_8859 {
                my $self = shift;
                my $value = decode_from_iso_8859_into_utf8(shift);
                $self->as_string($value, @_);
                }

                and then use SOAP::Data->type(iso_8859 => '������') instead of
                SOAP::Data->type(string => ....)

                Or even register this function in autotype lookup, so it'll detect
                128-255 characters and transcode those if required.

                > And how can I do something similar says in Delphi? Or I have to
                > convert all my strings in UTF8 before sending to server?
                Don't quite understand your question about Delphi. Did you mean COM
                interface? Something else?

                you may also find helpful my message on this topic
                (http://groups.yahoo.com/group/soaplite/message/904) if you missed
                it.

                Best wishes, Paul.


                __________________________________________________
                Do You Yahoo!?
                Make a great connection at Yahoo! Personals.
                http://personals.yahoo.com
              • Sergei Dolmatov
                ... Well, my english is very poor :( I d try to describe it more detailed - in perl client using SOAP::Lite I set encoding( iso-8859-1 ) property in request.
                Message 7 of 12 , Oct 19, 2001
                • 0 Attachment
                  On Wed, Oct 17, 2001 at 03:13:24PM -0700, Paul Kulchenko wrote:
                  > Hi, Sergei!
                  >
                  > > And how can I do something similar says in Delphi? Or I have to
                  > > convert all my strings in UTF8 before sending to server?
                  > Don't quite understand your question about Delphi. Did you mean COM
                  > interface? Something else?

                  Well, my english is very poor :(
                  I'd try to describe it more detailed - in perl client using SOAP::Lite I set encoding('iso-8859-1') property in request.
                  AFAIK, other clients may have not this value (so, I still didn't find something like that in Delphi's HTTPRio).
                  And I got error 500 - wide characters in request.
                  So question is - how can I set on server's side encoding property? And is it even possible?

                  >
                  > you may also find helpful my message on this topic
                  > (http://groups.yahoo.com/group/soaplite/message/904) if you missed
                  > it.

                  Yes, it's helpful, but as far as I understood, it's again about client side?..

                  --
                  Regards,
                  Sergei Dolmatov.
                • Paul Kulchenko
                  Hi, Sergei! ... Unfortunately can t tell you anything about Delphi, but you definitely can change encoding on server side for SOAP::Lite. use: my $server =
                  Message 8 of 12 , Oct 22, 2001
                  • 0 Attachment
                    Hi, Sergei!

                    --- Sergei Dolmatov <sergei@...> wrote:
                    > I'd try to describe it more detailed - in perl client using
                    > SOAP::Lite I set encoding('iso-8859-1') property in request.
                    > AFAIK, other clients may have not this value (so, I still didn't
                    > find something like that in Delphi's HTTPRio).
                    > And I got error 500 - wide characters in request.
                    > So question is - how can I set on server's side encoding property?
                    > And is it even possible?
                    Unfortunately can't tell you anything about Delphi, but you
                    definitely can change encoding on server side for SOAP::Lite. use:

                    my $server = SOAP::Transport::HTTP::CGI # or whatever server you have
                    -> encoding('isao-8859-1')
                    -> dispatch_to(...)
                    ;

                    You can also transcode your data to utf8 and use default encoding.

                    > Yes, it's helpful, but as far as I understood, it's again about
                    > client side?..
                    There is no big difference between client and server side. Interface
                    is *slightly* different, but almost all that was said about client
                    side is true about server side also.

                    Best wishes, Paul.

                    __________________________________________________
                    Do You Yahoo!?
                    Make a great connection at Yahoo! Personals.
                    http://personals.yahoo.com
                  • Sergei Dolmatov
                    ... Well, I m really stupid... :( Output from error_log: Can t locate object method encoding via package SOAP::Transport::HTTP::CGI (perhaps you forgot to
                    Message 9 of 12 , Oct 22, 2001
                    • 0 Attachment
                      On Mon, Oct 22, 2001 at 07:46:26AM -0700, Paul Kulchenko wrote:
                      > Hi, Sergei!
                      >
                      > --- Sergei Dolmatov <sergei@...> wrote:
                      > > I'd try to describe it more detailed - in perl client using
                      > > SOAP::Lite I set encoding('iso-8859-1') property in request.
                      > > AFAIK, other clients may have not this value (so, I still didn't
                      > > find something like that in Delphi's HTTPRio).
                      > > And I got error 500 - wide characters in request.
                      > > So question is - how can I set on server's side encoding property?
                      > > And is it even possible?
                      > Unfortunately can't tell you anything about Delphi, but you
                      > definitely can change encoding on server side for SOAP::Lite. use:
                      >
                      > my $server = SOAP::Transport::HTTP::CGI # or whatever server you have
                      > -> encoding('isao-8859-1')
                      > -> dispatch_to(...)
                      > ;
                      >

                      Well, I'm really stupid... :(

                      Output from error_log:

                      Can't locate object method "encoding" via package "SOAP::Transport::HTTP::CGI" (perhaps you forgot to load "SOAP::Transport::HTTP::CGI"?) at /home/sergei/public_html/SoapShop.cgi line 5.

                      Where am I wrong?

                      --
                      Regards,
                      Sergei Dolmatov.
                    • Paul Kulchenko
                      Hi, Sergei! ... No, you are not ;). Sorry, should read: my $server = SOAP::Transport::HTTP::CGI - dispatch_to(...) ;
                      Message 10 of 12 , Oct 23, 2001
                      • 0 Attachment
                        Hi, Sergei!

                        > > my $server = SOAP::Transport::HTTP::CGI # or whatever server you
                        > have
                        > > -> encoding('isao-8859-1')
                        > > -> dispatch_to(...)
                        > > ;
                        > >
                        >
                        > Well, I'm really stupid... :(
                        No, you are not ;). Sorry, should read:

                        my $server = SOAP::Transport::HTTP::CGI
                        -> dispatch_to(...)
                        ;
                        $server->serializer->encoding('iso-8859-1');
                        $server->handle;

                        Best wishes, Paul.

                        --- Sergei Dolmatov <sergei@...> wrote:
                        > On Mon, Oct 22, 2001 at 07:46:26AM -0700, Paul Kulchenko wrote:
                        > > Hi, Sergei!
                        > >
                        > > --- Sergei Dolmatov <sergei@...> wrote:
                        > > > I'd try to describe it more detailed - in perl client using
                        > > > SOAP::Lite I set encoding('iso-8859-1') property in request.
                        > > > AFAIK, other clients may have not this value (so, I still
                        > didn't
                        > > > find something like that in Delphi's HTTPRio).
                        > > > And I got error 500 - wide characters in request.
                        > > > So question is - how can I set on server's side encoding
                        > property?
                        > > > And is it even possible?
                        > > Unfortunately can't tell you anything about Delphi, but you
                        > > definitely can change encoding on server side for SOAP::Lite.
                        > use:
                        > >
                        > > my $server = SOAP::Transport::HTTP::CGI # or whatever server you
                        > have
                        > > -> encoding('isao-8859-1')
                        > > -> dispatch_to(...)
                        > > ;
                        > >
                        >
                        > Well, I'm really stupid... :(
                        >
                        > Output from error_log:
                        >
                        > Can't locate object method "encoding" via package
                        > "SOAP::Transport::HTTP::CGI" (perhaps you forgot to load
                        > "SOAP::Transport::HTTP::CGI"?) at
                        > /home/sergei/public_html/SoapShop.cgi line 5.
                        >
                        > Where am I wrong?
                        >
                        > --
                        > Regards,
                        > Sergei Dolmatov.
                        >
                        > ------------------------ Yahoo! Groups Sponsor
                        >
                        > To unsubscribe from this group, send an email to:
                        > soaplite-unsubscribe@yahoogroups.com
                        >
                        >
                        >
                        > Your use of Yahoo! Groups is subject to
                        > http://docs.yahoo.com/info/terms/
                        >
                        >


                        __________________________________________________
                        Do You Yahoo!?
                        Make a great connection at Yahoo! Personals.
                        http://personals.yahoo.com
                      • Bill Jones
                        Hi All :) I have a basic question, I ve been playing with only a few days now but I am not sure how to implement the few examples I ve seen both here and on
                        Message 11 of 12 , Oct 23, 2001
                        • 0 Attachment
                          Hi All :)

                          I have a basic question, I've been playing with only a few days now but I am
                          not sure how to 'implement' the few examples I've seen both here and on the
                          WWW.

                          I want to talk via SSL using basic authentication to:

                          http://207.203.95.95/bill/BrokerEmployee.asmx/ClassRoster?strBrokerID=ETB251
                          &strClassReferenceNumber=138887&strTerm=20021

                          (Sorry for the long URL, you may need to fix formatting errors caused by
                          posting.)

                          I posted a NTLM question the other day and got two very good responses about
                          various NTLM sites to get the protocol for implementing it in Perl (which I
                          am following up on :)

                          The remote server supports SOAP, GET, and POST - however, I am at a loss as
                          to how to use SOAP to do this. Ultimately I want to parse the XML - but I
                          am not sure I am even talking to the server correctly to start with.

                          Any thoughts or references would be most appreciated :)

                          -Sx-
                          William C (Bill) Jones
                          Lead, Courseware Support Analyst
                          (Lead e-Systems Developer)
                          Florida Community College at Jacksonville
                          501 West State Street, Rm 229
                          Jacksonville, Florida 32202-4030
                          wcjones@...
                          PHONE (904) 632-3089
                          FAX (904) 632-3007
                          __________________________________________________________________
                          "I know you believe you understand what you think I said, but I am
                          not sure you realize that what you heard is not what I meant."
                          ::: Richard Nixon
                        • alexukua
                          Example how to working with Ukraine text utf8 my $utf8= Åëåêòðîííà á³áë³îòåêà ; my $utf8 = pack U* , unpack U0U* , $utf8; my
                          Message 12 of 12 , Feb 18, 2009
                          • 0 Attachment
                            Example how to working with Ukraine text utf8

                            my $utf8='Åëåêòðîííà á³áë³îòåêà';

                            my $utf8 = pack 'U*', unpack 'U0U*', $utf8;

                            my $title=SOAP::Data->name('title')->type('string')->value($utf8);
                            --- In soaplite@yahoogroups.com, Bill Jones <sneex@...> wrote:
                            >
                            > Hi All :)
                            >
                            > I have a basic question, I've been playing with only a few days now
                            but I am
                            > not sure how to 'implement' the few examples I've seen both here and
                            on the
                            > WWW.
                            >
                            > I want to talk via SSL using basic authentication to:
                            >
                            >
                            http://207.203.95.95/bill/BrokerEmployee.asmx/ClassRoster?strBrokerID=ETB251
                            > &strClassReferenceNumber=138887&strTerm=20021
                            >
                            > (Sorry for the long URL, you may need to fix formatting errors caused by
                            > posting.)
                            >
                            > I posted a NTLM question the other day and got two very good
                            responses about
                            > various NTLM sites to get the protocol for implementing it in Perl
                            (which I
                            > am following up on :)
                            >
                            > The remote server supports SOAP, GET, and POST - however, I am at a
                            loss as
                            > to how to use SOAP to do this. Ultimately I want to parse the XML -
                            but I
                            > am not sure I am even talking to the server correctly to start with.
                            >
                            > Any thoughts or references would be most appreciated :)
                            >
                            > -Sx-
                            > William C (Bill) Jones
                            > Lead, Courseware Support Analyst
                            > (Lead e-Systems Developer)
                            > Florida Community College at Jacksonville
                            > 501 West State Street, Rm 229
                            > Jacksonville, Florida 32202-4030
                            > wcjones@...
                            > PHONE (904) 632-3089
                            > FAX (904) 632-3007
                            > __________________________________________________________________
                            > "I know you believe you understand what you think I said, but I am
                            > not sure you realize that what you heard is not what I meant."
                            > ::: Richard Nixon
                            >
                          Your message has been successfully submitted and would be delivered to recipients shortly.