Loading ...
Sorry, an error occurred while loading the content.

Re: [soaplite] SOAP::Lite architecture

Expand Messages
  • Dirk Eddelbuettel
    ... I d be far from claiming that my system is representative, but to deliver just one random data point edd@homebud:/usr/share/perl5 find . -name *.pm -size
    Message 1 of 9 , Jan 20, 2002
    • 0 Attachment
      On Sun, Jan 20, 2002 at 05:04:16PM -0500, Sam Tregar wrote:
      > Why is SOAP::Lite stored in one module file? I ask for two reasons:
      >
      > 1) It seems to me that memory usage could be lowered if modules were
      > only loaded into memory when actually used. This can be done through
      > require, autouse or (shudder) the AutoLoader.
      >
      > 2) I've never seen a module distribution this large in one file before.
      > I'm writing about CPAN module construction at the moment - if there
      > are reasons to recommend doing things this way I'd like to hear them!

      I'd be far from claiming that my system is representative, but to deliver
      just one random data point

      edd@homebud:/usr/share/perl5> find . -name \*.pm -size +100k | xargs ls -lGh
      -rw-r--r-- 1 root 203k Apr 15 2001 ./Date/Manip.pm
      -rw-r--r-- 1 root 112k Mar 10 2001 ./HTML/Element.pm
      -rw-r--r-- 1 root 157k Oct 22 09:42 ./SOAP/Lite.pm
      -rw-r--r-- 1 root 113k Oct 22 18:34 ./Spreadsheet/WriteExcel.pm

      we see that ar least one all-Perl module exceeds the file size of SOAP::Lite.

      Dirk

      --
      Good judgment comes from experience; experience comes from bad judgment.
      -- F. Brooks
    • Sam Tregar
      ... Interesting. And does Data::Manip also contain numerous sub-modules packaged interally? ... time passes as I install Date::Manip ... Nope. Its size
      Message 2 of 9 , Jan 20, 2002
      • 0 Attachment
        On Sun, 20 Jan 2002, Dirk Eddelbuettel wrote:

        > I'd be far from claiming that my system is representative, but to deliver
        > just one random data point
        >
        > edd@homebud:/usr/share/perl5> find . -name \*.pm -size +100k | xargs ls -lGh
        > -rw-r--r-- 1 root 203k Apr 15 2001 ./Date/Manip.pm
        > -rw-r--r-- 1 root 112k Mar 10 2001 ./HTML/Element.pm
        > -rw-r--r-- 1 root 157k Oct 22 09:42 ./SOAP/Lite.pm
        > -rw-r--r-- 1 root 113k Oct 22 18:34 ./Spreadsheet/WriteExcel.pm
        >
        > we see that ar least one all-Perl module exceeds the file size of SOAP::Lite.

        Interesting. And does Data::Manip also contain numerous sub-modules
        packaged interally? ... time passes as I install Date::Manip ... Nope.
        Its size seems to come from some radical manual inlining of logic (judging
        from the comment on line 3047) as well as some largish tables of
        locale-specific data. The latter in particular could probably be moved
        into locale-specific sub-modules or AutoLoaded. After all, why should I
        have to load the entire set of Italian date names into memory if I'll
        never use them?

        So, I guess I think my criticism stands even if it could also be applied
        to other CPAN modules. How arrogant, eh? Well, if there's fire below
        we're all gonna go; check out my HTML::Template for a good example of not
        following one's own advice (a single .pm >100k with a bunch of features
        that most people will never use).

        -sam
      • Paul Kulchenko
        Hi, Sam! Both points are valid. First of all, there WILL be redesign, but not VERY soon and I will ask for your suggestions re new design. Second, to my
        Message 3 of 9 , Jan 20, 2002
        • 0 Attachment
          Hi, Sam!

          Both points are valid. First of all, there WILL be redesign, but not
          VERY soon and I will ask for your suggestions re new design. Second,
          to my defense, only 2900 lines out of 5000 are actual code, the rest
          of it is documentation and is not processed, because it sits after
          __END__

          Memory usage can definitely be lowered, but not because of
          repackaging (see below), but slightly changing interactions between
          internal modules. I'm working on it.

          Psychological aspect. I started my module when SOAP.pm already
          existed and if you take a look into its distribution you'll find
          there SOAP.pm, sixteen (!) files in SOAP directory and 4 files for
          HTTP transport. I just got lost jumping between all of them and when
          I started my module I decided that it will NOT be like this.

          What we have now is one module for SOAP, one module for every
          transport and one test module. All other modules are loaded ONLY when
          needed.

          Let's take a look inside and see what's required for client and
          server.

          SOAP::Constants -- both
          SOAP::Utils -- both
          SOAP::Transport -- client (50 lines)
          SOAP::Fault -- server, both in the future
          SOAP::Data -- both
          SOAP::Header -- both
          SOAP::Serializer -- both
          SOAP::Parser -- both
          SOAP::MIMEParser -- server, not always (80 lines)
          SOAP::SOM -- both
          SOAP::Deserializer -- both
          SOAP::Client -- client (10 lines)
          SOAP::Server::* -- server (220 lines)
          SOAP::Trace -- both
          SOAP::Custom::XML::* -- both, not always (50 lines)
          SOAP::Schema::* -- client, not always (250 lines)

          we can save about 300 lines for client and server, approx 10%.

          > Also, any reason why SOAP::Lite doesn't use its own namespace for
          > internal
          > modules (for example by using SOAP::Trace rather than
          > SOAP::Lite::Trace)?
          No particular reason. I thought it's not convenient to write
          SOAP::Lite::Data, SOAP::Lite::Serializer, SOAP::Lite::Parser, and so
          on, and if you don't do it for some module, it doesn't make much
          sense to do it for others. Even considering this I was very careful
          to not interfere with modules from SOAP module (Keith Brown,
          DevelopMentor), the only SOAP module for Perl available at that time
          and I'm pretty sure even now you can use them in one script (though I
          didn't try it with last versions).

          Let me know if you think it's unreasonable, I'll definitely consider
          what can be done to fix this.

          One more reason why it's better to keep modules in separate files, is
          that you can use 'use/require' in this case. It can't be easily done
          when modules are packaged in one file. I would like to hear other
          reasons why do NOT keep them together ;). Thank you.

          Best wishes, Paul.

          --- Sam Tregar <sam@...> wrote:
          > Why is SOAP::Lite stored in one module file? I ask for two
          > reasons:
          >
          > 1) It seems to me that memory usage could be lowered if modules
          > were
          > only loaded into memory when actually used. This can be done
          > through
          > require, autouse or (shudder) the AutoLoader.
          >
          > 2) I've never seen a module distribution this large in one file
          > before.
          > I'm writing about CPAN module construction at the moment - if
          > there
          > are reasons to recommend doing things this way I'd like to hear
          > them!
          >
          > Also, any reason why SOAP::Lite doesn't use its own namespace for
          > internal
          > modules (for example by using SOAP::Trace rather than
          > SOAP::Lite::Trace)?
          >
          > -sam
          >
          >
          >
          > ------------------------ Yahoo! Groups Sponsor
          >
          > To unsubscribe from this group, send an email to:
          > soaplite-unsubscribe@yahoogroups.com
          >
          >
          >
          > Your use of Yahoo! Groups is subject to
          > http://docs.yahoo.com/info/terms/
          >
          >



          __________________________________________________
          Do You Yahoo!?
          Send FREE video emails in Yahoo! Mail!
          http://promo.yahoo.com/videomail/
        • Sam Tregar
          ... Sounds good to me. ... Ok, I guess this works out to an esthetic judgment then. I had an opposite reaction while working my way through the SOAP::Lite
          Message 4 of 9 , Jan 20, 2002
          • 0 Attachment
            On Sun, 20 Jan 2002, Paul Kulchenko wrote:

            > Both points are valid. First of all, there WILL be redesign, but not
            > VERY soon and I will ask for your suggestions re new design.

            Sounds good to me.

            > Psychological aspect. I started my module when SOAP.pm already
            > existed and if you take a look into its distribution you'll find
            > there SOAP.pm, sixteen (!) files in SOAP directory and 4 files for
            > HTTP transport. I just got lost jumping between all of them and when
            > I started my module I decided that it will NOT be like this.

            Ok, I guess this works out to an esthetic judgment then. I had an
            opposite reaction while working my way through the SOAP::Lite source - I
            wished it was more compartmentalized so that I could read through it in
            smaller chunks. I can see that this is more of judgment call than
            anything else...

            > we can save about 300 lines for client and server, approx 10%.

            This could be a significant saving - maybe 1MB or more depending on what's
            actually in those lines.

            > No particular reason. I thought it's not convenient to write
            > SOAP::Lite::Data, SOAP::Lite::Serializer, SOAP::Lite::Parser, and so
            > on, and if you don't do it for some module, it doesn't make much
            > sense to do it for others. Even considering this I was very careful
            > to not interfere with modules from SOAP module (Keith Brown,
            > DevelopMentor), the only SOAP module for Perl available at that time
            > and I'm pretty sure even now you can use them in one script (though I
            > didn't try it with last versions).

            It's definitely more convenient. However, considering that you don't
            "own" the SOAP:: namespace, it's not very polite to create modules there
            without registering them. If someone else wants to write a SOAP tracing
            package they have to know that SOAP::Lite has claimed SOAP::Trace already ,
            for example. The more popular the SOAP protocol gets the more likely this
            sort of problem is to occur.

            Unfortunately I think this would be a hard problem to fix - SOAP::Lite
            exposes a number of SOAP:: namespaces through its interface (SOAP::Data
            for example).

            > One more reason why it's better to keep modules in separate files, is
            > that you can use 'use/require' in this case. It can't be easily done
            > when modules are packaged in one file.

            I don't understand what you mean by this. I've written modules that use
            multiple namespaces and are separated into multiple files. Typically the
            user does:

            use Foo;

            And then in Foo.pm you have:

            package Foo;
            use Foo::Required;

            And later, when Foo::Optional is needed:

            require Foo::Optional; import Foo::Optional;

            Is there a problem here that I don't know about?

            -sam
          • Paul Kulchenko
            Hi, Sam! ... I agree. ... Hardly that much, but I ll do some tests to find exact number. ... There are two different problems. Every module uses at least one
            Message 5 of 9 , Jan 20, 2002
            • 0 Attachment
              Hi, Sam!

              --- Sam Tregar <sam@...> wrote:
              > smaller chunks. I can see that this is more of judgment call than
              > anything else...
              I agree.

              > > we can save about 300 lines for client and server, approx 10%.
              > This could be a significant saving - maybe 1MB or more depending on
              > what's actually in those lines.
              Hardly that much, but I'll do some tests to find exact number.

              > It's definitely more convenient. However, considering that you
              > don't
              > "own" the SOAP:: namespace, it's not very polite to create modules
              > there
              > without registering them. If someone else wants to write a SOAP
              > tracing
              > package they have to know that SOAP::Lite has claimed SOAP::Trace
              > already ,
              > for example. The more popular the SOAP protocol gets the more
              > likely this sort of problem is to occur.
              There are two different problems. Every module uses at least one
              namespace, and SOAP::Lite uses about ten of them. Since most of them
              are packaged in one file, it's not easy to find that namespace is
              already taken. Later is easy to address. POD file for every module
              can be created, so it'll be visible for CPAN searches. In addition to
              that, I would expect that SOAP developer that decides to create
              implementation in Perl will be quite familiar with other
              implementations and won't be taken by a surprise.

              First problem is more difficult. Why HTTP::Daemon namespace was used?
              Why not HTTP::LWP::Daemon or LWP::HTTP::Daemon? There are several
              reasons. What if another HTTP-based daemon will be written (in fact
              we have several. One of them is Net::Daemon? Will it create any
              problem? Maybe yes, maybe no. Was Gisle Aas impolite with taking
              HTTP::Daemon namespace? hardly. How about XML::Parser? Can't imagine
              there will be only one.

              You can't easily create several implementations behind one interface
              (unless this interface is already well established, like XML::SAX).
              It's (IMHO) reasonable to expect that there will be several competing
              implementations and some of them will use "better" namespaces. There
              is no "best practices" on thi topic, but I'd like to know different
              opinions, so next time I'll do it right.

              > I don't understand what you mean by this. I've written modules
              > that use multiple namespaces and are separated into multiple files.
              I just meant that if you have:

              -- Foo.pm
              package Foo;

              package Bar;

              you'll be able to do 'use Foo', but not 'use Bar' without doing
              tricks with %INC.

              Best wishes, Paul.

              __________________________________________________
              Do You Yahoo!?
              Send FREE video emails in Yahoo! Mail!
              http://promo.yahoo.com/videomail/
            • Sam Tregar
              ... I ll stick my neck out - I think this one was a bad choice. It probably should have been XML::Parser::Expat or just XML::Expat. But, hey, it s Larry
              Message 6 of 9 , Jan 20, 2002
              • 0 Attachment
                On Sun, 20 Jan 2002, Paul Kulchenko wrote:

                > How about XML::Parser? Can't imagine there will be only one.

                I'll stick my neck out - I think this one was a bad choice. It probably
                should have been XML::Parser::Expat or just XML::Expat. But, hey, it's
                Larry Wall, who's going to argue? ;)

                > There is no "best practices" on thi topic, but I'd like to know
                > different opinions, so next time I'll do it right.

                I strongly disagree. There is a well documented "best practice" - write
                to modules@... and register your namespaces with them. I suspect
                you could have avoided squatting on SOAP:: if you'd gotten their input
                when you started the module!

                > I just meant that if you have:
                >
                > -- Foo.pm
                > package Foo;
                >
                > package Bar;
                >
                > you'll be able to do 'use Foo', but not 'use Bar' without doing
                > tricks with %INC.

                Ah, right. Yes, this is a small problem but it's not a big deal when you
                consider that users shouldn't usually be loading sub-modules directly.

                -sam
              • Paul Kulchenko
                Hi, Sam! ... Actually that s precisely what I did (you can find my message in the archive). And I carefully checked archives on that and related topics. In
                Message 7 of 9 , Jan 21, 2002
                • 0 Attachment
                  Hi, Sam!

                  --- Sam Tregar <sam@...> wrote:
                  > I strongly disagree. There is a well documented "best practice" -
                  > write
                  > to modules@... and register your namespaces with them. I
                  > suspect
                  > you could have avoided squatting on SOAP:: if you'd gotten their
                  > input when you started the module!
                  Actually that's precisely what I did (you can find my message in the
                  archive). And I carefully checked archives on that and related
                  topics. In most cases you register MAIN namespace for your module,
                  but I never heard about registering internal modules and any
                  recommendations as for namespaces they should use.

                  Best wishes, Paul.

                  __________________________________________________
                  Do You Yahoo!?
                  Send FREE video emails in Yahoo! Mail!
                  http://promo.yahoo.com/videomail/
                • Sam Tregar
                  ... What is the point of registering a namespace if you don t register all that you intend to use? As I understand it, the point behind registering namespaces
                  Message 8 of 9 , Jan 21, 2002
                  • 0 Attachment
                    On Mon, 21 Jan 2002, Paul Kulchenko wrote:

                    > Actually that's precisely what I did (you can find my message in the
                    > archive). And I carefully checked archives on that and related
                    > topics. In most cases you register MAIN namespace for your module,
                    > but I never heard about registering internal modules and any
                    > recommendations as for namespaces they should use.

                    What is the point of registering a namespace if you don't register all
                    that you intend to use? As I understand it, the point behind registering
                    namespaces is so that people will know that you "own" that name. Now,
                    most modules can get away with just registering the base name since they
                    keep their private modules below their base - i.e. registering Foo::Bar
                    and then including Foo::Bar::Baz and Foo::Bar::Bif. However, the same
                    does not apply to registering Foo::Bar and then including Foo::Baz and
                    Foo::Bif in your distribution.

                    But I think you understand what I'm saying - I don't mean to beat you over
                    the head with it. What's done is done, right?

                    -sam
                  Your message has been successfully submitted and would be delivered to recipients shortly.