Loading ...
Sorry, an error occurred while loading the content.

SOAP::Lite architecture

Expand Messages
  • Sam Tregar
    Why is SOAP::Lite stored in one module file? I ask for two reasons: 1) It seems to me that memory usage could be lowered if modules were only loaded into
    Message 1 of 9 , Jan 20, 2002
    • 0 Attachment
      Why is SOAP::Lite stored in one module file? I ask for two reasons:

      1) It seems to me that memory usage could be lowered if modules were
      only loaded into memory when actually used. This can be done through
      require, autouse or (shudder) the AutoLoader.

      2) I've never seen a module distribution this large in one file before.
      I'm writing about CPAN module construction at the moment - if there
      are reasons to recommend doing things this way I'd like to hear them!

      Also, any reason why SOAP::Lite doesn't use its own namespace for internal
      modules (for example by using SOAP::Trace rather than SOAP::Lite::Trace)?

      -sam
    • Dirk Eddelbuettel
      ... I d be far from claiming that my system is representative, but to deliver just one random data point edd@homebud:/usr/share/perl5 find . -name *.pm -size
      Message 2 of 9 , Jan 20, 2002
      • 0 Attachment
        On Sun, Jan 20, 2002 at 05:04:16PM -0500, Sam Tregar wrote:
        > Why is SOAP::Lite stored in one module file? I ask for two reasons:
        >
        > 1) It seems to me that memory usage could be lowered if modules were
        > only loaded into memory when actually used. This can be done through
        > require, autouse or (shudder) the AutoLoader.
        >
        > 2) I've never seen a module distribution this large in one file before.
        > I'm writing about CPAN module construction at the moment - if there
        > are reasons to recommend doing things this way I'd like to hear them!

        I'd be far from claiming that my system is representative, but to deliver
        just one random data point

        edd@homebud:/usr/share/perl5> find . -name \*.pm -size +100k | xargs ls -lGh
        -rw-r--r-- 1 root 203k Apr 15 2001 ./Date/Manip.pm
        -rw-r--r-- 1 root 112k Mar 10 2001 ./HTML/Element.pm
        -rw-r--r-- 1 root 157k Oct 22 09:42 ./SOAP/Lite.pm
        -rw-r--r-- 1 root 113k Oct 22 18:34 ./Spreadsheet/WriteExcel.pm

        we see that ar least one all-Perl module exceeds the file size of SOAP::Lite.

        Dirk

        --
        Good judgment comes from experience; experience comes from bad judgment.
        -- F. Brooks
      • Sam Tregar
        ... Interesting. And does Data::Manip also contain numerous sub-modules packaged interally? ... time passes as I install Date::Manip ... Nope. Its size
        Message 3 of 9 , Jan 20, 2002
        • 0 Attachment
          On Sun, 20 Jan 2002, Dirk Eddelbuettel wrote:

          > I'd be far from claiming that my system is representative, but to deliver
          > just one random data point
          >
          > edd@homebud:/usr/share/perl5> find . -name \*.pm -size +100k | xargs ls -lGh
          > -rw-r--r-- 1 root 203k Apr 15 2001 ./Date/Manip.pm
          > -rw-r--r-- 1 root 112k Mar 10 2001 ./HTML/Element.pm
          > -rw-r--r-- 1 root 157k Oct 22 09:42 ./SOAP/Lite.pm
          > -rw-r--r-- 1 root 113k Oct 22 18:34 ./Spreadsheet/WriteExcel.pm
          >
          > we see that ar least one all-Perl module exceeds the file size of SOAP::Lite.

          Interesting. And does Data::Manip also contain numerous sub-modules
          packaged interally? ... time passes as I install Date::Manip ... Nope.
          Its size seems to come from some radical manual inlining of logic (judging
          from the comment on line 3047) as well as some largish tables of
          locale-specific data. The latter in particular could probably be moved
          into locale-specific sub-modules or AutoLoaded. After all, why should I
          have to load the entire set of Italian date names into memory if I'll
          never use them?

          So, I guess I think my criticism stands even if it could also be applied
          to other CPAN modules. How arrogant, eh? Well, if there's fire below
          we're all gonna go; check out my HTML::Template for a good example of not
          following one's own advice (a single .pm >100k with a bunch of features
          that most people will never use).

          -sam
        • Paul Kulchenko
          Hi, Sam! Both points are valid. First of all, there WILL be redesign, but not VERY soon and I will ask for your suggestions re new design. Second, to my
          Message 4 of 9 , Jan 20, 2002
          • 0 Attachment
            Hi, Sam!

            Both points are valid. First of all, there WILL be redesign, but not
            VERY soon and I will ask for your suggestions re new design. Second,
            to my defense, only 2900 lines out of 5000 are actual code, the rest
            of it is documentation and is not processed, because it sits after
            __END__

            Memory usage can definitely be lowered, but not because of
            repackaging (see below), but slightly changing interactions between
            internal modules. I'm working on it.

            Psychological aspect. I started my module when SOAP.pm already
            existed and if you take a look into its distribution you'll find
            there SOAP.pm, sixteen (!) files in SOAP directory and 4 files for
            HTTP transport. I just got lost jumping between all of them and when
            I started my module I decided that it will NOT be like this.

            What we have now is one module for SOAP, one module for every
            transport and one test module. All other modules are loaded ONLY when
            needed.

            Let's take a look inside and see what's required for client and
            server.

            SOAP::Constants -- both
            SOAP::Utils -- both
            SOAP::Transport -- client (50 lines)
            SOAP::Fault -- server, both in the future
            SOAP::Data -- both
            SOAP::Header -- both
            SOAP::Serializer -- both
            SOAP::Parser -- both
            SOAP::MIMEParser -- server, not always (80 lines)
            SOAP::SOM -- both
            SOAP::Deserializer -- both
            SOAP::Client -- client (10 lines)
            SOAP::Server::* -- server (220 lines)
            SOAP::Trace -- both
            SOAP::Custom::XML::* -- both, not always (50 lines)
            SOAP::Schema::* -- client, not always (250 lines)

            we can save about 300 lines for client and server, approx 10%.

            > Also, any reason why SOAP::Lite doesn't use its own namespace for
            > internal
            > modules (for example by using SOAP::Trace rather than
            > SOAP::Lite::Trace)?
            No particular reason. I thought it's not convenient to write
            SOAP::Lite::Data, SOAP::Lite::Serializer, SOAP::Lite::Parser, and so
            on, and if you don't do it for some module, it doesn't make much
            sense to do it for others. Even considering this I was very careful
            to not interfere with modules from SOAP module (Keith Brown,
            DevelopMentor), the only SOAP module for Perl available at that time
            and I'm pretty sure even now you can use them in one script (though I
            didn't try it with last versions).

            Let me know if you think it's unreasonable, I'll definitely consider
            what can be done to fix this.

            One more reason why it's better to keep modules in separate files, is
            that you can use 'use/require' in this case. It can't be easily done
            when modules are packaged in one file. I would like to hear other
            reasons why do NOT keep them together ;). Thank you.

            Best wishes, Paul.

            --- Sam Tregar <sam@...> wrote:
            > Why is SOAP::Lite stored in one module file? I ask for two
            > reasons:
            >
            > 1) It seems to me that memory usage could be lowered if modules
            > were
            > only loaded into memory when actually used. This can be done
            > through
            > require, autouse or (shudder) the AutoLoader.
            >
            > 2) I've never seen a module distribution this large in one file
            > before.
            > I'm writing about CPAN module construction at the moment - if
            > there
            > are reasons to recommend doing things this way I'd like to hear
            > them!
            >
            > Also, any reason why SOAP::Lite doesn't use its own namespace for
            > internal
            > modules (for example by using SOAP::Trace rather than
            > SOAP::Lite::Trace)?
            >
            > -sam
            >
            >
            >
            > ------------------------ Yahoo! Groups Sponsor
            >
            > To unsubscribe from this group, send an email to:
            > soaplite-unsubscribe@yahoogroups.com
            >
            >
            >
            > Your use of Yahoo! Groups is subject to
            > http://docs.yahoo.com/info/terms/
            >
            >



            __________________________________________________
            Do You Yahoo!?
            Send FREE video emails in Yahoo! Mail!
            http://promo.yahoo.com/videomail/
          • Sam Tregar
            ... Sounds good to me. ... Ok, I guess this works out to an esthetic judgment then. I had an opposite reaction while working my way through the SOAP::Lite
            Message 5 of 9 , Jan 20, 2002
            • 0 Attachment
              On Sun, 20 Jan 2002, Paul Kulchenko wrote:

              > Both points are valid. First of all, there WILL be redesign, but not
              > VERY soon and I will ask for your suggestions re new design.

              Sounds good to me.

              > Psychological aspect. I started my module when SOAP.pm already
              > existed and if you take a look into its distribution you'll find
              > there SOAP.pm, sixteen (!) files in SOAP directory and 4 files for
              > HTTP transport. I just got lost jumping between all of them and when
              > I started my module I decided that it will NOT be like this.

              Ok, I guess this works out to an esthetic judgment then. I had an
              opposite reaction while working my way through the SOAP::Lite source - I
              wished it was more compartmentalized so that I could read through it in
              smaller chunks. I can see that this is more of judgment call than
              anything else...

              > we can save about 300 lines for client and server, approx 10%.

              This could be a significant saving - maybe 1MB or more depending on what's
              actually in those lines.

              > No particular reason. I thought it's not convenient to write
              > SOAP::Lite::Data, SOAP::Lite::Serializer, SOAP::Lite::Parser, and so
              > on, and if you don't do it for some module, it doesn't make much
              > sense to do it for others. Even considering this I was very careful
              > to not interfere with modules from SOAP module (Keith Brown,
              > DevelopMentor), the only SOAP module for Perl available at that time
              > and I'm pretty sure even now you can use them in one script (though I
              > didn't try it with last versions).

              It's definitely more convenient. However, considering that you don't
              "own" the SOAP:: namespace, it's not very polite to create modules there
              without registering them. If someone else wants to write a SOAP tracing
              package they have to know that SOAP::Lite has claimed SOAP::Trace already ,
              for example. The more popular the SOAP protocol gets the more likely this
              sort of problem is to occur.

              Unfortunately I think this would be a hard problem to fix - SOAP::Lite
              exposes a number of SOAP:: namespaces through its interface (SOAP::Data
              for example).

              > One more reason why it's better to keep modules in separate files, is
              > that you can use 'use/require' in this case. It can't be easily done
              > when modules are packaged in one file.

              I don't understand what you mean by this. I've written modules that use
              multiple namespaces and are separated into multiple files. Typically the
              user does:

              use Foo;

              And then in Foo.pm you have:

              package Foo;
              use Foo::Required;

              And later, when Foo::Optional is needed:

              require Foo::Optional; import Foo::Optional;

              Is there a problem here that I don't know about?

              -sam
            • Paul Kulchenko
              Hi, Sam! ... I agree. ... Hardly that much, but I ll do some tests to find exact number. ... There are two different problems. Every module uses at least one
              Message 6 of 9 , Jan 20, 2002
              • 0 Attachment
                Hi, Sam!

                --- Sam Tregar <sam@...> wrote:
                > smaller chunks. I can see that this is more of judgment call than
                > anything else...
                I agree.

                > > we can save about 300 lines for client and server, approx 10%.
                > This could be a significant saving - maybe 1MB or more depending on
                > what's actually in those lines.
                Hardly that much, but I'll do some tests to find exact number.

                > It's definitely more convenient. However, considering that you
                > don't
                > "own" the SOAP:: namespace, it's not very polite to create modules
                > there
                > without registering them. If someone else wants to write a SOAP
                > tracing
                > package they have to know that SOAP::Lite has claimed SOAP::Trace
                > already ,
                > for example. The more popular the SOAP protocol gets the more
                > likely this sort of problem is to occur.
                There are two different problems. Every module uses at least one
                namespace, and SOAP::Lite uses about ten of them. Since most of them
                are packaged in one file, it's not easy to find that namespace is
                already taken. Later is easy to address. POD file for every module
                can be created, so it'll be visible for CPAN searches. In addition to
                that, I would expect that SOAP developer that decides to create
                implementation in Perl will be quite familiar with other
                implementations and won't be taken by a surprise.

                First problem is more difficult. Why HTTP::Daemon namespace was used?
                Why not HTTP::LWP::Daemon or LWP::HTTP::Daemon? There are several
                reasons. What if another HTTP-based daemon will be written (in fact
                we have several. One of them is Net::Daemon? Will it create any
                problem? Maybe yes, maybe no. Was Gisle Aas impolite with taking
                HTTP::Daemon namespace? hardly. How about XML::Parser? Can't imagine
                there will be only one.

                You can't easily create several implementations behind one interface
                (unless this interface is already well established, like XML::SAX).
                It's (IMHO) reasonable to expect that there will be several competing
                implementations and some of them will use "better" namespaces. There
                is no "best practices" on thi topic, but I'd like to know different
                opinions, so next time I'll do it right.

                > I don't understand what you mean by this. I've written modules
                > that use multiple namespaces and are separated into multiple files.
                I just meant that if you have:

                -- Foo.pm
                package Foo;

                package Bar;

                you'll be able to do 'use Foo', but not 'use Bar' without doing
                tricks with %INC.

                Best wishes, Paul.

                __________________________________________________
                Do You Yahoo!?
                Send FREE video emails in Yahoo! Mail!
                http://promo.yahoo.com/videomail/
              • Sam Tregar
                ... I ll stick my neck out - I think this one was a bad choice. It probably should have been XML::Parser::Expat or just XML::Expat. But, hey, it s Larry
                Message 7 of 9 , Jan 20, 2002
                • 0 Attachment
                  On Sun, 20 Jan 2002, Paul Kulchenko wrote:

                  > How about XML::Parser? Can't imagine there will be only one.

                  I'll stick my neck out - I think this one was a bad choice. It probably
                  should have been XML::Parser::Expat or just XML::Expat. But, hey, it's
                  Larry Wall, who's going to argue? ;)

                  > There is no "best practices" on thi topic, but I'd like to know
                  > different opinions, so next time I'll do it right.

                  I strongly disagree. There is a well documented "best practice" - write
                  to modules@... and register your namespaces with them. I suspect
                  you could have avoided squatting on SOAP:: if you'd gotten their input
                  when you started the module!

                  > I just meant that if you have:
                  >
                  > -- Foo.pm
                  > package Foo;
                  >
                  > package Bar;
                  >
                  > you'll be able to do 'use Foo', but not 'use Bar' without doing
                  > tricks with %INC.

                  Ah, right. Yes, this is a small problem but it's not a big deal when you
                  consider that users shouldn't usually be loading sub-modules directly.

                  -sam
                • Paul Kulchenko
                  Hi, Sam! ... Actually that s precisely what I did (you can find my message in the archive). And I carefully checked archives on that and related topics. In
                  Message 8 of 9 , Jan 21, 2002
                  • 0 Attachment
                    Hi, Sam!

                    --- Sam Tregar <sam@...> wrote:
                    > I strongly disagree. There is a well documented "best practice" -
                    > write
                    > to modules@... and register your namespaces with them. I
                    > suspect
                    > you could have avoided squatting on SOAP:: if you'd gotten their
                    > input when you started the module!
                    Actually that's precisely what I did (you can find my message in the
                    archive). And I carefully checked archives on that and related
                    topics. In most cases you register MAIN namespace for your module,
                    but I never heard about registering internal modules and any
                    recommendations as for namespaces they should use.

                    Best wishes, Paul.

                    __________________________________________________
                    Do You Yahoo!?
                    Send FREE video emails in Yahoo! Mail!
                    http://promo.yahoo.com/videomail/
                  • Sam Tregar
                    ... What is the point of registering a namespace if you don t register all that you intend to use? As I understand it, the point behind registering namespaces
                    Message 9 of 9 , Jan 21, 2002
                    • 0 Attachment
                      On Mon, 21 Jan 2002, Paul Kulchenko wrote:

                      > Actually that's precisely what I did (you can find my message in the
                      > archive). And I carefully checked archives on that and related
                      > topics. In most cases you register MAIN namespace for your module,
                      > but I never heard about registering internal modules and any
                      > recommendations as for namespaces they should use.

                      What is the point of registering a namespace if you don't register all
                      that you intend to use? As I understand it, the point behind registering
                      namespaces is so that people will know that you "own" that name. Now,
                      most modules can get away with just registering the base name since they
                      keep their private modules below their base - i.e. registering Foo::Bar
                      and then including Foo::Bar::Baz and Foo::Bar::Bif. However, the same
                      does not apply to registering Foo::Bar and then including Foo::Baz and
                      Foo::Bif in your distribution.

                      But I think you understand what I'm saying - I don't mean to beat you over
                      the head with it. What's done is done, right?

                      -sam
                    Your message has been successfully submitted and would be delivered to recipients shortly.