Loading ...
Sorry, an error occurred while loading the content.

RE: [xml-dbms] Framework for B2B solutions?

Expand Messages
  • adam flinton
    ... Hello Kevin, ... I am just in the middle (quite literally as I am the hub (being the warehouse)) of doing a large XML/JMS based system for a large Chain of
    Message 1 of 7 , May 20 9:35 AM
    • 0 Attachment
      > -----Original Message-----
      > From: kevincompton@... [mailto:kevincompton@...]
      > Sent: 19 May 2001 18:57
      > To: xml-dbms@yahoogroups.com
      > Subject: [xml-dbms] Framework for B2B solutions?
      >
      >
      > Hello All,
      >

      Hello Kevin,

      > I am currently working on a project to automate the procurement
      > process for a large cable company. I've been using xml-dbms to get
      > the PO into an acceptable xml format. Now I need some advice or
      > direction as to any sources or frameworks we can use to provide a
      > flexible & extendable format to add & change business processes
      > between the messaging going on between Customer & Supplier? Pretty
      > lofty request but any personal insights or resources would be much
      > appreciated.
      >

      I am just in the middle (quite literally as I am the hub (being the
      warehouse)) of doing a large XML/JMS based system for a large Chain of
      Supermarkets. We are using Java & XML-DBMS & the reality is that all the new
      stuff I have put into XML-DBMS have been put in precisely coz I faced / face
      the same set of "challenges" (don't you love euphemisms <G>).

      OK. In terms of adopting an existing set of DTD'es etc you have (at the
      moment) basically 3 choices:

      RosettaNet/OAGI (What we've gone for)
      EbXML (Appeals to the EDI folks as they wrote it)
      BizTalk - Microsoft.

      Points to bear in mind if you adopt a "pseudo/wannabe standard":

      1) There will be a huge inflation in message size as you may only need a
      small subset of a DTD but it will have lots of required elements that you
      need to shove in if only as empty elements. If you're dealing with a huge
      number of messages think about using Zip somehwere in a the chain (XML zips
      suberbly due to the vast amount of repetition).

      2) MAP WHAT YOU HAVE First. This allows your guys who know about "their"
      data structures to produce data they know & can see is right. Also what is
      the point in mapping an element when you have nowhere to get it from or (if
      incoming) to store it?

      3) Use XSLT to transform to/from the lovely but huge Std doc to the doc
      which you're actually going to use / produce (see (1 & 2)). This is why I
      have built JMS & XSLT into the latest beta release of XML-DMBS (even
      supports SonicMQ...<G>). This also allows you to easily produce documents to
      any of the 3 frontrunning pseudo-stds.

      4) Have a long chat with your coders about XML. Seriously. The reason is
      that if they are coming from a DB based background then they often have a
      hard time coping with the concept of "optional". DB columns ALWAYS exist.
      Wether they contain data or not is secondary. However.....optional means
      optional. This has caused us more grief than just about anything else as one
      system may be using "optional" elements as a key part of the process & as
      such in reality they aren't optional. Now if they were to send out the DTD &
      someone else was to build from that.....then there will be failures. Luckily
      with XSLT you can quickly pad out doc's with "required garbage" to keep the
      code @ the other end happy (e.g. test & if the element is not there/ empty
      shove in <element>Default_Value </element>).

      5) Don't try to leap straight into something like Biztalk/OAGI/EbXML DTD'es
      & XML @ the same time. Build your nice & simple doc's (points 1 & 2). Test
      them etc. Then ID which DTD'es are closest to what you want & then do some
      form of mapping. Going straight to a damn great DTD leads to coders who
      don't fully understand either XML or the DTD which they're writing to &
      commonly both.

      6) Remember 1 thing which is that the trickiest thing with what you are
      approaching is dispelling any ambiguity between the various parties. 2
      people can both build doc's which validate against the same DTD but that's
      about as far as the commonality goes. Don't depend on DTD'es to be anything
      more than a "guide".


      > (Also, as an aside, does anyone have any real world examples
      > that would suggest that we should use a J2EE app server or is this
      > overkill?)
      >

      The most current code has been tested in:

      Weblogic 5.1 & 6 & HP Bluestone (the one of their website). I would recomend
      Bluestone as it carries none of the Tengah (i.e. pre j2EE) legacy stuff
      which both Weblogic (far less in v6 btw) & Websphere (not Tengah but legacy)
      do). I would also just simply recomend Bluestone fullstop for development.

      For J2EE'ing all you need is to put a little bit of code into the
      "on_message()" method of either a message driven bean (mdb) or a
      messagelistener (the Xml-dbms code comes with an example mdb & the main
      class contains a "receive" method which is usefull simply for hooking up to
      topics & watching what comes through (either tail the session or (on
      Windows) save the startcommand in a bat file & run that with >log.txt as the
      last part in the command (e.g. java xmldbms.jar etc.etc > log.txt) if you
      want a file which you can copy & paste into say an XML Editor or simply to
      save a a "test.xml").

      I might chuck some example code into the example mdb.

      In essence all you're doing is piping text (string) from one process to the
      next e.g incoming might be

      message_text > XSLT (to turn into your simple doc) > TransferEngine (call
      the store method which also accepts a string)

      going out would be use the TransferEngine Transfer which returns a string >
      XSLT > JMS Send.


      Re is a J2EE server overkill. Try using a lightweight one first (e.g
      BlueStone). They do offer some very usefull things but make sure it's J2EE
      2.0 compliant re MessageDrivenBeans as they make Messaging very simple.

      Also remember that you could cut the job up into manageable chunks by using
      the XSLT part of XMLDBMS (it comes with a built in object cache so you only
      load & process a XSLT sheet once (i.e. the first time a specific type of
      message (or a message on a given topic) comes through / needs to be sent
      then it's loaded (url or local is handled) @ that point) on it's own.

      i.e. you could use it to set up a server as an XML transform server (a kind
      of "preprocessor") which then fires formatted messages into another server
      using XMLDBMS or gets the message from said server & wraps it in
      "OAGI-bodness" or Biztalk or EbXML etc.

      This also allows to use the same topic for a given document wether it comes
      in as an OAGI bod, Biztalk or EbXML.



      > Currently we are considering using JMS/SonicMQ for the messaging
      > piece but need to build the structure around it to implement the
      > RosettaNet standard.
      >

      The latest XML-DBMS has all of the tech part.e.g. the code happily supports
      SonicMQ even though it uses a completely different method of loading
      (doesn't use JNDI).


      The bit's regarding talking to your Developers (re keeping things simple) &
      other parties (re ambiguities) is up to you.

      Adam
    • Dewayne McNair
      ... recomend ... legacy) ... I want to throw my $0.02 in on this one... I m using XML-DBMS (first version) in JBoss, and it s working great. JBoss is an
      Message 2 of 7 , May 20 9:46 AM
      • 0 Attachment
        > The most current code has been tested in:
        >
        > Weblogic 5.1 & 6 & HP Bluestone (the one of their website). I would
        recomend
        > Bluestone as it carries none of the Tengah (i.e. pre j2EE) legacy stuff
        > which both Weblogic (far less in v6 btw) & Websphere (not Tengah but
        legacy)
        > do). I would also just simply recomend Bluestone fullstop for development.

        I want to throw my $0.02 in on this one... I'm using XML-DBMS (first
        version) in JBoss, and it's working great. JBoss is an open-source J2EE app
        server that you should really look at. I haven't tried the latest code on
        it, but, there's no reason to think it wouldn't work just fine -- JBoss has
        MDB support, among other of the latest J2EE specs.

        -- Dewayne
      • kevincompton@go.com
        Thanks, Adam for your input! It is nice to know that you are going through the same process. Could you please direct me to your latest xml-dbms work &
        Message 3 of 7 , May 21 8:39 AM
        • 0 Attachment
          Thanks, Adam for your input!
          It is nice to know that you are going through the same process.

          Could you please direct me to your latest xml-dbms work &
          documentation?

          Right now, I had envisioned using xml-dbms just for transferring DB
          data to & perhaps from xml documents.
          It seems that the package is playing a more central role in your
          implementation..(?)

          What type of setup are you using to handle the business logic? (eg.
          tier 1 customer gets 10% discount or some type of bulk deal) (Have
          you designed a "pipeline" to separate app code from business logic?)

          What I would like to do is have a framework that allows the
          flexibility to change (or add) business rules (both like above and
          those related to conforming to different transfer standards
          (Rosettanet)) without having to change my application code (ideally
          at least ;>).

          I know that BEA Weblogic has a process & workflow integrator piece &
          MS BizTalk server has an orchestration piece... this is sorta what I
          am talking about.
          How are you planning on facilitating this?


          Of what I have been able to wrap my mind around.. thus far.. is that
          everything revolves arount the messaging middle-ware ...
          And I wonder which would be the "best" way to go.. 1) provide a web
          interface or 2) have a client JMS/Sonic piece for each supplier to
          communicate with the queue.
          Any thoughts about that? R U using any type of web interface?

          Also.. I am still a bit fuzzy on the need for an application server..
          could you elaborate on some of the benefits that exist using MDB
          compared to coding message listeners? R U taking advantage of any of
          the other benefits provided by an app server ? (such as security,
          transactions, etc.) If so how?


          Thank-You VERY MUCH,

          Kevin Compton






          --- In xml-dbms@y..., adam flinton <aflinton@a...> wrote:
          >
          > > -----Original Message-----
          > > From: kevincompton@g... [mailto:kevincompton@g...]
          > > Sent: 19 May 2001 18:57
          > > To: xml-dbms@y...
          > > Subject: [xml-dbms] Framework for B2B solutions?
          > >
          > >
          > > Hello All,
          > >
          >
          > Hello Kevin,
          >
          > > I am currently working on a project to automate the
          procurement
          > > process for a large cable company. I've been using xml-dbms to
          get
          > > the PO into an acceptable xml format. Now I need some advice or
          > > direction as to any sources or frameworks we can use to provide a
          > > flexible & extendable format to add & change business processes
          > > between the messaging going on between Customer & Supplier?
          Pretty
          > > lofty request but any personal insights or resources would be
          much
          > > appreciated.
          > >
          >
          > I am just in the middle (quite literally as I am the hub (being the
          > warehouse)) of doing a large XML/JMS based system for a large Chain
          of
          > Supermarkets. We are using Java & XML-DBMS & the reality is that
          all the new
          > stuff I have put into XML-DBMS have been put in precisely coz I
          faced / face
          > the same set of "challenges" (don't you love euphemisms <G>).
          >
          > OK. In terms of adopting an existing set of DTD'es etc you have (at
          the
          > moment) basically 3 choices:
          >
          > RosettaNet/OAGI (What we've gone for)
          > EbXML (Appeals to the EDI folks as they wrote it)
          > BizTalk - Microsoft.
          >
          > Points to bear in mind if you adopt a "pseudo/wannabe standard":
          >
          > 1) There will be a huge inflation in message size as you may only
          need a
          > small subset of a DTD but it will have lots of required elements
          that you
          > need to shove in if only as empty elements. If you're dealing with
          a huge
          > number of messages think about using Zip somehwere in a the chain
          (XML zips
          > suberbly due to the vast amount of repetition).
          >
          > 2) MAP WHAT YOU HAVE First. This allows your guys who know
          about "their"
          > data structures to produce data they know & can see is right. Also
          what is
          > the point in mapping an element when you have nowhere to get it
          from or (if
          > incoming) to store it?
          >
          > 3) Use XSLT to transform to/from the lovely but huge Std doc to the
          doc
          > which you're actually going to use / produce (see (1 & 2)). This is
          why I
          > have built JMS & XSLT into the latest beta release of XML-DMBS (even
          > supports SonicMQ...<G>). This also allows you to easily produce
          documents to
          > any of the 3 frontrunning pseudo-stds.
          >
          > 4) Have a long chat with your coders about XML. Seriously. The
          reason is
          > that if they are coming from a DB based background then they often
          have a
          > hard time coping with the concept of "optional". DB columns ALWAYS
          exist.
          > Wether they contain data or not is secondary. However.....optional
          means
          > optional. This has caused us more grief than just about anything
          else as one
          > system may be using "optional" elements as a key part of the
          process & as
          > such in reality they aren't optional. Now if they were to send out
          the DTD &
          > someone else was to build from that.....then there will be
          failures. Luckily
          > with XSLT you can quickly pad out doc's with "required garbage" to
          keep the
          > code @ the other end happy (e.g. test & if the element is not
          there/ empty
          > shove in <element>Default_Value </element>).
          >
          > 5) Don't try to leap straight into something like
          Biztalk/OAGI/EbXML DTD'es
          > & XML @ the same time. Build your nice & simple doc's (points 1 &
          2). Test
          > them etc. Then ID which DTD'es are closest to what you want & then
          do some
          > form of mapping. Going straight to a damn great DTD leads to coders
          who
          > don't fully understand either XML or the DTD which they're writing
          to &
          > commonly both.
          >
          > 6) Remember 1 thing which is that the trickiest thing with what you
          are
          > approaching is dispelling any ambiguity between the various
          parties. 2
          > people can both build doc's which validate against the same DTD but
          that's
          > about as far as the commonality goes. Don't depend on DTD'es to be
          anything
          > more than a "guide".
          >
          >
          > > (Also, as an aside, does anyone have any real world examples
          > > that would suggest that we should use a J2EE app server or is
          this
          > > overkill?)
          > >
          >
          > The most current code has been tested in:
          >
          > Weblogic 5.1 & 6 & HP Bluestone (the one of their website). I would
          recomend
          > Bluestone as it carries none of the Tengah (i.e. pre j2EE) legacy
          stuff
          > which both Weblogic (far less in v6 btw) & Websphere (not Tengah
          but legacy)
          > do). I would also just simply recomend Bluestone fullstop for
          development.
          >
          > For J2EE'ing all you need is to put a little bit of code into the
          > "on_message()" method of either a message driven bean (mdb) or a
          > messagelistener (the Xml-dbms code comes with an example mdb & the
          main
          > class contains a "receive" method which is usefull simply for
          hooking up to
          > topics & watching what comes through (either tail the session or (on
          > Windows) save the startcommand in a bat file & run that with
          >log.txt as the
          > last part in the command (e.g. java xmldbms.jar etc.etc > log.txt)
          if you
          > want a file which you can copy & paste into say an XML Editor or
          simply to
          > save a a "test.xml").
          >
          > I might chuck some example code into the example mdb.
          >
          > In essence all you're doing is piping text (string) from one
          process to the
          > next e.g incoming might be
          >
          > message_text > XSLT (to turn into your simple doc) > TransferEngine
          (call
          > the store method which also accepts a string)
          >
          > going out would be use the TransferEngine Transfer which returns a
          string >
          > XSLT > JMS Send.
          >
          >
          > Re is a J2EE server overkill. Try using a lightweight one first (e.g
          > BlueStone). They do offer some very usefull things but make sure
          it's J2EE
          > 2.0 compliant re MessageDrivenBeans as they make Messaging very
          simple.
          >
          > Also remember that you could cut the job up into manageable chunks
          by using
          > the XSLT part of XMLDBMS (it comes with a built in object cache so
          you only
          > load & process a XSLT sheet once (i.e. the first time a specific
          type of
          > message (or a message on a given topic) comes through / needs to be
          sent
          > then it's loaded (url or local is handled) @ that point) on it's
          own.
          >
          > i.e. you could use it to set up a server as an XML transform server
          (a kind
          > of "preprocessor") which then fires formatted messages into another
          server
          > using XMLDBMS or gets the message from said server & wraps it in
          > "OAGI-bodness" or Biztalk or EbXML etc.
          >
          > This also allows to use the same topic for a given document wether
          it comes
          > in as an OAGI bod, Biztalk or EbXML.
          >
          >
          >
          > > Currently we are considering using JMS/SonicMQ for the
          messaging
          > > piece but need to build the structure around it to implement the
          > > RosettaNet standard.
          > >
          >
          > The latest XML-DBMS has all of the tech part.e.g. the code happily
          supports
          > SonicMQ even though it uses a completely different method of loading
          > (doesn't use JNDI).
          >
          >
          > The bit's regarding talking to your Developers (re keeping things
          simple) &
          > other parties (re ambiguities) is up to you.
          >
          > Adam
        • kevincompton@go.com
          ... We have looked into their biztalk server. The price seemed a bit high (50K). Of course, that was back when the scale of our solution was going to be quite
          Message 4 of 7 , May 21 8:49 AM
          • 0 Attachment
            --- In xml-dbms@y..., "Guillermo Molina" <gmolina@t...> wrote:
            > Did you check this software?
            >
            > http://www.microsoft.com/biztalk/default.asp
            >


            We have looked into their biztalk server. The price seemed a bit high
            (50K). Of course, that was back when the scale of our solution was
            going to be quite a bit smaller than it seems to have grown into.

            I also fear that using a MS specific product might make it a bit
            harder for us to market our solution & services to the next supplier
            down the line. It seems that using a solution that is mostly platform
            independent would help us to abstract the hosting environment out of
            the equation a bit.

            Others will be playing a role in the decision process too.. perhaps
            we will end up going this way.

            Thanks,
            Kevin Compton
          • adam flinton
            ... @ the moment it s stored on the newsgroup in the binaries section. I am looking to upload the latest (in fact I might do it now). The only diff with the
            Message 5 of 7 , May 22 10:31 AM
            • 0 Attachment
              > -----Original Message-----
              > From: kevincompton@... [mailto:kevincompton@...]
              > Sent: 21 May 2001 16:39
              > To: xml-dbms@yahoogroups.com
              > Subject: [xml-dbms] Re: Framework for B2B solutions?
              >
              >
              >
              > Thanks, Adam for your input!
              > It is nice to know that you are going through the same process.
              >
              > Could you please direct me to your latest xml-dbms work &
              > documentation?
              >

              @ the moment it's stored on the newsgroup in the "binaries" section. I am
              looking to upload the latest (in fact I might do it now). The only diff with
              the previous is that it has JAXP as a ParserUtil / Document factory

              > Right now, I had envisioned using xml-dbms just for transferring DB
              > data to & perhaps from xml documents.
              > It seems that the package is playing a more central role in your
              > implementation..(?)
              >

              You need 3 things:

              XSLT - Built in (with an object cache)
              JMS - Built in though for J2EE stuff I would advise using Message Driven
              Beans for receiving but use this class for sending.
              XMLDBMS - for SQL<>XML


              > What type of setup are you using to handle the business logic? (eg.
              > tier 1 customer gets 10% discount or some type of bulk deal) (Have
              > you designed a "pipeline" to separate app code from business logic?)
              >

              Yup. OK. Message comes in. Apply XSLT if you need it to strip off the
              garbage. You now have a nice & compact XML representation of the tables you
              will be inserted into (pass the string into the parserUtil methods which
              accept a string & return a DOM.

              If you want to do serious business logic if you then go & look @ any of the
              Java Data Binding code trees (e.g. the Zeus @ enhydra stuff which we are
              using). Hopefully JAXB will be along in a while...<:-((( This allows you
              to convert from an XML DOM tree into an Object Graph

              You now do your stuff. Then reverse the process to build back into an XML
              doc which agrees with your mapfile. Then you use XMLDBMS to insert into the
              DB.

              > What I would like to do is have a framework that allows the
              > flexibility to change (or add) business rules (both like above and
              > those related to conforming to different transfer standards
              > (Rosettanet)) without having to change my application code (ideally
              > at least ;>).
              >

              Difficult where business logic is concerned as you will have to convert it
              into some form of object (e.g. date or integer) if you're going to evaluate
              etc. There is a surprizing amount that XPath/XSLT allows you to do but if if
              it's anything like as involved as our Business Logic then in effect casting
              the XML strings to their relevant object types & then working on them is the
              best.

              > I know that BEA Weblogic has a process & workflow integrator piece &
              > MS BizTalk server has an orchestration piece... this is sorta what I
              > am talking about.
              > How are you planning on facilitating this?
              >
              >

              The Bea bit is not very good (IMHO) & ours is better. Don't just look @ Nice
              GUI tools as often they mask a pile of trouble below.

              But yes in effect a combination of the tools above with Message Topics does
              allow you to build little bits of code rather than a huge monolith. E.g. I
              get from topic A. I do something & chuick out a message onto Topic B where
              it is picked up & something else happens.

              > Of what I have been able to wrap my mind around.. thus far.. is that
              > everything revolves arount the messaging middle-ware ...
              > And I wonder which would be the "best" way to go.. 1) provide a web
              > interface or 2) have a client JMS/Sonic piece for each supplier to
              > communicate with the queue.
              > Any thoughts about that? R U using any type of web interface?
              >

              The former would be better unless you are looking @ having you suppliers
              type everything in. If you want an automated system then going directly to
              JMS is the best option. It's very simple & has a shed load of error handling
              built in (though you do have to decide what you want to use from that &
              how).


              > Also.. I am still a bit fuzzy on the need for an application server..
              > could you elaborate on some of the benefits that exist using MDB
              > compared to coding message listeners? R U taking advantage of any of
              > the other benefits provided by an app server ? (such as security,
              > transactions, etc.) If so how?
              >

              Connection Pooling (to the DB) is good. You also get a bunch of in built
              stuff which otherwise you might have to write yourself (e.g.
              fall-over/restart (of the app) etc.etc). However it is a horses for courses
              thing. I have test listeners which have yet to fallover despite around 4-5
              months of day in day out use (quite heavily @ times) running on a Solaris
              box.

              It's up to you. The nice thing is that you can happily write something which
              uses XMLDBMS/XSLT etc as a standalone & then shoving inside a MessageDriven
              bean really is cut & paste.

              >
              > Thank-You VERY MUCH,
              >

              No probs.

              Adam
            Your message has been successfully submitted and would be delivered to recipients shortly.