Loading ...
Sorry, an error occurred while loading the content.
 

pearifying metabase: phase 1

Expand Messages
  • Lukas Smith
    Hi All, Just trying to get a list together here; any comments greatly appreciated: Structural differences (or better what will need to be added to metabase to
    Message 1 of 6 , Feb 1, 2002
      Hi All,

      Just trying to get a list together here; any comments greatly
      appreciated:
      Structural differences (or better what will need to be added to metabase
      to include or be compatible with pear db's features):

      You will find my naming of the "todos" to be a bit optimistic (resolved,
      minor, small, large, unsure) but that is just a way to categorize the
      different things

      This list might see some expansion in the future. Under resolved I will
      list anything that will soon we included in Metbase, has just been
      included or has been included for some time .. or more exact: anything
      we don't need to worry about. The other categories should be self
      explanatory. If needed we can always add another category: "extra large"
      :-)

      So here it goes:
      Resolved:
      - Missing Drivers
      Sybase (resolved: is coming)
      - get* family (resolved: with the latest release)
      - placeholder function (resolved: metabase actually has had this for
      quite some time)

      Minor:
      - fetchmode (minor)
      - DSN support (minor)
      - Missing Drivers
      Frontbase (minor?: will the frontbase guys write one/modify the current
      for us?),
      LDAP (minor: I want it myself, but the driver is fairly new and not that
      "feature rich" yet, so we should be able to port it)
      - tableInfo (minor: metabase allready has some of those features)
      - flexible default behavior (minor: shouldn't be too big of a deal to
      implement)

      Small:
      - Error support (small?: maybe not a big deal through the use if
      customer error handler feature of metabase)
      - transaction (small?: I haven't look at either metabases' nor pear db's
      implementation)
      - DB_result object (minor)
      - error messages (small?)
      - getListOf (small?: Thomas was not sure about the status inside of PEAR
      of these features)

      Large:
      - LOB support (large?: I haven't look at either metabases' nor pear db's
      implementation)
      - API changess like function naming, parameter orders, coding style
      stuff (large: a lot of work but nothing too complicated)

      Unsure:
      - storage.php (?: is this used a lot? Will this work out of the box once
      we get API compatibility with PEAR DB? This could basically be
      understood as another level of abstraction . maybe this is where the
      LDAP handler should actually "attack" rather than as it is now in the
      level of PEAR DB core that was meant for relational rdbms)
      - different ways of how data is stored in the DB for emulated features:
      sequences, NULL ..? (?: this is actually one of those most difficult
      areas .. hopefully a simply convert script could take care of it . but
      its definitely ugly if certain features have been emulated differently
      in the db)

      Approach:
      Dunno if this is crazy, but I would much rather not start off building
      wrappers, but to actually work on a merged version that is easy to make
      backward compatible with both abstraction layers (again as a warning
      this is my first shot at merging two fairly large and estabhlished
      projects - especially with opensource you gotta please the public or you
      will face forks with would defeat the purpose of this merger - so I am a
      bit nervous).

      Probably I will attempt at getting Pears DB.php to play nice with the
      metabase_database.php :-)

      Best regards,
      Lukas Smith
      smith@...

      PS: For anyone hoping for a faster pace with this development:
      unfortunately I do have very limited time to work on this. So any help
      is greatly appreciated.
      _______________________________
      DybNet Internet Solutions GbR
      Alt Moabit 89
      10559 Berlin
      Germany
      Tel. : +49 30 83 22 50 00
      Fax : +49 30 83 22 50 07
      www.dybnet.de info@...
      _______________________________
    • Manuel Lemos
      Hello, ... This can easily be emulated because Metabase supports random access to database rows. ... This was planned to be added to Metabase similarly to JDBC
      Message 2 of 6 , Feb 4, 2002
        Hello,

        Lukas Smith wrote:
        > Minor:
        > - fetchmode (minor)

        This can easily be emulated because Metabase supports random access to
        database rows.


        > - DSN support (minor)

        This was planned to be added to Metabase similarly to JDBC and it seems
        to PEAR-DB. I just disagree that the database name be required in the
        DSN as it seems that PEAR-DB requires. Metabase lets you change the
        database to each you want to connect in the with the same driver object
        instance. Actually you may perform queries to a server that has no
        databases installed. So what PEAR-DB does? I haven't checked, but does
        it fails to connect because you did not specify a database name?


        > - Missing Drivers
        > Frontbase (minor?: will the frontbase guys write one/modify the current
        > for us?),
        > LDAP (minor: I want it myself, but the driver is fairly new and not that
        > "feature rich" yet, so we should be able to port it)

        Yes, this is a long shot that in the end you may realize that doesn't
        quite fit in a database abstraction layer that is supposed to execute
        SQL queries.


        > - tableInfo (minor: metabase allready has some of those features)

        Actually Metabase does not have this yet. I want to add it because I
        want Metabase manager to be able to reverse engineer schemas of
        applications already installed by traditional means, to encourage people
        to migrate to Metabase and use the database independent schema
        definition.



        > - flexible default behavior (minor: shouldn't be too big of a deal to
        > implement)

        What does this mean?


        > Small:
        > - Error support (small?: maybe not a big deal through the use if
        > customer error handler feature of metabase)

        Yes, Metabase error handler callback should be sufficient to implement
        whatever error handling methods provides. Metabase passes more
        information to the error handler callback than just the error string
        returned by the Error() driver method.


        > - transaction (small?: I haven't look at either metabases' nor pear db's
        > implementation)

        I don't see a problem here. Metabase eventually will implement more
        transaction related things soon.


        > - DB_result object (minor)

        I think you have all in Metabase that you need to emulate PEAR-DB and
        probably more.


        > - error messages (small?)

        Error messages is not a problem. What Metabase does not provide is error
        number abstraction.

        The way I see it, errors are fatal conditions and so in each situation
        they should be handled the same way regardless of the type of error that
        happened. So, I never implemented error abstraction because I don't see
        much point in having a way to distinguish error types programatically.
        Usually an error string is enough for logging so the programmer can read
        the exact error message that the database back end returned along with
        some context information provided by Metabase but all in text because
        that is what is useful for humans (programmers).

        Also, it seems to me that error abstraction is quite utopic because an
        operation can fail in thousands of ways and a database can return
        different error codes for each so it will be an hell to categorize all
        the codes even if the driver can antecipate the meaning of all of them.

        Another thing that I disagree in PEAR-DB design is that certain
        conditions that are not errors but just status information is abstracted
        as errors. For those conditions there should be functions to retrieve
        them. For instance, querying affected rows or if a driver supports a
        certain capacity.

        If a programmer calls a function that is not implemented for a certain
        driver, that is a programming error, not a run-time error. The program
        should handle that as a fatal error with any need to distinguish what
        type of error was that. The right way for programmers to handle the
        (lack of ) ability of a driver to implement a certain capacity is to
        query the driver before trying to use it. So there is not need for
        abstracting this type error either.


        > - getListOf (small?: Thomas was not sure about the status inside of PEAR
        > of these features)

        I don't know what is this.

        > Large:
        > - LOB support (large?: I haven't look at either metabases' nor pear db's
        > implementation)

        AFAIK, PEAR-DB doesn't quite handle LOBs. It simply treats them as
        normal text fields and so it retrieves data all at once. This is
        inadequate because if you very large fields you will blow your computer
        memory trying to store or retrieve all at once.

        The proper way to handle LOBs is with a streaming interface. PHP lacks
        of streams abstraction, I mean you sure have streams for files,
        networking connections, but you can't redefine those streams in a way
        that you can use the same interface functions to store or retrieve data
        to them. This is something that is very well done in Java.

        Anyway, Metabase just provides a set a separate functions to abstract
        LOB streams. For instance you can execute a SELECT query that returns a
        LOB field and connect it to an INSERT or UPDATE prepared query that
        inserts the data in another LOB field.


        > - API changess like function naming, parameter orders, coding style
        > stuff (large: a lot of work but nothing too complicated)

        Forget it for now. Keep in mind that at this point you should be
        concentrated on making the proof of concept which is wrapping PEAR-DB
        API around Metabase API and demonstrate that it works as needed so
        nobody developing PEAR-DB applications need to change their code and the
        same for Metabase based applications.


        > Unsure:
        > - storage.php (?: is this used a lot? Will this work out of the box once
        > we get API compatibility with PEAR DB? This could basically be
        > understood as another level of abstraction . maybe this is where the
        > LDAP handler should actually "attack" rather than as it is now in the
        > level of PEAR DB core that was meant for relational rdbms)

        Yes, that seems to be an higher level abstraction that just sits on
        PEAR-DB API. There is nothing to do about it.



        > - different ways of how data is stored in the DB for emulated features:
        > sequences, NULL ..? (?: this is actually one of those most difficult
        > areas .. hopefully a simply convert script could take care of it . but
        > its definitely ugly if certain features have been emulated differently
        > in the db)

        Yes, personally this is another thing that I quite don't agree on some
        PEAR-DB implementations. PEAR-DB should not be executing implicit schema
        management functions on demand because for security reasons it may not
        be desired that the current application user have privileges to do it.
        For instance, it is wrong to create sequence emulation tables on demand
        because as I said the application user may not have privileges to create
        tables.

        The proper way to do it is to have separate schema management functions
        like Metabase provides for things creating/dropping/altering
        tables/fields/indexes/sequences. Of course PEAR-DB does those things
        like on demand schema management because it lacks of explicit functions
        to do so by design.

        Honestly I don't know what to do about it because there is probably a
        reasonable number of PEAR-DB users relying on that behaviour. I think
        that PEAR-DB functions that rely on that behaviour should be deprecated
        but I'm afraid the problem won't go away doing just that.


        > Approach:
        > Dunno if this is crazy, but I would much rather not start off building
        > wrappers, but to actually work on a merged version that is easy to make
        > backward compatible with both abstraction layers (again as a warning
        > this is my first shot at merging two fairly large and estabhlished
        > projects - especially with opensource you gotta please the public or you
        > will face forks with would defeat the purpose of this merger - so I am a
        > bit nervous).

        I think that is unrealistic. You may waste very easily many months
        rewriting everything and in the end realize that you may still not make
        the proof of concept of the merget. I advise that you just stick to the
        minimum necessary to make the proof of concept.


        > Probably I will attempt at getting Pears DB.php to play nice with the
        > metabase_database.php :-)

        That is more realistic.


        > PS: For anyone hoping for a faster pace with this development:
        > unfortunately I do have very limited time to work on this. So any help
        > is greatly appreciated.

        That may also be a problem. If you are not able to make the proof of
        concept soon, the actual merger never start and more releases of PEAR-DB
        and Metabase are made with new and incompatible feature implementations
        making the goal even harder. Just try to no wonder too much on things
        for which you not get quick answers.

        Regards,
        Manuel Lemos
      • Lukas Smith
        ... seems ... object ... so I will not worry about doing this myself for now ... people ... Well you can get the column names in metabase iirc. But if you are
        Message 3 of 6 , Feb 4, 2002
          > > - DSN support (minor)
          >
          > This was planned to be added to Metabase similarly to JDBC and it
          seems
          > to PEAR-DB. I just disagree that the database name be required in the
          > DSN as it seems that PEAR-DB requires. Metabase lets you change the
          > database to each you want to connect in the with the same driver
          object
          > instance. Actually you may perform queries to a server that has no
          > databases installed. So what PEAR-DB does? I haven't checked, but does
          > it fails to connect because you did not specify a database name?

          so I will not worry about doing this myself for now

          > > - tableInfo (minor: metabase allready has some of those features)
          >
          > Actually Metabase does not have this yet. I want to add it because I
          > want Metabase manager to be able to reverse engineer schemas of
          > applications already installed by traditional means, to encourage
          people
          > to migrate to Metabase and use the database independent schema
          > definition.

          Well you can get the column names in metabase iirc.
          But if you are working on this I will also not worry about this for now.

          > > - getListOf (small?: Thomas was not sure about the status inside of
          PEAR
          > > of these features)
          >
          > I don't know what is this.

          PEAR DB manual:
          DB::getListOf() -- list internal DB info
          I read somewhere else that this method gives data about users and the
          database etc ...
          But Thomas was not sure how far development in this method has gotten.

          What is the status of the LOB support in PEAR DB .. is this experimental
          right now? Is this used? Do I have to worry about backward compatibility
          or is it enough to provide the functionality that metabase offers (which
          workes quite well for me)

          I will do some function renaming, but I will only do that for my testing
          enviorment. So I will only do this for mysql (I think our DB testing
          server is currently being used for something else so I don't have
          anything else easily available) so this will not be all that much work.
          Generally I am mainly familiar with mysql (I have done some development
          on Postgresql and a bit of tyoing around with Oracle and DB2. So if you
          guys can think of any pitfalls that are specific to certain RDBMS
          drivers but that need special handling inside the basic package (well
          theoretically this should not matter since the RDBMS specific driver
          should allways be able to overwrite any core methods ...) please let me
          know.

          Best regards,
          Lukas Smith
          smith@...
          _______________________________
          DybNet Internet Solutions GbR
          Alt Moabit 89
          10559 Berlin
          Germany
          Tel. : +49 30 83 22 50 00
          Fax : +49 30 83 22 50 07
          www.dybnet.de info@...
          _______________________________
        • Lukas Smith
          ... Just refering to the long list of defines in PEAR-DB where you can define the default functionality of a number of methods. Best regards, Lukas Smith
          Message 4 of 6 , Feb 4, 2002
            >> - flexible default behavior (minor: shouldn't be too big of a deal to
            >> implement)
            >
            >What does this mean?

            Just refering to the long list of defines in PEAR-DB where you can
            define the default functionality of a number of methods.

            Best regards,
            Lukas Smith
            smith@...
            _______________________________
            DybNet Internet Solutions GbR
            Alt Moabit 89
            10559 Berlin
            Germany
            Tel. : +49 30 83 22 50 00
            Fax : +49 30 83 22 50 07
            www.dybnet.de info@...
            _______________________________
          • Manuel Lemos
            Hello, ... Yes, it is basically calling parse_url and assigning the result values to arguments array of MetabaseSetupDatabase. ... Yes, that is for result set
            Message 5 of 6 , Feb 13, 2002
              Hello,

              Lukas Smith wrote:
              >
              > > > - DSN support (minor)
              > >
              > > This was planned to be added to Metabase similarly to JDBC and it
              > seems
              > > to PEAR-DB. I just disagree that the database name be required in the
              > > DSN as it seems that PEAR-DB requires. Metabase lets you change the
              > > database to each you want to connect in the with the same driver
              > object
              > > instance. Actually you may perform queries to a server that has no
              > > databases installed. So what PEAR-DB does? I haven't checked, but does
              > > it fails to connect because you did not specify a database name?
              >
              > so I will not worry about doing this myself for now

              Yes, it is basically calling parse_url and assigning the result values
              to arguments array of MetabaseSetupDatabase.


              > > > - tableInfo (minor: metabase allready has some of those features)
              > >
              > > Actually Metabase does not have this yet. I want to add it because I
              > > want Metabase manager to be able to reverse engineer schemas of
              > > applications already installed by traditional means, to encourage
              > people
              > > to migrate to Metabase and use the database independent schema
              > > definition.
              >
              > Well you can get the column names in metabase iirc.

              Yes, that is for result set metadata not for table metadata.


              > But if you are working on this I will also not worry about this for now.

              Yes, but it will not be ready soon at least upto the level I want it
              which is to reverse engineer schemas installed by traditional methods. I
              need to give some thought about how to resolve data type mapping
              ambiguities.


              Regards,
              Manuel Lemos
            Your message has been successfully submitted and would be delivered to recipients shortly.