Loading ...
Sorry, an error occurred while loading the content.

14RE: [OracleWarehouseBuilder] Version control for data

Expand Messages
  • Rick Hall
    Nov 3, 2003
    • 0 Attachment
      Arnoud -

      Thanks for your suggestion. I was under the (apparently mistaken)
      impression that Workspace Manager allowed one to handle and
      **compare** multiple versions of metadata, not data.

      I'll double check the data comparison functionality of Oracle
      Workspace Manager and see if it lives up to the claims of this
      description.

      - Rick


      --- Arnoud Otte <arnoud.otte@...> wrote:
      > Hi Rick,
      >
      > For that kind of data version control, why not have a more detailed
      > look
      > at Oracle Workspace Manager. It comes with 9iR2. From what I have
      > read -
      > no hands on experience - it seems to hold a lot of potential in
      > solving
      > your particular problem within Oracle itself.
      >
      >
      http://otn.oracle.com/products/workspace_mgr/htdocs/Workspace_Manager_Da
      > taSheet92.html
      >
      > Excerpt:
      >
      > "Oracle Workspace Manager is a unique and versatile feature of the
      > Oracle9i Database for application developers and DBAs. It provides
      > a
      > virtual environment (workspaces) in which to: create multiple
      > versions
      > of table data; isolate collections of changes to table data for
      > long
      > duration transactions lasting days or weeks; and keep a persistent
      > history of changes.
      > ...
      > Manage a collection of updates and inserts as a unit before turning
      > them
      > into production / public data.
      > Workspace Manager lets you modify, review and rollback changes
      > before
      > making them public. Until you make the changes public, they are
      > invisible to other users of the database, who will access only the
      > regular production data. You can organize the changes in a simple
      > set of
      > workspaces or in a complex workspace hierarchy. Two examples of
      > this are
      > 1. A life sciences application in which Workspace Manager supports
      > the
      > discovery and quality assurance processes by managing a collection
      > of
      > gene annotations in a workspace and making it public after the
      > successful completion of the findings. 2. A HR application that
      > collects and isolates changes to salaries and job titles in a
      > workspace,
      > and on approval merges them into production data.
      > ...
      > Savepoints
      > Savepoints are milestones in the workspace to which row changes in
      > version-enabled tables can be rolled back to, and to which users
      > can go
      > to see the state of the database as it existed as of a particular
      > point
      > in time. Savepoints are usually created in response to business
      > events,
      > such as the completion of a design or the end of a billing period.
      > Modifying a row after a savepoint is created causes a new version
      > of
      > that row to be created. Subsequent changes are applied to this new
      > version in the workspace until another savepoint is created. Users
      > can
      > compare differences between any two savepoints."
      >
      > Arnoud.
      >
      > -----Original Message-----
      > From: Rick Hall [mailto:rphall1@...]
      > Sent: Saturday, October 11, 2003 08:10
      > To: OracleWarehouseBuilder@yahoogroups.com
      > Subject: [OracleWarehouseBuilder] Version control for data
      >
      >
      > I'm new to Oracle Warehouse Builder. I am wondering if it would be
      > suitable for the following task.
      >
      > A client of mine, a bio-informatics company, is in the business of
      > aggregating, correlating, curating, and reselling genomic,
      > proteonic,
      > and interaction data. The company receives data from various
      > public-source and proprietary databases, and using its own
      > proprietary applications, imports the third-party data into its own
      > database. The company adds value for its customers by simplifying
      > their access to biological data. Instead of researching genes,
      > proteins and interactions using 30 or so separate databases, the
      > company's customers can use just one relational database for
      > research.
      >
      > The company's product is essentially pure data. The amount of data
      > is
      > not large, perhaps a few hundred MB when exported to an Oracle dump
      > file.
      >
      > The problem that the company faces is quality control. Essentially,
      > they have to maintain not only their proprietary import
      > applications
      > under version control, but they also have to maintain their
      > **data**
      > under version control. For example, if a bug is discovered in some
      > import application, they must be able to rollback to an earlier
      > version of the data and reapply a fixed version of the application.
      > Things can get complicated because of the large number of source
      > databases (not all of which are relational -- even particularly
      > rational) and the corresponding number of import applications.
      >
      > Until now, the bio-informatics company has relied on binary formats
      > for creating checkpoints of their data. For example, they regularly
      > backup their data and use Oracle Export to create snapshots. The
      > problem is that binary formats are too coarse-grained for effective
      > version control of the database. For example, sometimes only a few
      > rows in one table might change as part of the data curation
      > process.
      > When that incremental change is archived by Oracle Export, it
      > becomes
      > quite opaque, which complicates quality control processes.
      >
      > I'm in the process of trying to improve the version control system
      > for this company. I've proposed moving to text-based snapshots of
      > the
      > database, which would be stored in CVS and which could be easily
      > diff'ed as part of a quality control program.
      >
      > I see that OWB allows one to target text files as output. But this
      > isn't quite enough. Right now, the company's schema is evolving
      > rapidly, as new sources are added and as customers request new
      > features. The company needs an easy way to export all data
      > (sequences, procedures, tables, views, etc) from a single schema
      > without extensively reconfiguring some conversion process.
      >
      > So, in a nutshell, will OWB allow one to easily export the metadata
      > and data from all objects in a single schema to a set of text-based
      > files?
      >
      > Thanks in advance.
      >
      > - Rick Hall
      >
      >
      > __________________________________
      > Do you Yahoo!?
      > The New Yahoo! Shopping - with improved product search
      > http://shopping.yahoo.com
      >
      >
      >
      > Yahoo! Groups Sponsor
      >
      >
      <http://rd.yahoo.com/M=259395.3614674.4902533.1261774/D=egroupweb/S=1705
      >
      115386:HM/A=1524963/R=0/SIG=12o885gmo/*http://hits.411web.com/cgi-bin/au
      > toredir?camp=556&lineid=3614674&prop=egroupweb&pos=HM>
      >
      >
      <http://us.adserver.yahoo.com/l?M=259395.3614674.4902533.1261774/D=egrou
      > pmail/S=:HM/A=1524963/rand=560635249>
      >
      > To unsubscribe from this group, send an email to:
      > OracleWarehouseBuilder-unsubscribe@yahoogroups.com
      >
      >
      >
      > Your use of Yahoo! Groups is subject to the Yahoo! Terms of Service
      > <http://docs.yahoo.com/info/terms/> .
      >
      >
      >

      __________________________________
      Do you Yahoo!?
      Exclusive Video Premiere - Britney Spears
      http://launch.yahoo.com/promos/britneyspears/
    • Show all 5 messages in this topic