Loading ...
Sorry, an error occurred while loading the content.

2523Re: NLP Buddies... and NLP Babies!

Expand Messages
  • Jim Bromer
    Aug 3, 2004
      I prefer not the term vaporware when referring to my programming.

      I have no professional background to speak of, although I am doing
      some data base stuff and even a little programming for some people.

      I have been studying programming since 1982. I have been studying
      subjects related to this AI project since 1991.

      Right now I am trying to debug the file database management system
      that I have written for my program before I start working on the
      actual AI. However, I have so many different ideas about writing AI
      algorithms that I am reasonably confident that I will accomplish
      something.

      I believe that you have to explore both positive and critical
      aspects of a theory. But it is also important to explore
      alternative theories as well. These alternative theories may
      overlap rather than compete directly. But the situation is more
      complicated than that. Any reference may refer to a complicated
      referent. A complicated referent may contain or otherwise be
      relevant to systems of other referents. The references used in
      representations of ideas may refer to things like material objects,
      relations, abstractions, imaginary objects or any other object or
      relation from thought and nature.

      Symbolic references may contain logical expressions or other kinds
      of relations. However, the referents of the symbolic references may
      themselves contain or refer to relational properties that could make
      the symbolic expression invalid, untrue or even nonsensical. In
      other words, thoughts are not always true, accurate, valid or even
      meaningful. The problem here is that if further reasoning is based
      on nonsense or other "noise", the products of that reasoning may be
      incomprehensibly meaningless. There has to be a way to separate the
      noise from the meaningful and reasonable.

      I believe then that a system of education or other interactive
      learning is necessary in AI. But, the program has to be able to
      solve some of the problems that it encounters itself.

      If we knew all the rules of learning or if we knew the fundamental
      rules that would enable the program to derive the rest, we could
      write them out as some kind of formal rule system. But we obviously
      don't know them. We have to learn what they are. The program has
      to be able to learn new rules, but it also has to integrate them
      within a complicated system of knowledge. I suspect that this
      integration of new rules of learning and new pieces of knowledge is
      where the real complications lurk.

      I can describe what I call flexible rules that use different
      responses for different contexts. However, I do not know how the
      system could create new variations of flexible rules, validate their
      use and integrate them without some kind of feedback. This feedback
      may be determined through passive observation, interaction with an
      instructor or by interacting with the objects of an environment.

      My idea is that an AI program will develop theories or theory-like
      things that explain observed events. These theories will need to
      conform to various observations derived from interactions with the
      input-output environment.

      Right now I suspect that the program has to use a trial and error
      approach to try different ways to connect ideas to see if they can
      combine to define possible and reasonable relations between the
      referent ideas of an expression (or between the referent objects of
      related observations). An instructor will not be able to define the
      thousands of little connections between ideas that would be required
      to fully represent a complex idea, but the computer could use
      overlapping examples to get a sense of the core ideas significant to
      a particular subject. However, this reasonable solution to the
      problem is made more difficult (and perhaps more unreasonable)
      because a symbolic expression like natural language uses reference
      terms to designate the subject matter. Since there is more than one
      way to refer to a subject, this means that the program has to be
      able to figure out what subject is being referred to in an
      expression. Therefore, a simpler means to designate the subject of
      a reference term has to be used in primal learning or in an
      elementary check to try to make sure that the computer understands
      the primary subjects of an expression. The problem is also relevant
      in non-symbolic environments, because a recognized object acts like
      a symbol or index to related objects. A leaf may be taken as a
      symbol of the tree for example. In non-linguistic interactions,
      like a robot's interactions with the environment, this primary
      subject reference might involve simple interactions with objects to
      examine them to make sure that they produce the effects that the
      robot AI had assumed to be relevant in its theories about the
      environment. In passive observations, like celestial observations,
      other means would have to be used to try corroborate theories and
      observations. In an interactive IO environment like language the
      program and the instructor would have to establish a way to
      communicate fundamental relations between references. I have
      expressed this last sentence carefully in order to cover different
      manifestations of this process. In traditional linguistic learning,
      this primary relational association might be between word and object
      where the object is observed through visual input. For instance the
      word "water" might be associated with the visual observation of a
      glass of water. In a text only environment, the primary relational
      associations might be constructed with words that refer to a
      fundamental or important relation of a common subject, like "drink"
      and "water". Although the traditional sense of human learning is
      presumably based on the sound of the word and the image or touch of
      the object, on a more profound examination you can see that it truly
      involves the establishment of a relation between two referent
      objects. But these two references are actually comprised of
      complicated particles of input data. Therefore, I concluded that
      the AI has to be able to figure some of the problems out for itself
      before it could recognize that two complicated referent objects may
      be relevantly related, and it needs some way to test fundamental and
      significant associations between referent objects.

      As more information became available to the AI, it would be able to
      test different expressions that can be related to some common
      subjects. If this integration process produces conflicts they can
      be examined to try to find the reasons for the conflicts.

      So my yet-to-be tested theory of AI currently incorporates three
      basic methods to attempt to corroborate the AI's theories about the
      relations between objects of reference. The first is established
      through a primal or elementary communication with an instructor or
      with the objects of an environment to try to detect regularly
      occurring patterns or representations of objects their features and
      their relations. These co-occurring regular patterns may be used as
      primal references. The second step in establishing the validity of
      the AI's theories about the world will consist of establishing
      alternative theories about the subjects of interest in an attempt to
      find groups of theories that best conform to observed input. The
      third step then will involve the establishment and testing of
      integrated expressions or actions which will allow the system to
      again try to confirm its theories by utilizing them in focused
      patterns of examination to test the mutual consistency of its
      theories within the context of the subject matter. This is not an
      easy step since inconsistency outside of a strict logical system is
      not easily reduced to point to a particular cause for the conflict.

      In all three stages the computer has to be able to solve some of the
      problems itself. When learning something new, the program could
      rely on simple generalization techniques. At the more advanced
      level of primary learning it can use previously established
      communication or interaction techniques. At the highest level it
      would use previously learned theoretical systems to explore the
      complex interrelations of an integrated network of references.

      Jim Bromer

      --- In artificialintelligencegroup@yahoogroups.com, "toddpierce1968"
      <todd_pierce@h...> wrote:
      > Jim,
      >
      > I knew you were going to come through with a message describing the
      > pursuit of something just as insane as what I am pursuing.
      >
      > The description of what you're doing is very insightful. Based on
      the
      > limited description of what I'm doing myself, you managed to peg my
      > project right on the tail of the donkey.
      >
      > In fact, it's quite ironic that I, somewhat trained in linguistics,
      > would be pursuing a more reductive computational model whereas you,
      > apparently a business programmer from what I can tell, would be
      using
      > a biological/developmental model. By the way, I would like to
      hear a
      > bit about your background training so I know what vocabulary to
      use as
      > we discuss these things.
      >
      > What you are doing is definitely much more interesting. I can
      imagine
      > it's hard to fit values normally encoded in a child's brain into a
      > standard database format. Even with my model which is based on a
      > minimal parametric approach of the simplest of feature matrices,
      > designing the database was a nightmare.
      >
      > It was difficult to prioritize what features about a word or a
      > 'concept' computationally provided the greatest contribution to
      coming
      > up with the right result. I've had to revise my database a couple
      of
      > times in the past year, and that's after six years of planning...
      and
      > that's just for the lexicon!
      >
      > As I imagine you already know, since all of these 'features' need
      to
      > be specifically set by the trainer, I have to write interactive
      > subroutines to support every database table. Everything has to be
      > explicity explained to the computer.
      >
      > I do think that we're both discovering interesting abstractions
      about
      > human language as we sort through the various different factors
      that
      > ultimately make human language the best one invented yet. We may
      even
      > be discovering things that nobody else has discovered before.
      >
      > I do have some training in developmental linguistics so maybe I
      could
      > be of some contribution. Regardless of my theoretic experience,
      your
      > practical implementation of language acquisition sounds
      fascinating.
      > I may even be able to use some of the information in my own
      > implementation.
      >
      > I'd be very interested not only in the current incarnation of your
      > database but also in what is vaporware. Is this all in your head
      > still or do you have it documented?
      >
      > You see, this is one reason I figured conducting some of this
      > conversation outside of the forum. We're both dealing with systems
      > and concepts that are so outrageously huge that just the two of us
      > could take over the floor.
      >
      > I'm fine with it for the time being, but remember, I do keep
      backups
      > of my work as well as the documentation on a website, and I don't
      plan
      > to post the address in a forum. This is not because it's secret,
      it's
      > because I don't need to invite armchair critics from all over the
      > world :)
      >
      > That point aside, what interests me most is the language
      acquisition
      > rules you have come up with so far and the resulting database
      > architecture, either implemented or planned. Naturally, if that's
      too
      > big a question to answer here you know you have been invited to
      mail
      > me at todd_pierce at hotmail dot com.
      >
      > -Todd
      >
      > --- In artificialintelligencegroup@yahoogroups.com, "Jim Bromer"
      > <jbromer@i...> wrote:
      > > All of your comments seem pretty familiar to me. I am still
      working
      > > on the database management part of my program and I am starting
      with a
      > > lexicon of particles of input text. I have decided not to start
      with
      > > a list of words because I want to figure out how the program can
      learn
      > > to identify meaningful words and then to determine their
      utilization
      > > meanings. I believe that this is the fundamental contemporary
      problem
      > > of artificial intelligence and that the effort to jump start
      learning
      > > with an established data base of words and relationships is only
      > > pushing the problem back to the next stage of development. What
      I
      > > mean is that the trial and error algorithms and the interactions
      that
      > > may help the program to evaluate them is the significant problem
      in
      > > contemporary AI, and the thing is that this problem exists at all
      > > levels of intelligence. Let me put it this way: if an infant
      must use
      > > some special means to initially learn language and to figure
      basic
      > > things about the environment out, then he will always possess
      those
      > > special skills. We have a bias against initial learning because
      while
      >
      >
      > > we cannot understand it, we ourselves have progressed to more
      advanced
      > > issues ourselves. However, I believe that this naïve learning
      may be
      > > comprised of a system of associating data objects both
      immediately
      > > perceived and stored in memory in such a way so that they play
      certain
      > > roles in understanding. My interest then is to find the
      simplest set
      > > of roles that would be necessary for intelligence to emerge
      through
      > > this kind of process. However, this is a logistical nightmare
      for me
      > > too. If this set of what I have sometimes called ideological
      roles
      > > becomes too extensive, then it would be too unwieldy to use.
      > >
      > > I would like to continue using the artificialintelligencegroup to
      > > discuss these ideas so that others could join in if they want to.
      > >
      > > Jim Bromer
      > >
      > >
      > > --- In
      artificialintelligencegroup@yahoogroups.com, "toddpierce1968"
      > > <todd_pierce@h...> wrote:
      > > > Sam, Jim,
      > > >
      > > > Well, the project is already started (and not a secret in any
      way).
      > > > The one part that is pretty much complete is handling the
      lexicon.
      > > >
      > > > I decided to implement a custom database of sorts and use a
      > > > syntactically tagged corpus I bought from Celex. The program
      can now
      > > > store vocabulary, import vocaulary and there is a subroutine to
      > > > interactively refine syntactic features that are not set yet.
      > > >
      > > > I'm planning to implement feature matrices in as many places as
      > > > possible in the system; a brute force method that few people
      have
      > > > really taken full advantage of.
      > > >
      > > > So basically where I'm at is I'm sitting on a database system
      that,
      > > > given a word in text, like 'walked', will return a hell of a
      lot of
      > > > information about 'walked', and in addition, about the root
      word
      > > > 'walk' as well. Root forms and derived forms are stored in
      two
      > > > different tables.
      > > >
      > > > This whole project is logistically insane. That's why I just
      do it
      > > > bit by bit in my spare time. But I have managed to mostly
      take care
      > > > of the lexicon that way and as of a few weeks ago, it's
      structure and
      > > > support functions (LISP) are done.
      > > >
      > > > So now it's on to a parser and knowlege representation. I had
      planned
      > > > to proceed using a model I learned in college, but over the
      years,
      > > > I've made a lot of improvements on it just drunken
      brainstorming.
      > > >
      > > > Which is why I decided to post a message. If I can improve on
      his
      > > > model just brainstorming by myself, maybe I could make a lot
      more
      > > > progress if I had someone to discuss it with. Maybe I could
      come up
      > > > with a different model alltogether. If nothing else, I know
      I'd be
      > > > less likely to forget something I'd have to go back and fix
      later.
      > > >
      > > > Since it's planned to be a binary feature intensive system,
      what sorts
      > > > of features would be computationally useful for words and the
      objects
      > > > they identify? living/non-living, physical/non-physical, etc.
      > > >
      > > > And what about turning sentences into some sort of 'internal
      > > > representation'? How abstract does it have to be? Must it be
      hard
      > > > logic or is something resembling natural language phrase
      structure
      > > > good enough? Do some things simply need to be stored
      literally as
      > > > they text typed in?
      > > >
      > > > And speaking of this parsing process, must it be a computer
      > > > programmer's dream of lexicon->parser->knowlege or must it be
      blurred,
      > > > where each step must depend on every level of representation?
      Can the
      > > > process of generating a sentence use the same machinery but
      backwards?
      > > > i.e knowlege->parser->lexicon?
      > > >
      > > > Is there any way neural networks would be uniquely qualified
      for any
      > > > of this?
      > > >
      > > > And what cues can we take from nature? For example, we do
      know that
      > > > humans do have a 'working memory' with a dedicated buffer just
      for
      > > > language. We also know the concept of time is key when
      dealing with
      > > > language and knowlege. And all higher animals have a concept
      of
      > > > Euclidian geometry. Most language happens in discourse, which
      usually
      > > > provides many cues to parsing and meaning, how can I leverage
      off of
      > > > that? Furthermore, is it important to forget things?
      > > >
      > > > So, what cognititive machinery could help in this effort?
      > > >
      > > > If those are the sorts of things you don't mind mulling about
      in the
      > > > back of your head for a few days a week, feel free to e-mail
      me at
      > > > todd_pierce@h...
      > > >
      > > > I do have a document that goes back years which is dedicated
      to this
      > > > sort of brainstorming and I don't mind recording even the
      craziest of
      > > > ideas. After all, the document did result in a rather nice
      lexicon.
      > > >
      > > > -Todd
      > > >
      > > >
      > > > --- In artificialintelligencegroup@yahoogroups.com, Samuel Buhr
      > > > <sam_lonester@y...> wrote:
      > > > > Wow! NLP! How do you propose to start on this
      > > > > project? Tell me more, without giving up valuable
      > > > > information, of course.
      > > > >
      > > > > Sam
      > > > >
      > > > >
      > > > > --- toddpierce1968 <todd_pierce@h...> wrote:
      > > > >
      > > > > > Once again, I'm posting a message looking for
      > > > > > anybody who is
      > > > > > interested in NLP or Linguistics.
      > > > > >
      > > > > > I'm working on my own natural language processor so
      > > > > > it would be nice
      > > > > > to take up dialogue with anybody who has done this
      > > > > > or is interested in it.
      > > > > >
      > > > > > Furthermore, maybe I would make fewer mistakes
      > > > > > during development :)
      > > > > >
      > > > > > -T
      > > > > >
      > > > > >
      > > > > >
      > > > > > ------------------------ Yahoo! Groups Sponsor
      > > > > > --------------------~-->
      > > > > > Yahoo! Domains - Claim yours for only $14.70
      > > > > >
      > > > > http://us.click.yahoo.com/Z1wmxD/DREIAA/yQLSAA/7brrlB/TM
      > > > > >
      > > > >
      > > -----------------------------------------------------------------
      ---~->
      > > > > >
      > > > > >
      > > > > >
      > > > > > Yahoo! Groups Links
      > > > > >
      > > > > >
      > > > > http://groups.yahoo.com/group/artificialintelligencegroup/
      > > > > >
      > > > > >
      > > > > >
      > > > > artificialintelligencegroup-unsubscribe@yahoogroups.com
      > > > > >
      > > > > >
      > > > > >
      > > > > >
      > > > >
      > > > >
      > > > >
      > > > >
      > > > > __________________________________
      > > > > Do you Yahoo!?
      > > > > New and Improved Yahoo! Mail - Send 10MB messages!
      > > > > http://promotions.yahoo.com/new_mail
    • Show all 13 messages in this topic