Loading ...
Sorry, an error occurred while loading the content.

Re: Linguists? NLP Buddies?

Expand Messages
  • Jim Bromer
    All of your comments seem pretty familiar to me. I am still working on the database management part of my program and I am starting with a lexicon of
    Message 1 of 13 , Aug 1, 2004
    • 0 Attachment
      All of your comments seem pretty familiar to me. I am still working
      on the database management part of my program and I am starting with a
      lexicon of particles of input text. I have decided not to start with
      a list of words because I want to figure out how the program can learn
      to identify meaningful words and then to determine their utilization
      meanings. I believe that this is the fundamental contemporary problem
      of artificial intelligence and that the effort to jump start learning
      with an established data base of words and relationships is only
      pushing the problem back to the next stage of development. What I
      mean is that the trial and error algorithms and the interactions that
      may help the program to evaluate them is the significant problem in
      contemporary AI, and the thing is that this problem exists at all
      levels of intelligence. Let me put it this way: if an infant must use
      some special means to initially learn language and to figure basic
      things about the environment out, then he will always possess those
      special skills. We have a bias against initial learning because while
      we cannot understand it, we ourselves have progressed to more advanced
      issues ourselves. However, I believe that this naïve learning may be
      comprised of a system of associating data objects both immediately
      perceived and stored in memory in such a way so that they play certain
      roles in understanding. My interest then is to find the simplest set
      of roles that would be necessary for intelligence to emerge through
      this kind of process. However, this is a logistical nightmare for me
      too. If this set of what I have sometimes called ideological roles
      becomes too extensive, then it would be too unwieldy to use.

      I would like to continue using the artificialintelligencegroup to
      discuss these ideas so that others could join in if they want to.

      Jim Bromer


      --- In artificialintelligencegroup@yahoogroups.com, "toddpierce1968"
      <todd_pierce@h...> wrote:
      > Sam, Jim,
      >
      > Well, the project is already started (and not a secret in any way).
      > The one part that is pretty much complete is handling the lexicon.
      >
      > I decided to implement a custom database of sorts and use a
      > syntactically tagged corpus I bought from Celex. The program can now
      > store vocabulary, import vocaulary and there is a subroutine to
      > interactively refine syntactic features that are not set yet.
      >
      > I'm planning to implement feature matrices in as many places as
      > possible in the system; a brute force method that few people have
      > really taken full advantage of.
      >
      > So basically where I'm at is I'm sitting on a database system that,
      > given a word in text, like 'walked', will return a hell of a lot of
      > information about 'walked', and in addition, about the root word
      > 'walk' as well. Root forms and derived forms are stored in two
      > different tables.
      >
      > This whole project is logistically insane. That's why I just do it
      > bit by bit in my spare time. But I have managed to mostly take care
      > of the lexicon that way and as of a few weeks ago, it's structure and
      > support functions (LISP) are done.
      >
      > So now it's on to a parser and knowlege representation. I had planned
      > to proceed using a model I learned in college, but over the years,
      > I've made a lot of improvements on it just drunken brainstorming.
      >
      > Which is why I decided to post a message. If I can improve on his
      > model just brainstorming by myself, maybe I could make a lot more
      > progress if I had someone to discuss it with. Maybe I could come up
      > with a different model alltogether. If nothing else, I know I'd be
      > less likely to forget something I'd have to go back and fix later.
      >
      > Since it's planned to be a binary feature intensive system, what sorts
      > of features would be computationally useful for words and the objects
      > they identify? living/non-living, physical/non-physical, etc.
      >
      > And what about turning sentences into some sort of 'internal
      > representation'? How abstract does it have to be? Must it be hard
      > logic or is something resembling natural language phrase structure
      > good enough? Do some things simply need to be stored literally as
      > they text typed in?
      >
      > And speaking of this parsing process, must it be a computer
      > programmer's dream of lexicon->parser->knowlege or must it be blurred,
      > where each step must depend on every level of representation? Can the
      > process of generating a sentence use the same machinery but backwards?
      > i.e knowlege->parser->lexicon?
      >
      > Is there any way neural networks would be uniquely qualified for any
      > of this?
      >
      > And what cues can we take from nature? For example, we do know that
      > humans do have a 'working memory' with a dedicated buffer just for
      > language. We also know the concept of time is key when dealing with
      > language and knowlege. And all higher animals have a concept of
      > Euclidian geometry. Most language happens in discourse, which usually
      > provides many cues to parsing and meaning, how can I leverage off of
      > that? Furthermore, is it important to forget things?
      >
      > So, what cognititive machinery could help in this effort?
      >
      > If those are the sorts of things you don't mind mulling about in the
      > back of your head for a few days a week, feel free to e-mail me at
      > todd_pierce@h...
      >
      > I do have a document that goes back years which is dedicated to this
      > sort of brainstorming and I don't mind recording even the craziest of
      > ideas. After all, the document did result in a rather nice lexicon.
      >
      > -Todd
      >
      >
      > --- In artificialintelligencegroup@yahoogroups.com, Samuel Buhr
      > <sam_lonester@y...> wrote:
      > > Wow! NLP! How do you propose to start on this
      > > project? Tell me more, without giving up valuable
      > > information, of course.
      > >
      > > Sam
      > >
      > >
      > > --- toddpierce1968 <todd_pierce@h...> wrote:
      > >
      > > > Once again, I'm posting a message looking for
      > > > anybody who is
      > > > interested in NLP or Linguistics.
      > > >
      > > > I'm working on my own natural language processor so
      > > > it would be nice
      > > > to take up dialogue with anybody who has done this
      > > > or is interested in it.
      > > >
      > > > Furthermore, maybe I would make fewer mistakes
      > > > during development :)
      > > >
      > > > -T
      > > >
      > > >
      > > >
      > > > ------------------------ Yahoo! Groups Sponsor
      > > > --------------------~-->
      > > > Yahoo! Domains - Claim yours for only $14.70
      > > >
      > > http://us.click.yahoo.com/Z1wmxD/DREIAA/yQLSAA/7brrlB/TM
      > > >
      > >
      --------------------------------------------------------------------~->
      > > >
      > > >
      > > >
      > > > Yahoo! Groups Links
      > > >
      > > >
      > > http://groups.yahoo.com/group/artificialintelligencegroup/
      > > >
      > > >
      > > >
      > > artificialintelligencegroup-unsubscribe@yahoogroups.com
      > > >
      > > >
      > > >
      > > >
      > >
      > >
      > >
      > >
      > > __________________________________
      > > Do you Yahoo!?
      > > New and Improved Yahoo! Mail - Send 10MB messages!
      > > http://promotions.yahoo.com/new_mail
    • toddpierce1968
      Jim, I knew you were going to come through with a message describing the pursuit of something just as insane as what I am pursuing. The description of what
      Message 2 of 13 , Aug 1, 2004
      • 0 Attachment
        Jim,

        I knew you were going to come through with a message describing the
        pursuit of something just as insane as what I am pursuing.

        The description of what you're doing is very insightful. Based on the
        limited description of what I'm doing myself, you managed to peg my
        project right on the tail of the donkey.

        In fact, it's quite ironic that I, somewhat trained in linguistics,
        would be pursuing a more reductive computational model whereas you,
        apparently a business programmer from what I can tell, would be using
        a biological/developmental model. By the way, I would like to hear a
        bit about your background training so I know what vocabulary to use as
        we discuss these things.

        What you are doing is definitely much more interesting. I can imagine
        it's hard to fit values normally encoded in a child's brain into a
        standard database format. Even with my model which is based on a
        minimal parametric approach of the simplest of feature matrices,
        designing the database was a nightmare.

        It was difficult to prioritize what features about a word or a
        'concept' computationally provided the greatest contribution to coming
        up with the right result. I've had to revise my database a couple of
        times in the past year, and that's after six years of planning... and
        that's just for the lexicon!

        As I imagine you already know, since all of these 'features' need to
        be specifically set by the trainer, I have to write interactive
        subroutines to support every database table. Everything has to be
        explicity explained to the computer.

        I do think that we're both discovering interesting abstractions about
        human language as we sort through the various different factors that
        ultimately make human language the best one invented yet. We may even
        be discovering things that nobody else has discovered before.

        I do have some training in developmental linguistics so maybe I could
        be of some contribution. Regardless of my theoretic experience, your
        practical implementation of language acquisition sounds fascinating.
        I may even be able to use some of the information in my own
        implementation.

        I'd be very interested not only in the current incarnation of your
        database but also in what is vaporware. Is this all in your head
        still or do you have it documented?

        You see, this is one reason I figured conducting some of this
        conversation outside of the forum. We're both dealing with systems
        and concepts that are so outrageously huge that just the two of us
        could take over the floor.

        I'm fine with it for the time being, but remember, I do keep backups
        of my work as well as the documentation on a website, and I don't plan
        to post the address in a forum. This is not because it's secret, it's
        because I don't need to invite armchair critics from all over the
        world :)

        That point aside, what interests me most is the language acquisition
        rules you have come up with so far and the resulting database
        architecture, either implemented or planned. Naturally, if that's too
        big a question to answer here you know you have been invited to mail
        me at todd_pierce at hotmail dot com.

        -Todd

        --- In artificialintelligencegroup@yahoogroups.com, "Jim Bromer"
        <jbromer@i...> wrote:
        > All of your comments seem pretty familiar to me. I am still working
        > on the database management part of my program and I am starting with a
        > lexicon of particles of input text. I have decided not to start with
        > a list of words because I want to figure out how the program can learn
        > to identify meaningful words and then to determine their utilization
        > meanings. I believe that this is the fundamental contemporary problem
        > of artificial intelligence and that the effort to jump start learning
        > with an established data base of words and relationships is only
        > pushing the problem back to the next stage of development. What I
        > mean is that the trial and error algorithms and the interactions that
        > may help the program to evaluate them is the significant problem in
        > contemporary AI, and the thing is that this problem exists at all
        > levels of intelligence. Let me put it this way: if an infant must use
        > some special means to initially learn language and to figure basic
        > things about the environment out, then he will always possess those
        > special skills. We have a bias against initial learning because while


        > we cannot understand it, we ourselves have progressed to more advanced
        > issues ourselves. However, I believe that this naïve learning may be
        > comprised of a system of associating data objects both immediately
        > perceived and stored in memory in such a way so that they play certain
        > roles in understanding. My interest then is to find the simplest set
        > of roles that would be necessary for intelligence to emerge through
        > this kind of process. However, this is a logistical nightmare for me
        > too. If this set of what I have sometimes called ideological roles
        > becomes too extensive, then it would be too unwieldy to use.
        >
        > I would like to continue using the artificialintelligencegroup to
        > discuss these ideas so that others could join in if they want to.
        >
        > Jim Bromer
        >
        >
        > --- In artificialintelligencegroup@yahoogroups.com, "toddpierce1968"
        > <todd_pierce@h...> wrote:
        > > Sam, Jim,
        > >
        > > Well, the project is already started (and not a secret in any way).
        > > The one part that is pretty much complete is handling the lexicon.
        > >
        > > I decided to implement a custom database of sorts and use a
        > > syntactically tagged corpus I bought from Celex. The program can now
        > > store vocabulary, import vocaulary and there is a subroutine to
        > > interactively refine syntactic features that are not set yet.
        > >
        > > I'm planning to implement feature matrices in as many places as
        > > possible in the system; a brute force method that few people have
        > > really taken full advantage of.
        > >
        > > So basically where I'm at is I'm sitting on a database system that,
        > > given a word in text, like 'walked', will return a hell of a lot of
        > > information about 'walked', and in addition, about the root word
        > > 'walk' as well. Root forms and derived forms are stored in two
        > > different tables.
        > >
        > > This whole project is logistically insane. That's why I just do it
        > > bit by bit in my spare time. But I have managed to mostly take care
        > > of the lexicon that way and as of a few weeks ago, it's structure and
        > > support functions (LISP) are done.
        > >
        > > So now it's on to a parser and knowlege representation. I had planned
        > > to proceed using a model I learned in college, but over the years,
        > > I've made a lot of improvements on it just drunken brainstorming.
        > >
        > > Which is why I decided to post a message. If I can improve on his
        > > model just brainstorming by myself, maybe I could make a lot more
        > > progress if I had someone to discuss it with. Maybe I could come up
        > > with a different model alltogether. If nothing else, I know I'd be
        > > less likely to forget something I'd have to go back and fix later.
        > >
        > > Since it's planned to be a binary feature intensive system, what sorts
        > > of features would be computationally useful for words and the objects
        > > they identify? living/non-living, physical/non-physical, etc.
        > >
        > > And what about turning sentences into some sort of 'internal
        > > representation'? How abstract does it have to be? Must it be hard
        > > logic or is something resembling natural language phrase structure
        > > good enough? Do some things simply need to be stored literally as
        > > they text typed in?
        > >
        > > And speaking of this parsing process, must it be a computer
        > > programmer's dream of lexicon->parser->knowlege or must it be blurred,
        > > where each step must depend on every level of representation? Can the
        > > process of generating a sentence use the same machinery but backwards?
        > > i.e knowlege->parser->lexicon?
        > >
        > > Is there any way neural networks would be uniquely qualified for any
        > > of this?
        > >
        > > And what cues can we take from nature? For example, we do know that
        > > humans do have a 'working memory' with a dedicated buffer just for
        > > language. We also know the concept of time is key when dealing with
        > > language and knowlege. And all higher animals have a concept of
        > > Euclidian geometry. Most language happens in discourse, which usually
        > > provides many cues to parsing and meaning, how can I leverage off of
        > > that? Furthermore, is it important to forget things?
        > >
        > > So, what cognititive machinery could help in this effort?
        > >
        > > If those are the sorts of things you don't mind mulling about in the
        > > back of your head for a few days a week, feel free to e-mail me at
        > > todd_pierce@h...
        > >
        > > I do have a document that goes back years which is dedicated to this
        > > sort of brainstorming and I don't mind recording even the craziest of
        > > ideas. After all, the document did result in a rather nice lexicon.
        > >
        > > -Todd
        > >
        > >
        > > --- In artificialintelligencegroup@yahoogroups.com, Samuel Buhr
        > > <sam_lonester@y...> wrote:
        > > > Wow! NLP! How do you propose to start on this
        > > > project? Tell me more, without giving up valuable
        > > > information, of course.
        > > >
        > > > Sam
        > > >
        > > >
        > > > --- toddpierce1968 <todd_pierce@h...> wrote:
        > > >
        > > > > Once again, I'm posting a message looking for
        > > > > anybody who is
        > > > > interested in NLP or Linguistics.
        > > > >
        > > > > I'm working on my own natural language processor so
        > > > > it would be nice
        > > > > to take up dialogue with anybody who has done this
        > > > > or is interested in it.
        > > > >
        > > > > Furthermore, maybe I would make fewer mistakes
        > > > > during development :)
        > > > >
        > > > > -T
        > > > >
        > > > >
        > > > >
        > > > > ------------------------ Yahoo! Groups Sponsor
        > > > > --------------------~-->
        > > > > Yahoo! Domains - Claim yours for only $14.70
        > > > >
        > > > http://us.click.yahoo.com/Z1wmxD/DREIAA/yQLSAA/7brrlB/TM
        > > > >
        > > >
        > --------------------------------------------------------------------~->
        > > > >
        > > > >
        > > > >
        > > > > Yahoo! Groups Links
        > > > >
        > > > >
        > > > http://groups.yahoo.com/group/artificialintelligencegroup/
        > > > >
        > > > >
        > > > >
        > > > artificialintelligencegroup-unsubscribe@yahoogroups.com
        > > > >
        > > > >
        > > > >
        > > > >
        > > >
        > > >
        > > >
        > > >
        > > > __________________________________
        > > > Do you Yahoo!?
        > > > New and Improved Yahoo! Mail - Send 10MB messages!
        > > > http://promotions.yahoo.com/new_mail
      • Jim Bromer
        I prefer not the term vaporware when referring to my programming. I have no professional background to speak of, although I am doing some data base stuff and
        Message 3 of 13 , Aug 3, 2004
        • 0 Attachment
          I prefer not the term vaporware when referring to my programming.

          I have no professional background to speak of, although I am doing
          some data base stuff and even a little programming for some people.

          I have been studying programming since 1982. I have been studying
          subjects related to this AI project since 1991.

          Right now I am trying to debug the file database management system
          that I have written for my program before I start working on the
          actual AI. However, I have so many different ideas about writing AI
          algorithms that I am reasonably confident that I will accomplish
          something.

          I believe that you have to explore both positive and critical
          aspects of a theory. But it is also important to explore
          alternative theories as well. These alternative theories may
          overlap rather than compete directly. But the situation is more
          complicated than that. Any reference may refer to a complicated
          referent. A complicated referent may contain or otherwise be
          relevant to systems of other referents. The references used in
          representations of ideas may refer to things like material objects,
          relations, abstractions, imaginary objects or any other object or
          relation from thought and nature.

          Symbolic references may contain logical expressions or other kinds
          of relations. However, the referents of the symbolic references may
          themselves contain or refer to relational properties that could make
          the symbolic expression invalid, untrue or even nonsensical. In
          other words, thoughts are not always true, accurate, valid or even
          meaningful. The problem here is that if further reasoning is based
          on nonsense or other "noise", the products of that reasoning may be
          incomprehensibly meaningless. There has to be a way to separate the
          noise from the meaningful and reasonable.

          I believe then that a system of education or other interactive
          learning is necessary in AI. But, the program has to be able to
          solve some of the problems that it encounters itself.

          If we knew all the rules of learning or if we knew the fundamental
          rules that would enable the program to derive the rest, we could
          write them out as some kind of formal rule system. But we obviously
          don't know them. We have to learn what they are. The program has
          to be able to learn new rules, but it also has to integrate them
          within a complicated system of knowledge. I suspect that this
          integration of new rules of learning and new pieces of knowledge is
          where the real complications lurk.

          I can describe what I call flexible rules that use different
          responses for different contexts. However, I do not know how the
          system could create new variations of flexible rules, validate their
          use and integrate them without some kind of feedback. This feedback
          may be determined through passive observation, interaction with an
          instructor or by interacting with the objects of an environment.

          My idea is that an AI program will develop theories or theory-like
          things that explain observed events. These theories will need to
          conform to various observations derived from interactions with the
          input-output environment.

          Right now I suspect that the program has to use a trial and error
          approach to try different ways to connect ideas to see if they can
          combine to define possible and reasonable relations between the
          referent ideas of an expression (or between the referent objects of
          related observations). An instructor will not be able to define the
          thousands of little connections between ideas that would be required
          to fully represent a complex idea, but the computer could use
          overlapping examples to get a sense of the core ideas significant to
          a particular subject. However, this reasonable solution to the
          problem is made more difficult (and perhaps more unreasonable)
          because a symbolic expression like natural language uses reference
          terms to designate the subject matter. Since there is more than one
          way to refer to a subject, this means that the program has to be
          able to figure out what subject is being referred to in an
          expression. Therefore, a simpler means to designate the subject of
          a reference term has to be used in primal learning or in an
          elementary check to try to make sure that the computer understands
          the primary subjects of an expression. The problem is also relevant
          in non-symbolic environments, because a recognized object acts like
          a symbol or index to related objects. A leaf may be taken as a
          symbol of the tree for example. In non-linguistic interactions,
          like a robot's interactions with the environment, this primary
          subject reference might involve simple interactions with objects to
          examine them to make sure that they produce the effects that the
          robot AI had assumed to be relevant in its theories about the
          environment. In passive observations, like celestial observations,
          other means would have to be used to try corroborate theories and
          observations. In an interactive IO environment like language the
          program and the instructor would have to establish a way to
          communicate fundamental relations between references. I have
          expressed this last sentence carefully in order to cover different
          manifestations of this process. In traditional linguistic learning,
          this primary relational association might be between word and object
          where the object is observed through visual input. For instance the
          word "water" might be associated with the visual observation of a
          glass of water. In a text only environment, the primary relational
          associations might be constructed with words that refer to a
          fundamental or important relation of a common subject, like "drink"
          and "water". Although the traditional sense of human learning is
          presumably based on the sound of the word and the image or touch of
          the object, on a more profound examination you can see that it truly
          involves the establishment of a relation between two referent
          objects. But these two references are actually comprised of
          complicated particles of input data. Therefore, I concluded that
          the AI has to be able to figure some of the problems out for itself
          before it could recognize that two complicated referent objects may
          be relevantly related, and it needs some way to test fundamental and
          significant associations between referent objects.

          As more information became available to the AI, it would be able to
          test different expressions that can be related to some common
          subjects. If this integration process produces conflicts they can
          be examined to try to find the reasons for the conflicts.

          So my yet-to-be tested theory of AI currently incorporates three
          basic methods to attempt to corroborate the AI's theories about the
          relations between objects of reference. The first is established
          through a primal or elementary communication with an instructor or
          with the objects of an environment to try to detect regularly
          occurring patterns or representations of objects their features and
          their relations. These co-occurring regular patterns may be used as
          primal references. The second step in establishing the validity of
          the AI's theories about the world will consist of establishing
          alternative theories about the subjects of interest in an attempt to
          find groups of theories that best conform to observed input. The
          third step then will involve the establishment and testing of
          integrated expressions or actions which will allow the system to
          again try to confirm its theories by utilizing them in focused
          patterns of examination to test the mutual consistency of its
          theories within the context of the subject matter. This is not an
          easy step since inconsistency outside of a strict logical system is
          not easily reduced to point to a particular cause for the conflict.

          In all three stages the computer has to be able to solve some of the
          problems itself. When learning something new, the program could
          rely on simple generalization techniques. At the more advanced
          level of primary learning it can use previously established
          communication or interaction techniques. At the highest level it
          would use previously learned theoretical systems to explore the
          complex interrelations of an integrated network of references.

          Jim Bromer

          --- In artificialintelligencegroup@yahoogroups.com, "toddpierce1968"
          <todd_pierce@h...> wrote:
          > Jim,
          >
          > I knew you were going to come through with a message describing the
          > pursuit of something just as insane as what I am pursuing.
          >
          > The description of what you're doing is very insightful. Based on
          the
          > limited description of what I'm doing myself, you managed to peg my
          > project right on the tail of the donkey.
          >
          > In fact, it's quite ironic that I, somewhat trained in linguistics,
          > would be pursuing a more reductive computational model whereas you,
          > apparently a business programmer from what I can tell, would be
          using
          > a biological/developmental model. By the way, I would like to
          hear a
          > bit about your background training so I know what vocabulary to
          use as
          > we discuss these things.
          >
          > What you are doing is definitely much more interesting. I can
          imagine
          > it's hard to fit values normally encoded in a child's brain into a
          > standard database format. Even with my model which is based on a
          > minimal parametric approach of the simplest of feature matrices,
          > designing the database was a nightmare.
          >
          > It was difficult to prioritize what features about a word or a
          > 'concept' computationally provided the greatest contribution to
          coming
          > up with the right result. I've had to revise my database a couple
          of
          > times in the past year, and that's after six years of planning...
          and
          > that's just for the lexicon!
          >
          > As I imagine you already know, since all of these 'features' need
          to
          > be specifically set by the trainer, I have to write interactive
          > subroutines to support every database table. Everything has to be
          > explicity explained to the computer.
          >
          > I do think that we're both discovering interesting abstractions
          about
          > human language as we sort through the various different factors
          that
          > ultimately make human language the best one invented yet. We may
          even
          > be discovering things that nobody else has discovered before.
          >
          > I do have some training in developmental linguistics so maybe I
          could
          > be of some contribution. Regardless of my theoretic experience,
          your
          > practical implementation of language acquisition sounds
          fascinating.
          > I may even be able to use some of the information in my own
          > implementation.
          >
          > I'd be very interested not only in the current incarnation of your
          > database but also in what is vaporware. Is this all in your head
          > still or do you have it documented?
          >
          > You see, this is one reason I figured conducting some of this
          > conversation outside of the forum. We're both dealing with systems
          > and concepts that are so outrageously huge that just the two of us
          > could take over the floor.
          >
          > I'm fine with it for the time being, but remember, I do keep
          backups
          > of my work as well as the documentation on a website, and I don't
          plan
          > to post the address in a forum. This is not because it's secret,
          it's
          > because I don't need to invite armchair critics from all over the
          > world :)
          >
          > That point aside, what interests me most is the language
          acquisition
          > rules you have come up with so far and the resulting database
          > architecture, either implemented or planned. Naturally, if that's
          too
          > big a question to answer here you know you have been invited to
          mail
          > me at todd_pierce at hotmail dot com.
          >
          > -Todd
          >
          > --- In artificialintelligencegroup@yahoogroups.com, "Jim Bromer"
          > <jbromer@i...> wrote:
          > > All of your comments seem pretty familiar to me. I am still
          working
          > > on the database management part of my program and I am starting
          with a
          > > lexicon of particles of input text. I have decided not to start
          with
          > > a list of words because I want to figure out how the program can
          learn
          > > to identify meaningful words and then to determine their
          utilization
          > > meanings. I believe that this is the fundamental contemporary
          problem
          > > of artificial intelligence and that the effort to jump start
          learning
          > > with an established data base of words and relationships is only
          > > pushing the problem back to the next stage of development. What
          I
          > > mean is that the trial and error algorithms and the interactions
          that
          > > may help the program to evaluate them is the significant problem
          in
          > > contemporary AI, and the thing is that this problem exists at all
          > > levels of intelligence. Let me put it this way: if an infant
          must use
          > > some special means to initially learn language and to figure
          basic
          > > things about the environment out, then he will always possess
          those
          > > special skills. We have a bias against initial learning because
          while
          >
          >
          > > we cannot understand it, we ourselves have progressed to more
          advanced
          > > issues ourselves. However, I believe that this naïve learning
          may be
          > > comprised of a system of associating data objects both
          immediately
          > > perceived and stored in memory in such a way so that they play
          certain
          > > roles in understanding. My interest then is to find the
          simplest set
          > > of roles that would be necessary for intelligence to emerge
          through
          > > this kind of process. However, this is a logistical nightmare
          for me
          > > too. If this set of what I have sometimes called ideological
          roles
          > > becomes too extensive, then it would be too unwieldy to use.
          > >
          > > I would like to continue using the artificialintelligencegroup to
          > > discuss these ideas so that others could join in if they want to.
          > >
          > > Jim Bromer
          > >
          > >
          > > --- In
          artificialintelligencegroup@yahoogroups.com, "toddpierce1968"
          > > <todd_pierce@h...> wrote:
          > > > Sam, Jim,
          > > >
          > > > Well, the project is already started (and not a secret in any
          way).
          > > > The one part that is pretty much complete is handling the
          lexicon.
          > > >
          > > > I decided to implement a custom database of sorts and use a
          > > > syntactically tagged corpus I bought from Celex. The program
          can now
          > > > store vocabulary, import vocaulary and there is a subroutine to
          > > > interactively refine syntactic features that are not set yet.
          > > >
          > > > I'm planning to implement feature matrices in as many places as
          > > > possible in the system; a brute force method that few people
          have
          > > > really taken full advantage of.
          > > >
          > > > So basically where I'm at is I'm sitting on a database system
          that,
          > > > given a word in text, like 'walked', will return a hell of a
          lot of
          > > > information about 'walked', and in addition, about the root
          word
          > > > 'walk' as well. Root forms and derived forms are stored in
          two
          > > > different tables.
          > > >
          > > > This whole project is logistically insane. That's why I just
          do it
          > > > bit by bit in my spare time. But I have managed to mostly
          take care
          > > > of the lexicon that way and as of a few weeks ago, it's
          structure and
          > > > support functions (LISP) are done.
          > > >
          > > > So now it's on to a parser and knowlege representation. I had
          planned
          > > > to proceed using a model I learned in college, but over the
          years,
          > > > I've made a lot of improvements on it just drunken
          brainstorming.
          > > >
          > > > Which is why I decided to post a message. If I can improve on
          his
          > > > model just brainstorming by myself, maybe I could make a lot
          more
          > > > progress if I had someone to discuss it with. Maybe I could
          come up
          > > > with a different model alltogether. If nothing else, I know
          I'd be
          > > > less likely to forget something I'd have to go back and fix
          later.
          > > >
          > > > Since it's planned to be a binary feature intensive system,
          what sorts
          > > > of features would be computationally useful for words and the
          objects
          > > > they identify? living/non-living, physical/non-physical, etc.
          > > >
          > > > And what about turning sentences into some sort of 'internal
          > > > representation'? How abstract does it have to be? Must it be
          hard
          > > > logic or is something resembling natural language phrase
          structure
          > > > good enough? Do some things simply need to be stored
          literally as
          > > > they text typed in?
          > > >
          > > > And speaking of this parsing process, must it be a computer
          > > > programmer's dream of lexicon->parser->knowlege or must it be
          blurred,
          > > > where each step must depend on every level of representation?
          Can the
          > > > process of generating a sentence use the same machinery but
          backwards?
          > > > i.e knowlege->parser->lexicon?
          > > >
          > > > Is there any way neural networks would be uniquely qualified
          for any
          > > > of this?
          > > >
          > > > And what cues can we take from nature? For example, we do
          know that
          > > > humans do have a 'working memory' with a dedicated buffer just
          for
          > > > language. We also know the concept of time is key when
          dealing with
          > > > language and knowlege. And all higher animals have a concept
          of
          > > > Euclidian geometry. Most language happens in discourse, which
          usually
          > > > provides many cues to parsing and meaning, how can I leverage
          off of
          > > > that? Furthermore, is it important to forget things?
          > > >
          > > > So, what cognititive machinery could help in this effort?
          > > >
          > > > If those are the sorts of things you don't mind mulling about
          in the
          > > > back of your head for a few days a week, feel free to e-mail
          me at
          > > > todd_pierce@h...
          > > >
          > > > I do have a document that goes back years which is dedicated
          to this
          > > > sort of brainstorming and I don't mind recording even the
          craziest of
          > > > ideas. After all, the document did result in a rather nice
          lexicon.
          > > >
          > > > -Todd
          > > >
          > > >
          > > > --- In artificialintelligencegroup@yahoogroups.com, Samuel Buhr
          > > > <sam_lonester@y...> wrote:
          > > > > Wow! NLP! How do you propose to start on this
          > > > > project? Tell me more, without giving up valuable
          > > > > information, of course.
          > > > >
          > > > > Sam
          > > > >
          > > > >
          > > > > --- toddpierce1968 <todd_pierce@h...> wrote:
          > > > >
          > > > > > Once again, I'm posting a message looking for
          > > > > > anybody who is
          > > > > > interested in NLP or Linguistics.
          > > > > >
          > > > > > I'm working on my own natural language processor so
          > > > > > it would be nice
          > > > > > to take up dialogue with anybody who has done this
          > > > > > or is interested in it.
          > > > > >
          > > > > > Furthermore, maybe I would make fewer mistakes
          > > > > > during development :)
          > > > > >
          > > > > > -T
          > > > > >
          > > > > >
          > > > > >
          > > > > > ------------------------ Yahoo! Groups Sponsor
          > > > > > --------------------~-->
          > > > > > Yahoo! Domains - Claim yours for only $14.70
          > > > > >
          > > > > http://us.click.yahoo.com/Z1wmxD/DREIAA/yQLSAA/7brrlB/TM
          > > > > >
          > > > >
          > > -----------------------------------------------------------------
          ---~->
          > > > > >
          > > > > >
          > > > > >
          > > > > > Yahoo! Groups Links
          > > > > >
          > > > > >
          > > > > http://groups.yahoo.com/group/artificialintelligencegroup/
          > > > > >
          > > > > >
          > > > > >
          > > > > artificialintelligencegroup-unsubscribe@yahoogroups.com
          > > > > >
          > > > > >
          > > > > >
          > > > > >
          > > > >
          > > > >
          > > > >
          > > > >
          > > > > __________________________________
          > > > > Do you Yahoo!?
          > > > > New and Improved Yahoo! Mail - Send 10MB messages!
          > > > > http://promotions.yahoo.com/new_mail
        • toddpierce1968
          Jim, ... Sorry, I just wanted you to know that your ideas are just as important to me as your code. ... Wow... that s a pretty small amount. ... Nothing wrong
          Message 4 of 13 , Aug 3, 2004
          • 0 Attachment
            Jim,

            >>I prefer not the term vaporware when referring to my programming.

            Sorry, I just wanted you to know that your ideas are just as important
            to me as your code.

            >>I have no professional background to speak of,

            Wow... that's a pretty small amount.

            >> although I am doing some data base stuff and even a
            >> little programming for some people.

            Nothing wrong with using your brain to help people with their machines.

            >>However, I have so many different ideas about writing AI
            >>algorithms that I am reasonably confident that I will accomplish
            >>something.

            The place to start is to have an idea. The second thing to think
            about is a database. So, in a way, you're on the right track.
            Algorithms are tough. But remember, there is stuff out there on the
            Web that is free for the taking.

            >>But the situation is more
            >>complicated than that. Any reference may refer
            >>to a complicated referent. A complicated referent may
            >>contain or otherwise be relevant to systems of other referents.
            >>The references used in representations of ideas may refer to
            >>things like material objects, relations, abstractions, imaginary
            >>objects or any other object or relation from thought and nature.

            Encoding an electro-chemically driven network of neurons does not fit
            well into a database. Nor are the algorithms clean. Though I do
            think you're killing yourself trying to fit a square peg into a round
            hole, one interesting thing that you mention is that not everything
            presented to a machine intelligence necessarily needs to be true.

            In fact, I specifically plan to have everything presented to the
            machine and placed in its database to be marked as an 'assertion'
            about reality. If the machine wants to confirm it, then it can ask.
            Otherwise, that information will not be considered 'true'. Other AI
            systems have used this mechanism.

            >>Symbolic references may contain logical expressions or other kinds
            >>of relations. However, the referents of the symbolic references may
            >>themselves contain or refer to relational properties that could make
            >>the symbolic expression invalid, untrue or even nonsensical. In
            >>other words, thoughts are not always true, accurate, valid or even
            >>meaningful. The problem here is that if further reasoning is based
            >>on nonsense or other "noise", the products of that reasoning may be
            >>incomprehensibly meaningless. There has to be a way to separate the
            >>noise from the meaningful and reasonable.

            Or does there? Our president, at least, doesn't have that
            functionality installed. He's running the country based on the Bible
            as a user manual, so obviously it is possible to survive without
            making decisions that have to do with reality.

            Noticing this, however, smart people came up with a way to check an
            assertion to see if new information is consistent, makes sense and
            even provides anything that is interesting:
            http://www.cogsci.ed.ac.uk/~jbos/comsem/download/nasslli.pdf

            I, for one, don't have any intention of using strict logic either...
            after all, I don't have any good logical human role models to follow.
            But I do recognize that machine intelligence is not human
            intelligence and maybe it's just fine being what it is.

            >>If we knew all the rules of learning or if we knew the fundamental
            >>rules that would enable the program to derive the rest, we could
            >>write them out as some kind of formal rule system. But we obviously
            >>don't know them.

            Electro-chemical systems don't follow rules in the sense you're
            talking about. If we implement what we do know about these systems,
            we come up with something just as pathetic as humans.

            >>We have to learn what they are.

            I guess you get an 'A' for conviction :)

            >>My idea is that an AI program will develop theories or theory-like
            >>things that explain observed events. These theories will need to
            >>conform to various observations derived from interactions with the
            >>input-output environment.

            As opposed to systems without input and output? Hehe... anyways,
            you're correct... all biological lifeforms are indeed data-driven.

            I handled half of the message. You obviously have come to almost all
            of the same conclusions that computer scientists, linguists,
            biologists and cognitive scientists came to fifty years ago.

            You are using biological metaphors heavily.

            Many of these 'elusive' rules you speak of are not elusive at all. We
            just don't have computers that are fast enough to take advantage of
            all of them. My system is very simple and it still takes an hour and
            a half for the machine to add one word to its vocabulary.

            But don't let it stop you from thinking. Personally I imagine I'll be
            able to get a new faster machine eventually. And you, sir, had better
            research neural networks if you plan to stick with your biological
            metaphor.

            If you need any info on anything, just ask.

            -Todd

            --- In artificialintelligencegroup@yahoogroups.com, "Jim Bromer"
            <jbromer@i...> wrote:
            > I prefer not the term vaporware when referring to my programming.
            >
            > I have no professional background to speak of, although I am doing
            > some data base stuff and even a little programming for some people.
            >
            > I have been studying programming since 1982. I have been studying
            > subjects related to this AI project since 1991.
            >
            > Right now I am trying to debug the file database management system
            > that I have written for my program before I start working on the
            > actual AI. However, I have so many different ideas about writing AI
            > algorithms that I am reasonably confident that I will accomplish
            > something.
            >
            > I believe that you have to explore both positive and critical
            > aspects of a theory. But it is also important to explore
            > alternative theories as well. These alternative theories may
            > overlap rather than compete directly. But the situation is more
            > complicated than that. Any reference may refer to a complicated
            > referent. A complicated referent may contain or otherwise be
            > relevant to systems of other referents. The references used in
            > representations of ideas may refer to things like material objects,
            > relations, abstractions, imaginary objects or any other object or
            > relation from thought and nature.
            >
            > Symbolic references may contain logical expressions or other kinds
            > of relations. However, the referents of the symbolic references may
            > themselves contain or refer to relational properties that could make
            > the symbolic expression invalid, untrue or even nonsensical. In
            > other words, thoughts are not always true, accurate, valid or even
            > meaningful. The problem here is that if further reasoning is based
            > on nonsense or other "noise", the products of that reasoning may be
            > incomprehensibly meaningless. There has to be a way to separate the
            > noise from the meaningful and reasonable.
            >
            > I believe then that a system of education or other interactive
            > learning is necessary in AI. But, the program has to be able to
            > solve some of the problems that it encounters itself.
            >
            > If we knew all the rules of learning or if we knew the fundamental
            > rules that would enable the program to derive the rest, we could
            > write them out as some kind of formal rule system. But we obviously
            > don't know them. We have to learn what they are. The program has
            > to be able to learn new rules, but it also has to integrate them
            > within a complicated system of knowledge. I suspect that this
            > integration of new rules of learning and new pieces of knowledge is
            > where the real complications lurk.
            >
            > I can describe what I call flexible rules that use different
            > responses for different contexts. However, I do not know how the
            > system could create new variations of flexible rules, validate their
            > use and integrate them without some kind of feedback. This feedback
            > may be determined through passive observation, interaction with an
            > instructor or by interacting with the objects of an environment.
            >
            > My idea is that an AI program will develop theories or theory-like
            > things that explain observed events. These theories will need to
            > conform to various observations derived from interactions with the
            > input-output environment.
            >
            > Right now I suspect that the program has to use a trial and error
            > approach to try different ways to connect ideas to see if they can
            > combine to define possible and reasonable relations between the
            > referent ideas of an expression (or between the referent objects of
            > related observations). An instructor will not be able to define the
            > thousands of little connections between ideas that would be required
            > to fully represent a complex idea, but the computer could use
            > overlapping examples to get a sense of the core ideas significant to
            > a particular subject. However, this reasonable solution to the
            > problem is made more difficult (and perhaps more unreasonable)
            > because a symbolic expression like natural language uses reference
            > terms to designate the subject matter. Since there is more than one
            > way to refer to a subject, this means that the program has to be
            > able to figure out what subject is being referred to in an
            > expression. Therefore, a simpler means to designate the subject of
            > a reference term has to be used in primal learning or in an
            > elementary check to try to make sure that the computer understands
            > the primary subjects of an expression. The problem is also relevant
            > in non-symbolic environments, because a recognized object acts like
            > a symbol or index to related objects. A leaf may be taken as a
            > symbol of the tree for example. In non-linguistic interactions,
            > like a robot's interactions with the environment, this primary
            > subject reference might involve simple interactions with objects to
            > examine them to make sure that they produce the effects that the
            > robot AI had assumed to be relevant in its theories about the
            > environment. In passive observations, like celestial observations,
            > other means would have to be used to try corroborate theories and
            > observations. In an interactive IO environment like language the
            > program and the instructor would have to establish a way to
            > communicate fundamental relations between references. I have
            > expressed this last sentence carefully in order to cover different
            > manifestations of this process. In traditional linguistic learning,
            > this primary relational association might be between word and object
            > where the object is observed through visual input. For instance the
            > word "water" might be associated with the visual observation of a
            > glass of water. In a text only environment, the primary relational
            > associations might be constructed with words that refer to a
            > fundamental or important relation of a common subject, like "drink"
            > and "water". Although the traditional sense of human learning is
            > presumably based on the sound of the word and the image or touch of
            > the object, on a more profound examination you can see that it truly
            > involves the establishment of a relation between two referent
            > objects. But these two references are actually comprised of
            > complicated particles of input data. Therefore, I concluded that
            > the AI has to be able to figure some of the problems out for itself
            > before it could recognize that two complicated referent objects may
            > be relevantly related, and it needs some way to test fundamental and
            > significant associations between referent objects.
            >
            > As more information became available to the AI, it would be able to
            > test different expressions that can be related to some common
            > subjects. If this integration process produces conflicts they can
            > be examined to try to find the reasons for the conflicts.
            >
            > So my yet-to-be tested theory of AI currently incorporates three
            > basic methods to attempt to corroborate the AI's theories about the
            > relations between objects of reference. The first is established
            > through a primal or elementary communication with an instructor or
            > with the objects of an environment to try to detect regularly
            > occurring patterns or representations of objects their features and
            > their relations. These co-occurring regular patterns may be used as
            > primal references. The second step in establishing the validity of
            > the AI's theories about the world will consist of establishing
            > alternative theories about the subjects of interest in an attempt to
            > find groups of theories that best conform to observed input. The
            > third step then will involve the establishment and testing of
            > integrated expressions or actions which will allow the system to
            > again try to confirm its theories by utilizing them in focused
            > patterns of examination to test the mutual consistency of its
            > theories within the context of the subject matter. This is not an
            > easy step since inconsistency outside of a strict logical system is
            > not easily reduced to point to a particular cause for the conflict.
            >
            > In all three stages the computer has to be able to solve some of the
            > problems itself. When learning something new, the program could
            > rely on simple generalization techniques. At the more advanced
            > level of primary learning it can use previously established
            > communication or interaction techniques. At the highest level it
            > would use previously learned theoretical systems to explore the
            > complex interrelations of an integrated network of references.
            >
            > Jim Bromer
            >
            > --- In artificialintelligencegroup@yahoogroups.com, "toddpierce1968"
            > <todd_pierce@h...> wrote:
            > > Jim,
            > >
            > > I knew you were going to come through with a message describing the
            > > pursuit of something just as insane as what I am pursuing.
            > >
            > > The description of what you're doing is very insightful. Based on
            > the
            > > limited description of what I'm doing myself, you managed to peg my
            > > project right on the tail of the donkey.
            > >
            > > In fact, it's quite ironic that I, somewhat trained in linguistics,
            > > would be pursuing a more reductive computational model whereas you,
            > > apparently a business programmer from what I can tell, would be
            > using
            > > a biological/developmental model. By the way, I would like to
            > hear a
            > > bit about your background training so I know what vocabulary to
            > use as
            > > we discuss these things.
            > >
            > > What you are doing is definitely much more interesting. I can
            > imagine
            > > it's hard to fit values normally encoded in a child's brain into a
            > > standard database format. Even with my model which is based on a
            > > minimal parametric approach of the simplest of feature matrices,
            > > designing the database was a nightmare.
            > >
            > > It was difficult to prioritize what features about a word or a
            > > 'concept' computationally provided the greatest contribution to
            > coming
            > > up with the right result. I've had to revise my database a couple
            > of
            > > times in the past year, and that's after six years of planning...
            > and
            > > that's just for the lexicon!
            > >
            > > As I imagine you already know, since all of these 'features' need
            > to
            > > be specifically set by the trainer, I have to write interactive
            > > subroutines to support every database table. Everything has to be
            > > explicity explained to the computer.
            > >
            > > I do think that we're both discovering interesting abstractions
            > about
            > > human language as we sort through the various different factors
            > that
            > > ultimately make human language the best one invented yet. We may
            > even
            > > be discovering things that nobody else has discovered before.
            > >
            > > I do have some training in developmental linguistics so maybe I
            > could
            > > be of some contribution. Regardless of my theoretic experience,
            > your
            > > practical implementation of language acquisition sounds
            > fascinating.
            > > I may even be able to use some of the information in my own
            > > implementation.
            > >
            > > I'd be very interested not only in the current incarnation of your
            > > database but also in what is vaporware. Is this all in your head
            > > still or do you have it documented?
            > >
            > > You see, this is one reason I figured conducting some of this
            > > conversation outside of the forum. We're both dealing with systems
            > > and concepts that are so outrageously huge that just the two of us
            > > could take over the floor.
            > >
            > > I'm fine with it for the time being, but remember, I do keep
            > backups
            > > of my work as well as the documentation on a website, and I don't
            > plan
            > > to post the address in a forum. This is not because it's secret,
            > it's
            > > because I don't need to invite armchair critics from all over the
            > > world :)
            > >
            > > That point aside, what interests me most is the language
            > acquisition
            > > rules you have come up with so far and the resulting database
            > > architecture, either implemented or planned. Naturally, if that's
            > too
            > > big a question to answer here you know you have been invited to
            > mail
            > > me at todd_pierce at hotmail dot com.
            > >
            > > -Todd
            > >
            > > --- In artificialintelligencegroup@yahoogroups.com, "Jim Bromer"
            > > <jbromer@i...> wrote:
            > > > All of your comments seem pretty familiar to me. I am still
            > working
            > > > on the database management part of my program and I am starting
            > with a
            > > > lexicon of particles of input text. I have decided not to start
            > with
            > > > a list of words because I want to figure out how the program can
            > learn
            > > > to identify meaningful words and then to determine their
            > utilization
            > > > meanings. I believe that this is the fundamental contemporary
            > problem
            > > > of artificial intelligence and that the effort to jump start
            > learning
            > > > with an established data base of words and relationships is only
            > > > pushing the problem back to the next stage of development. What
            > I
            > > > mean is that the trial and error algorithms and the interactions
            > that
            > > > may help the program to evaluate them is the significant problem
            > in
            > > > contemporary AI, and the thing is that this problem exists at all
            > > > levels of intelligence. Let me put it this way: if an infant
            > must use
            > > > some special means to initially learn language and to figure
            > basic
            > > > things about the environment out, then he will always possess
            > those
            > > > special skills. We have a bias against initial learning because
            > while
            > >
            > >
            > > > we cannot understand it, we ourselves have progressed to more
            > advanced
            > > > issues ourselves. However, I believe that this naïve learning
            > may be
            > > > comprised of a system of associating data objects both
            > immediately
            > > > perceived and stored in memory in such a way so that they play
            > certain
            > > > roles in understanding. My interest then is to find the
            > simplest set
            > > > of roles that would be necessary for intelligence to emerge
            > through
            > > > this kind of process. However, this is a logistical nightmare
            > for me
            > > > too. If this set of what I have sometimes called ideological
            > roles
            > > > becomes too extensive, then it would be too unwieldy to use.
            > > >
            > > > I would like to continue using the artificialintelligencegroup to
            > > > discuss these ideas so that others could join in if they want to.
            > > >
            > > > Jim Bromer
            > > >
            > > >
            > > > --- In
            > artificialintelligencegroup@yahoogroups.com, "toddpierce1968"
            > > > <todd_pierce@h...> wrote:
            > > > > Sam, Jim,
            > > > >
            > > > > Well, the project is already started (and not a secret in any
            > way).
            > > > > The one part that is pretty much complete is handling the
            > lexicon.
            > > > >
            > > > > I decided to implement a custom database of sorts and use a
            > > > > syntactically tagged corpus I bought from Celex. The program
            > can now
            > > > > store vocabulary, import vocaulary and there is a subroutine to
            > > > > interactively refine syntactic features that are not set yet.
            > > > >
            > > > > I'm planning to implement feature matrices in as many places as
            > > > > possible in the system; a brute force method that few people
            > have
            > > > > really taken full advantage of.
            > > > >
            > > > > So basically where I'm at is I'm sitting on a database system
            > that,
            > > > > given a word in text, like 'walked', will return a hell of a
            > lot of
            > > > > information about 'walked', and in addition, about the root
            > word
            > > > > 'walk' as well. Root forms and derived forms are stored in
            > two
            > > > > different tables.
            > > > >
            > > > > This whole project is logistically insane. That's why I just
            > do it
            > > > > bit by bit in my spare time. But I have managed to mostly
            > take care
            > > > > of the lexicon that way and as of a few weeks ago, it's
            > structure and
            > > > > support functions (LISP) are done.
            > > > >
            > > > > So now it's on to a parser and knowlege representation. I had
            > planned
            > > > > to proceed using a model I learned in college, but over the
            > years,
            > > > > I've made a lot of improvements on it just drunken
            > brainstorming.
            > > > >
            > > > > Which is why I decided to post a message. If I can improve on
            > his
            > > > > model just brainstorming by myself, maybe I could make a lot
            > more
            > > > > progress if I had someone to discuss it with. Maybe I could
            > come up
            > > > > with a different model alltogether. If nothing else, I know
            > I'd be
            > > > > less likely to forget something I'd have to go back and fix
            > later.
            > > > >
            > > > > Since it's planned to be a binary feature intensive system,
            > what sorts
            > > > > of features would be computationally useful for words and the
            > objects
            > > > > they identify? living/non-living, physical/non-physical, etc.
            > > > >
            > > > > And what about turning sentences into some sort of 'internal
            > > > > representation'? How abstract does it have to be? Must it be
            > hard
            > > > > logic or is something resembling natural language phrase
            > structure
            > > > > good enough? Do some things simply need to be stored
            > literally as
            > > > > they text typed in?
            > > > >
            > > > > And speaking of this parsing process, must it be a computer
            > > > > programmer's dream of lexicon->parser->knowlege or must it be
            > blurred,
            > > > > where each step must depend on every level of representation?
            > Can the
            > > > > process of generating a sentence use the same machinery but
            > backwards?
            > > > > i.e knowlege->parser->lexicon?
            > > > >
            > > > > Is there any way neural networks would be uniquely qualified
            > for any
            > > > > of this?
            > > > >
            > > > > And what cues can we take from nature? For example, we do
            > know that
            > > > > humans do have a 'working memory' with a dedicated buffer just
            > for
            > > > > language. We also know the concept of time is key when
            > dealing with
            > > > > language and knowlege. And all higher animals have a concept
            > of
            > > > > Euclidian geometry. Most language happens in discourse, which
            > usually
            > > > > provides many cues to parsing and meaning, how can I leverage
            > off of
            > > > > that? Furthermore, is it important to forget things?
            > > > >
            > > > > So, what cognititive machinery could help in this effort?
            > > > >
            > > > > If those are the sorts of things you don't mind mulling about
            > in the
            > > > > back of your head for a few days a week, feel free to e-mail
            > me at
            > > > > todd_pierce@h...
            > > > >
            > > > > I do have a document that goes back years which is dedicated
            > to this
            > > > > sort of brainstorming and I don't mind recording even the
            > craziest of
            > > > > ideas. After all, the document did result in a rather nice
            > lexicon.
            > > > >
            > > > > -Todd
            > > > >
            > > > >
            > > > > --- In artificialintelligencegroup@yahoogroups.com, Samuel Buhr
            > > > > <sam_lonester@y...> wrote:
            > > > > > Wow! NLP! How do you propose to start on this
            > > > > > project? Tell me more, without giving up valuable
            > > > > > information, of course.
            > > > > >
            > > > > > Sam
            > > > > >
            > > > > >
            > > > > > --- toddpierce1968 <todd_pierce@h...> wrote:
            > > > > >
            > > > > > > Once again, I'm posting a message looking for
            > > > > > > anybody who is
            > > > > > > interested in NLP or Linguistics.
            > > > > > >
            > > > > > > I'm working on my own natural language processor so
            > > > > > > it would be nice
            > > > > > > to take up dialogue with anybody who has done this
            > > > > > > or is interested in it.
            > > > > > >
            > > > > > > Furthermore, maybe I would make fewer mistakes
            > > > > > > during development :)
            > > > > > >
            > > > > > > -T
            > > > > > >
            > > > > > >
            > > > > > >
            > > > > > > ------------------------ Yahoo! Groups Sponsor
            > > > > > > --------------------~-->
            > > > > > > Yahoo! Domains - Claim yours for only $14.70
            > > > > > >
            > > > > > http://us.click.yahoo.com/Z1wmxD/DREIAA/yQLSAA/7brrlB/TM
            > > > > > >
            > > > > >
            > > > -----------------------------------------------------------------
            > ---~->
            > > > > > >
            > > > > > >
            > > > > > >
            > > > > > > Yahoo! Groups Links
            > > > > > >
            > > > > > >
            > > > > > http://groups.yahoo.com/group/artificialintelligencegroup/
            > > > > > >
            > > > > > >
            > > > > > >
            > > > > > artificialintelligencegroup-unsubscribe@yahoogroups.com
            > > > > > >
            > > > > > >
            > > > > > >
            > > > > > >
            > > > > >
            > > > > >
            > > > > >
            > > > > >
            > > > > > __________________________________
            > > > > > Do you Yahoo!?
            > > > > > New and Improved Yahoo! Mail - Send 10MB messages!
            > > > > > http://promotions.yahoo.com/new_mail
          • Samuel Buhr
            Jim, Sounds like reinforcement learning to me. Have you found any good books on the subject? Sam ... === message truncated ===
            Message 5 of 13 , Aug 3, 2004
            • 0 Attachment
              Jim,

              Sounds like reinforcement learning to me. Have you
              found any good books on the subject?

              Sam


              --- Jim Bromer <jbromer@...> wrote:

              > I prefer not the term vaporware when referring to my
              > programming.
              >
              > I have no professional background to speak of,
              > although I am doing
              > some data base stuff and even a little programming
              > for some people.
              >
              > I have been studying programming since 1982. I have
              > been studying
              > subjects related to this AI project since 1991.
              >
              > Right now I am trying to debug the file database
              > management system
              > that I have written for my program before I start
              > working on the
              > actual AI. However, I have so many different ideas
              > about writing AI
              > algorithms that I am reasonably confident that I
              > will accomplish
              > something.
              >
              > I believe that you have to explore both positive and
              > critical
              > aspects of a theory. But it is also important to
              > explore
              > alternative theories as well. These alternative
              > theories may
              > overlap rather than compete directly. But the
              > situation is more
              > complicated than that. Any reference may refer to a
              > complicated
              > referent. A complicated referent may contain or
              > otherwise be
              > relevant to systems of other referents. The
              > references used in
              > representations of ideas may refer to things like
              > material objects,
              > relations, abstractions, imaginary objects or any
              > other object or
              > relation from thought and nature.
              >
              > Symbolic references may contain logical expressions
              > or other kinds
              > of relations. However, the referents of the
              > symbolic references may
              > themselves contain or refer to relational properties
              > that could make
              > the symbolic expression invalid, untrue or even
              > nonsensical. In
              > other words, thoughts are not always true, accurate,
              > valid or even
              > meaningful. The problem here is that if further
              > reasoning is based
              > on nonsense or other "noise", the products of that
              > reasoning may be
              > incomprehensibly meaningless. There has to be a way
              > to separate the
              > noise from the meaningful and reasonable.
              >
              > I believe then that a system of education or other
              > interactive
              > learning is necessary in AI. But, the program has
              > to be able to
              > solve some of the problems that it encounters
              > itself.
              >
              > If we knew all the rules of learning or if we knew
              > the fundamental
              > rules that would enable the program to derive the
              > rest, we could
              > write them out as some kind of formal rule system.
              > But we obviously
              > don't know them. We have to learn what they are.
              > The program has
              > to be able to learn new rules, but it also has to
              > integrate them
              > within a complicated system of knowledge. I suspect
              > that this
              > integration of new rules of learning and new pieces
              > of knowledge is
              > where the real complications lurk.
              >
              > I can describe what I call flexible rules that use
              > different
              > responses for different contexts. However, I do not
              > know how the
              > system could create new variations of flexible
              > rules, validate their
              > use and integrate them without some kind of
              > feedback. This feedback
              > may be determined through passive observation,
              > interaction with an
              > instructor or by interacting with the objects of an
              > environment.
              >
              > My idea is that an AI program will develop theories
              > or theory-like
              > things that explain observed events. These theories
              > will need to
              > conform to various observations derived from
              > interactions with the
              > input-output environment.
              >
              > Right now I suspect that the program has to use a
              > trial and error
              > approach to try different ways to connect ideas to
              > see if they can
              > combine to define possible and reasonable relations
              > between the
              > referent ideas of an expression (or between the
              > referent objects of
              > related observations). An instructor will not be
              > able to define the
              > thousands of little connections between ideas that
              > would be required
              > to fully represent a complex idea, but the computer
              > could use
              > overlapping examples to get a sense of the core
              > ideas significant to
              > a particular subject. However, this reasonable
              > solution to the
              > problem is made more difficult (and perhaps more
              > unreasonable)
              > because a symbolic expression like natural language
              > uses reference
              > terms to designate the subject matter. Since there
              > is more than one
              > way to refer to a subject, this means that the
              > program has to be
              > able to figure out what subject is being referred to
              > in an
              > expression. Therefore, a simpler means to designate
              > the subject of
              > a reference term has to be used in primal learning
              > or in an
              > elementary check to try to make sure that the
              > computer understands
              > the primary subjects of an expression. The problem
              > is also relevant
              > in non-symbolic environments, because a recognized
              > object acts like
              > a symbol or index to related objects. A leaf may be
              > taken as a
              > symbol of the tree for example. In non-linguistic
              > interactions,
              > like a robot's interactions with the environment,
              > this primary
              > subject reference might involve simple interactions
              > with objects to
              > examine them to make sure that they produce the
              > effects that the
              > robot AI had assumed to be relevant in its theories
              > about the
              > environment. In passive observations, like
              > celestial observations,
              > other means would have to be used to try corroborate
              > theories and
              > observations. In an interactive IO environment like
              > language the
              > program and the instructor would have to establish a
              > way to
              > communicate fundamental relations between
              > references. I have
              > expressed this last sentence carefully in order to
              > cover different
              > manifestations of this process. In traditional
              > linguistic learning,
              > this primary relational association might be between
              > word and object
              > where the object is observed through visual input.
              > For instance the
              > word "water" might be associated with the visual
              > observation of a
              > glass of water. In a text only environment, the
              > primary relational
              > associations might be constructed with words that
              > refer to a
              > fundamental or important relation of a common
              > subject, like "drink"
              > and "water". Although the traditional sense of
              > human learning is
              > presumably based on the sound of the word and the
              > image or touch of
              > the object, on a more profound examination you can
              > see that it truly
              > involves the establishment of a relation between two
              > referent
              > objects. But these two references are actually
              > comprised of
              > complicated particles of input data. Therefore, I
              > concluded that
              > the AI has to be able to figure some of the problems
              > out for itself
              > before it could recognize that two complicated
              > referent objects may
              > be relevantly related, and it needs some way to test
              > fundamental and
              > significant associations between referent objects.
              >
              > As more information became available to the AI, it
              > would
              === message truncated ===




              __________________________________
              Do you Yahoo!?
              New and Improved Yahoo! Mail - Send 10MB messages!
              http://promotions.yahoo.com/new_mail
            • Jim Bromer
              I think that reinforcement schemes alone would be inadequate to develop intelligence because natural symbolic references are complicated. Although you could
              Message 6 of 13 , Aug 5, 2004
              • 0 Attachment
                I think that reinforcement schemes alone would be inadequate to
                develop intelligence because natural symbolic references are
                complicated. Although you could reinforce a symbolic expression or
                an association between a symbolic expression and a referent
                observation or action, a clear meaning of the reinforcement would
                emerge in only a few rather weak kinds of situations. Most symbolic
                references are context laden, meaning that meaning and relevance may
                vary wildly around even similar uses of a symbolic reference term
                and that different symbolic expressions may point toward a common
                subject matter.

                If I try to positively reinforce your expression on this group you
                might then feel emboldened to argue that your suggestion of
                reinforcement was so absolutely right that my argument could only be
                absolute nonsense. On the other hand if I am too negative in my
                response, you might become less willing to participate in these
                discussions. These two examples are not the extremes but only two
                of many possible situations that could be drawn from (or imagined
                in) this particular situation. So instead of relying on a
                simplistic reinforcement, I try to use words to both explain why I
                don't think that reinforcement schemes are too effective, while
                letting you know that I have come to different conclusions only
                because I thought the concept was important enough and potentially
                subtle enough to have spent a lot of time thinking about it.

                A reinforcement scheme by nature is a simplification of the problem
                of explanation. I want to develop a program that will learn to
                converse in a natural language meaning that it is supposed to
                eventually be able to use language to communicate. It will use a
                simple reinforcement scheme as one of its means of learning. It
                will also use simple logic, crude probability, pattern matching and
                comparison, categorization and reasoning as tools of learning. I do
                not think that one simplistic means of training or defining
                relations is sufficient to explain learning in a complicated
                environment.

                I was able to find a number of resources on reinforcement learning
                on the internet.
                Jim Bromer

                --- In artificialintelligencegroup@yahoogroups.com, Samuel Buhr
                <sam_lonester@y...> wrote:
                > Jim,
                >
                > Sounds like reinforcement learning to me. Have you
                > found any good books on the subject?
                >
                > Sam
                >
                >
                > --- Jim Bromer <jbromer@i...> wrote:
                >
                > > I prefer not the term vaporware when referring to my
                > > programming.
                > >
                > > I have no professional background to speak of,
                > > although I am doing
                > > some data base stuff and even a little programming
                > > for some people.
                > >
                > > I have been studying programming since 1982. I have
                > > been studying
                > > subjects related to this AI project since 1991.
                > >
                > > Right now I am trying to debug the file database
                > > management system
                > > that I have written for my program before I start
                > > working on the
                > > actual AI. However, I have so many different ideas
                > > about writing AI
                > > algorithms that I am reasonably confident that I
                > > will accomplish
                > > something.
                > >
                > > I believe that you have to explore both positive and
                > > critical
                > > aspects of a theory. But it is also important to
                > > explore
                > > alternative theories as well. These alternative
                > > theories may
                > > overlap rather than compete directly. But the
                > > situation is more
                > > complicated than that. Any reference may refer to a
                > > complicated
                > > referent. A complicated referent may contain or
                > > otherwise be
                > > relevant to systems of other referents. The
                > > references used in
                > > representations of ideas may refer to things like
                > > material objects,
                > > relations, abstractions, imaginary objects or any
                > > other object or
                > > relation from thought and nature.
                > >
                > > Symbolic references may contain logical expressions
                > > or other kinds
                > > of relations. However, the referents of the
                > > symbolic references may
                > > themselves contain or refer to relational properties
                > > that could make
                > > the symbolic expression invalid, untrue or even
                > > nonsensical. In
                > > other words, thoughts are not always true, accurate,
                > > valid or even
                > > meaningful. The problem here is that if further
                > > reasoning is based
                > > on nonsense or other "noise", the products of that
                > > reasoning may be
                > > incomprehensibly meaningless. There has to be a way
                > > to separate the
                > > noise from the meaningful and reasonable.
                > >
                > > I believe then that a system of education or other
                > > interactive
                > > learning is necessary in AI. But, the program has
                > > to be able to
                > > solve some of the problems that it encounters
                > > itself.
                > >
                > > If we knew all the rules of learning or if we knew
                > > the fundamental
                > > rules that would enable the program to derive the
                > > rest, we could
                > > write them out as some kind of formal rule system.
                > > But we obviously
                > > don't know them. We have to learn what they are.
                > > The program has
                > > to be able to learn new rules, but it also has to
                > > integrate them
                > > within a complicated system of knowledge. I suspect
                > > that this
                > > integration of new rules of learning and new pieces
                > > of knowledge is
                > > where the real complications lurk.
                > >
                > > I can describe what I call flexible rules that use
                > > different
                > > responses for different contexts. However, I do not
                > > know how the
                > > system could create new variations of flexible
                > > rules, validate their
                > > use and integrate them without some kind of
                > > feedback. This feedback
                > > may be determined through passive observation,
                > > interaction with an
                > > instructor or by interacting with the objects of an
                > > environment.
                > >
                > > My idea is that an AI program will develop theories
                > > or theory-like
                > > things that explain observed events. These theories
                > > will need to
                > > conform to various observations derived from
                > > interactions with the
                > > input-output environment.
                > >
                > > Right now I suspect that the program has to use a
                > > trial and error
                > > approach to try different ways to connect ideas to
                > > see if they can
                > > combine to define possible and reasonable relations
                > > between the
                > > referent ideas of an expression (or between the
                > > referent objects of
                > > related observations). An instructor will not be
                > > able to define the
                > > thousands of little connections between ideas that
                > > would be required
                > > to fully represent a complex idea, but the computer
                > > could use
                > > overlapping examples to get a sense of the core
                > > ideas significant to
                > > a particular subject. However, this reasonable
                > > solution to the
                > > problem is made more difficult (and perhaps more
                > > unreasonable)
                > > because a symbolic expression like natural language
                > > uses reference
                > > terms to designate the subject matter. Since there
                > > is more than one
                > > way to refer to a subject, this means that the
                > > program has to be
                > > able to figure out what subject is being referred to
                > > in an
                > > expression. Therefore, a simpler means to designate
                > > the subject of
                > > a reference term has to be used in primal learning
                > > or in an
                > > elementary check to try to make sure that the
                > > computer understands
                > > the primary subjects of an expression. The problem
                > > is also relevant
                > > in non-symbolic environments, because a recognized
                > > object acts like
                > > a symbol or index to related objects. A leaf may be
                > > taken as a
                > > symbol of the tree for example. In non-linguistic
                > > interactions,
                > > like a robot's interactions with the environment,
                > > this primary
                > > subject reference might involve simple interactions
                > > with objects to
                > > examine them to make sure that they produce the
                > > effects that the
                > > robot AI had assumed to be relevant in its theories
                > > about the
                > > environment. In passive observations, like
                > > celestial observations,
                > > other means would have to be used to try corroborate
                > > theories and
                > > observations. In an interactive IO environment like
                > > language the
                > > program and the instructor would have to establish a
                > > way to
                > > communicate fundamental relations between
                > > references. I have
                > > expressed this last sentence carefully in order to
                > > cover different
                > > manifestations of this process. In traditional
                > > linguistic learning,
                > > this primary relational association might be between
                > > word and object
                > > where the object is observed through visual input.
                > > For instance the
                > > word "water" might be associated with the visual
                > > observation of a
                > > glass of water. In a text only environment, the
                > > primary relational
                > > associations might be constructed with words that
                > > refer to a
                > > fundamental or important relation of a common
                > > subject, like "drink"
                > > and "water". Although the traditional sense of
                > > human learning is
                > > presumably based on the sound of the word and the
                > > image or touch of
                > > the object, on a more profound examination you can
                > > see that it truly
                > > involves the establishment of a relation between two
                > > referent
                > > objects. But these two references are actually
                > > comprised of
                > > complicated particles of input data. Therefore, I
                > > concluded that
                > > the AI has to be able to figure some of the problems
                > > out for itself
                > > before it could recognize that two complicated
                > > referent objects may
                > > be relevantly related, and it needs some way to test
                > > fundamental and
                > > significant associations between referent objects.
                > >
                > > As more information became available to the AI, it
                > > would
                > === message truncated ===
                >
                >
                >
                >
                > __________________________________
                > Do you Yahoo!?
                > New and Improved Yahoo! Mail - Send 10MB messages!
                > http://promotions.yahoo.com/new_mail
              • ssit_mcet
                Hello Todd, I am very much interested in NLP.As I have done a project on Natural Language Interface to Database where I used PERL language to build a Parser
                Message 7 of 13 , Aug 12, 2004
                • 0 Attachment
                  Hello Todd,

                  I am very much interested in NLP.As I have done a project
                  on "Natural Language Interface to Database" where I used PERL
                  language to build a Parser to process user query based on some
                  restricted set of grammer.
                  Your proposed project make me enthusiast in this again.

                  I can contribute(probably) if u make it clear to me more elaborately.

                  Regards,
                  Sahasrangshu
                • toddpierce1968
                  Sahasrangshu, It would be very nice to hear about what you did as well. I m having problems proceeding, mostly in terms of determining the scope of my project
                  Message 8 of 13 , Aug 13, 2004
                  • 0 Attachment
                    Sahasrangshu,

                    It would be very nice to hear about what you did as well. I'm having
                    problems proceeding, mostly in terms of determining the scope of my
                    project and prioritizing semantic relations... obviously the biggest
                    problems in NLP to begin with :)

                    Perhaps the best way to proceed would be to send me an e-mail. I have
                    some stuff posted on the web that explains the progress I've made so
                    far.

                    My email address is todd_pierce at hotmail.com and I can send you a
                    link at your convenience.

                    -Todd

                    --- In artificialintelligencegroup@yahoogroups.com, ssit_mcet
                    <no_reply@y...> wrote:
                    > Hello Todd,
                    >
                    > I am very much interested in NLP.As I have done a project
                    > on "Natural Language Interface to Database" where I used PERL
                    > language to build a Parser to process user query based on some
                    > restricted set of grammer.
                    > Your proposed project make me enthusiast in this again.
                    >
                    > I can contribute(probably) if u make it clear to me more elaborately.
                    >
                    > Regards,
                    > Sahasrangshu
                  • Marc Vincent Irvin
                    Todd, I am also working on a project very similar, after fifteen minutes of reviewing this series of emails, to yours. . I have just recently decided to start
                    Message 9 of 13 , Nov 4, 2004
                    • 0 Attachment
                      Todd,

                      I am also working on a project very similar, after fifteen minutes of
                      reviewing this series of emails, to yours. .

                      I have just recently decided to start sharing my visions with others.
                      To simplify my explaination of the way I intend to go about making a
                      language that posseses intellect I have given it a name, and set up a
                      Introductory GROUP site at http://groups.yahoo.com/group/Buddy_Rex/.

                      I heard you say something about an English Grammar parser. I've built
                      six so far with ever increasing levels of grammatical understanding. If
                      you go to my YAHOO group and look at my code samples you will see that I
                      have tackled the Grammar problem from many angles. Having examples of
                      my many angles in on e place might be helpful to you or others.


                      toddpierce1968 wrote:

                      >Once again, I'm posting a message looking for anybody who is
                      >interested in NLP or Linguistics.
                      >
                      >I'm working on my own natural language processor so it would be nice
                      >to take up dialogue with anybody who has done this or is interested in it.
                      >
                      >Furthermore, maybe I would make fewer mistakes during development :)
                      >
                      >-T
                      >
                      >
                      >
                      >
                      >
                      >Yahoo! Groups Links
                      >
                      >
                      >
                      >
                      >
                      >
                      >
                      >
                    Your message has been successfully submitted and would be delivered to recipients shortly.