Loading ...
Sorry, an error occurred while loading the content.

Re: [neat] Re: Multiple networks

Expand Messages
  • Ian Badcoe
    ... Hi, Although I m seriously considering Hayek as source material for techniques I will experiment with, I only thought about this so far, I didn t
    Message 1 of 16 , Apr 1 3:20 AM
      At 09:46 31/03/2004 +0000, you wrote:
      >Hi Ian, I've read the first few sections of the Baum paper in detail
      >and it's really fascinating stuff. I want to implement a system
      >like this with NEAT. Of course there are logistical questions about
      >putting NEAT inside such a system, because Hayek wants control of
      >who gets to reproduce but so does NEAT..so it would have to be
      >One thing I wonder is how the system deals with bidding
      >inconsistency. The function that is used to determine the bid value
      >is potentially quite complex, meaning that a bidder that makes
      >several rational bids can later make a totally crazy bid that makes
      >no sense. Thus, such a system can be noisy and constantly needing
      >to reorganize. How serious do you see this problem?

      Although I'm seriously considering Hayek as "source material" for
      techniques I will experiment with, I only thought about this so far, I
      didn't implement anything. And from experience I know very well that
      implementing something always reveals 2-3 nasty holes in the theory you
      thought you knew.

      So what you ask is a classical example of a hole in what I'd
      thought through. It's a very good point, and maybe I misread Eric's intent
      -- he did, after all, always use fixed bids in his experiments.

      I think I'll ask him, he hangs out there

      There is some control, they are not allowed to bid more than they
      own and maybe not more than the reward for the objective (if known).

      Beyond that, a few new thoughts occur to me (sorry for
      stream-of-consciousness format):

      1) what you have spotted is an additional cost to using function-bids over
      fixed-bids -- e.g. that functional bids will take more generations to
      evolve, precisely because it will take along while to eliminate any
      individuals with rare aberrant bidding behaviour

      2) one could maybe limit this problem by having a separately evolving
      max-bid parameter on each individual

      3) the pattern-recognition approach (which Eric's examples use to choose
      y/n on a fixed, evolved bid) has an advantage in this, because I can
      imagine it easier to evolve a pattern matching certain cases (certainly a
      subset of ones the individual's strategy could _theoretically_ be applied
      to) than to evolve a correct bid for any case

      4) you could use pattern-recognition and function-bids -- so that the
      function would not be called on to bid outside its "expertise"

      5) or you could use a network for pattern recognition (in addition to one
      for making actions and maybe one for bidding)

      6) or it may just be, that in a system with function-bids, the overall
      system of interacting bids becomes more chaotic -- this brings us back to
      the problem I listed before, where a "rogue trader" can bankrupt the whole
      system, requiring one to rewind the whole universe and try again. It's a
      real problem if the universe cannot be rewound, which it cannot, for
      example, if it contains another evolved system...

      7) but all in all I think I'm starting to like fixed bids for simple cases
      (of course, I don't like simple cases...)

      This all ties in to (what I believe are) the other weaknesses of Hayek. I
      don't mean weaknesses in any sort of critical sense, but there are areas it
      naturally addresses, and others it does not.

      a - convoluted objectives -- if an objective may not be achievable, then in
      order to know whether to bid on working towards it, an individual would
      need to know whether the individuals which followed it were going to
      succeed (or bid highly)

      b - pleasure and pain -- in a complex environment, positive and negative
      stimuli can be expected. Some of these will be the results of (chains of)
      individual actions and need to be credited to an individual (the last in
      the chain; if the system is working on multiple objectives then there may
      be a separate chain for each one; the concpet of a "chain" only exists to
      the human observer). I don't see how to determine which individual(s)
      receive positive stimuli. Negative stimuli are even worse, as there's no
      amortization of the cost along the chain -- one "mug" gets the whole blame...

      c - internal state -- as well as performing acts on the world, it can be
      usefuly for an individual to perform acts internal tot he system. An
      example might be calculating the average of some data and storing it. The
      existence of such calculated data in the systems internal store could be
      regarded as increasing the value of the world and thus providing profit to
      the individual that calculated it, but as a general principal, can we
      expect that "averaging module" to know whether some other module wants an
      average or not? I've wondered at the need for a "commissioning" system,
      where one individual can request data and offer payment for it...

      d - multiple objectives -- For simple cases, Hayek, scales well to multiple
      objectives, as each individual simply bids for how much value it thinks it
      can contribute to whichever objective it likes -- so the most
      profitable/progressable objective is automatically estimated. However, if
      the case becomes more complex, especially if there is a wide choice of
      similar objectives, then deciding which to pursue becomes a hard problem
      much as in (a)

      Anyway, that's my take, my hope is that, having identified these
      limits, eventually I'll arrive at a reformulation which addresses them

      Hope this vague verbosity helps,

      Ian Badcoe

      Living@Home - Open Source Evolving Organisms -
    Your message has been successfully submitted and would be delivered to recipients shortly.