Loading ...
Sorry, an error occurred while loading the content.

Creating a fitness function for koza-gp.lisp

Expand Messages
  • Glenn M. Lewis
    Hi all! I finally bit the bullet and started writing my own GP framework because I was having difficulties figuring out how to apply my particular problem
    Message 1 of 2 , Nov 14, 2006
    • 0 Attachment
      Hi all! I finally bit the bullet and started writing my own GP
      framework because I was having difficulties figuring out how to
      apply my particular problem (financial market trading by rule
      creation) to any of the existing GP frameworks.

      I chose to implement the GP framework in REBOL
      (http://www.rebol.com/) because its speed impressed me in the past
      and also because it has the nice property of "code is data is code",
      like Common Lisp. I would have chosen Common Lisp if I weren't such
      a dog-gonned newbie at it.

      So I finally got the random tree code generation working and the
      fitness function so that I can generate and evaluate random
      individuals for fitness. At this point, I generate a population of
      100 individuals all of which generate a positive profit (by
      regenerating each individual with a new random tree until it proves
      profitable).

      I'm at the point where I need to start writing all the crossover,
      mutation, etc. stuff that would truly make it a GP framework, and
      I'm a bit intimidated now.

      I finally found a version of John Koza's "koza-gp.lisp" on the
      internet that compiles with Lisp In A Box (Franz Allegro Lisp), and
      suddenly had the idea that maybe it is time for me to port my half-
      GP framework over to Common Lisp so that I can take advantage of all
      the cool work that John Koza did.

      So my question is... is it possible to convert the fitness function
      listed below to a Common Lisp version that would be compatible
      with "koza-gp.lisp"? My concern is that I don't think it allows for
      a fitness function to iterate over a matrix of data (which
      represents market history) like I am doing below.

      Thank you very much!
      -- Glenn Lewis


      commission: 10.00
      scale-factor: 100.00

      fitness-func: func [
      tree [number! block!] "Randomly generated function tree."
      market [block!] "Pre-processed market data."
      actual [block!] "Actual market prices for calculating profit."
      /days numdays [number!] "Number of days of data to feed to GP at
      a time."
      /verbose
      /local ent-price ent-price-actual cur-contracts val t
      ] [
      ;;; verbose: 1 ;;; Override verbose setting for now...
      numdays: any [numdays 100]
      cur-contracts: 0
      ent-price: 0.0
      ent-price-actual: 0.0
      profit: 0.00
      ;;; Set up the array to pass the market data to the GP tree...
      m: array numdays
      repeat i numdays [
      poke m i market/(numdays - i + 1)
      ]
      for day numdays ((length? market) - 1) 1 [
      ;;; Update the matrix with today's data
      poke m 1 market/:day
      ;;; ent-price must be updated before each run of the tree so
      that its value is relative
      ;;; to the current closing price...
      t: actual/:day
      if verbose [ print t/date ]
      if ent-price-actual > 0.0 [
      ent-price: log-e (ent-price-actual / t/close)
      ]
      ;;; Process the data through the tree, interpret the
      results, play the market
      val: do tree
      ;;; Three cases... currently out of the market, currently
      long, or currently short...
      case [
      zero? cur-contracts [
      ;; Currently not in the market... we can go long, go
      short, or stay out...
      case [
      zero? val [
      ;; Stay out of the market
      if verbose [ print "Stay out of the
      market." ]
      ]
      val > 0.0 [
      ;; Go long
      t: actual/(day + 1)
      ent-price-actual: t/open
      cur-contracts: 1
      profit: profit - commission
      if verbose [ print rejoin [ "Going long...
      at " ent-price-actual ]]
      ]
      val < 0.0 [
      ;; Go short
      t: actual/(day + 1)
      ent-price-actual: t/open
      cur-contracts: -1
      profit: profit - commission
      if verbose [ print rejoin [ "Going short...
      at " ent-price-actual ]]
      ]
      ]
      ]
      cur-contracts > 0 [
      ;; Currently long in the market... we can stay in
      the market or exit our long position
      case [
      zero? val [
      ;; Stay long
      if verbose [ print "Staying long" ]
      ]
      val > 0.0 [
      ;; Stay long
      if verbose [ print "Staying long." ]
      ]
      val < 0.0 [
      ;; Exit market
      t: actual/(day + 1)
      profit: profit - commission
      profit: profit + (scale-factor * (t/open -
      ent-price-actual))
      if verbose [ print rejoin [ "Exiting long
      position... at " t/open ", profit=" (scale-factor * (t/open - ent-
      price-actual))]]
      ent-price-actual: 0.0
      cur-contracts: 0
      ]
      ]
      ]
      cur-contracts < 0 [
      ;; Currently short in the market... we can stay in
      the market or exit our short position
      case [
      zero? val [
      ;; Stay short
      if verbose [ print "Staying short" ]
      ]
      val > 0.0 [
      ;; Exit market
      t: actual/(day + 1)
      profit: profit - commission
      profit: profit + (scale-factor * (ent-price-
      actual - t/open))
      if verbose [ print rejoin [ "Exiting short
      position... at " t/open ", profit=" (scale-factor * (ent-price-
      actual - t/open))]]
      ent-price-actual: 0.0
      cur-contracts: 0
      ]
      val < 0.0 [
      ;; Stay short
      if verbose [ print "Staying short." ]
      ]
      ]
      ]
      ]

      ;;; Now, shift the old data over to the right in preparation
      for the new day's data...
      my-shift-right m
      ]
      ;; Return the final profit in the account... the higher the
      better.
      profit
      ]
    • Lee Spector
      Glenn, Although I haven t digested your REBOL code I think I can say that the answer is yes -- you can iterate over whatever you want in a fitness function
      Message 2 of 2 , Nov 15, 2006
      • 0 Attachment
        Glenn,

        Although I haven't digested your REBOL code I think I can say that
        the answer is "yes" -- you can iterate over whatever you want in a
        fitness function for koza-gp.lisp, just as long as you return a
        standardized fitness value and a number of hits (the latter of which
        is used only in termination tests and in reports). Depending on the
        details of your problem you may or may not want your iteration to be
        handled as stepping through "fitness cases" as is done for Koza's
        regression example -- one alternative is to do all of your iteration
        in a single fitness case. In any event there's a lot of flexibility
        here, and I've used Koza's code for many problems of many different
        types.

        -Lee


        On Nov 14, 2006, at 3:38 PM, Glenn M. Lewis wrote:

        > Hi all! I finally bit the bullet and started writing my own GP
        > framework because I was having difficulties figuring out how to
        > apply my particular problem (financial market trading by rule
        > creation) to any of the existing GP frameworks.
        >
        > I chose to implement the GP framework in REBOL
        > (http://www.rebol.com/) because its speed impressed me in the past
        > and also because it has the nice property of "code is data is code",
        > like Common Lisp. I would have chosen Common Lisp if I weren't such
        > a dog-gonned newbie at it.
        >
        > So I finally got the random tree code generation working and the
        > fitness function so that I can generate and evaluate random
        > individuals for fitness. At this point, I generate a population of
        > 100 individuals all of which generate a positive profit (by
        > regenerating each individual with a new random tree until it proves
        > profitable).
        >
        > I'm at the point where I need to start writing all the crossover,
        > mutation, etc. stuff that would truly make it a GP framework, and
        > I'm a bit intimidated now.
        >
        > I finally found a version of John Koza's "koza-gp.lisp" on the
        > internet that compiles with Lisp In A Box (Franz Allegro Lisp), and
        > suddenly had the idea that maybe it is time for me to port my half-
        > GP framework over to Common Lisp so that I can take advantage of all
        > the cool work that John Koza did.
        >
        > So my question is... is it possible to convert the fitness function
        > listed below to a Common Lisp version that would be compatible
        > with "koza-gp.lisp"? My concern is that I don't think it allows for
        > a fitness function to iterate over a matrix of data (which
        > represents market history) like I am doing below.
        >
        > Thank you very much!
        > -- Glenn Lewis
        >
        > commission: 10.00
        > scale-factor: 100.00
        >
        > fitness-func: func [
        > tree [number! block!] "Randomly generated function tree."
        > market [block!] "Pre-processed market data."
        > actual [block!] "Actual market prices for calculating profit."
        > /days numdays [number!] "Number of days of data to feed to GP at
        > a time."
        > /verbose
        > /local ent-price ent-price-actual cur-contracts val t
        > ] [
        > ;;; verbose: 1 ;;; Override verbose setting for now...
        > numdays: any [numdays 100]
        > cur-contracts: 0
        > ent-price: 0.0
        > ent-price-actual: 0.0
        > profit: 0.00
        > ;;; Set up the array to pass the market data to the GP tree...
        > m: array numdays
        > repeat i numdays [
        > poke m i market/(numdays - i + 1)
        > ]
        > for day numdays ((length? market) - 1) 1 [
        > ;;; Update the matrix with today's data
        > poke m 1 market/:day
        > ;;; ent-price must be updated before each run of the tree so
        > that its value is relative
        > ;;; to the current closing price...
        > t: actual/:day
        > if verbose [ print t/date ]
        > if ent-price-actual > 0.0 [
        > ent-price: log-e (ent-price-actual / t/close)
        > ]
        > ;;; Process the data through the tree, interpret the
        > results, play the market
        > val: do tree
        > ;;; Three cases... currently out of the market, currently
        > long, or currently short...
        > case [
        > zero? cur-contracts [
        > ;; Currently not in the market... we can go long, go
        > short, or stay out...
        > case [
        > zero? val [
        > ;; Stay out of the market
        > if verbose [ print "Stay out of the
        > market." ]
        > ]
        > val > 0.0 [
        > ;; Go long
        > t: actual/(day + 1)
        > ent-price-actual: t/open
        > cur-contracts: 1
        > profit: profit - commission
        > if verbose [ print rejoin [ "Going long...
        > at " ent-price-actual ]]
        > ]
        > val < 0.0 [
        > ;; Go short
        > t: actual/(day + 1)
        > ent-price-actual: t/open
        > cur-contracts: -1
        > profit: profit - commission
        > if verbose [ print rejoin [ "Going short...
        > at " ent-price-actual ]]
        > ]
        > ]
        > ]
        > cur-contracts > 0 [
        > ;; Currently long in the market... we can stay in
        > the market or exit our long position
        > case [
        > zero? val [
        > ;; Stay long
        > if verbose [ print "Staying long" ]
        > ]
        > val > 0.0 [
        > ;; Stay long
        > if verbose [ print "Staying long." ]
        > ]
        > val < 0.0 [
        > ;; Exit market
        > t: actual/(day + 1)
        > profit: profit - commission
        > profit: profit + (scale-factor * (t/open -
        > ent-price-actual))
        > if verbose [ print rejoin [ "Exiting long
        > position... at " t/open ", profit=" (scale-factor * (t/open - ent-
        > price-actual))]]
        > ent-price-actual: 0.0
        > cur-contracts: 0
        > ]
        > ]
        > ]
        > cur-contracts < 0 [
        > ;; Currently short in the market... we can stay in
        > the market or exit our short position
        > case [
        > zero? val [
        > ;; Stay short
        > if verbose [ print "Staying short" ]
        > ]
        > val > 0.0 [
        > ;; Exit market
        > t: actual/(day + 1)
        > profit: profit - commission
        > profit: profit + (scale-factor * (ent-price-
        > actual - t/open))
        > if verbose [ print rejoin [ "Exiting short
        > position... at " t/open ", profit=" (scale-factor * (ent-price-
        > actual - t/open))]]
        > ent-price-actual: 0.0
        > cur-contracts: 0
        > ]
        > val < 0.0 [
        > ;; Stay short
        > if verbose [ print "Staying short." ]
        > ]
        > ]
        > ]
        > ]
        >
        > ;;; Now, shift the old data over to the right in preparation
        > for the new day's data...
        > my-shift-right m
        > ]
        > ;; Return the final profit in the account... the higher the
        > better.
        > profit
        > ]
        >
        >
        >

        --
        Lee Spector, Professor of Computer Science
        School of Cognitive Science, Hampshire College
        893 West Street, Amherst, MA 01002-3359
        lspector@..., http://hampshire.edu/lspector/
        Phone: 413-559-5352, Fax: 413-559-5438
      Your message has been successfully submitted and would be delivered to recipients shortly.