Loading ...
Sorry, an error occurred while loading the content.
 

HyperNEAT Parameters

Expand Messages
  • Jan v.d. Lugt
    Hi everyone, My name is Jan van der Lugt and I am doing some exploratory work together with Evert Haasdijk on the parameters used by HyperNEAT (version 2.6).
    Message 1 of 9 , Jul 8, 2010
      Hi everyone,

      My name is Jan van der Lugt and I am doing some exploratory work together with Evert Haasdijk on the parameters used by HyperNEAT (version 2.6). One of my concerns is the ranges for the parameters used by HyperNEAT. Some of them were quite easy to figure out (e.g. the probabilities), but for others I don't have the slightest clue about a sensible range. Below are the parameters and my ideas about sensible ranges:

      PopulationSize 120.0 [positive int]
      MaxGenerations 600.0 [positive int]
      DisjointCoefficient 2.0 [?]
      ExcessCoefficient 2.0 [?]
      WeightDifferenceCoefficient 1.0 [?]
      FitnessCoefficient 0.0 [?]
      CompatibilityThreshold 6.0 [?]
      CompatibilityModifier 0.3 [?]
      SpeciesSizeTarget 8.0 1 < x < PopulationSize
      DropoffAge 15.0 [?, probably generations?]
      AgeSignificance 1.0 [?]
      SurvivalThreshold 0.2 [?]
      MutateAddNodeProbability 0.03 [0.0-1.0]
      MutateAddLinkProbability 0.05 [0.0-1.0]
      MutateDemolishLinkProbability 0.00 [0.0-1.0]
      MutateLinkWeightsProbability 0.8 [0.0-1.0]
      MutateOnlyProbability 0.25 [0.0-1.0]
      MutateLinkProbability 0.1 [0.0-1.0]
      AllowAddNodeToRecurrentConnection 0.0 [boolean]
      SmallestSpeciesSizeWithElitism 5.0 1 < x < PopulationSize
      MutateSpeciesChampionProbability 0.0 [0.0-1.0]
      MutationPower 2.5 [?]
      AdultLinkAge 18.0 [?]
      AllowRecurrentConnections 0.0 [boolean]
      AllowSelfRecurrentConnections 0.0 [boolean]
      ForceCopyGenerationChampion 1.0 [boolean]
      LinkGeneMinimumWeightForPhentoype 0.0 [?, spelling error?]
      GenerationDumpModulo 10.0 [?]
      RandomSeed -1.0 [positive int, -1 for random]
      ExtraActivationFunctions 1.0 [boolean]
      AddBiasToHiddenNodes 0.0 [boolean]
      SignedActivation 1.0 [boolean]
      ExtraActivationUpdates 9.0 [?]
      OnlyGaussianHiddenNodes 0.0 [boolean]
      ExperimentType 15.0 [0, 6, 9, 10, 11, 12, 13, 15, 16, 24, 26]

      If anyone can (partially) complete this list, I would be very grateful! I don't need detailed explanations for each parameter, just some sensible ranges (ideally not too big) where good values might lie.

      Regards,
      Jan van der Lugt
    • Jeff Clune
      Hello. I am glad you are interested in HyperNEAT. I have not played with most of those parameters, but one thing I can tell you is that the experiment type
      Message 2 of 9 , Jul 12, 2010
        Hello. I am glad you are interested in HyperNEAT. I have not played with
        most of those parameters, but one thing I can tell you is that the
        experiment type parameter determines which experiment you run (e.g., one of
        them is my quadruped robot experiment, another is my modularity experiment,
        another is the boxes problem, etc.), so you do not want to sweep that at all
        (it should be set to your experiment).

        You can see which parameter values I used in my papers (it is published in
        most of them), but since I have not changed most of them I have not learned
        what appropriate ranges are. I also know the parameter values are listed in
        most/all of the UCF papers.

        Are you planning on doing sweeps by hand or using an automated algorithm?
        There is a CMA-ES algorithm for evolving evolutionary operators that I could
        point you too that has been successful at finding good parameter values in
        the past...if you do any work along those lines I would be fascinated to
        hear how it turns out.

        Good luck!

        PS. Sorry for the delayed response. A lot of the regulars on this list have
        been at GECCO all week.

        Best regards,
        Jeff Clune

        Digital Evolution Lab, Michigan State University
        jclune@...
        www.msu.edu/~jclune




        > From: "Jan v.d. Lugt" <janlugt@...>
        > Reply-To: "neat@yahoogroups.com" <neat@yahoogroups.com>
        > Date: Thu, 8 Jul 2010 14:51:34 +0200
        > To: "neat@yahoogroups.com" <neat@yahoogroups.com>
        > Subject: [neat] HyperNEAT Parameters
        >
        > Hi everyone,
        >
        > My name is Jan van der Lugt and I am doing some exploratory work together
        > with Evert Haasdijk on the parameters used by HyperNEAT (version 2.6). One
        > of my concerns is the ranges for the parameters used by HyperNEAT. Some of
        > them were quite easy to figure out (e.g. the probabilities), but for others
        > I don't have the slightest clue about a sensible range. Below are the
        > parameters and my ideas about sensible ranges:
        >
        > PopulationSize 120.0 [positive int]
        > MaxGenerations 600.0 [positive int]
        > DisjointCoefficient 2.0 [?]
        > ExcessCoefficient 2.0 [?]
        > WeightDifferenceCoefficient 1.0 [?]
        > FitnessCoefficient 0.0 [?]
        > CompatibilityThreshold 6.0 [?]
        > CompatibilityModifier 0.3 [?]
        > SpeciesSizeTarget 8.0 1 < x < PopulationSize
        > DropoffAge 15.0 [?, probably generations?]
        > AgeSignificance 1.0 [?]
        > SurvivalThreshold 0.2 [?]
        > MutateAddNodeProbability 0.03 [0.0-1.0]
        > MutateAddLinkProbability 0.05 [0.0-1.0]
        > MutateDemolishLinkProbability 0.00 [0.0-1.0]
        > MutateLinkWeightsProbability 0.8 [0.0-1.0]
        > MutateOnlyProbability 0.25 [0.0-1.0]
        > MutateLinkProbability 0.1 [0.0-1.0]
        > AllowAddNodeToRecurrentConnection 0.0 [boolean]
        > SmallestSpeciesSizeWithElitism 5.0 1 < x < PopulationSize
        > MutateSpeciesChampionProbability 0.0 [0.0-1.0]
        > MutationPower 2.5 [?]
        > AdultLinkAge 18.0 [?]
        > AllowRecurrentConnections 0.0 [boolean]
        > AllowSelfRecurrentConnections 0.0 [boolean]
        > ForceCopyGenerationChampion 1.0 [boolean]
        > LinkGeneMinimumWeightForPhentoype 0.0 [?, spelling error?]
        > GenerationDumpModulo 10.0 [?]
        > RandomSeed -1.0 [positive int, -1 for random]
        > ExtraActivationFunctions 1.0 [boolean]
        > AddBiasToHiddenNodes 0.0 [boolean]
        > SignedActivation 1.0 [boolean]
        > ExtraActivationUpdates 9.0 [?]
        > OnlyGaussianHiddenNodes 0.0 [boolean]
        > ExperimentType 15.0 [0, 6, 9, 10, 11, 12, 13, 15, 16, 24, 26]
        >
        > If anyone can (partially) complete this list, I would be very grateful! I
        > don't need detailed explanations for each parameter, just some sensible
        > ranges (ideally not too big) where good values might lie.
        >
        > Regards,
        > Jan van der Lugt
      • Ken
        ... These are the standard NEAT compatibility coefficients for the compatibility equation the determines how far apart two individuals are for the purposes of
        Message 3 of 9 , Jul 13, 2010
          Hi Jan, I can comment on some of these:

          --- In neat@yahoogroups.com, "Jan v.d. Lugt" <janlugt@...> wrote:
          >
          > Hi everyone,
          >
          > My name is Jan van der Lugt and I am doing some exploratory work together
          > with Evert Haasdijk on the parameters used by HyperNEAT (version 2.6). One
          > of my concerns is the ranges for the parameters used by HyperNEAT. Some of
          > them were quite easy to figure out (e.g. the probabilities), but for others
          > I don't have the slightest clue about a sensible range. Below are the
          > parameters and my ideas about sensible ranges:
          >
          > PopulationSize 120.0 [positive int]
          > MaxGenerations 600.0 [positive int]

          > DisjointCoefficient 2.0 [?]
          > ExcessCoefficient 2.0 [?]
          > WeightDifferenceCoefficient 1.0 [?]

          These are the standard NEAT compatibility coefficients for the compatibility equation the determines how far apart two individuals are for the purposes of speciation. In general, the question is how many gene differences are worth an average distance of 1.0 in weights. The numbers above say that an average difference of 1.0 in weights is like having two genes not shared between the two individuals. That is pretty reasonable.

          > FitnessCoefficient 0.0 [?]

          > CompatibilityThreshold 6.0 [?]

          This parameter relates to the coefficients above. It can be thought of as, how many genes not shared does it take for individuals to be considered in a different species. However, it is only relevant at the start because the next parameter lets it change.

          > CompatibilityModifier 0.3 [?]

          This number means the threshold changes at a rate of 0.3 per generation, which is reasonable. It does not change if the SpeciesSize Target is already met.

          > SpeciesSizeTarget 8.0 1 < x < PopulationSize

          > DropoffAge 15.0 [?, probably generations?]

          Yes, in 15 generations a species will be penalized if it is not making progress.

          > AgeSignificance 1.0 [?]

          > SurvivalThreshold 0.2 [?]

          Only the top 20% of each species is allowed to reproduce. Controls greediness within species.

          > MutateAddNodeProbability 0.03 [0.0-1.0]
          > MutateAddLinkProbability 0.05 [0.0-1.0]
          > MutateDemolishLinkProbability 0.00 [0.0-1.0]
          > MutateLinkWeightsProbability 0.8 [0.0-1.0]
          > MutateOnlyProbability 0.25 [0.0-1.0]
          > MutateLinkProbability 0.1 [0.0-1.0]
          > AllowAddNodeToRecurrentConnection 0.0 [boolean]
          > SmallestSpeciesSizeWithElitism 5.0 1 < x < PopulationSize
          > MutateSpeciesChampionProbability 0.0 [0.0-1.0]
          > MutationPower 2.5 [?]

          Mutations of weight go up to 2.5 in a single mutation. You wouldn't want it over 5.0 or so.

          > AdultLinkAge 18.0 [?]
          > AllowRecurrentConnections 0.0 [boolean]
          > AllowSelfRecurrentConnections 0.0 [boolean]
          > ForceCopyGenerationChampion 1.0 [boolean]
          > LinkGeneMinimumWeightForPhentoype 0.0 [?, spelling error?]
          > GenerationDumpModulo 10.0 [?]
          > RandomSeed -1.0 [positive int, -1 for random]
          > ExtraActivationFunctions 1.0 [boolean]
          > AddBiasToHiddenNodes 0.0 [boolean]
          > SignedActivation 1.0 [boolean]
          > ExtraActivationUpdates 9.0 [?]
          > OnlyGaussianHiddenNodes 0.0 [boolean]
          > ExperimentType 15.0 [0, 6, 9, 10, 11, 12, 13, 15, 16, 24, 26]
          >
          > If anyone can (partially) complete this list, I would be very grateful! I
          > don't need detailed explanations for each parameter, just some sensible
          > ranges (ideally not too big) where good values might lie.
          >

          Jason Gauci should also be able to comment on other variables that I didn't cover.

          ken
        • Jeff Clune
          Hi Ken. This question comes up a lot, and these answers are helpful. Would it be worthwhile to put this information on the NEAT users page under a What does
          Message 4 of 9 , Jul 14, 2010
            Hi Ken. This question comes up a lot, and these answers are helpful. Would
            it be worthwhile to put this information on the NEAT users page under a
            "What does each parameter do?" section? Apologies if that already exists
            somewhere.


            Best regards,
            Jeff Clune

            Digital Evolution Lab, Michigan State University
            jclune@...
            www.msu.edu/~jclune




            > From: Ken <kstanley@...>
            > Reply-To: "neat@yahoogroups.com" <neat@yahoogroups.com>
            > Date: Wed, 14 Jul 2010 06:52:28 -0000
            > To: "neat@yahoogroups.com" <neat@yahoogroups.com>
            > Subject: [neat] Re: HyperNEAT Parameters
            >
            >
            >
            > Hi Jan, I can comment on some of these:
            >
            > --- In neat@yahoogroups.com, "Jan v.d. Lugt" <janlugt@...> wrote:
            >>
            >> Hi everyone,
            >>
            >> My name is Jan van der Lugt and I am doing some exploratory work together
            >> with Evert Haasdijk on the parameters used by HyperNEAT (version 2.6). One
            >> of my concerns is the ranges for the parameters used by HyperNEAT. Some of
            >> them were quite easy to figure out (e.g. the probabilities), but for others
            >> I don't have the slightest clue about a sensible range. Below are the
            >> parameters and my ideas about sensible ranges:
            >>
            >> PopulationSize 120.0 [positive int]
            >> MaxGenerations 600.0 [positive int]
            >
            >> DisjointCoefficient 2.0 [?]
            >> ExcessCoefficient 2.0 [?]
            >> WeightDifferenceCoefficient 1.0 [?]
            >
            > These are the standard NEAT compatibility coefficients for the compatibility
            > equation the determines how far apart two individuals are for the purposes of
            > speciation. In general, the question is how many gene differences are worth
            > an average distance of 1.0 in weights. The numbers above say that an average
            > difference of 1.0 in weights is like having two genes not shared between the
            > two individuals. That is pretty reasonable.
            >
            >> FitnessCoefficient 0.0 [?]
            >
            >> CompatibilityThreshold 6.0 [?]
            >
            > This parameter relates to the coefficients above. It can be thought of as,
            > how many genes not shared does it take for individuals to be considered in a
            > different species. However, it is only relevant at the start because the next
            > parameter lets it change.
            >
            >> CompatibilityModifier 0.3 [?]
            >
            > This number means the threshold changes at a rate of 0.3 per generation, which
            > is reasonable. It does not change if the SpeciesSize Target is already met.
            >
            >> SpeciesSizeTarget 8.0 1 < x < PopulationSize
            >
            >> DropoffAge 15.0 [?, probably generations?]
            >
            > Yes, in 15 generations a species will be penalized if it is not making
            > progress.
            >
            >> AgeSignificance 1.0 [?]
            >
            >> SurvivalThreshold 0.2 [?]
            >
            > Only the top 20% of each species is allowed to reproduce. Controls greediness
            > within species.
            >
            >> MutateAddNodeProbability 0.03 [0.0-1.0]
            >> MutateAddLinkProbability 0.05 [0.0-1.0]
            >> MutateDemolishLinkProbability 0.00 [0.0-1.0]
            >> MutateLinkWeightsProbability 0.8 [0.0-1.0]
            >> MutateOnlyProbability 0.25 [0.0-1.0]
            >> MutateLinkProbability 0.1 [0.0-1.0]
            >> AllowAddNodeToRecurrentConnection 0.0 [boolean]
            >> SmallestSpeciesSizeWithElitism 5.0 1 < x < PopulationSize
            >> MutateSpeciesChampionProbability 0.0 [0.0-1.0]
            >> MutationPower 2.5 [?]
            >
            > Mutations of weight go up to 2.5 in a single mutation. You wouldn't want it
            > over 5.0 or so.
            >
            >> AdultLinkAge 18.0 [?]
            >> AllowRecurrentConnections 0.0 [boolean]
            >> AllowSelfRecurrentConnections 0.0 [boolean]
            >> ForceCopyGenerationChampion 1.0 [boolean]
            >> LinkGeneMinimumWeightForPhentoype 0.0 [?, spelling error?]
            >> GenerationDumpModulo 10.0 [?]
            >> RandomSeed -1.0 [positive int, -1 for random]
            >> ExtraActivationFunctions 1.0 [boolean]
            >> AddBiasToHiddenNodes 0.0 [boolean]
            >> SignedActivation 1.0 [boolean]
            >> ExtraActivationUpdates 9.0 [?]
            >> OnlyGaussianHiddenNodes 0.0 [boolean]
            >> ExperimentType 15.0 [0, 6, 9, 10, 11, 12, 13, 15, 16, 24, 26]
            >>
            >> If anyone can (partially) complete this list, I would be very grateful! I
            >> don't need detailed explanations for each parameter, just some sensible
            >> ranges (ideally not too big) where good values might lie.
            >>
            >
            > Jason Gauci should also be able to comment on other variables that I didn't
            > cover.
            >
            > ken
            >
            >
          • jgmath2000
            Hey there, Many of the parameters used in HyperNEAT are taken from NEAT and you can read about them in Ken s resposne (or in more detail in his dissertation).
            Message 5 of 9 , Jul 14, 2010
              Hey there,

              Many of the parameters used in HyperNEAT are taken from NEAT and you can read about them in Ken's resposne (or in more detail in his dissertation). Here is information on the parameters that you won't find there:

              FitnessCoefficient 0.0 [?]

              This is a compatibility coefficient based on the fitness. If two individuals have different fitness, they will be put into different species. If you wanted to use this, you would have to set it low enough that is done not overpower the other coefficients. I didn't find it very effective, so I suggest leaving it at 0.

              SpeciesSizeTarget 8.0 1 < x < PopulationSize

              This is the expected # of species. The compatibility threshold will dynamically adjust to try to fit the species into that size. I recommend PopulationSize/8. This will put about 8 individuals per species.

              AgeSignificance 1.0 [?]

              This is a multiplier to the fitness of young individuals. It does not mix will with elitism since it makes fitness based on individual age which is independent of the evaluation, so I recommend leaving it at 1, effectively disabling age significance.

              AdultLinkAge 18.0 [?]

              If an individual is chosen to be mutated, links that are less than this number in age are mutated 100% of the time, whereas the other links are mutated based on the mutation rates.

              AllowRecurrentConnections 0.0 [boolean]
              AllowSelfRecurrentConnections 0.0 [boolean]

              These allow for recurrent and self recurrent (self loop) connections.

              ForceCopyGenerationChampion

              This turns on/off elitism

              LinkGeneMinimumWeightForPhentoype 0.0

              If the absolute value of a link is less than this number, the link will not be expressed in the neural network, even though it exists in the genome.

              GenerationDumpModulo 10.0

              Every <x> generations in the xml output will contain every individual in the population. All other generations (except the final one) only contain the generation champion. This saves disk space and makes searching through the xml file faster.

              ExtraActivationUpdates 9.0

              When a network is first created, there are zeroes in all of the activation levels. This parameter encodes the number of times to activate the network when activating a fresh network.

              ExperimentType 15.0

              This parameter is discussed in the HyperNEAT manual. Basically, it is used in the ExperimentRun class to decide which experiment to run.

              Hope this helps, let me know if you have any additional questions,

              Jason G.


              --- In neat@yahoogroups.com, "Jan v.d. Lugt" <janlugt@...> wrote:
              >
              > Hi everyone,
              >
              > My name is Jan van der Lugt and I am doing some exploratory work together
              > with Evert Haasdijk on the parameters used by HyperNEAT (version 2.6). One
              > of my concerns is the ranges for the parameters used by HyperNEAT. Some of
              > them were quite easy to figure out (e.g. the probabilities), but for others
              > I don't have the slightest clue about a sensible range. Below are the
              > parameters and my ideas about sensible ranges:
              >
              > PopulationSize 120.0 [positive int]
              > MaxGenerations 600.0 [positive int]
              > DisjointCoefficient 2.0 [?]
              > ExcessCoefficient 2.0 [?]
              > WeightDifferenceCoefficient 1.0 [?]
              > FitnessCoefficient 0.0 [?]
              > CompatibilityThreshold 6.0 [?]
              > CompatibilityModifier 0.3 [?]
              > SpeciesSizeTarget 8.0 1 < x < PopulationSize
              > DropoffAge 15.0 [?, probably generations?]
              > AgeSignificance 1.0 [?]
              > SurvivalThreshold 0.2 [?]
              > MutateAddNodeProbability 0.03 [0.0-1.0]
              > MutateAddLinkProbability 0.05 [0.0-1.0]
              > MutateDemolishLinkProbability 0.00 [0.0-1.0]
              > MutateLinkWeightsProbability 0.8 [0.0-1.0]
              > MutateOnlyProbability 0.25 [0.0-1.0]
              > MutateLinkProbability 0.1 [0.0-1.0]
              > AllowAddNodeToRecurrentConnection 0.0 [boolean]
              > SmallestSpeciesSizeWithElitism 5.0 1 < x < PopulationSize
              > MutateSpeciesChampionProbability 0.0 [0.0-1.0]
              > MutationPower 2.5 [?]
              > AdultLinkAge 18.0 [?]
              > AllowRecurrentConnections 0.0 [boolean]
              > AllowSelfRecurrentConnections 0.0 [boolean]
              > ForceCopyGenerationChampion 1.0 [boolean]
              > LinkGeneMinimumWeightForPhentoype 0.0 [?, spelling error?]
              > GenerationDumpModulo 10.0 [?]
              > RandomSeed -1.0 [positive int, -1 for random]
              > ExtraActivationFunctions 1.0 [boolean]
              > AddBiasToHiddenNodes 0.0 [boolean]
              > SignedActivation 1.0 [boolean]
              > ExtraActivationUpdates 9.0 [?]
              > OnlyGaussianHiddenNodes 0.0 [boolean]
              > ExperimentType 15.0 [0, 6, 9, 10, 11, 12, 13, 15, 16, 24, 26]
              >
              > If anyone can (partially) complete this list, I would be very grateful! I
              > don't need detailed explanations for each parameter, just some sensible
              > ranges (ideally not too big) where good values might lie.
              >
              > Regards,
              > Jan van der Lugt
              >
            • Ken
              Jeff, that is a good idea. However, one point of confusion is that the particular parameters differ from one implementation to the next. Perhaps it s still
              Message 6 of 9 , Jul 14, 2010
                Jeff, that is a good idea. However, one point of confusion is that the particular parameters differ from one implementation to the next. Perhaps it's still worth including these answers anyway just to give some idea.

                ken

                --- In neat@yahoogroups.com, Jeff Clune <jclune@...> wrote:
                >
                > Hi Ken. This question comes up a lot, and these answers are helpful. Would
                > it be worthwhile to put this information on the NEAT users page under a
                > "What does each parameter do?" section? Apologies if that already exists
                > somewhere.
                >
                >
                > Best regards,
                > Jeff Clune
                >
                > Digital Evolution Lab, Michigan State University
                > jclune@...
                > www.msu.edu/~jclune
                >
                >
                >
                >
                > > From: Ken <kstanley@...>
                > > Reply-To: "neat@yahoogroups.com" <neat@yahoogroups.com>
                > > Date: Wed, 14 Jul 2010 06:52:28 -0000
                > > To: "neat@yahoogroups.com" <neat@yahoogroups.com>
                > > Subject: [neat] Re: HyperNEAT Parameters
                > >
                > >
                > >
                > > Hi Jan, I can comment on some of these:
                > >
                > > --- In neat@yahoogroups.com, "Jan v.d. Lugt" <janlugt@> wrote:
                > >>
                > >> Hi everyone,
                > >>
                > >> My name is Jan van der Lugt and I am doing some exploratory work together
                > >> with Evert Haasdijk on the parameters used by HyperNEAT (version 2.6). One
                > >> of my concerns is the ranges for the parameters used by HyperNEAT. Some of
                > >> them were quite easy to figure out (e.g. the probabilities), but for others
                > >> I don't have the slightest clue about a sensible range. Below are the
                > >> parameters and my ideas about sensible ranges:
                > >>
                > >> PopulationSize 120.0 [positive int]
                > >> MaxGenerations 600.0 [positive int]
                > >
                > >> DisjointCoefficient 2.0 [?]
                > >> ExcessCoefficient 2.0 [?]
                > >> WeightDifferenceCoefficient 1.0 [?]
                > >
                > > These are the standard NEAT compatibility coefficients for the compatibility
                > > equation the determines how far apart two individuals are for the purposes of
                > > speciation. In general, the question is how many gene differences are worth
                > > an average distance of 1.0 in weights. The numbers above say that an average
                > > difference of 1.0 in weights is like having two genes not shared between the
                > > two individuals. That is pretty reasonable.
                > >
                > >> FitnessCoefficient 0.0 [?]
                > >
                > >> CompatibilityThreshold 6.0 [?]
                > >
                > > This parameter relates to the coefficients above. It can be thought of as,
                > > how many genes not shared does it take for individuals to be considered in a
                > > different species. However, it is only relevant at the start because the next
                > > parameter lets it change.
                > >
                > >> CompatibilityModifier 0.3 [?]
                > >
                > > This number means the threshold changes at a rate of 0.3 per generation, which
                > > is reasonable. It does not change if the SpeciesSize Target is already met.
                > >
                > >> SpeciesSizeTarget 8.0 1 < x < PopulationSize
                > >
                > >> DropoffAge 15.0 [?, probably generations?]
                > >
                > > Yes, in 15 generations a species will be penalized if it is not making
                > > progress.
                > >
                > >> AgeSignificance 1.0 [?]
                > >
                > >> SurvivalThreshold 0.2 [?]
                > >
                > > Only the top 20% of each species is allowed to reproduce. Controls greediness
                > > within species.
                > >
                > >> MutateAddNodeProbability 0.03 [0.0-1.0]
                > >> MutateAddLinkProbability 0.05 [0.0-1.0]
                > >> MutateDemolishLinkProbability 0.00 [0.0-1.0]
                > >> MutateLinkWeightsProbability 0.8 [0.0-1.0]
                > >> MutateOnlyProbability 0.25 [0.0-1.0]
                > >> MutateLinkProbability 0.1 [0.0-1.0]
                > >> AllowAddNodeToRecurrentConnection 0.0 [boolean]
                > >> SmallestSpeciesSizeWithElitism 5.0 1 < x < PopulationSize
                > >> MutateSpeciesChampionProbability 0.0 [0.0-1.0]
                > >> MutationPower 2.5 [?]
                > >
                > > Mutations of weight go up to 2.5 in a single mutation. You wouldn't want it
                > > over 5.0 or so.
                > >
                > >> AdultLinkAge 18.0 [?]
                > >> AllowRecurrentConnections 0.0 [boolean]
                > >> AllowSelfRecurrentConnections 0.0 [boolean]
                > >> ForceCopyGenerationChampion 1.0 [boolean]
                > >> LinkGeneMinimumWeightForPhentoype 0.0 [?, spelling error?]
                > >> GenerationDumpModulo 10.0 [?]
                > >> RandomSeed -1.0 [positive int, -1 for random]
                > >> ExtraActivationFunctions 1.0 [boolean]
                > >> AddBiasToHiddenNodes 0.0 [boolean]
                > >> SignedActivation 1.0 [boolean]
                > >> ExtraActivationUpdates 9.0 [?]
                > >> OnlyGaussianHiddenNodes 0.0 [boolean]
                > >> ExperimentType 15.0 [0, 6, 9, 10, 11, 12, 13, 15, 16, 24, 26]
                > >>
                > >> If anyone can (partially) complete this list, I would be very grateful! I
                > >> don't need detailed explanations for each parameter, just some sensible
                > >> ranges (ideally not too big) where good values might lie.
                > >>
                > >
                > > Jason Gauci should also be able to comment on other variables that I didn't
                > > cover.
                > >
                > > ken
                > >
                > >
                >
              • Ken
                Jeff, I followed your suggestion and pasted some of the answers from this thread into the NEAT Users Page under, Can you explain some of the NEAT Parameters?
                Message 7 of 9 , Jul 18, 2010
                  Jeff, I followed your suggestion and pasted some of the answers from this thread into the NEAT Users Page under, "Can you explain some of the NEAT Parameters?" I didn't try to exhaustively explain all parameters, but included some of the explanations. (It's tough to explain all parameters since they vary by implementation.)

                  ken

                  --- In neat@yahoogroups.com, Jeff Clune <jclune@...> wrote:
                  >
                  > Hi Ken. This question comes up a lot, and these answers are helpful. Would
                  > it be worthwhile to put this information on the NEAT users page under a
                  > "What does each parameter do?" section? Apologies if that already exists
                  > somewhere.
                  >
                  >
                  > Best regards,
                  > Jeff Clune
                  >
                  > Digital Evolution Lab, Michigan State University
                  > jclune@...
                  > www.msu.edu/~jclune
                  >
                  >
                  >
                  >
                  > > From: Ken <kstanley@...>
                  > > Reply-To: "neat@yahoogroups.com" <neat@yahoogroups.com>
                  > > Date: Wed, 14 Jul 2010 06:52:28 -0000
                  > > To: "neat@yahoogroups.com" <neat@yahoogroups.com>
                  > > Subject: [neat] Re: HyperNEAT Parameters
                  > >
                  > >
                  > >
                  > > Hi Jan, I can comment on some of these:
                  > >
                  > > --- In neat@yahoogroups.com, "Jan v.d. Lugt" <janlugt@> wrote:
                  > >>
                  > >> Hi everyone,
                  > >>
                  > >> My name is Jan van der Lugt and I am doing some exploratory work together
                  > >> with Evert Haasdijk on the parameters used by HyperNEAT (version 2.6). One
                  > >> of my concerns is the ranges for the parameters used by HyperNEAT. Some of
                  > >> them were quite easy to figure out (e.g. the probabilities), but for others
                  > >> I don't have the slightest clue about a sensible range. Below are the
                  > >> parameters and my ideas about sensible ranges:
                  > >>
                  > >> PopulationSize 120.0 [positive int]
                  > >> MaxGenerations 600.0 [positive int]
                  > >
                  > >> DisjointCoefficient 2.0 [?]
                  > >> ExcessCoefficient 2.0 [?]
                  > >> WeightDifferenceCoefficient 1.0 [?]
                  > >
                  > > These are the standard NEAT compatibility coefficients for the compatibility
                  > > equation the determines how far apart two individuals are for the purposes of
                  > > speciation. In general, the question is how many gene differences are worth
                  > > an average distance of 1.0 in weights. The numbers above say that an average
                  > > difference of 1.0 in weights is like having two genes not shared between the
                  > > two individuals. That is pretty reasonable.
                  > >
                  > >> FitnessCoefficient 0.0 [?]
                  > >
                  > >> CompatibilityThreshold 6.0 [?]
                  > >
                  > > This parameter relates to the coefficients above. It can be thought of as,
                  > > how many genes not shared does it take for individuals to be considered in a
                  > > different species. However, it is only relevant at the start because the next
                  > > parameter lets it change.
                  > >
                  > >> CompatibilityModifier 0.3 [?]
                  > >
                  > > This number means the threshold changes at a rate of 0.3 per generation, which
                  > > is reasonable. It does not change if the SpeciesSize Target is already met.
                  > >
                  > >> SpeciesSizeTarget 8.0 1 < x < PopulationSize
                  > >
                  > >> DropoffAge 15.0 [?, probably generations?]
                  > >
                  > > Yes, in 15 generations a species will be penalized if it is not making
                  > > progress.
                  > >
                  > >> AgeSignificance 1.0 [?]
                  > >
                  > >> SurvivalThreshold 0.2 [?]
                  > >
                  > > Only the top 20% of each species is allowed to reproduce. Controls greediness
                  > > within species.
                  > >
                  > >> MutateAddNodeProbability 0.03 [0.0-1.0]
                  > >> MutateAddLinkProbability 0.05 [0.0-1.0]
                  > >> MutateDemolishLinkProbability 0.00 [0.0-1.0]
                  > >> MutateLinkWeightsProbability 0.8 [0.0-1.0]
                  > >> MutateOnlyProbability 0.25 [0.0-1.0]
                  > >> MutateLinkProbability 0.1 [0.0-1.0]
                  > >> AllowAddNodeToRecurrentConnection 0.0 [boolean]
                  > >> SmallestSpeciesSizeWithElitism 5.0 1 < x < PopulationSize
                  > >> MutateSpeciesChampionProbability 0.0 [0.0-1.0]
                  > >> MutationPower 2.5 [?]
                  > >
                  > > Mutations of weight go up to 2.5 in a single mutation. You wouldn't want it
                  > > over 5.0 or so.
                  > >
                  > >> AdultLinkAge 18.0 [?]
                  > >> AllowRecurrentConnections 0.0 [boolean]
                  > >> AllowSelfRecurrentConnections 0.0 [boolean]
                  > >> ForceCopyGenerationChampion 1.0 [boolean]
                  > >> LinkGeneMinimumWeightForPhentoype 0.0 [?, spelling error?]
                  > >> GenerationDumpModulo 10.0 [?]
                  > >> RandomSeed -1.0 [positive int, -1 for random]
                  > >> ExtraActivationFunctions 1.0 [boolean]
                  > >> AddBiasToHiddenNodes 0.0 [boolean]
                  > >> SignedActivation 1.0 [boolean]
                  > >> ExtraActivationUpdates 9.0 [?]
                  > >> OnlyGaussianHiddenNodes 0.0 [boolean]
                  > >> ExperimentType 15.0 [0, 6, 9, 10, 11, 12, 13, 15, 16, 24, 26]
                  > >>
                  > >> If anyone can (partially) complete this list, I would be very grateful! I
                  > >> don't need detailed explanations for each parameter, just some sensible
                  > >> ranges (ideally not too big) where good values might lie.
                  > >>
                  > >
                  > > Jason Gauci should also be able to comment on other variables that I didn't
                  > > cover.
                  > >
                  > > ken
                  > >
                  > >
                  >
                • Jeff Clune
                  Thanks jason. This is very helpful. A question below. ... What is the unit here? Is that 18 generations? Seems like that could really raise the mutation rate!
                  Message 8 of 9 , Jul 19, 2010
                    Thanks jason. This is very helpful. A question below.

                    > AdultLinkAge 18.0 [?]
                    >
                    > If an individual is chosen to be mutated, links that are less than this number
                    > in age are mutated 100% of the time, whereas the other links are mutated based
                    > on the mutation rates.

                    What is the unit here? Is that 18 generations? Seems like that could really
                    raise the mutation rate! Have you found that this helps performance? What
                    value do you recommend?


                    >
                    >
                    > --- In neat@yahoogroups.com, "Jan v.d. Lugt" <janlugt@...> wrote:
                    >>
                    >> Hi everyone,
                    >>
                    >> My name is Jan van der Lugt and I am doing some exploratory work together
                    >> with Evert Haasdijk on the parameters used by HyperNEAT (version 2.6). One
                    >> of my concerns is the ranges for the parameters used by HyperNEAT. Some of
                    >> them were quite easy to figure out (e.g. the probabilities), but for others
                    >> I don't have the slightest clue about a sensible range. Below are the
                    >> parameters and my ideas about sensible ranges:
                    >>
                    >> PopulationSize 120.0 [positive int]
                    >> MaxGenerations 600.0 [positive int]
                    >> DisjointCoefficient 2.0 [?]
                    >> ExcessCoefficient 2.0 [?]
                    >> WeightDifferenceCoefficient 1.0 [?]
                    >> FitnessCoefficient 0.0 [?]
                    >> CompatibilityThreshold 6.0 [?]
                    >> CompatibilityModifier 0.3 [?]
                    >> SpeciesSizeTarget 8.0 1 < x < PopulationSize
                    >> DropoffAge 15.0 [?, probably generations?]
                    >> AgeSignificance 1.0 [?]
                    >> SurvivalThreshold 0.2 [?]
                    >> MutateAddNodeProbability 0.03 [0.0-1.0]
                    >> MutateAddLinkProbability 0.05 [0.0-1.0]
                    >> MutateDemolishLinkProbability 0.00 [0.0-1.0]
                    >> MutateLinkWeightsProbability 0.8 [0.0-1.0]
                    >> MutateOnlyProbability 0.25 [0.0-1.0]
                    >> MutateLinkProbability 0.1 [0.0-1.0]
                    >> AllowAddNodeToRecurrentConnection 0.0 [boolean]
                    >> SmallestSpeciesSizeWithElitism 5.0 1 < x < PopulationSize
                    >> MutateSpeciesChampionProbability 0.0 [0.0-1.0]
                    >> MutationPower 2.5 [?]
                    >> AdultLinkAge 18.0 [?]
                    >> AllowRecurrentConnections 0.0 [boolean]
                    >> AllowSelfRecurrentConnections 0.0 [boolean]
                    >> ForceCopyGenerationChampion 1.0 [boolean]
                    >> LinkGeneMinimumWeightForPhentoype 0.0 [?, spelling error?]
                    >> GenerationDumpModulo 10.0 [?]
                    >> RandomSeed -1.0 [positive int, -1 for random]
                    >> ExtraActivationFunctions 1.0 [boolean]
                    >> AddBiasToHiddenNodes 0.0 [boolean]
                    >> SignedActivation 1.0 [boolean]
                    >> ExtraActivationUpdates 9.0 [?]
                    >> OnlyGaussianHiddenNodes 0.0 [boolean]
                    >> ExperimentType 15.0 [0, 6, 9, 10, 11, 12, 13, 15, 16, 24, 26]
                    >>
                    >> If anyone can (partially) complete this list, I would be very grateful! I
                    >> don't need detailed explanations for each parameter, just some sensible
                    >> ranges (ideally not too big) where good values might lie.
                    >>
                    >> Regards,
                    >> Jan van der Lugt
                    >>
                    >
                    >
                  • jgmath2000
                    Hey there, It is 18 generations. It seemed to help explore the search space more effectively when I tested it with Xor and a couple of other simple problems
                    Message 9 of 9 , Jul 20, 2010
                      Hey there,

                      It is 18 generations. It seemed to help explore the search space more effectively when I tested it with Xor and a couple of other simple problems back in 2006, but it would not hurt to make sure that this parameter setting is still valid now that we are evolving CPPNs and not ANNs. Of course, the same is true for all of the parameters when it comes down to it.

                      --- In neat@yahoogroups.com, Jeff Clune <jclune@...> wrote:
                      >
                      > Thanks jason. This is very helpful. A question below.
                      >
                      > > AdultLinkAge 18.0 [?]
                      > >
                      > > If an individual is chosen to be mutated, links that are less than this number
                      > > in age are mutated 100% of the time, whereas the other links are mutated based
                      > > on the mutation rates.
                      >
                      > What is the unit here? Is that 18 generations? Seems like that could really
                      > raise the mutation rate! Have you found that this helps performance? What
                      > value do you recommend?
                      >
                      >
                      > >
                      > >
                      > > --- In neat@yahoogroups.com, "Jan v.d. Lugt" <janlugt@> wrote:
                      > >>
                      > >> Hi everyone,
                      > >>
                      > >> My name is Jan van der Lugt and I am doing some exploratory work together
                      > >> with Evert Haasdijk on the parameters used by HyperNEAT (version 2.6). One
                      > >> of my concerns is the ranges for the parameters used by HyperNEAT. Some of
                      > >> them were quite easy to figure out (e.g. the probabilities), but for others
                      > >> I don't have the slightest clue about a sensible range. Below are the
                      > >> parameters and my ideas about sensible ranges:
                      > >>
                      > >> PopulationSize 120.0 [positive int]
                      > >> MaxGenerations 600.0 [positive int]
                      > >> DisjointCoefficient 2.0 [?]
                      > >> ExcessCoefficient 2.0 [?]
                      > >> WeightDifferenceCoefficient 1.0 [?]
                      > >> FitnessCoefficient 0.0 [?]
                      > >> CompatibilityThreshold 6.0 [?]
                      > >> CompatibilityModifier 0.3 [?]
                      > >> SpeciesSizeTarget 8.0 1 < x < PopulationSize
                      > >> DropoffAge 15.0 [?, probably generations?]
                      > >> AgeSignificance 1.0 [?]
                      > >> SurvivalThreshold 0.2 [?]
                      > >> MutateAddNodeProbability 0.03 [0.0-1.0]
                      > >> MutateAddLinkProbability 0.05 [0.0-1.0]
                      > >> MutateDemolishLinkProbability 0.00 [0.0-1.0]
                      > >> MutateLinkWeightsProbability 0.8 [0.0-1.0]
                      > >> MutateOnlyProbability 0.25 [0.0-1.0]
                      > >> MutateLinkProbability 0.1 [0.0-1.0]
                      > >> AllowAddNodeToRecurrentConnection 0.0 [boolean]
                      > >> SmallestSpeciesSizeWithElitism 5.0 1 < x < PopulationSize
                      > >> MutateSpeciesChampionProbability 0.0 [0.0-1.0]
                      > >> MutationPower 2.5 [?]
                      > >> AdultLinkAge 18.0 [?]
                      > >> AllowRecurrentConnections 0.0 [boolean]
                      > >> AllowSelfRecurrentConnections 0.0 [boolean]
                      > >> ForceCopyGenerationChampion 1.0 [boolean]
                      > >> LinkGeneMinimumWeightForPhentoype 0.0 [?, spelling error?]
                      > >> GenerationDumpModulo 10.0 [?]
                      > >> RandomSeed -1.0 [positive int, -1 for random]
                      > >> ExtraActivationFunctions 1.0 [boolean]
                      > >> AddBiasToHiddenNodes 0.0 [boolean]
                      > >> SignedActivation 1.0 [boolean]
                      > >> ExtraActivationUpdates 9.0 [?]
                      > >> OnlyGaussianHiddenNodes 0.0 [boolean]
                      > >> ExperimentType 15.0 [0, 6, 9, 10, 11, 12, 13, 15, 16, 24, 26]
                      > >>
                      > >> If anyone can (partially) complete this list, I would be very grateful! I
                      > >> don't need detailed explanations for each parameter, just some sensible
                      > >> ranges (ideally not too big) where good values might lie.
                      > >>
                      > >> Regards,
                      > >> Jan van der Lugt
                      > >>
                      > >
                      > >
                      >
                    Your message has been successfully submitted and would be delivered to recipients shortly.