Variable number of inputs
- I have run into at least two situations where I have a variable number of inputs. For example, one is a list of a list of values. All of the lists can be of different length, including the outer list. It seems to me that HyperNEAT could handle this.
When requesting the weights from the CPPN, could I use coordinates normalized to the size of the list meaning:
2 0.5, 1
3 0.33, 0.66, 1
4 0.25, 0.5, 0.75, 1
And do this for both the inner and outer lists.
Any comments on this type of approach or of handling variable sized inputs in general?
- The variable-length list idea is interesting because it is possible that HyperNEAT could help with this kind of situation in principle, but you have to be careful in practice because it will be very sensitive to the precise meaning of the data in the lists. That is, if the item at the start of the list means the same thing no matter how long the list, that situation is different than if the items don't really have a precise meaning tied to their positions.
I would guess that there are a lot of potential pitfalls in general because what you are doing during evolution is training the network to treat certain positions in a certain way. So if an item at x-position 0.25 has been used by the evolved substrate in a certain way and suddenly position 0.25 disappears when the list expands, then the network may not process the item at the same cardinal position in the list in the same way, for whatever that is worth.
However, in some cases, it may be that there is an aggregate kind of view of the whole list in which precise positions are not important or only relative relationships matter. In that case it is possible the idea can work. A situation like that is changing the resolution of an eye, wherein the image is spread across all the pixel positions in a similar way regardless of the resolution.
So it really requires some insight into the domain itself and will likely be tricky to get working.
--- In firstname.lastname@example.org, "joelshellman" <joel@...> wrote:
> I have run into at least two situations where I have a variable number of inputs. For example, one is a list of a list of values. All of the lists can be of different length, including the outer list. It seems to me that HyperNEAT could handle this.
> When requesting the weights from the CPPN, could I use coordinates normalized to the size of the list meaning:
> Length Coordinates
> 1 1
> 2 0.5, 1
> 3 0.33, 0.66, 1
> 4 0.25, 0.5, 0.75, 1
> And do this for both the inner and outer lists.
> Any comments on this type of approach or of handling variable sized inputs in general?
> Thank you.
- I find it curious that I have not found any information about handling
variable sized inputs (except one might consider a recurrent/temporal
network to provide for one limited degree of that) while the first two
applications (each in an unrelated domain from each other) for which
I'm seriously considering using neural nets both involve variable
sized inputs. Maybe I'm just lucky that way :)
Okay, so here are some more untested thoughts on the subject...
> then the network may not process the item at the same cardinal position in the list in the sameAh, and I think you've hit on something we can do safely assume, I
> way, ... in which precise positions are not important or only relative relationships matter...
think. If we have a variable sized list as input then absolute
position should not matter. If absolute position did matter, then we
- take all the absolute positions that did matter and create a fixed
- transform it into a variable length input made up of data and position tuples
So... I would suggest that cardinal position is irrelevant in this
case since if it was important, we could transform the input into a
fixed width input, or a variable input where cardinal position doesn't
What I'm trying to do then is have something that is capable of
summarizing the data in every possible way and find the ways that are
most important. So it should be able to:
- find any linear combination of all the data
- find how many transitions across a set of thresholds
- find the percentage of inputs that are greater than/less than/in
between a set of thresholds
- find patterns across thresholds
- find patterns of change
So, is there a way to have the above accomplished with variable sized
input... ie. would the network generalize between 0.25 and 0.2 and
0.33 in a desirable way?
Another thing to consider is adding additional "coordinates". If I
remember right, it seems that the checkers example mentioned including
x, y, and distance from the center as inputs to the CPPN. Similarly,
let's say for example the first and last items in a variable sized
list are particularly important. We could add another coordinate that
was -1 for the first item in the list, 1 for the last, and 0 for all
- Hi Joel,
I did some similar experiments in the past. I think what I did was quite similar to what Ken suggests: the positions are not related to the absolute index in the list, but are always between a fixed range (0-1). If you have just 3 items, then the positions will be 0, 0.5 and 1, if you have a hundred elements, then 0, 0.01...0.99,1
Unfortunately I did not have to much time to complete the experiment. The domain was a quite tricky one anyway, so the results that I got were not too reassuring, especially if the size of the list had a great variance. So I absolutely agree with Ken that it really has a lot of potential pitfalls.
Of course it does not mean that if you have enough time it's impossible to achieve some better results.
I also agree with him that it's really similar to changing the "resolution" and I think there would be a great potential in it (if anyone had a working solution): if you eventually have really big networks, which also means very high computation cost the evolution could be speed up a bit by starting with low resolution, and slowly increase it when you have some promising networks. So at the end you would probably need more generations, but less time.
BTW it's probably a bit easier to do then to work with the variable size list, as the later one is more or less equivalent to having the already evolved networks (I mean the one that works on the substrate) and change the resolution of the substrate. At this point evolution can not help you any more, that's why I think it's even harder...
I think some guys did some similar experiments, I just can't remember who they were and what was their results.