Loading ...
Sorry, an error occurred while loading the content.

61248Xilinx Neuron chip (was Re: [beam] Memristor)

Expand Messages
  • Richard Piotter
    Dec 26, 2013
    • 0 Attachment
      One addendum in regards to the decay line... Consider the decay line to be more similar to the general state of a neuron. Neurons are not merely controlled by electro-chemically generated synaptic impulses alone. There are purely chemical neurotransmitters, both in the synapse and in the general chemistry of the brain. I would consider the decay line to be a more global factor, a sort of chemical state of sorts that allows a neuron to be more or less in a state of conditionability vs forgettability.

      That said, sourcing it from other neuronal outputs is absolutely fine, and ultimately expected. One needs to realize the key to such circuits comes in fine balance. A pulse chain fed to the decay line that is too rapid will utterly wipe out any ability for other neuron to learn whatsoever. A pulse chain too slow would result in network saturation, the BEAM equivalent to a robo-seizure.

      There is great complexity in neural networks, and I think monitoring activity of the whole is important to finding a base level to base that decay rate on. Let's take a vision system as an example. Say we want to detect motion. One needs to have a network pass the entire visual field detected to it's adjacent neurons (in an inhibitory manner), to suppress the "world motion" that occurs as the robot moves. When an individual sensory input does not match the inhibitory pulses that inhibit the world motion, you have a case of object motion. Reading all the values of the neurons in each neural layer is important. Say as the robot moves, one area is bright, and another dark. in the bright area, the sensors are highly active. We collect that activity with an integrating neuron and feed a higher decay rate pulse to the neuron chips in that sensory layer to inhibit saturation. Likewise, in a darker area, the sensory neurons are less active, and we would read that global lack of activity, reducing the delay line pulses to a trickle, preventing total desaturation.

      That's just an idea of how one might use the decay line in a real world application.

      Richard Piotter
      richfiles


      Begin forwarded message:
      > From: <connor_ramsey@...>
      > Subject: Re: [beam] Memristor
      > Date: December 25, 2013 9:05:05 PM CST
      > To: <beam@yahoogroups.com>
      > Reply-To: beam@yahoogroups.com
      >
      > I'd prefer to source the decay pulse via neural input. If you think about it, this way the decay pulse can also be utilized as a regenerative pulse. If a neuron outputs to its own decay input, then every time the neuron fires, it will decrement its weight, but then quickly increment it again, effectively regenerating the neuron's condition. Also, I would like to suggest an improvement on your design: a multiple input NOR gate hooked up to the weight accumulator, so that when the weight is 0, the NOR gate triggers an oscillator that bypasses the comparator to produce a constant series of output pulses. This makes sense because the neuron then requires 0 input pulses to give an output pulse.
      > Anyway, the way I wanted to connect the neurons in my ZISC computer was by having the most active neurons processed first, but for that I need a compact multiple input binary comparator design to integrate into the neurons. Since I don't have such a thing at my disposal, I can't currently do that. So I'm using Bruce's H-net to serially process all neurons in numerical order. Each output pulse is buffered and sent to two places: 1) a pulse encoder, that encodes the pulse into its neuron's address value and sends it to the main bus, and 2) directly to an H-gate input. The output of each H-gate enables its encoder section, and thus queues the output to be sent. Obviously the H-net will handle numerically smaller addresses first, working its way up by nearest activated neuron. If any preceding neurons are triggered during the process, the H-net will skip back to them before proceeding. So in a way, the H-net does allow the computer to branch.
      > Of course, I suppose that organizing the processes by weight would bear no difference, seeing that the transmission system has to get every output to every input before each global pulse anyway, meaning that it would have no advantage over an H-net based system. But yes, the goal is to bus everything everywhere in between global pulses, so that everything is sure to be processed.
      >
      > Enjoy, Connor

      Begin forwarded message:

      > From: Richard Piotter <richfiles1@...>
      > Subject: Re: [beam] Memristor
      > Date: December 24, 2013 2:03:34 AM CST
      > To: beam@yahoogroups.com
      > Reply-To: beam@yahoogroups.com
      >
      > The decay line is a low pulse rate that increments the weight registers to "forget" their conditioning. The pulse must be slow enough to allow for an equilibrium between active input activity and low input activity. It can be sourced by sensory or other neuronal outputs, or it could just be a very slow (compared to typical neuronal activity) global pulse.
      >
      > It's meant to represent time in the digital neuron. Since it uses asynchronous clocking derived by either propagation delay of the inputs, or by sampling inputs for highs and lows (based on configuration), there is no clear means to represent time as a function of de-conditioning, or forgetfulness. The decay line allows this to be loosely emulated. Every time the neuron fires, it takes one less cumulative excitatory pulse to fire the nest time. The first time the neuron fires, it would take 32 input pulses to trigger it. the next time, it takes 31 pulses, then 30, 29, 28, and so on, till it reaches a 1 to 1 input pulse to output fire ratio. The decay line increases the number of input pulses required for it to fire by one pulse increment with every trigger of the decay line.
      >
      > It's horribly low resolution, and very crude means to create a neuron, but the fact you can fit multiple neurons on even Xilinx's smallest 9536 CPLD, a $7 chip, is where the apparent appeal lies, in my opinion. Nv and Nu circuits are very nice, but there is a clear lack of adaptability in the area of neuron approximation. Yeah, you can have a sensor or logic tie different resistor values into an Nv circuit, but it really isn't much like neurons. There isn't a lot of asynchronous pulse streams that represent analog outputs in BEAM... It's mostly linear pulse trains and pulse length/order modification.
      >
      > I don't know that many people would use this type of digital neuron in their circuits, or that it'd be easy to get it to actually do anything, but I feel like it would be nice for it to actually exist, incase someone actually does want to play with it! :)
      >
      > This is a VERY crude block diagram of the neuron. Sadly, I seem to have lost the logic level schematic, and am looking for it. It's a simple enough design, that I should be able to easily redo it from scratch, should I get the time.
      >
      > http://richfiles.solarbotics.net/robots/XilinxNeuron.gif
      >
      > I'll be busy pretty much all week, so i may not get to it till later, but I have more pics of the memristor progress... Though as I stated earlier, I seem to have had a low success rate... I honestly don't even know if the silver based circuit pen traces will work in place of aluminum.
      >
      > Richard Piotter
      > richfiles
      >
      >
      > Begin forwarded message:
      >> From: <connor_ramsey@...>
      >> Subject: Re: [beam] Memristor
      >> Date: December 23, 2013 11:18:50 PM CST
      >> To: <beam@yahoogroups.com>
      >> Reply-To: beam@yahoogroups.com
      >>
      >> For now I'm just going to assume that the decay line in the Xilinx neuron you described connects like the other two synapses. Which I just figured out how I'm going to connect all of the neurons in my ZISC computer.
      >> If I used a ROM matrix to coordinate all of the synaptic connections either in series or parallel, I need 3n^2 bits of ROM, where n = neuron count. Not a problem for small networks, but say I have 256 neurons to account for, and that makes 576KbB of ROM. Granted that's not nearly a lot at first glance, but that's just with 8bit addresses. When you get to 32 bits, that's 147,456 TbBs! No way that's happening!
      >> So Instead, I going to use a PAL as the "synapse memory", where whenever a neuron fires, it outputs its address number, which is fed into the PAL, and then the pulse is sent to every other address that has the neuron's address as one of its terms. Basically the communication between neurons will work by processing the neurons with the lowest weight values first, and more decayed neurons last. Neurons with equal weights are processed in numerical order. I'm still working out the logic for this process, as I'm working on the computer right now, out of impulse.
      >> But anyway, this way, for the same 100 neurons, only 40 bytes of synapse memory are required, and for 2^32 neurons, that's only 64MbBs for the synapse connections, and 16MbBs in total weight memory(assuming 32bit neurons).
      >>
      >> Anyway, enjoy, Connor
      >>
      >>
      >
      >
      > ------------------------------------
      >
      > Yahoo Groups Links
      >
      >
      >
    • Show all 64 messages in this topic