Browse Groups

• ## Chapter 17: Utility with stochastical processes - bellmann functions

(1)
• NextPrevious
• Hey there, i`ve read through the german edition of AIMA, so please excuse if i don t always use correct english terminology. Chapter 17 describes how to
Message 1 of 1 , Aug 14, 2012
View Source
Hey there,

i`ve read through the german edition of AIMA, so please excuse if i don't always use correct english

terminology.
Chapter 17 describes how to calculate utilities for a process with non-deterministic, but defined states.
I'm currently working on an agent which can be decribed as follows:

- The agent works in a fully observable environment
- The actions have a probalistic outcome

The agent has to decide in regular timeframes (e.g. one-minute) whether or not to buy a certain good. The good

the agent has to buy is used from a process. The amount of usage is given by a trend-function overlayed with a

normal distributed function (this means the the mean of the usage is always known but uncertain; the

uncertainity is given by the variance of the overlayed normal distributed function).
The price for the good also follows a trendfunction overlayed by a normal dictributed function. If the agent

buys goods which are not used just in time there are costs for the warehouse.
The agent's objective is to assure that the using process always has enough goods to use and to buy the goods

as cheap as possible. If the agent fails to satisfy the process's need there is a high penalty. The agent can

only buy a given amount of the good at a time (the value is given by the current state of the warehouse and is

known by the agent but also uncertain for the future; there is a known maximum of goods the agent can buy at a

time).

My approach is to build an infinite binomial decision tree (actions: buy or not buy). The states are uncertain

and described by the above stated probalistic functions).

I think the utilities for every sequence of actions can be calculated using the bellman functions. However i'm unsure how to use these functions with probalistic outcomes.

Any hints or ideas?

Kind regards.
Sebastian
Your message has been successfully submitted and would be delivered to recipients shortly.
• Changes have not been saved
Press OK to abandon changes or Cancel to continue editing
• Your browser is not supported
Kindly note that Groups does not support 7.0 or earlier versions of Internet Explorer. We recommend upgrading to the latest Internet Explorer, Google Chrome, or Firefox. If you are using IE 9 or later, make sure you turn off Compatibility View.