5184Re: When we cannot waste 1MB per datum.
- Jun 25, 2012Nice to remind the world that I existed also at 2002. :-)
Obviously, representing a single FC position is not something that
should, need to or even could consume a full 1MB RAM.
What I had in mind was AI stuff.
On Mon, 2012-06-25 at 17:14 +0300, Shlomi Fish wrote:
> Hi all,
> Omer Zak (CCed to this message) wrote this old (2002) Hackers-IL message:
> Titled "How to exploit 1MB data structures?" this message reads:
> The future computers will have terabytes of memory (mostly in disks, but
> some in RAM). They will be able to manipulate megabytes at the time
> needed by yesterday's computers to manipulate single bytes.
> I wonder how can all those extra bytes be harnassed. (AI anyone?)
> Let's try to use 1MB to represent the string "The beep has booped".
> The data structure will contain:
> Now I'd like to give a view from the other side of the fence: when have
> milliards (10^9s) of data items and we need to fit them all in RAM (first -
> 128 GB - now I also got access to a machine with 512 GB), because storing and
> retrieving them on disk is far too slow. This is the case of Freecell Solver's
> dbm_fc_solver where I've spent a lot of time reducing its RAM consumption:
> * http://tech.groups.yahoo.com/group/fc-solve-discuss/message/1114
> And there's another optimisation coming up:
> * http://tech.groups.yahoo.com/group/fc-solve-discuss/message/1115
> If every position in the game’s graph consumed 1MB of RAM, then I could fit
> at most 512*1024 of them in 512 GB of RAM, which is all I have at the moment.
> And I can reach much more than 512K of positions on my underpowered home
> computers. Furthermore, manipulating these 1MB data will likely be much more
> time-consuming than the compact ones I'm using at the moment. Some of my scans
> reached 3 milliard stored positions, and such an amount would have consumed
> roughly 3 peta bytes (3 * 10^15), which I suspect are beyond any single HPC
> machine at the moment (to say nothing of consumer computers).
> I think that if we have large amounts of RAM resources at our disposal, we
> should not carelessly waste them like Omer suggests (unless of course we ran
> into a use-case that necessitates that).
> Shlomi Fish
PHP - the language of the Vogons.
My own blog is at http://www.zak.co.il/tddpirate/
My opinions, as expressed in this E-mail message, are mine alone.
They do not represent the official policy of any organization with which
I may be affiliated in any way.
WARNING TO SPAMMERS: at http://www.zak.co.il/spamwarning.html